Smart Mobile Communication & Artificial Intelligence: Proceedings of the 15th IMCL Conference – Volume 2 (Lecture Notes in Networks and Systems, 937) [2024 ed.] 3031560744, 9783031560743

Citation preview

Lecture Notes in Networks and Systems 937

Michael E. Auer Thrasyvoulos Tsiatsos   Editors

Smart Mobile Communication & Artificial Intelligence Proceedings of the 15th IMCL Conference – Volume 2

Lecture Notes in Networks and Systems

937

Series Editor Janusz Kacprzyk , Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

Advisory Editors Fernando Gomide, Department of Computer Engineering and Automation—DCA, School of Electrical and Computer Engineering—FEEC, University of Campinas— UNICAMP, São Paulo, Brazil Okyay Kaynak, Department of Electrical and Electronic Engineering, Bogazici University, Istanbul, Türkiye Derong Liu, Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, USA Institute of Automation, Chinese Academy of Sciences, Beijing, China Witold Pedrycz, Department of Electrical and Computer Engineering, University of Alberta, Alberta, Canada Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Marios M. Polycarpou, Department of Electrical and Computer Engineering, KIOS Research Center for Intelligent Systems and Networks, University of Cyprus, Nicosia, Cyprus Imre J. Rudas, Óbuda University, Budapest, Hungary Jun Wang, Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong

The series “Lecture Notes in Networks and Systems” publishes the latest developments in Networks and Systems—quickly, informally and with high quality. Original research reported in proceedings and post-proceedings represents the core of LNNS. Volumes published in LNNS embrace all aspects and subfields of, as well as new challenges in, Networks and Systems. The series contains proceedings and edited volumes in systems and networks, spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, Aerospace Systems, Automation, Manufacturing, Smart Grids, Nonlinear Systems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publication timeframe and the worldwide distribution and exposure which enable both a wide and rapid dissemination of research output. The series covers the theory, applications, and perspectives on the state of the art and future developments relevant to systems and networks, decision making, control, complex processes and related areas, as embedded in the fields of interdisciplinary and applied sciences, engineering, computer science, physics, economics, social, and life sciences, as well as the paradigms and methodologies behind them. Indexed by SCOPUS, INSPEC, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science. For proposals from Asia please contact Aninda Bose ([email protected]).

Michael E. Auer · Thrasyvoulos Tsiatsos Editors

Smart Mobile Communication & Artificial Intelligence Proceedings of the 15th IMCL Conference – Volume 2

Editors Michael E. Auer CTI Global Frankfurt/Main, Germany

Thrasyvoulos Tsiatsos Department of Informatics Aristotle University of Thessaloniki Thessaloniki, Greece

ISSN 2367-3370 ISSN 2367-3389 (electronic) Lecture Notes in Networks and Systems ISBN 978-3-031-56074-3 ISBN 978-3-031-56075-0 (eBook) https://doi.org/10.1007/978-3-031-56075-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Preface

IMCL2023 was the 15th edition of the International Conference on Interactive Mobile Communication, Technologies and Learning. This interdisciplinary conference is part of an international initiative to promote technology-enhanced learning and online engineering world-wide. The IMCL2023 covered all aspects of mobile learning as well as the emergence of mobile communication technologies, infrastructures and services and their implications for education, business, governments and society. The IMCL conference series actually aims to promote the development of mobile learning, to provide a forum for education and knowledge transfer, to expose students to latest ICT technologies and to encourage the study and implementation of mobile applications in teaching and learning. The conference was also a platform for critical debates on theories, approaches, principles and applications of mobile learning among educators, developers, researchers, practitioners and policymakers. IMCL2023 has been again organized by Aristotle University of Thessaloniki, Greece, from 09 to 10 November 2023. This year’s theme of the conference was “Smart Mobile Communication & Artificial Intelligence”. Again, outstanding scientists from around the world accepted the invitation for keynote speeches: • Minjuan Wang, Professor and Program Head, San Diego State University; Editorin-Chief, IEEE Transactions on Learning Technologies (TLT), USA: The Impact of Metaverse and Generative AI on Education. • Michalis Giannakos, Professor at Norwegian University of Science and Technology (NTNU), Norway: Multimodal Learning Analytics to Future Learning Systems. • Stavros Demetriadis, Professor at School of Informatics, Aristotle University of Thessaloniki, Greece: Harmonizing Minds: Navigating Human-AI Symbiosis in Learning Environments with Conversational AI. Furthermore, interesting workshops and tutorials have been organized: • Since its beginning this conference is devoted to new approaches in learning with a focus to mobile learning, mobile communication, mobile technologies and engineering education. • We are currently witnessing a significant transformation in the development of working and learning environments with a focus to mobile online communication. Therefore, the following main topics have been discussed during the conference in detail: • Mobile Learning Issues: • Dynamic learning experiences • Large scale adoption of mobile learning

vi

Preface

• • • • • • • • •

Ethical and legal issues Research methods and evaluation in mobile learning Mobile learning models, theory and pedagogy Life-long and informal learning using mobile devices Open and distance mobile learning Social implications of mobile learning Cost effective management of mobile Learning processes Quality in mobile learning Case studies in mobile learning

• Interactive Communication Technologies and Infrastructures: • • • • • • • • •

Wearables and Internet of Things (IoT) Tangible, embedded and embodied interaction Location-based integration Cloud computing Emerging mobile technologies and standards Interactive and collaborative mobile learning environments Crowd sensing 5G Network Infrastructure Platforms to support students’ mobility

• Mobile Applications: • • • • • • • • • • • •

Augmented-, Virtual-, Mixed- & Cross-Reality apps Smart cities Remote and Online laboratories Serious games and Gamification Mobile health care, healthy lifestyle and training Mobile apps for sports Mobile credentials, badges and Blockchain Learning analytics Mobile learning in cultural institutions and open spaces Mobile systems and services for opening up education Social networking applications Mobile Learning Management Systems (mLMS)

The following Special Sessions have been organized: • Interactive Learning Interfaces for Meaning and Expression (iLIME2023), Chair: Dionysios Politis, Aristotle University of Thessaloniki, Greece. • From Headsets to Mindsets: Human-Centred Extended Reality for Fostering Participation, Engagement and Co-Creation, Chairs: Petros Lameras, Centre for Post Digital Cultures, Coventry University, Coventry, UK, Nektarios Moumoutzis, Lab of Distributed Multimedia Information Systems and Applications, School of Electronics and Computer Engineering, Technical University of Crete, Sylvester Arnab, Centre for Post Digital Cultures, Coventry University, Coventry, UK, and Panagiotis Petridis, Aston University, UK.

Preface

vii

• Empowering Young Women in ICT by Fostering an Inclusive Technological Thinking (GIFT – IT), Chairs Ciupe Aurelia, Technical University of Cluj-Napoca, Romania. • Digital Technologies for Health and Sports, Chairs: Stella Douka, Aristotle University of Thessaloniki, Greece, and Thrasyvoulos Tsiatsos, Aristotle University of Thessaloniki, Greece. Also, a Doctoral Consortium has been organized in the context of IMCL2023, chaired by Christos Katsanos, Aristotle University of Thessaloniki, Greece, and Jenny Pange, University of Ioannina, Greece. As submission types have been accepted: • Full Paper, Short Paper, Doctoral Consortium Work in Progress, (with in person or distant/pre-recorded presentation) • Poster • Special Sessions • Round Table Discussions, Workshops, Tutorials and Students’ Competition All contributions were subject to a double-blind review. The review process was very competitive. We had to review about 146 submissions. A team of about 78 reviewers did this terrific job. Our special thanks go to all of them. Due to the time and conference schedule restrictions, we could finally accept only the best 77 submissions for presentation. The best papers were the following: • Category “Full Paper”: “Evaluation of Explainable Artificial Intelligence methods in Language Learning Classification of Spanish Tertiary Students” by Grigorios Tzionis (1), Gerasimos Antzoulatos (1), Periklis Papaioannou (1), Athanasios Mavropoulos (1), Ilias Gialampoukidis (1), Marta González Burgos (2), Stefanos Vrochidis (1), Ioannis Kompatsiaris (1) and Maro Vlachopoulou (3). Organization(s): (1): CERTH, Greece; (2): Metodo Estudios Consultores, Spain; (3): University of Macedonia, Greece. • Category “Short Paper”: “VR as a Tool for Enhancing Public Speaking Skills” by Aurelia Ciupe, Technical University of Cluj-Napoca, Romania. • Category “Work-in-Progress”: “Work-in-Progress: "Smart Print Automation" Remote Lab and Cloud Connector” by Christian Madritsch, Pierre Hohenberger, Benjamin Heindl and Valentin Smoly, Carinthia University of Applied Sciences, Austria. Our conference had again more than 144 participants from 30 countries. IMCL2025 will be held again at Aristotle University of Thessaloniki, Greece. Michael E. Auer IMCL Steering Committee Chair Thrasyvoulos Tsiatsos IMCL General Chair

Committees

Steering Committee Chair Michael E. Auer

CTI Global, Frankfurt/M., Germany

General Conference Chair Thrasyvoulos Tsiatsos

Aristotle University of Thessaloniki, Greece

International Chairs Samir A. El-Seoud Neelakshi. C. Premawardhena Alexander Kist Alaa Ashmawy David Guralnick Uriel Cukierman

The British University in Egypt (Africa) University of Kelaniya, Sri Lanka (Asia) University of Southern Queensland, Australia (Australia/Oceania) American University in Dubai (Middle East) Kaleidoscope Learning New York, USA (North America) University of Buenos Aires, Argentina (South America)

Technical Program Chairs Ioannis Stamelos Stavros Demetriadis Sebastian Schreiter

Aristotle University of Thessaloniki, Greece Aristotle University of Thessaloniki, Greece IAOE, France

Workshop, Tutorial and Special Sessions Chairs Andreas Pester Thrasyvoulos Tsiatsos

The British University in Egypt, Cairo, Egypt Aristotle University of Thessaloniki, Greece

x

Committees

Publication Chair Sebastian Schreiter

IAOE, France

Local Organization Chair Stella Douka

Aristotle University of Thessaloniki, Greece

Local Organization Committee Members Christos Temertzoglou

Aristotle University of Thessaloniki, Greece

Program Committee Members (TBC) Abul Azad Agisilaos Konidaris Anastasios Economides Anastasios Karakostas Anastasios Mikropoulos Apostolos Gkamas Carlos Travieso-Gonzalez Charalampos Karagiannidis Christos Bouras Christos Katsanos Christos Douligeris Christos Pierrakeas Daphne Economou David Jennings Demetrios Sampson Despo Ktoridou Dieter Wuttke Dimitrios Kalles Dionysios Politis Dominik May Doru Ursutiu George Magoulas George Palaigeorgiou

Northern Illinois University, USA Ionian University, Greece University of Macedonia, Greece Information Technologies Institute, Greece University of Ioannina, Greece University Ecclesiastical Academy of Vella of Ioannina, Greece Universidad de Las Palmas de Gran Canaria, Spain University of Thessaly, Greece University of Patras, Greece Aristotle University of Thessaloniki, Greece University of Piraeus, Greece University of Patras, Greece University of Westminster, UK University College, Ireland University of Piraeus, Greece University of Nicosia, Cyprus Technical University Ilmenau, Germany Hellenic Open University, Greece Aristotle University of Thessaloniki, Greece University of Georgia, Athens, USA University Transylvania Brasov, Romania Birkbeck College, UK University of Western Macedonia, Greece

Committees

Giasemi Vavoula Golberi S. Ferreira Gustavo Alves Helen Karatza Kostas Apostolou Maiga Chang María Isabel Pozzo Manuel Castro Maya Satratzemi Michail Giannakos Michalis Xenos Monica Divitini Nektarios Moumoutzis Nikolaos Avouris Nikolaos Tselios Olga Viberg Panagiotis Bamidis Panagiotis Petridis Petros Lameras Petros Nicopolitidis Rhena Delport Santi Caballé Stelios Xinogalos Stavros Nikou Stamatios Papadakis Tharenos Bratitsis Ting-Ting Wu Valery Varney Vassilis Komis

xi

University of Leicester, UK CEFET/SC, Brazil Polytechnic of Porto, Portugal Aristotle University of Thessaloniki, Greece McMaster University, Canada Athabasca University, Canada National Technological University, Argentina Universidad Nacional de Educación a Distancia, Spain University of Macedonia, Greece Norwegian University of Science and Technology, Norway University of Patras, Greece Norwegian University of Science and Technology, Norway Technical University of Crete, Greece University of Patras, Greece University of Patras, Greece KTH Royal Institute of Technology, Sweden Aristotle University of Thessaloniki, Greece Aston University, UK The Serious Games Institute, UK Aristotle University of Thessaloniki, Greece University of Pretoria, South Africa Open University of Catalonia, Spain University of Macedonia, Greece University of Strathclyde, UK The University of Crete, Greece University of Western Macedonia, Greece National Yunlin University of Science and Technology, Taiwan TH Cologne, Germany University of Patras, Greece

Contents

Women in ICT Empowering Tech Talents Among Young Women: Key Takeaways from a Panel Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aurelia Ciupe and Daniela Popescu

3

Pivoting Towards European University Alliances Supporting Technology Entrepreneurship for Women….. Insights from a Feminist Perspective . . . . . . . . Deirdre McQuillan

11

Women in Higher Education in Bulgaria: Achievements and Possibilities . . . . . . Yoana Pavlova Empowering Students in Technical Higher Education Through Teamwork and Alternative Assessment Methods. A Case Study . . . . . . . . . . . . . . . . . . . . . . . . Irina Duma, Andreia Molea, Nicolae Vlad Burnete, and Dan Moldovanu

25

33

Real World Experiences Parameterization and Modeling of Structural Designs for the Transformation of a Smart City . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anastasia Gasidou, Dimitrios Kotsifakos, and Christos Douligeris

45

Smart m-Observation and Study of Orthodox Art . . . . . . . . . . . . . . . . . . . . . . . . . . Desislava Paneva-Marinova, Detelin Luchev, Maxim Goynov, Emanuela Mitreva, Radoslav Pavlov, and Dušan Tati´c

57

Interactive Chatbot for Improving the Text Classification Data Quality . . . . . . . . Doaa S. Elzanfaly, Nada Amr Mohamed, and Nermin Abdelhakim Othman

63

Cultivating Computational Thinking in Early Years Through Board Games. The Cthink.it Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tharrenos Bratitsis, Maria Tsapara, Kiriaki Melliou, Leonard Busuttil, Diane Vassallo, James Callus, Gonçalo Meireles, Iro Koliakou, Nabil Tarraf Kojok, and Sofia Sousa Board Game Design by Children as an Assessment Mechanism in Kindergarten. A Case Study About Disability and Vulnerability . . . . . . . . . . . . Maria Tsapara and Tharrenos Bratitsis

78

90

xiv

Contents

Progressive Healthcare Pedagogy: An Application Merging ChatGPT and AI-Video Technologies for Gamified and Cost-Effective Scenario-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Matthew Pears, Cherry Poussa, and Stathis Th. Konstantinidis A Project Overview: The Implementation of a Native Android App for Mobile Signal Detection and a PHP Laravel Web-Based Platform for Real-Time Monitoring and Analysis of Wireless Communication Networks for Educational Purposes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Georgios Roussos and Petros Nicopolitidis Identifying the Most Mobile Content Sections Within a Course of Biosensors from the Last Decades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Cristian Ravariu, Gabriel Dima, Musala Sarada, Avireni Srinivasulu, and Bhargav Appasani Work-in-Progress: SYNERGIA, Towards an Online Communication and Collaboration Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Hippokratis Apostolidis, Spyridon Armatas, George Tsantikis, and Thrasyvoulos Tsiatsos The Development of Interdisciplinary Digital Learning Platform to Advance Digital Learning Strategic Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Fei Geng and Daniel D’Souza Artificial Intelligence in Power Distribution Systems . . . . . . . . . . . . . . . . . . . . . . . 151 Razvan Alexandru Moise and Aurel Fratu Adaptation of Internet of Things Technology to the Management of Educational Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Abdelghani Ait Ben Braim, Mustapha Raoufi, and Mohammed Skouri Motivation to Learn in Immersive Web Environments: Pilot Study . . . . . . . . . . . . 168 Bárbara Cleto, Carlos Santos, and Mário Vairinhos Music Recommendation Based on Face Emotion Recognition . . . . . . . . . . . . . . . . 180 Pallavi Ramsaran and Leckraj Nagowah Utilizing New Technologies for Children with Communication and Swallowing Disorders: A Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Eugenia I. Toki, Soultana Papadopoulou, and Jenny Pange Fostering a Co-creation Process for the Development of an Extended Reality Healthcare Education Resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Juliana Samson, Petros Lameras, Natasha Taylor, and Rosie Kneafsey

Contents

xv

Mobile Health Care, Healthy Lifestyle and Training Enhanced Web Platform for Optimizing Medical Fundraising for a Charitable Fund . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Nurkhan Issin, Azamat Salamat, Assanali Aidarkhan, and Mariza Tsakalerou A Brain-Computer Interface Application Based on P300 Evoked EEG Potentials for Enabling the Communication Between Users and Chat GPT . . . . . 226 Oana Andreea Rusanu Self-management of Type-2 Diabetes Using a Mobile Application: A Pilot Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Soulakshmee D. Nagowah, Abha Jodheea-Jutton, Kavi Kumar Khedo, Shakuntala Baichoo, Sudha Cheerkoot-Jalim, Leckraj Nagowah, and Zahra Mungloo-Dilmohamud Plant Disease Prediction Using Deep Learning Techniques . . . . . . . . . . . . . . . . . . 251 Widaad Fayid Hulkury and Leckraj Nagowah From Headsets to Mindsets – Human-Centred Extended Reality 3D Transformation of 2D Captured Museum Objects at Risk . . . . . . . . . . . . . . . . . 267 Maxim Goynov, Dušan Tati´c, Desislava Paneva-Marinova, Radomir S. Stankovi´c, Detelin Luchev, Emanuela Mitreva, and Lilia Pavlova From Data Abundance to Informed Citizenship: The Empowering Potential of the Dali Life Game for Data Literacy . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Petros Lameras, Sylvester Arnab, and Mark Lewis Rollick Games: A Formal Language Based Platform for Location-Based Pervasive Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 George Anestis, Nektarios Gioldasis, Stefanos Karasavvidis, Tzanis Palioudakis, Nektarios Moumoutzis, and Stavros Christodoulakis Perceptions and Challenges of Implementing XR Technologies in Education: A Survey-Based Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Filip Škola, Alexandra Karanasiou, Mike Triantafillou, Haris Zacharatos, and Fotis Liarokapis

xvi

Contents

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids Aged 7–12 Using Gaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Panagiotis Petridis, Vladlena Benson, Mariam Garibyan, Gonçalo Meireles, Alex Carpov, Dimitra Dimitrakopoulou, and Marisa Teles Remote and Online Laboratories Work-in-Progress: “Smart Print Automation” Remote Lab and Cloud Connector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 C. Madritsch, P. Hohenberger, B. Heindl, and V. Smoly Mobile Application Facilitating Agricultural Monitoring . . . . . . . . . . . . . . . . . . . . 326 D. Delioglani and A. Karakostas Open-Source Based Remote Control of Thermo-Optical Plant uDAQ28/LT . . . . 334 Ján Šefˇcík and Katarína Žáková Development of Teaching Methods Using Artificial Intelligence Techniques . . . 342 Abdelali El Gourari, Abdelghani Ait Ben Braim, Mustapha Raoufi, and Mohammed Skouri Work-in-Progress: Realization of IoT Lab Benches Controlled by a Raspberry Pi Accessible via the Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 Amadou Dahirou Gueye and Lamine Yade Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

Women in ICT

Empowering Tech Talents Among Young Women: Key Takeaways from a Panel Discussion Aurelia Ciupe(B)

and Daniela Popescu

Technical University of Cluj-Napoca, Cluj-Napoca, Romania [email protected]

Abstract. Promoting equality, equity, and inclusion – as reflections of societal diversity – aims to strengthen potential in IT&C through both education and research. New initiatives that encourage women to pursue prospective career paths in technology should be promoted as examples of good practices. These could further develop into pilot initiatives, raising awareness and revealing the importance of women’s representation within the DNA of innovation. Adopting an ecosystembased perspective for piloting a “Manifesto for Women in Tech” could highlight the role of women as key contributors to the field of digital technology. This reflection represents the qualities of good practice through its core principles, objectives, audience, supporting stakeholders, approach, and its potential for upscaling. Insights obtained from a ‘Women in Tech’ panel discussion underpin this strategy, offering unique perspectives from influential tech leaders with the goal of inspiring the next generation of Female tech innovators. Keywords: panel discussion · industry engagement · technology careers

1 Background and Context The underrepresentation of women in the European VC landscape represents a major constraint to the development of a strong innovation ecosystem, according to the premises that underpin a fragmented financial market in the New European Innovation Agenda [1]. Based on data from 2020 and 2021, a significant gender financing gap has been spotlighted in the European VC markets. In 2020, only 1.7% of the capital raised in European VC markets was captured by tech companies led by female founders. In 2021, male-only firms accounted for respectively about 90% and 84% of capital and deals concluded, against 1.1% and 5.4% reported for women-led companies, respectively. The gap remains in the spotlight when considering companies with male-female co-founders, which captured only 8.8% of the capital raised in 2021. With respect to Deep Tech Talents, one of the flagships that drive the New European Innovation Agenda, several premises address the gender gap and promote the diversity and inclusiveness in the tech sector. The European Commission is committed to reducing gender differences in study fields chosen that will help create a more equal society and workplace. Efforts have been undertaken to increase the number of female students in engineering, manufacturing and construction, and in ICT, as well as in increasing the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 3–10, 2024. https://doi.org/10.1007/978-3-031-56075-0_1

4

A. Ciupe and D. Popescu

number of male students in art and humanities, health and welfare, and in education. From an entrepreneurial perspective, the proportion of women starting businesses has increased in recent years, the rate of entrepreneurship is still higher for men than for women. In the EU, the women’s entrepreneurship rate is half that of men, where despite the wide variation in the rate of self-employment across EU countries, this inequality is present in all member countries. A panel discussion was conducted on three main challenges universal to women in science: gender wage gap, cultural views and support for young girls in science, and the necessity for women in the field [2]. It was clarified that ‘science’ covers all forms of the discipline, without distinction between hard and soft sciences. A session was held that highlighted women’s historic contributions to science and technology. It included the reading of draft monologues and a panel discussion. Audience interaction assisted in the development of the monologues and addressed emerging queries. The ultimate goal was a play about individual women scientists, to be performed independently as part of a larger, funded project. This aimed to educate, enlighten, and inspire through dramatizing their accomplishments. Another example of reporting a panel discussion [4] investigates the link between the diversity of management, particularly female top executives and board members, and a company’s performance. Using data from the 2,500 largest Danish firms from 1993 to 2001, the study found a positive impact of female representation in top management roles on firm performance. The positive effects depend on the qualifications of these female leaders. This study is notable for its substantial statistical evidence and effective control for causality direction, courtesy of a large and diverse data set. 1.1 PILOTING a “Manifesto for Women in TECH “ Piloting a Manifesto for Women in TECH has been proposed as a key measure within the Gender Equality Strategic Plan, under the EUt + alliance [2], framing the EUt + (https://www.univ-tech.eu/) mission towards a model of good practices for institutional integration, inclusion, and regional connectivity. The commitment of the 8 consortium partners on May, 19th, 2022 promoted its adoption as a core value of the EUt + identity and recognition as a flagship action to support a human-centered model of technology [5]. The presence of women in the TECH sector has been approached from 8 perspectives including: ● Female entrepreneurship ● Rewarding and financial compensations ● Work-life balance ● Gender equality ● Organizational culture ● Transparency and fairness in promotion ● Reintegration in the work environment after long-term breaks. A piloting framework in the form of an initiative has been proposed at national level, with the national support of the Authority for Digitization of Romania, as presented in Table.1.

Empowering Tech Talents Among Young Women

5

Table 1. EUT + Manifesto for Women in TECH Supporting principles P1. Women as drivers of a fresh perspectives and an innovative frame of mind in all environments and the technology field welcomes unlimited innovation P2. Championing the effort of rooting out inequality and discrimination against women will require us all to enact change starting today P3. Proving the capabilities of creatively finding new and comprehensive portfolios of roles for women in tech P4. Working towards consolidating STEM education and helping to reduce abandonment of tech studies, jobs and careers by women P5. Providing help, promoting role models, delivering mentoring, providing sponsoring opportunities, empowering skills, transferability and career advancing guidance for women across the tech world P6. Helping women forge a better path in technology, in a local, regional or continental, global and multi-generational context Objectives

O1. Stimulating and supporting access, development and maintenance of careers in TECH for young women in Romania, starting with the educational stage and including integration on relevant labor market

O2. Successful integration of young women in the TECH field, in order to increase their contribution to the field by promoting a mentoring and training system

O3. Eliminating gender discrimination, occupational barriers and prejudice that persist in this sector, both in the public and private sectors

Key action vectors

A1. Change of perspective

A2. Act against stereotypes

A3. Act against gender discrimination, occupational barriers and prejudice that persist in this sector, both in the public and private sectors

A4. Empower professional confidence and motivation

A5. Provide team support

A6. Organized and active communication through campaigns that make an impact (continued)

6

A. Ciupe and D. Popescu Table 1. (continued) A7. Empower women entrepreneurship in technology

A8. Increase A9. Involve governmental multi-stakeholder response and support effort to the needs of including more women in technology

A10. Commit to the cause of promoting diversity and inclusion just as important as reaching financial performance Roadmap

Phase 1. An Initiative Conference was held under the Innowave Summit 2022, in Cluj-Napoca, Romania, where the Manifesto Initiative and its roadmap were presented, focusing on highlighting, empowering, recognizing and promoting women and girls as tech talents. Ambassadors in technological fields in Romania were appointed from key industry players, as drivers for sharing experiences and success stories in the tech fields Phase 2. Series of workshops dedicated to women with interests in the tech industry with specific aims at: ● Encouraging participation in ICT education and continuing education in the field throughout life, in order to increase the chances of career development in this field characterized by accelerated innovation ● Supporting women’s entrepreneurial initiatives in the ICT field (setting up companies, identifying specializations, leadership and business communication team identification and motivation, sales and marketing, teamwork, etc.) ● Identifying and assuming professional opportunities, new roles, additional tasks, innovative projects and technologies ● Promoting an inclusive, non-discriminatory environment for women in the field Phase 3. Mentoring sessions with the scope of upskilling for a successful career in the tech industry, aiming at: ● Organizing internships/exchanges for women ICT students or who carry out professional activity in the field carried out at organizations / companies in the field ● Organizing experience exchange visits in locations that proved to successfully integrate women in the the TECH ● Allocating scholarships for women who want to develop a career in ICT Phase 4. Other supporting actions including common initiatives within the EUt + consortium level, with industry/community/institutional engagement, in accordance to the GEP strategic plan and the institutional ethos

Empowering Tech Talents Among Young Women

7

2 Key Take Aways from the “Women Leaders Sharing Personal Journeys in Tech, Engineering and Production Research” Panel Discussion The conference panel titled ‘Women Leaders sharing personal journeys in Tech, Engineering, and Production Research’ featured six accomplished female executives representing renowned global companies. They openly discussed their professional experiences, their involvement in noteworthy initiatives and programs, and the mix of opportunities and challenges that marked their careers. The conversation also allowed them to share insights on the industry-specific skills necessary for future growth in these fields. Their inspiring stories painted a vivid picture of their trajectory in the tech, engineering, and production research sectors and provided invaluable lessons for all attendees. 2.1 Panelists Sharing Their Backgrounds The Managing Director of NTT Romania (1st Panelist – P1) embarked on her technology career over two decades ago. Her fluency in various languages, rather than specific information technology skills, initiated her journey. Initially joining as a quality assurance tester, she gradually accumulated experience and ultimately advanced to the position of CEO. She underscores the pivotal role that talent management and recruitment skills play in the IT domain, in addition to stressing the necessity of technical prowess. The Education Director of Microsoft Romania’s (2nd Panelist – P2) initial fascination with mathematics led to a degree in programming. An unpredictable pivot in her interests directed her towards business school, coincidentally in-sync with Microsoft Romania’s inception post-revolution. What began as a collaborative project with Microsoft developed into a continuous 25-year association. Presently, she is engaged in the education sector, with a goal to make substantial contributions. The General Manager of Emerson in Cluj (3rd Panelist – P3) boasts a career extending over twenty years within the tech sector. Having begun her journey as an engineer for a local firm, she climbed the professional ladder to become a commercial director. Despite initial setbacks, her tenacity secured her a position at Emerson, where she eventually assumed responsibility for dual locations. The Commercial Lead at the Bosch Cluj Plant (4th Panelist – P4) traversed a unique trajectory, transitioning her attention from economics to technology. Her achievement in establishing the controlling department from inception at Danone culminated in a 13-year tenure and significant roles. Her career track grabbed the attention of Bosch, where she now contributes to company growth. A professor and chair at Ben Gurion University in Israel (5th Panelist – P5) expresses apprehension regarding the expanding influence of the government. She underscores potential implications on fundamental democratic values like academic research and women’s rights, and appeals for sustained initiatives to protect these privileges. The Vice-Rector at the Technical University (6th Panelist – P6) advocates for women to consider careers in production research and IT. As a core member of the International Foundation for Production Research’s board, her objective is to broaden the talent spectrum across diverse regions and promote grassroots initiatives.

8

A. Ciupe and D. Popescu

2.2 Panelists Sharing Key Strategies and Initiatives They Are Involved In The panel discussion highlighted various strategies and initiatives aimed at promoting gender diversity and empowering women in the tech industry. Several key takeaways emerged from the discussion: Balancing Gender Representation: P1 emphasized the importance of maintaining gender balance across all levels of an organization. NTT’s success in achieving nearequal gender representation, even in male-dominated sectors like automotive and manufacturing, serves as a benchmark for successful business operations. Training and Skill Development: P2 discussed post-pandemic efforts to provide free digital skills training in collaboration with Private Foundations. Initiatives like the “Women for IT” program empower women by equipping them with digital skills for tech jobs, fostering career growth and enhancing recognition through certifications. Collaborative Support: P3 highlighted collaborative efforts to support women in tech careers. Initiatives at regional and international levels, including partnerships with educational institutions and women’s engineering organizations, create an embracing environment for women. Mentorship, networking, and skill development programs play a pivotal role in this endeavor. Diversity Drives Innovation: P4 stressed the significance of diversity in driving innovation and competitiveness. Bosch’s decade-long initiatives to boost women’s presence, including the “Women at Bosch” project, focus on fostering women’s representation in leadership roles through inclusive training and cultural adaptability programs. Addressing Academic Disparities: P5 shared an initiative at Vancouver University to address gender disparities in academia, particularly at higher ranks. By mandating gender equality in major committees, the university aims to counter the “scissors diagram” trend where women’s representation decreases at higher academic levels, fostering a more equitable academic environment. Empowering Younger Generations and Broader Ambitions: P6 referred to multiple stakeholders to empower young girls and women in tech. Through initiatives in high schools and universities, the project aims to foster early interest and confidence in tech careers. The focus is on creating an embracing environment and providing role models for aspiring women in tech. Broader ambitions, extending beyond the local level to European consortiums and universities were highlighted. These initiatives reflect a commitment to adapting to rapid societal changes and equipping women with the skills to succeed in the evolving tech landscape.

2.3 Panelists Sharing Opportunities and Challenges Across Their Career Paths Responses reflected the challenges women face in overcoming biases and preconceptions, but also showcase positive changes and the determination to inspire and support the next generation of women in tech. P1 discussed the challenge of establishing her personal standards as valid benchmarks for success. Transitioning from looking externally for validation to defining her

Empowering Tech Talents Among Young Women

9

own standards was a difficult process. However, as CEO, she successfully transformed the management style to be more collaborative and participative. This shift allowed for an environment where any gender can thrive and develop equally, contributing to professional growth. P2 highlighted biases that persist in corporate settings, even within diverse cultures. She shared an example of encountering biases when joining the top management board, as the youngest and only female member. She recounts through an anectode facing biases over minor matters, related to personal belonging. However, she is proud to have contributed to changing the board’s perspective and believes that different views, including those of women, are essential for progress. Additionally, she emphasized the importance of engaging with high schools to encourage young girls to consider careers in technology. As a female leader in Eastern Europe, P3 revealed the multifaceted challenges encountered by women in leadership roles. Her story unveils the complex process of overcoming gender stereotypes and regional biases that persist in this context. Her experience underscores the need for perseverance and courage to challenge preconceptions. Her narrative also highlights the crucial role of fostering a supportive environment that encourages innovation, learning from mistakes, and cultivating a culture of continuous growth. P4 reflected on challenges and the need to fight against gender biases. Despite these obstacles, she made reference to statistics indicating a positive trend of women leading in IT&C. She emphasized the significance of inclusive leadership and fostering a fair culture within companies. According to her perspective, women possess qualities like effective communication, empathy, and resilience, which contribute to collaboration and inclusive management. She encourages women to maintain curiosity, self-confidence, and the ability to voice their opinions, while also supporting the next generation. P5 highlighted the challenge of using her voice effectively, both as a department chair and on faculty committees. She found that practice, supportive colleagues, and practical techniques like the “Wonder Woman posture” helped her overcome this challenge. She believes that while difficult, using one’s voice can be mastered, offering advice for other women to conquer this obstacle. P6 shared her personal experiences of being a woman in industry and academia. Despite being the only woman in her department and faculty for many years, she never felt discriminated against. She attributes this to her ability to take on leadership roles, becoming the dean of her faculty and later, the vice-rector. P6 highlighted women’s unique management style, including qualities like empathy, communication, and generosity. She also mentioned the blend of strength and softness that women often bring to their roles. 2.4 Panelists Sharing Skills for Careers in Technology P3 emphasized the importance of focusing on soft skills in addition to technical skills. She highlighted qualities such as dealing with ambiguity, embracing change, and positioning oneself effectively in a rapidly changing environment. She also recognizes the need for constant self-improvement and resilience.

10

A. Ciupe and D. Popescu

P4 reflected on the evolving nature of industries like automotive, which require continuous adaptation to new technical skills. She underscored the importance of curiosity, staying informed about current trends, and developing a flexible mindset to thrive in changing circumstances. P5 suggested that women should engage in politics and unions to influence policies that promote inclusion and broader societal change. P6 referred to the ability to learn how to learn is a crucial skill for the future, given the rapid pace of change. She highlighted the importance of adaptability and the capacity to acquire new knowledge and skills efficiently. P2 outlined four types of skills needed for the digital economy: basic digital skills, productivity skills, technological skills related to cloud computing, and soft skills. She also emphasizes the growing importance of cybersecurity and artificial intelligence as part of the digital alphabet. P1 added that mental well-being and mental hygiene are crucial skills in the modern world. She emphasizes the need to teach oneself and the younger generation how to maintain mental health in the face of information overload and changing dynamics.

3 Conclusions Within the context of the WiT initiative of the European University of Technology, a European University Alliance, the panel discussion among women representatives in executive positions in leading tech and engineering companies, underscored the importance of gender diversity in the tech industry and showcased a range of collaborative initiatives, mentorship programs, and inclusive practices that can help women thrive and contribute to the field’s innovation and growth. The collected insights define a focus group in the form of a panel of experts and is to be used as quantitative input for a multi-method further study.

References 1. European Commission, Communication From The Commission: A New European Innovation Agenda, COM/2022/332 final. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX: 52022DC0332. Accessed 25 July 2023 2. Van Staden, A., Ahmed, N., Getachew, Y., Gledhill, I., Kanjere, M., Khuluse-Makhanya, S., Das, S.: Gender shouldn’t matter because we are all scientists here’: a narration of the panel discussion at the 2nd international women in science without borders conference. South African J. Sci. 115(3–4), 1–4 (2016) 3. Schrader, C.B., Brown, J., Lubamersky, L., Madsen-Brooks, L., Pyke, P., Reeder, H.: Panel discussion: off the record-untold stories of women, science, and engineering. In: 2012 ASEE Annual Conference & Exposition, pp. 25–1022 (2012) 4. Smith, N., Smith, V., Verner, M.: Do women in top management affect firm performance? a panel study of 2,500 Danish firms. Int. J. Prod. Perform. Manag. 55(7), 569–593 (2006) 5. EUt+ Women in Tech Manifesto https://www.univ-tech.eu/medias/fichier/manifesto1_1653 403227208-pdf?ID_FICHE=28473&INLINE=FALSE. Accessed 25 July 2023

Pivoting Towards European University Alliances Supporting Technology Entrepreneurship for Women….. Insights from a Feminist Perspective Deirdre McQuillan(B) Technological University Dublin, Grangegorman, Dublin 7, Ireland [email protected]

Abstract. Interesting career possibilities for women emerge from technology entrepreneurship. Although the trend is improving, women entrepreneurs starting technology-based firms lag far behind men. The traditionally masculine identity of the ‘entrepreneur’ represents a major challenge for women interested to start a technology business. In the university ecosystem there is a ‘triple gendered’ situation – technology, technology studies and the gendered environment of academic incubation hubs that support start-up. This paper draws on a structured literature review (SLR) to frame the gender issues of women technology entrepreneurs within the university ecosystem. The aim of the paper is to develop insights for the emerging European University Alliances such as the European University of Technology (EUt +) to help fostering entrepreneurial ecosystems for women in technology fields. Keywords: entrepreneurship · start-up · women · gender · feminist theories

1 Introduction “For younger children, there is no difference in the use of computers, but at the age of 16 or 17 the gender gap grows. Computer software is usually developed for (and by) males, which may explain why boys tend to find computers more appealing and more easily develop confidence in using them.” (Rajahonka and Villman 2019, p15). Start-ups by women are low and women entrepreneurs starting technology-based firms fall far behind men (Cansiz and Teknec 2018). However, women-led high-tech start-ups are on the rise, albeit from a low base (Gupta and Etzkowitz 2021). Technology start-ups led by women are expected to boom in the coming years due to better awareness, education and infrastructure (EY 2020). This new generation of women technology entrepreneurs is recognized for its potential contribution to the technology industry in particular (Ozgen and Sanderson, 2006), and for the successful implementation of innovation policies (Demiralp et al. 2018). This rising trend in women led technology © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 11–24, 2024. https://doi.org/10.1007/978-3-031-56075-0_2

12

D. McQuillan et al.

start-ups can make an important contribution to social and economic evolution in general (Demartini 2018). Multiple studies suggest a link between technology-oriented firms led by women and higher growth (Azoulay et al. 2020; Hecker 2005). The term for such women founders “high-impact female entrepreneurs” has been coined by Aidis and Weeks (2016). These women are usually college-educated women who are growth oriented, constantly seeking out and expanding into new markets. How and why such women start and grow their companies is little understood however as few studies explore the phenomenon of high growth oriented women technology entrepreneurs (Swarz 2022). The importance of high growth technology companies to digital transition and job creation demands more data and research for deeper understanding and explanations. Governments and educators have long encouraged girls to pursue STEM subjects through wide ranging supports and initiatives. This gender gap in STEM education extends to computing. For example, in Ireland only one-in-five Irish computer science graduates are women (SFI 2021). Those statistics influence numbers of women entrepreneurs coming from science and technology fields. Only 17% of start-up founders in Europe are women (European Parliament 2021; Mari et al. 2021). Only 14.8% of ICT technology start-up founders are women in the EU (EPRS 2023). A substantive challenge lies in the traditionally masculine identity of the ‘entrepreneur’. For women, they attempt to ‘fit’ this norm or to feminize entrepreneurial identity (Ekinsmyth 2014; Lewis 2013; Mayes et al. 2020). For university education systems, while there may be an improving trend on women studying technology programmes, the persistent association of technology with masculinity within the educational system ensures that women will encounter systematic biases (Andrews 2019; Wajcman 2009). The necessity to explore stereotypes and gendered practices that discourages women participation and start-up career paths for women in technology education is gaining traction (e.g. Andrews 2019; Pujol and Montenegro 2015). For potential women technology entrepreneurs that enter academic incubator environments for the support necessary to overcome gender issues, they also meet gendered environments there. Reyes and Neergaard (2023) refer to this as a triple gendered situation – gendered technology, gendered technology studies and the gendered environment of academic incubation hubs that support start-up. Given the multidimensional challenges, gender plays a huge role in this field. It is important therefore to draw on feminist theory to understand women technology entrepreneurs in the university ecosystem. The paper draws on a structured literature review (SLR) to draw out gender issues of women technology entrepreneurs within the university ecosystem, a context has not yet received explicit focus. The aim of the paper is to develop insights for the emerging European University Alliances such as the European University of Technology (EUt + ) so they can foster strong entrepreneurial ecosystems for women in technology fields across university environments.

2 Literature Review Research on women technology entrepreneurship is usually engrained in the mainstream literature whether in the entrepreneurship field or in gender studies (Reyes and Neergard 2023). Therefore, research has not specifically addressed the idiosyncrasies and

Pivoting Towards European University Alliances Supporting Technology

13

complexities of the technology industry, its practices and the associated gender and power relations. That prevents women from participating in the wider university ecosystem and in many opportunities to explore and examine new topics concerning gender and to inform policy. Policymakers and the OECD are promoting the opportunities for economic growth and employment that could be driven by better rates in technology entrepreneurship by women (Treanor 2019). Traditionally gender and entrepreneurship as a focus of study are constructed from two comparatively young academic domains (Holmquist and Sundin 2020). The mainstream agenda in both the fields of entrepreneurship and in gender studies fails to acknowledge that entrepreneurship is increasingly normalised for women in terms of the workings of entrepreneurship and in terms of how women shape their working lives. Instead gender is usually studied as an added variable or with a comparative lens in the discourse. Theories and insights of how women explore and exploit technology start-up opportunities are not well understood. Scholars need to focus on generating domainspecific theories for the field. Empirical studies are needed adopting both quantitative and qualitative methods. There is also a need to account for differences in entrepreneurial contexts, whether social, geographic or demographic where gender and intersectionality result in various forms of discrimination. To lay the groundwork for exploring and examining important questions for women technology entrepreneurship within the university ecosystem, it is necessary to begin from what exists. To understand which feminist approaches are used in this particular problem of women’s technology entrepreneurship and to help in the analysis, the literature review covers three domains of feminist theory: liberal feminism, social feminism and poststructuralist feminism. These groups encapsulate the main feminist approaches in the literature (Reyes and Neergaard 2023; Alvesson and Billing 1997; Neergaard and Marlow 2011). 2.1 Liberal Feminism The argument of the liberal feminist tradition that institutional adjustments and policy shifts are necessary to open up traditionally male dominated areas is well established among entrepreneurship scholars (e.g. Calás et al. 2009; Ahl et al. 2006; Mayes and Norgaard 2023). Liberal feminism is also known as feminist empiricism or gender-as-avariable (Campbell 2004; Henry and Marlow 2014). It considers women and men to have similar capacities and that structures hinder women. For example, in entrepreneurship, studies indicate that it could be barriers to start a firm, access to financing, education and experience, that limits the success of women entrepreneurs (Ahl 2006; Fischer et al. 1993; Poggesi et al. 2016). By this reasoning it is necessary to eliminate barriers and change structures so that women are afforded the same opportunities as men (Ahl and Marlow 2012; Calas et al. 2009; Campbell 2004; Muntean and Ozkazanc-Pan 2015). They can then reach their start-up potential (Calas et al. 2009). For the liberal feminist, research focuses on comparing women to men in technology start-up and different dimensions in the stages of firm start-up and growth (Mirchandani 1999). The view does not consider that women may have unique experience and be exposed to different socialization processes (Poggesi et al. 2016).

14

D. McQuillan et al.

2.2 Social Feminism Conversely, social feminism is a feminist movement advocates for social rights and special accommodations for women. It considers women different to men based on their lived experience of how patriarchal societal structures oppress them (Ahl 2006; Campbell 2004). For scholars studying technology entrepreneurship from this dimension, gender differences must be incorporated into the research (Muntean and Ozkazanc-Pan 2015). Research in this perspective is often characterized by its discussion of women’s unique individual characteristics, their experiences, perspectives and contributions working as a technology entrepreneur (Bruni et al. 2005; Bui et al. 2018). 2.3 Post Structuralism Feminism Poststructuralist feminism considers gender to be socially-constructed and focuses on how gendered social practices construct dominant discourses (Bruni et al. 2005; Cal as et al. 2009; Henry and Marlow 2014). Poststructuralist research focuses on sociallyconstructed gendered practices leading to the emergence of groups of dominants and dominated, and individual experiences of gendered practices in entrepreneurship (Ahl and Marlow 2012; Bruni et al. 2004; Garcıa and Welter 2013). For example, Gupta and Etzkowitz (2020) adopt a post-structural feminist approach to explore identity formation and context modification within the socio-cultural context of Indian women’s high-tech entrepreneurial experiences. 2.4 Feminist Approaches to Women in Technology Entrepreneurship Within the women in technology entrepreneurship literature studies are emerging that confront entrepreneurship from a feminist perspective. Forty nine were identified in high quality peer reviewed journals in a 2022 literature search by Reyes and Neergaard (2023). Their findings by inductive analysis were classified similarly according to the three feminist perspectives of liberal feminism, social feminism and post structural feminism. The studies were then inductively aggregated into four dimensions: (1) antecedents (individual characteristics of women that influence how do they act entrepreneurially), (2) challenges (gender issues that women confront), (3) outcomes (what happens to the women and their firms as a result from gender discrimination); and (4) solutions (what the women do in response). Individual characteristics represent a key antecedent because they guide how women socialize and they play an important role in choices women feel they can make, how they act entrepreneurially, and how they perceive their entrepreneurial opportunity ecosystem (Birley 1989; Langowitz and Minniti 2007). For challenges, the authors highlight how the current literature captures challenges of discrimination and power relations at every stage of entrepreneurship. For outcomes the authors identify different levels of outcomes including individual outcomes, factors influencing careers choices and perceptions of others within the macro, meso and micro environment. Solutions according to Reyes and Neergaard emphasize women’s coping strategies and various recommendations to address gender issues for women pursuing careers as entrepreneurs in technology-based industries.

Pivoting Towards European University Alliances Supporting Technology

15

3 Methodology Past literature reviews focus on women entrepreneurs in technology fields (Reyes and Neergaard 2023; Pugalia and Cetindamar 2022; Kuschel and Lepeley 2016b) but not specifically focusing on technology entrepreneurship in higher education institutions and drawing on feminist perspectives. This review combines three areas in order to understand women technology entrepreneurs so that a deeper understanding of the issues for women can emerge within the higher education context. This understanding can help European University alliances such as EUt + to create ecosystems that encourage and foster future start-ups by women in technology and deep tech sectors, a growing priority of the European Commission.. Our literature review therefore includes 3 areas: (1) women in technology entrepreneurship, (2) researching women in technology entrepreneurship and (3) entrepreneurship education for women in STEM sectors. This paper uses a transparent and replicable procedure for the SLR that ensures objectivity in the choice of articles (Jesson et al. 2011). The aim of the SLR is to produce a qualitative judgement of papers selected. The author followed similar guidelines used for other women technology entrepreneurship studies (e.g. Reyes and Neergaard 2023) and also consulted the reference lists of all articles included to ensure all relevant scholarship on the topics was identified. The SLR methodology comprised five steps. First, a keyword search to gather literature sources on the topic of women technology entrepreneurship was conducted in three databases: (1) Business Source Complete, (2) Scopus and (3) Web of Science. The search strings used the following keywords: woman, women, female, gender AND tech*, hightech*, ICT AND entrepreneur*, founder*, owner*, venture*, start-up*. These terms were searched for in titles, abstracts and keywords for each article in each database, considering the following inclusion criteria (Step 1): (1) (2) (3) (4)

Publications in Academic Journals; Peer-reviewed journal articles; Published in all years between 2019 and 2023; Articles in English

This search process resulted in 3280 articles. The data set was cleaned up of duplicate articles within and across the three databases (Step 2). This reduced articles to 2186. Further exclusion criteria was applied in Step 3: (1) Sector: articles dealing with women entrepreneurs in fields other than technology were discarded. (2) The use of ICT: articles dealing the use of ICT, technology adoption were discarded; (3) Employees: articles focused on women employees, managers, top managers, women executives and women in boards of directors in firms were discarded; (4) Digital entrepreneurship and entrepreneurial intentions were only included to the extent that the informed our exploration of entrepreneurship education among technology students. Many articles were deleted in Step 3 as they did not fit the themes and indeed many were not about entrepreneurship despite being picked up in Step 1. Step 3 process resulted in 112 articles. After reading the papers to determine their inclusion, the lists

16

D. McQuillan et al.

were filtered to include articles published only in journals in the Academic Journal Guide (AJG) 2021. This guide signifies the scope and the quality of journals in the business and management domain. This infers quality and trustworthiness from the underlying studies (Step 4). After Step 4, 58 articles were left. Furthermore, we reviewed the reference and citation lists for the selected articles to identify other relevant articles, adding 36 articles to the topic of women technology entrepreneurship (Step 5). Step 6 organised the papers according to the areas of interest, namely a final data set of articles in the topic of women technology entrepreneurship, articles in the topic of researching women in technology, and articles in the topic of entrepreneurship education for women in technology within HEIs. Similar to the approach of Reyes and Neergaard (2023), the publications were firstly analysed to identify their feminist approach and then an inductive thematic analysis was used to map concepts. For the thematic analysis, articles were read in full. Excerpts were then coded, sub coded and combined to allow concepts and themes to emerge. The overarching themes represent a broad category of codes sharing similar concepts or ideas. Finally, using table and graphs we refined the themes and codes to establish theoretical underpinnings from the analysis (Creswell 2012; Miles and Huberman 1994).

4 Findings 4.1 The Importance of Context and Intersectionality in Universities There are calls for more research on gendered spaces and places as regards technology entrepreneurship (Treanor 2022). Many studies are qualitative and explored within geographic regions. It is recognised that findings will not be easily applicable in different country contexts, particularly in emerging or mature economies, or from high patriarchal societies. Recognising differences in geographic contexts is an important consideration for European University Alliances that are drawn from univerisites across all corners of Europe. For example, there are differences in the context in which a woman starts a firm depending on whether she starts it in Ireland, the USA, Romania or Kenya highlighting differences between disciplines and empirical settings. Moreover, the reliance on FDI multinationals in Ireland and strong supports for start-ups from the government has a particular dynamic for enterprise start-up relative to Pakistan for example. The Global Entrepreneurship Monitor that examines entrepreneurship on a global basis shows the vast range of different conditions between countries; legal, social, cultural and other conditions (GEP 2022). Studies show for example in the Canadian context, governments and non-governmental organizations strongly support women entrepreneurs including formal women’s only networks like incubators, accelerators, and associations through investment (Woodward, Worth, Schnar 2021). A fundamental way in which gender is expressed in any society is through technology. Technical skills and domains of expertise are distributed between and within the sexes shaping masculinities and femininities (Bray, 2007). Constraints referring to the largely established conception of male dominance can change. They are influenced by the wider context where new areas of negotiation can emerge as well as effective platforms for exchange of ideas and supportive networks (Törenli 2010; Güney-Frahm 2018).

Pivoting Towards European University Alliances Supporting Technology

17

Context is therefore significant in constructing entrepreneurial opportunity, and in navigating challenges of gender and entrepreneurship in universities. For example, in the process of construction of an entrepreneurial identity in India, women innovators modify their context to ensure positive results within an academic incubator both for gender dynamics and enhancing an emergent entrepreneurial culture (Gupta and Ecktzwitz 2021). On the importance of intersectionality, our understanding is challenged by the consideration that women entrepreneurs are usually treated as a homogenous group both in practice and in research. To explore how women’s entrepreneurial performance as a homogenous groups differs from their normative male counterparts reinforces a gender binary and reproduces gendered inequalities (Ahl 2006). Owalla et al. (2021) explore the intersection of gender, ethnicity, place and innovation, highlighting the scarcity and geographical clustering of women-led, technology sector SMEs in the UK. Treanor (2022) reminds us not only of the importance of place and regional access to institutional supports, but also how such access and opportunities differ due to social positioning. Inspired by previous studies (Adler and Izraeli 1994; Cetindamar and Beyhan 2019), Pugalia et al. (2021) categorizes the a range of causes from the literature to explain the lower participation of immigrant women in the technology sector by individual, firm and societal level barriers. 4.2 Technological Intentions for Students and Academics in Technology Venturing The gender-related barriers women face in this highly masculinised context of technology entrepreneurship are evidenced from the literature (Poggesi et al. 2020; Kuschel et al. 2020). Although entrepreneurship education can affect one piece of the puzzle – female STEM students’ intentions for technology venturing, the social desirability of such a career for women is still embedded in broader societal norms and expectations that those students need to navigate. There is world-wide evidence that women pursue different organizational and economic missions, and that there are significant variations in motivations (necessity versus opportunity) and intentions to start a business (Jennings and Brush 2013; Ladge et al. 2019). A recent study by Nikou, Brandwick, Carsou and Brush (2019) highlight differences by gender in the pathways that predict intentions to start-up a business. The technology industry is interesting because the conditions that women entrepreneurs meet in general are exacerbated by the “triple masculinity” and ingrained gendered practices (Reyes and Neergard 2023). Studies have found in high-tech entrepreneurship a professional site providing new career opportunities for women, as exemplified in the StartX accelerator at Stanford University, where gender-neutral environment has significantly enhanced women’s participation in start-ups (Etzkowitz 2013a).

18

D. McQuillan et al.

4.3 Start-Up Education in STEM Sectors Feminist Theory argues that systemic and contextual factors deprive women of certain resources in the socialization process. Of particular note is education and networks (Brush et al. 2018). Being deprived of those resources suggests that women may have different configurations of variables leading to entrepreneurial intentions relative to those for men (Treanor 2022). Considering the different views, a liberal feminist might argue for supports on education or network support (Brush et al. 2018) while the social feminist perspective would argue that the women’s early and ongoing socialization process might influence the formation of entrepreneurial self-efficacy (Bird and Brush, 2002; Piperopoulos and Dimov 2015). A recent study by Pereglova et al. (2023) shows that entrepreneurship education, measured as participation in an entrepreneurship course, has a stronger impact on feasibility, desirability and intentions for technology entrepreneurship for female STEM students compared to men. This study attests to the importance that entrepreneurship education can play for women students that may have entrepreneurial intentions. It supports the notion that entrepreneurship education could be part of a solution to counteract societal norms concerning technology entrepreneurship as a less desirable and feasible route for women students in STEM fields. Overall gendered studies about entrepreneurship education for technology academics and students remain scarce. There have been recent calls for unpacking the types of entrepreneurship education activities that work for women (Padilla-Angulo et al. 2022). Some scholars have questioned the ability of entrepreneurship education to motivate female students given the dominance of the “heroic male” narratives and case examples as well as the normative assumptions. Scholars identify some of these as equating to stereotypically male characteristics such as risk-taking, aggressiveness, where women are positioned as deficient and in need of extra support (Ahl 2007; Jones 2014; Siivonen et al. 2022; Sharen and McGowan 2018). Therefore, it is important to address this tension and assess whether entrepreneurship education can have the potential to motivate female students to pursue technology entrepreneurship. Studies suggesting that mentoring may not be as beneficial for women students within the package of educational supports for start-up in the HEI ecosystem (Henry, 2020) might have this particular problem of a lack of role models and male narrative stereotypes. Women who do engage in technology entrepreneurship demand research attention to showcase their achievements, thus providing those much needed role models for other women (Poggesi et al. 2020). Careful attention needs to be made concerning decisions about entrepreneurship education content. Course designers must understand and consider what course designs can deliver the most benefits for women students. 4.4 Academic Incubators for Women in Technology Many agree that one way of increasing women’s participation in technology entrepreneurship is support from business incubators (JPMorgan Chase 2017; NWBCCouncil 2017). Research however also suggests that incubators are gendered environments (Armitage and Feldman 2017; Buche and Scillitoe 2007).

Pivoting Towards European University Alliances Supporting Technology

19

Studies suggest that academic incubator environments tend to adopt a uniform approach to client support that does not recognise distinct needs or challenges of their women tenants (Treanor and Henry 2019). They are gendered environments. Treanor (2022) identifies gendered organisational practices within and amongst technology incubators, in tandem with societal norms. Similarly, Sperber and Linder (2022) in their study of blockchain start-ups explain how the gender imbalance supports the reproduction of the normative male tech entrepreneur. While maintaining such gendered organisational and sectoral cultures results in the marginalisation of women (Ozkazanc-Pan and Muntean 2018). Some of the issues emerging from studies finds that women do not receive sufficient mentorship and networking support to overcome various gender barriers (Lalkala 2003). Examples might be biases held by gatekeepers that result in their offering different support or putting emphasis on “fitting in” rather than revealing or challenging gender or other diversity and inclusion issues (Treanor 2022).

5 Limitations and Future Research This paper supports other scholars calling out the need for more empirical studies both qualitative and quantitative in women technology entrepreneurship. However the paper itself relies on arguments derived from a structured literature review. The aim of the paper is build some groundwork and ensure that the newly emerging European University alliances can evolve as models of success in fostering technology entrepreneurship among women, that they can support research that will give a real understanding from the feminist perspective. The paper promotes feminist perspectives as a powerful force for change to influence ecosystem development noting that attracting more women to technology entrepreneurship is a priority of the European Commission. Future research can support new empirical insights and grounded research that can build on the SLR contributing to new and enhanced theory.

6 Conclusion To conclude, technologies are transforming how business is done, opening up opportunities for women entrepreneurs to enter global value chains. Increasing women in technology is a strategic priority for European Commission and HEI’s are a cornerstone of facilitating this. Investment in women technology entrepreneurs can lead to lower levels of global poverty, better lives for children, and greater creativity and innovation. Women that want to start a technology venture in academia face the ‘triple masculinity’ challenge, navigating not only the world of tech and STEM education, but also the highly masculinised norms of entrepreneurship and academic incubators. Studies are emerging however to respond to these problems from feminist approaches that have the power to deeply explore and understand the challenges in HEIs, European University alliances and beyond. Finishing also appropriately with a quote from Dixon et al (2014) paper on gendered space - “gender shapes the construction and meanings of technology… But technology in turn also shapes gender roles” (Dixon et al. 2014, p994).

20

D. McQuillan et al.

References Adler, N.J., Izraeli, D.: Competitive Frontiers: Women Managers in a Global Economy. Blackwell, Cambridge, MA (1994) Ahl, H.: Why research on women entrepreneurs needs new directions. Entrep. Theory Pract. 30(5), 595–621 (2006) Aidis, R., Weeks, J.: Mapping the gendered ecosystem: the evolution of measurement tools for comparative high-impact female entrepreneur development. Int. J. Gend. Entrep. 8(4), 330–352 (2016). https://doi.org/10.1108/IJGE-12-2015-0044 Andrews, L.: The technology enterprise: systemic bias against women. UC Irvine Law Rev. 9(5) Women, Law, Society, & Technology (2019). https://scholarship.law.uci Armitage, C.J., Conner, M.: Efficacy of the theory of planned behavior: a meta-analytic review. Br. J. Soc. Psychol. 40(4), 471–499 (2001) Azoulay, P., Jones, B.F., Kim, J.D., Miranda, J.: Age and high-growth entrepreneurship. Am. Econ. Rev. Insights 2(1), 65–82 (2020). https://doi.org/10.1257/aeri.20180582 Bird, B.: Implementing entrepreneurial ideas: the case for intention. Acad. Manag. Rev. 13(3), 442–453 (1988). https://doi.org/10.5465/amr.1988.4306970 Birley, S.: Female entrepreneurs: are they really any different? J. Small Bus. Manage. 27(1), 32 (1989) Brush, C.G., Carter, N.M., Gatewood, E.J., Greene, P.G., Hart, M.M.: The use of bootstrapping by women entrepreneurs in positioning for growth. Ventur. Cap. 8(1), 15–31 (2006). https:// doi.org/10.1080/13691060500433975[2] Bray, F.: Gender and technology. Ann. Rev. Anthropol. 36(1), 37–53 (2007) Bruni, A., Gherardi, S., Poggio, B.: Entrepreneur-mentality, gender and the study of women entrepreneurs. J. Organ. Chang. Manag. 17(3), 256–268 (2004). https://doi.org/10.1108/095 34810410538315 Bruni, A., Silvia, G., Barbara, P.: Gender and Entrepreneurship: An Ethnographic Approach (2005). https://doi.org/10.4324/9780203698891 Bui, H.T., Kuan, A., Chu, T.T.: Female entrepreneurship in patriarchal society: motivation and challenges. J. Small Bus. Entrep. 30(4), 325–343 (2018) Buche, M.W., Scillitoe, J.L.: Influence of gender and social networks on organizational learning within technology incubators. Mid-Am. J. Bus. 22(1), 59–67 (2007). https://doi.org/10.1108/ 19355181200700005 Calás, M. B., Smircich, L., Bourne, K.A.: Extending the boundaries: reframing entrepreneurship as social change through feminist perspectives. Acad. Manag. Rev. 34(3), 552–569 (2009). http://www.jstor.org/stable/27760019 Campbell, K.: Quilting a feminist map to guide the study of women entrepreneurs. In: Narrative and Discursive Approaches in Entrepreneurship: A Second Movements in Entrepreneurship Book, Edward Elgar Publishing, Cheltenham (2004) Cansiz, M., Teknec_I, P.D.: Innovative and technology-based women entrepreneurs in Turkey: capital and performance. J. Econ. Cult. Soc. 151–183 (2018). 10.26650/ jecs372449 Carol Ekinsmyth.: Mothers business, work/life and the politics of mumpreneurship. Gend. Place Cult. 21(10), 1230–1248 (2014). https://doi.org/10.1080/0966369X.2013.817975 Cetindamar, D., Beyhan, B.: Using women entrepreneurship in technology ventures: gendered organization and gendered society interactions. In: Appio, F.P., Therin, F., Yoon, H. (Eds.) Handbook of Research on Techno-Entrepreneurial Ecosystems, 3rd ed., Edward Elgar, Cheltenham, UK (2019) Creswell, J.W.: Educational research: planning, conducting, and evaluating quantitative and qualitative research, 4th ed., Boston, MA: Pearson (2012)

Pivoting Towards European University Alliances Supporting Technology

21

Demartini, P.: Innovative female-led start-ups. Do women in business underperform? Adm. Sci. 8 (2018). https://doi.org/10.3390/admsci8040070 Demiralp, B., Morrison, L.T.R., Zayed, S.: On the commercialization path: entrepreneurship and intellectual property outputs among women in STEM. Technol Innov 19(4), 707–726 (2018). https://doi.org/10.21300/19.4.2018.707 Dixon, L.J., et al.: Gendered space: the digital divide between male and female users in internet public access sites. J. Comput. Mediat. Comm. 19, 991–1009 (2014). https://doi.org/10.1111/ jcc4.12088 Etzkowitz, H.: StartX and the ‘paradox of success’: filling the gap in Stanford’s entrepreneurial culture. Soc. Sci. Inf. 52(4), 605–627 (2013) European Parliament, On promoting gender equality in Science, Technology, Engineering and Mathematics (STEM) education and careers (2021). https://www.europarl.europa.eu/doceo/ document/A-9-2021-0163_EN.html EPRS. Women in the digital sector. At at Glance, European Parliamentary Research Service (2023). https://www.europarl.europa.eu/RegData/etudes/ATAG/2023/739380/EPRS_A TA(2023)739380_EN.pdf EY Global Review 2020; How will we reframe our future? Available at: https://assets.ey.com/con tent/dam/ey-sites/ey-com/en_gl/topics/global-review/2020/ey-global-review-2020-v3.pdf Fischer, E., Reuber, R., Dyke, L.S.: A theoretical overview and extension of research on sex, gender, and entrepreneurship. J. Bus. Ventur. 8(2), 151–168 (1993) Franco, M., Garcia, M.: Drivers of ICT acceptance and implementation in micro-firms in the estate agent sector: influence on organizational performance. Inf. Technol. Dev. 24(4), 658–680 (2018) García, M.-C. D., Welter, F.: Gender identities and practices: interpreting women entrepreneurs narratives. Int. Small Bus. J. 31(4), 384–404 (2013). https://doi.org/10.1177/026624261142 2829 GEM (2022). https://www.gemconsortium.org/reports/latest-global-report Gupta, N., Eckowitz, H.: Women founders in a high-tech incubator: negotiating entrepreneurial identity in the Indian socio-cultural context. Int. J. Gend. Entrep. 13(4), 353–372 (2021) Hecker, D.E.: High-technology employment: a NAICS-based update. Monthly Labor Rev. 128(7), 57–72 (2005). http://www.jstor.org/stable/23805498 Henry, C., Marlow, S.: Exploring the intersection of gender, feminism and entrepreneurship. In: Handbook of Research on Entrepreneurship, Edward Elgar Publishing, Cheltenham (2014) Henry, C., Lawton Smith, H., Meschitti, V., Foss, L., McGowan, P.: Networking, dender and academia: an ecosystems approach. In: H. Lawton Smith, C. Henry, H. Etzkowitz & A. Poulovassilis (Eds.) Gender, Science & Innovation: New perspectives, Cheltenham: Edward Elgar Publishing, pp.146–167 (2020) Holmquist, C., Sunden, E.: Is there a place for gender questions in studies on entrepreneurship, or for entrepreneurship questions in gender studies?. Int. J. Gender Entrepreneurship 12(2), 89–101 (2020) Jennings, J., Brush, C.: Research on women entrepreneurs: challenges to (and from) the broader entrepreneurship literature? Acad. Manag. Ann. 7(1), 663–715 (2013) Jesson, J., Matheson, L., Lacey, F.M.: Doing Your Literature Review: Traditional and Systematic Techniques. SAGE Publications, London (2011) JPMorganChase Creating inclusive high-tech incubators and accelerators: strategies to increase participation rates of women and minority entrepreneurs (2017). http://icic.org/wp-content/upl oads/2016/05/ICIC_JPMC_Incubators_post.pdf Jones, S.: Gendered discourses of entrepreneurship in UK higher education: The fictive entrepreneur and the fictive student. Int. Small Bus. J. 32(3), 237–258 (2014). https://doi. org/10.1177/0266242612453933

22

D. McQuillan et al.

Kuschel, K., Ettl, K., D_ıaz-Garc_ıa, C., Alsos, G.A.: Stemming the gender gap in STEM entrepreneurship – insights into women’s entrepreneurship in science, technology, engineering and mathematics. Int. Entrepreneurship Manag. J. 16, 1–15 (2020). https://doi.org/10.1007/s11 365-020-00642-5 Kuschel, K., Ettl, K., D_ıaz-Garc_ıa, C., Alsos, G.A.: Stemming the gender gap in STEM entrepreneurship – insights into women’s entrepreneurship in science, technology, engineering and mathematics. Int. Entrepreneurship Manag. J. 16(1), 1–15 (2020) Ladge, J., Eddleston, K.A., Sugiyama, K.: Am I an entrepreneur? How imposter fears hinder women entrepreneurs’ business growth”, Business Horizons (2019). https://doi.org/ https:// doi.org/10.1016/j.bushor.2019.05.001 Langowitz, N., Minniti, M.: The entrepreneurial propensity of women. Entrepreneurship Theory Pract. 31(3), 341–364 (2007). https://doi.org/10.1111/j.1540-6520.2007.00177.x Lewis, P.: The search for an authentic entrepreneurial identity: difference and professionalism among women business owners. Gend. Work Organ. 20, 252–266 (2013) Mayes and Norgaard 2023, [typo - Reyes and Neergaard, 2023 - in reference list] Mayes et al. 2020 [typo - Reyes et al] Mari, M., Poggesi, S., Foss, R.: Women’s Entrepreneurship in STEM Disciplines: Issues and Perspectives, Springer Nature, Canton of Zug (2021) Miles, M. B., Huberman, A.M.: Qualitative data analysis: an expanded sourcebook. thousand oaks, CA: Sage Publications (1994) Mirchandani, K.: Feminist insight on gendered work: new directions in research on women and entrepreneurship. Gend. Work Organ. 6(4), 224–235 (1999). https://doi.org/10.1111/14680432.00085 Mohanty, E., Mishra, A.: Understanding the gendered nature of developing country MSMEs’ access, adoption and use of information and communication technologies for development (ICT4D). Int. J. Gend. Entrep. 12(3), 273–295 (2020) Muntean, S., Ozkazanc-Pan, B.: A Gender integrative conceptualization of entrepreneurship. New England J. Entrepreneurship 18(1), 27–40 (2015). https://doi.org/10.1108/NEJE-18-01-2015B002 Neergaard, H., Marlow, S.: The emperor’s new clothes: rendering a feminist theory of entrepreneurship visible. In: Paper Presented at the 56th ICSB Conference, Stockholm (2011) Nikou, S., Brännback, M., Carsrud, A., Brush, C.: Entrepreneurial intentions and gender: pathways to start-up. Int. J. Gend. Entrep. 11(3), 348–372 (2019) NWBCCouncil. Women’s participation in business incubators and accelerators (2017). https://cdn.www.nwbc.gov/wp-content/uploads/2017/03/13133748/Womens-articipationin- Business-Incubators-and-Ac celerators_FINAL.pdf örenli, N.: The potential of ICT to generate ‘solidaristic’ practices among women home-based workers in Turkey. Technol. Work Employ. 25(1), 49–62 (2010) Orser, B., Riding, A., Li, Y.: Technology adoption and gender-inclusive entrepreneurship education and training. Int. J. Gend. Entrep. 11(3), 273–298 (2019) Owalla, B., Nyanzu, E., Vorley, T.: Intersections of gender, ethnicity, place and innovation: mapping the diversity of women-led SMEs in the United Kingdom. Int. Small Bus. J. 39(7), 681–706 (2021) Ozgen, E., Sanderson, S.: DO men and women entrepreneurs differ in their reliance on sources of information in opportunity recognition in technical fields? Acad. Entrepreneurship J. 12(2), 47–65 (2006) Ozkazanc-Pan, B., Muntean, S.: Networking towards (in)equality: women entrepreneurs in technology. Gend. Work Organ. 25(4), 379–400 (2018) Padilla-Angulo, L., Garc_ıa-Cabrera, A.M., Lucia-Casademunt, A.M.: Unpacking entrepreneurial education: learning activities, students’ gender, and attitude toward entrepreneurship. Acad. Manag. Learn. Educ. 21(4), 532-560 (2022).https://doi.org/10.5465/amle.2020.0043

Pivoting Towards European University Alliances Supporting Technology

23

Pergelova, A., Angulo-Ruiz, F., Manolova, T.S., Yordanova, D.: Entrepreneurship education and its gendered effects on feasibility, desirability and intentions for technology entrepreneurship among STEM students. Int. J. Gend. Entrep. 15(2), 191–228 (2023). https://doi.org/10.1108/ IJGE-08-2022-0139 Piperopoulos, P., Dimov, D.: Burst bubbles or build steam? entrepreneurship education, entrepreneurial self-efficacy, and entrepreneurial intentions. J. Small Bus. Manage. 53(4), 970–985 (2015) Poggesi, S., Mari, M., De Vita, L.: What’s new in female entrepreneurship research? answers from the literature. Int. Entrepreneurship Manag. J. 12(3), 735–764 (2016) Pugalia, S., Cetindamar, D.: Insights on the glass ceiling for immigrant women entrepreneurs in the technology sector. Int. J. Gend. Entrep. 14(1), 44–68 (2022) Pujol, J., Montenegro, M.: Technology and feminism: a strange couple. Rev. de Estudios Sociales 35(51), 173–185 (2015) Rajahonka, M., Villman, K.: Women managers and entrepreneurs and digitalization: on the verge of a new era or a nervous breakdown?. Technol. Innov. Manag. Rev. 9(6) (2019) Reyes, C., Neergaard, H.: Feminist perspectives on gender and technology entrepreneurship in incubator settings. Int. J. Gend. Entrep. 15(1), 64–92 (2023) Schwars, E., Skeepers, C., Tofae, T.: Women entrepreneurs’ opportunity identification of digital platform start-ups: emerging evidence from South Africa. Int. J. Gend. Entrep. 14(3), 352–374 (2022) SFI Just one-in-five Irish computer science graduates are women. Science Foundation Ireland Research News (2021). https://www.sfi.ie/research-news/news/codeplus-women-pathway/ Sharen, C., McGowan, R.: Invisible or clich_ed: how are women represented in business cases? J. Manag. Educ. 43(2), 129–173 (2018). https://doi.org/10.1177/1052562918812154 Siivonen, P.T., Komulainen, K., Kasanen, K., Kupiainen, P.: Aged and gendered master narratives on entrepreneurship in finnish higher education. Int. J. Gend. Entrep. 14(1), 26–43 (2022). https://doi.org/10.1108/IJGE-01-2021-0010 Sperber, S., Linder, C.: Gender bias in IT entrepreneurship: the self-referential role of male overrepresentation in digital businesses. Eur. J. Inf. Syst. 32(5), 902–919 (2022). https://doi.org/ 10.1080/0960085X.2022.2075801 Sveningsson, S., Alvesson, M.: Managing managerial identities: organizational fragmentation, discourse and identity struggle. Hum. Relat. 56(10), 1163–1193 (2003). https://doi.org/10. 1177/00187267035610001 Swartz, E.M., Scheepers, C.B., Toefy, T.: Women entrepreneurs’ opportunity identification of digital platform start-ups: emerging evidence from South Africa. Int. J. Gend. Entrep. 14(3), 352–374 (2022). https://doi.org/10.1108/IJGE-06-2021-0096 Treanor, L.: Gender, STEM women and entrepreneurship: a review and future research directions. Int. J. Gend. Entrep. 14(4), 499–520 (2022) Treanor, L.: Can Entrepreneurship Education Bridge the gap? STEMM Women’s Entrepreneurial and Commercialisation Activity. International Conference on Gender Research; Reading Apr (2019): 630-XXIV (2019) Wajcman, J.: Feminist theories of technology. Camb. J. Econ. 34(1), 143–152 (2009). https://doi. org/10.1093/cje/ben057 Wang, M.T., Degol, J.L.: Gender gap in science, technology, engineering, and mathematics (STEM): current knowledge, implications for practice, policy, and future direction. Education (2017)

24

D. McQuillan et al.

Wheadon, M., Duval-Couetil, N.: Token entrepreneurs: a review of gender, capital, and context in technology entrepreneurship. Entrep. Reg. Dev. 31(3/4), 308–336 (2019). https://doi.org/10. 1080/08985626.2018.1551795 Woodwark, M., Wood, A., Schnarr, K.: Standing on the shoulders of giantesses: how women technology founders use single and mixed gender networks for success and change. Int. J. Gend. Entrep. 13(4), 420–448 (2021)

Women in Higher Education in Bulgaria: Achievements and Possibilities Yoana Pavlova(B) Technical University of Sofia, 1000 Sofia, Bulgaria [email protected]

Abstract. The development of higher education is an ongoing process that responds to the needs and the stage of development of a given society. The modern development of Bulgarian education began after the liberation of the country from the Ottoman Empire. Initially, women’s access to higher education was obstructed by laws and regulations reflecting patriarchal perceptions of the roles of women and men in society. Thanks to the active work of the Bulgarian Women’s Union, Bulgarian women gained the right to study at university, and in the following years, efforts were directed toward expanding women’s career opportunities in higher education and science. The accession of our country as a Member State of the European Union has created further prerequisites for emphasizing gender equality in the academic sphere, as universities have developed their own Gender Equality Plans. The article analyses some of these plans and an available database on the academic staff of some of the country’s universities. The aim is to trace the changes that have taken place and to highlight the areas where more efforts need to be concentrated to achieve gender equality in higher education and science. Keywords: Strategy · Women in Academic career · High education

1 Women in Higher Education and Innovation in Bulgaria 1.1 Development of Higher Education The modern history of Bulgarian higher education began after the liberation of the country from the Ottoman Empire in 1878. The new rulers passed an Education Act and a special decree established the first higher education institution in the country. In 1904 it was renamed a university. When it was founded, the University consisted of a History and Philology Department with three main disciplines: History, Slavonic Philology and Philosophy with Pedagogy. In the academic year 1889/1890 the Physics and Mathematics Department was opened, and in 1892 the Faculty of Law was opened. Access to higher education was initially only possible for men. After petitions and protests organized by the Bulgarian Women’s Union, women were permitted first as listeners at the university, and from 1901 as full-time students. Despite the legal difficulties for the period 1888 1938, 3094 women managed to complete higher education. Their percentage increased to 21% by 1946 and 16% of them also managed to pursue a career [1]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 25–32, 2024. https://doi.org/10.1007/978-3-031-56075-0_3

26

Y. Pavlova

The process of expanding women’s access and training in higher education continued after the end of World War II. During the period of socialist rule in the country, access to higher education was determined by the state. A quota principle was established, subject to the ideology of “ loyalty” to the Party. In line with party policy, the percentage of women being admitted to universities increased. In line with the key communist postulate of ‘equality of the sexes’, women were encouraged to work in ‘male-dominated’ sectors such as Heavy Industry, which had a reflex effect on increasing interest in studying engineering. Despite the ideology of equality, under socialism, higher education was not available to everyone. The rulers developed so-called “social engineering” by wanting to form a mass of Party loyalists. The admission of students to Bulgarian universities reflects the communist ideology of a class-party approach, where priority is given to those who are useful for the development of socialism, such as the working class, the working peasantry and the people’s intelligentsia [2]. After the fall of the country’s totalitarian regime, access to higher education was liberalised. This favoured its development and the entry of new specialities and disciplines. New universities were established to train young specialists in response to the needs of the economy and society. The function of higher education is, besides preparing personnel for the labour market in various sectors of the economy, to produce educated and thinking people who will participate in the governance of the country. A significant change was observed in the educational status of the population in the period from 1934 to 2021. If in 1934 people with higher education were only 0.7% of the population, in 2021 they are 25.5%. Respectively, the percentage of people with primary and lower education is also decreasing from 89.7% in 1934 to 10.9% in 2021. Increasing education among the population is key for the development of any society [3]. 1.2 Women in Higher Education and Innovation The European Union’s education policies and targets are aimed at increasing the number of female and male graduates, on the one hand, and increasing the percentage of those going into research, studies and innovation, on the other. According to the Bulgarian National Statistical Institute (NSI), the number of persons engaged in research (Researchers) in the public sector (including the Bulgarian Academy of Sciences (BAS), Scientific Association (SAA), and other scientific organizations) and in the higher education sector has decreased in the period 2018–2020. There is a trend of outflow of young people who want to work and develop in the field of higher education and science. 51.2% of all researchers in the public sector and in the higher education sector are aged 35−54. Only 2.8% of young scientists are under the age of 25 and 18.5% are aged 25–34. The innovation sector is very competitive, which is one of the reasons why young people prefer to work in the private sector, where funding is more substantial. In the higher education sector, 3.5% of researchers are under 25 and nearly 18% are between 25−34. Out of these, women working in Research Sector are 2877 and in Higher Education Sector are 4673. Comparatively, there is relative equality in the number of women and men working in the Public Sector, even across age ranges. In the Higher Education sector, there is also parity between women and men developing in this

Women in Higher Education in Bulgaria

27

field, except for the age group 65 and above. There, men have a significant advantage, which can be explained by traditional conceptions of male and female occupations [4]. According to the statistics released by the Ministry of Education and Science, women researchers account for less than half of the total and about half of the remaining staff in the science and higher education sectors. The situation is similar when we look at the distribution by scientific field. The data show that women are actively involved in various scientific fields such as medicine, natural sciences, agricultural and veterinary sciences, etc. [4]. They represent half or more of the total number working in these scientific fields. The report shows that the number of people working in the field of research and innovation, 34 613 people are defined as researchers. The accepted definition of ‘researcher’ is ‘a professional engaged in the creation of new knowledge. He/she conducts research, refines and develops concepts, theories, models, technical equipment, software and operational methods’ [5]. The researcher category includes 16,779 women who are developing and/or refining new concepts in various fields of science. Analyzing the data, we can conclude that women represent almost half of the country’s scientific and research staff in various sectors. It is noticeable that women are predominant in the Humanities and Social Sciences, where 2925 women work out of a total of 4842. In the health sector, 5176 women work out of 8122 people. Such a preponderance is not surprising, because traditionally young girls have been encouraged to choose professions in the Humanities and Social Sciences, which is more in line with and closer to “their nature” related to motherhood and caring for others. The concepts of traditional women’s roles and professions are described in the works of Beauvoir, Bourdieur, Butler and Gilligan [6–8]. Despite the encroaching methods of STEM education and mandatory career guidance classes, the number of young women wishing to pursue science and engineering careers continues to be lower. Studies conducted by researchers such as Pavlova, Kolcheva, Dragneva, Doicheva, Katsunov, V., J. Ivanova, Tsv. Velichkova (Co-editor) show that the choice of a future career is largely determined by parents and imposed patriarchal attitudes of a “sub-par” profession for women. Women doctors, teachers, and lawyers are among the most preferred choices [9, 10]. Significant progress has been made in the number of women scientists working in the natural sciences, with 3,162 out of 5,847 researchers being women. Compared to other EU countries, Bulgaria is one of the countries with the most women working in IT. According to Eurostat, the share of women in ICT in Bulgaria is 28.9%. The second place is taken by Romania −25.2 and Estonia –24.5, while the lowest share of women is in the Czech Republic −10.9 and Hungary −13.6.The most drastic difference is in Technical Sciences, where there are 4181 women out of 13 395. Such a sharp difference can be explained by the existing stereotype about the difficulty of engineering science and the different skills that women and men have. Such perceptions continue to determine the career choices of young girls. For this reason, it is imperative to work continuously and deliberately to introduce STEM/STEAM education already in schools, which will be beneficial to increase the number of girls engaged in science and engineering [11].

28

Y. Pavlova

Attracting more girls into engineering is one of the goals the government has set for itself. Various strategies and plans have been developed. The “National Strategy for the Development of Scientific Research in the Republic of Bulgaria 2017–2030” aims to encourage the participation of women to engage in science, research, innovation and technology development. The Law on Higher Education and the Law on Pre-school and School Education set the basic frameworks and criteria for training at each stage, with an emphasis on developing young people’s digital competencies and critical thinking. It is envisaged to introduce STEM and STEM education and career guidance for students as measures to encourage young girls to pursue a career in science. All legal and strategic documents are aimed at encouraging young people to get educated and to choose science as their future career. Special programmes are envisaged to encourage young girls to pursue science and research in engineering. Various organisations such as Women in Tech- Bulgaria and Bulgarian Centre for Women in Technology (BCWT) are working to increase women’s participation in the digital industry, science, and entrepreneurship.

2 Strategies for Gender Equality in Higher Education Since our country’s accession to the European Union, a number of measures have been taken to promote and improve gender equality in various spheres of public life. An initial impulse was made with the adoption of the Law on Equality between Women and Men, the Law on Protection against Discrimination, and the Law on Higher Education. The “National Strategy for Equality between Women and Men in the Republic of Bulgaria 2022–2026”, in line with the European Commission’s (EC) Strategy for Gender Equality 2020–2025, is in force in our country. Gender equality in academia is promoted by the principles enshrined in Gender Equality in Academia and Research, EU Directives 79/7/EU, EU 2019/1158, the Charter for European Scientists, and the Code of Conduct for the Recruitment of Scientists, considering the recommendations of the European Commission to strengthen gender equality policies in universities outlined in the report Towards a vision for the future of universities in Europe 2030. In an effort to better implement gender equality policies in universities, a handbook has been developed “Guide to the preparation of gender equality plans under Horizon Europe” (EU 2021a) where the mandatory and recommended elements as well as the main areas to be addressed are described. A guide for universities on how to prepare gender equality plans has also been developed to support them (“Toolkit for Gender Equality in Academia and Research”). Measuring progress on gender equality policy is presented in the first EC report She Figures, where indicators and data on gender in EU research and innovation are provided (EC, 2021b). In Bulgaria, progress on gender equality policy implementation is reflected in an annual report of the Ministry of Labour and Social Policy. The European Commission launched the 2022 “European Gender Champion Award”. The initiative will be held annually with the aim of recognising achievements in gender equality in academic and scientific institutions. In response to legal requirements and strategies, universities in the country have developed their own plans to promote gender equality in higher education.

Women in Higher Education in Bulgaria

29

The implementation of national, European, and international laws and strategies for gender equality in the country is the subject of various studies and research. Special attention is paid to gender equality in terms of its cultural and economic significance (Ropova), as well as the problems, risks and difficulties that exist in the development and implementation of equality plans at the universities [12–14]. The analysis of the plans shows what measures the universities have set out and what the indicators of their success are. Among the main objectives they set themselves is to promote gender balance in university management and decision-making, career progression, participation in projects and research, and work-life balance. Universities indicate as a goal the creation of capacities within organisations to implement sustainable gender equality policies. Creating such capacity is unthinkable without building gender sensitivity and various issues relating to equality. The main indicators for their achievement focus on gathering information on employees in terms of gender and career development. Encouraging the participation of women in scientific projects and developments, promoting their academic achievements. Some universities provide for the collection of information on reports of gender-based violence. One of the key indicators is the provision of training on gender equality. Awareness-raising is at the heart of the successful implementation of strategies and policies. A good practice in encouraging women to pursue academic careers is the national scholarship program “For Women in Science” (National Commission for UNESCO Bulgaria, Sofia University “St. Kliment Ohridski and L’Oréal). The scholarships are designed for young women scientists and aim to honour their scientific potential and support them to realise their dreams of changing the world through science.

3 Professional Realization of Bulgarian Women in Academia The professional realization of women in higher education is associated with a number of difficulties. Initially, Bulgarian women are not allowed to work, regardless of their higher education. This is the case of women lawyers, architects, doctors and engineers. The first woman lecturer at a Bulgarian university was Teodora Raikova. She was appointed in 1918 as her father’s assistant at the Faculty of Physics and Mathematics. Slowly, women entered science. Data show that until 1944 women held only the positions of assistant professors and lecturers at the State University of Sofia. Their number was 48, with the largest number of women in the Faculty of Medicine (58.7%) and the Faculty of Physics and Mathematics (26.1%). Interestingly, the first female associate professor in Bulgaria, Elizaveta Karamikhailova, is from the Faculty of Physics and Mathematics. Women face difficulties in their realization in other higher education institutions and professions. In the following years, women made their way into academic careers [15]. Currently, there are 52 universities in Bulgaria, which train in 52 professional fields. The total number of students is over 200 000. An analysis of the management positions shows that there are only 11 female rectors of universities against 41 men. Women head the Agrarian University - Plovdiv, the Academy of Music, Dance and Fine Arts, Burgas University University “Prof. Dr. Asen Zlatarov”, the Higher School of Construction “Lyuben Karavelov”, the Higher School of Transport “Todor Kableshkov”,

30

Y. Pavlova

the Higher School of Agribusiness and Regional Development, the Higher School of Telecommunications and Posts, the Medical University - Plovdiv, the Economic Academy “Dimitar a. University of Library and Information Technologies (UniBIT), University of Chemical Technology and Metallurgy (CCTMU) - Sofia. It is evident from the universities that the fields of their studies are quite different, which shows that overcoming stereotypes of “female and male” spheres of development has begun. At three of the largest universities in the country, St. Kl. Ohridski”, University of National and Worldwide Esonomy, and Technical University of Sofia are still dominated by men in managerial positions. Rectors of the universities are men as women vicerectors are 5 in SU, 2 in UNWE and 1 in TU-Sofia. Of 16 faculties at SU “St. Kl. Ohridski,” only 6 of them are managed by women. At the UNWE there are 8 faculties and 3 are headed by women. TU-Sofia has 14 faculties and only 2 faculties have elected women as deans. There is still a predominance of men in leadership positions, which influences the formation of some of the policies and guidelines in universities. In an analysis of available information on the numerical composition of some universities, there is a balance and even a predominance of female faculty at the expense of men. Data from the University of Veliko Tarnovo “St. Cyril and Methodius” show that the number of habilitated female lecturers is 140 and of men - 120. The difference in senior assistant professors is in favour of women - 86 against 49 men. The trend is obviously stable because even in the case of the youngest lecturers with assistant professor positions, the data are - 36 women against 27 men. Women are more active in participating in research projects as well (women- 166, men- 83). At the Technical University of Gabrovo the academic staff includes 144 lecturers, of which 46% are women. A total of 130 lecturers holds research degrees, and the malefemale ratio with research degrees is 68% to 32% for women. There is an absolute 50/50 male-female ratio in management positions. Within the University in the last year, 30% of women have been involved in research projects. Eight women are currently studying for PhD degrees, representing 22% of all PhD students. Analysis of the data shows that the predominant proportion of female Ph.D. students who have successfully defended their Ph.D. in time are moving on to scientific careers after their studies [16]. Women also predominate in the academic staff of the Medical University of Pleven. There are 188 lecturers in total, of whom 104 are women and 84 are men. There are 50 assistant professors, of whom 29 are women and 21 men. Associate professors’ number 103, of whom 59 are women and 44 are men, and professors total 35, of whom 16 are women. In all categories of academic posts women predominate, except for professors, where men have a slight predominance. The ratio of men and women engaged in science at the University of Ruse is almost equal. A total of 462 lecturers work at the university. Of these, 248 are men and 214 are women. At the “D. A. Tsenov” Academy of Economics in Svishtov the total number of habilitated lecturers is 77, of which 41 are men and 36 women. The data show that in the scientific field of “Finance, money circulation, credit, and insurance” 68% of the

Women in Higher Education in Bulgaria

31

lecturers are men, but in the field of “Accounting, control, and analysis of economic activity” 65 percent of the lecturers are women against 35% men. The total number of academic staff at Trakia University in Stara Zagora is 573, of which 338 are women and 235 are men. The largest number of women among the lecturers is in the biggest structure of the university - the Medical Faculty, where there are 130 women to 73 men. The Faculty of Veterinary Medicine is the only one where men predominate - there are 58 men and 41 women lecturers. The Faculty of Engineering and Technology in Yambol has 32 women and 27 men, the Faculty of Pedagogy has 33 women lecturers and nine men, and the Faculty of Economics has 32 women to 18 men. The only unit that has approximately the same number of scholars is the Faculty of Agriculture, where 39 women and 37 men teach. The only university where women lecturers are a minority is the Higher Naval School. The Naval Academy has 148 faculty members, of which only 51 are women. At Varna Technical University, the number of female and male lecturers is almost equal. Their total number is 204 people, with 109 ladies, of whom 92 have a degree in education and science [16]. The analysis of the academic staff of some of the country’s universities shows that women have successfully made it in academia and the fields in which they teach are the most diverse.

4 Conclusion In conclusion, we can say that women have come a long way from winning the right to be educated at university to successful realization in academia. Women’s entry into science and higher education has involved effort and persistence in pursuing focused action. EU policies are a strong impetus for the development of gender equality in education and science. The development of ‘Gender Equality Strategies’ by universities has had a beneficial effect on building sensitivity to the issue and targeted measures for its implementation. In addition, it can be said that the analysis of data on the number of women and men working in higher education and science shows the positive trends and results achieved in terms of women’s participation in these fields. At the same time, it also highlights problematic areas that need special attention.

References 1. Dinkova, M.: Social Portrait of the Bulgarian Woman. Profizdat Publisher, Sofia (1980) 2. Boyadjieva, P.: University and Society. LIK Publisher, Sofia (1998) 3. National Statistical Institute: Annual Report (2021). https://www.nsi.bg/bg/content/2698/. Accessed 10 Sept 2023 4. Ministry of Education and Science: Report of the Comprehensive Preliminary Impact Assessment of the Law on Promotion of Scientific Research and Innovation (2022)

32

Y. Pavlova

5. HCI:https://www.nsi.bg/bg/content/2670/%D0%BD%D0%B0%D1%83%D1%87%D0% BD%D0%BE%D0%B8%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%BE% D0%B2%D0%B0%D1%82%D0%B5%D0%BB%D1%81%D0%BA%D0%B0-%D0% B8-%D1%80%D0%B0%D0%B7%D0%B2%D0%BE%D0%B9%D0%BD%D0%B0-% D0%B4%D0%B5%D0%B9%D0%BD%D0%BE%D1%81%D1%82-%D0%BD%D0% B8%D1%80%D0%B4). Accessed 17 Sept 2023 6. Bourdieu, P.: Male domination. LIC, S. (2002) 7. Butler, D.: Pregogor to Anxiety about gender LitterNet e-journal 5 (138) (2011). http://lit ernet.bg/publish27/judith-butler/predgovor.htm 8. Pavlova, Y., Kolcheva, N., Dragova, S.: The choice of profession among Bulgarian adolescents and their career orientation. In: Proceedings “Man - a Measure of all Things? The Challenges of the Post-Industrial Information Society, Technical University Press (2022) 9. Kolcheva, N., Dragova, S., Pavlova, Y., Doycheva, V.: Handbook of professions, Project G-Guidance (2023). ISBN 978–619–92482–0–1, https://eprints.nbu.bg/id/eprint/4780/ 10. Gilligan, C.: In a Different Voice: Psychological Theory and Women’s Development Paperback. Harvard University Press, London, England (2016) 11. Popova, L.S.: Gender equality in science - policies and practices, innovations in technology and education. In: Proceedings of the XIV International Scientific and Practical Conference. Volume 3. Kemerovo, Belovo, Novosibirsk, Veliko Tarnovo, Shumen, pp. 152–157 (2021) 12. Katsunov, V., Ivanova, J., Velichkova, T.S.V.: (Co-editor). Women in the History of Academic Science in Bulgaria. Sofia (2018) 13. Doneva, R., Gaftandzhieva, S., Somova, E., Mileva, N.: How to promote the change in the area of gender equality in academia and research – Bulgarian case, Conference name12th annual International Conference of Education, Research and Innovation (2019) 14. Serafimova, D.: Gender Equality Plans in Bulgarian Higher Education Institutions, Published by: Az-byki Press 31(1), 61–72 (2023) 15. Daskalova, K.: Women, Gender and Modernization in Bulgaria (1878–1944). Sofia University Kliment Ohridski Publisher, Sofia (2012) 16. BTA (2023). https://fakti.bg/bulgaria/754599--maria-gabriel-za-jenite-ucheni. Accessed 01 Sept 2023

Empowering Students in Technical Higher Education Through Teamwork and Alternative Assessment Methods. A Case Study Irina Duma(B) , Andreia Molea, Nicolae Vlad Burnete, and Dan Moldovanu Technical University of Cluj-Napoca, Cluj-Napoca, Romania [email protected]

Abstract. Given the continuous development of the socio-economic environment, as well as the dynamics of students’ population, it has become of outmost importance to engage and empower students in technical higher education from the very early stages of their learning paths. Therefore, the present paper aims to evaluate how teamwork, combined with self-assessment and colleagues evaluation contributes to the development of students professional and transversal skills, which are indispensable for their professional life and personal development. The study was carried out considering three technical disciplines from the field of Automotive Engineering, taught for bachelor’s and master’s students. Also, the engagement of female students was analysed to explore the possible barriers in their participation to technical higher education. Keywords: higher education · technical skills · soft skills · students empowering

1 Introduction 1.1 Necessity of the Proposed Topic Along with the creation of the European Higher Education Area (EHEA), as assumed by the ministries of education in Sorbonne Joint Declaration (1998) [3], followed by the Bologna Declaration (1999) [1], several objectives were proposed by the signatory states, mainly with the aim of developing higher education systems that would be compatible, comparable and that would encourage diversity in the student population and remove barriers to access higher education. One of the main objectives of the EHEA consists of enhancing employability, for which higher education institutions should play a proactive role. In the London Communique [2], it is emphasized that “higher education should play a strong role in fostering social cohesion, reducing inequalities and raising the level of knowledge, skills and competences”, while “the student body entering, participating in and completing higher education at all levels should reflect the diversity of our populations”. Therefore, it is of outmost importance to develop, in addition to national policies related to the social dimension of higher education, local practices that would enhance participation of students and their engagement in designing, participating © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 33–42, 2024. https://doi.org/10.1007/978-3-031-56075-0_4

34

I. Duma et al.

in and completing their own educational path. These practices should consider the proposed learning outcomes fit for each study level, as well as the individual needs of each student. Consequently, all stakeholders – higher education institutions, policy-makers, decision-makers, socio-economic partners and students, should work together in order to ensure that graduates possess competences suitable for a smooth transition to the labor market, which also enable them to develop new competences needed later in throughout their working lives [4]. Considering the constantly and rapidly changing labor markets, as well as the dynamics of student generations, it is important that higher education institutions foster an environment in which students remain engaged and have the sense of empowerment during their learning process. Thus, the new learning contexts encourage project-based approaches, in which students are encouraged to make use of digital tools, as well as to have the sense of ownership on their research work and disseminate it at the end of the task [6]. The specialty literature has explored the theoretical implications of students’ engagement in their learning process. Thus, in [9] it has been concluded that taking responsibility by students for a learning project, especially when there is a certain degree of uncertainty, allows a better engagement of students in the learning process. The same paper emphasizes the influence of the learning environment on student engagement, stating that the quality of social relations can ground education in ways that reach beyond technical reasoning. Moreover, in [10] it is highlighted the importance of student engagement for achievement and learning in higher education, considering several factors such as the institutional practices, individual psycho-social processes, the socio-cultural perspective. Working in teams, as a commonly met learning strategy in higher education applied to enhance deep learning and develop soft skills, is an important aspect in preparing students to actively participate in a globalized world, especially when being part of diverse groups [11]. When it comes to the usage of technology meant to contribute to the proper solving of tasks in an efficient and flexible way, it has been noticed that students may encounter difficulties in real-life cases [8], which may prevent them from performing in their career. Moreover, student engagement through collaborative learning contributes to fostering creativity in higher education and, in addition, enhancing professional competences and personal development [5]. Besides collaborative environments, it is also known that project-based learning contributes to the outcomes of students, both in terms of knowledge and engagement [7], which can be measured by questionnaires, interviews, observation and other alike tools. 1.2 Aim and Objectives of the Present Paper The aim of the present paper is to explore how teamwork, as part of the mandatory applicative activities within engineering specialty disciplines, together with selfassessment and evaluation of other colleagues, contributes to the academic performances and student engagement in classes. The research also proposes to provide an observationbased analysis of how female students are perceived by their peers, especially in a field which is not traditionally chosen by them when entering higher education.

Empowering Students in Technical Higher Education

35

Moreover, it is important to note students’ perception on this kind of activity and how their sense of engagement and belonging to the group was influenced by applying similar methods of engagement and evaluation for the three chosen disciplines.

2 Case Study 2.1 Methods Used To raise the engagement of students for laboratory and project work, as well as to empower students to have a sense of involvement within their education course, a method for evaluating student performance was applied. Considering the particularities of each discipline where this method was used, the main characteristic lies with the formation of small groups of 2–4 students, who had to deliver a certain theme or project, after working the whole semester. The evaluation consisted of the teacher’s grade with respect to the initial inquiries and the presentation delivered by the group, as well as the grades given by students to each other within a certain group. Therefore, it was possible to explore how objectivity can be maintained by students while evaluating their own peers. Moreover, students were encouraged to objectively evaluate their own colleagues’ progress during the whole semester as well as their own involvement in each task, thus empowering them to actively take part in all aspects of the disciplines’ activities. For the present paper, three disciplines were selected, in the field of Automotive Engineering within the Technical University of Cluj-Napoca, two of them being taught for the 3rd year bachelor’s students and one discipline for the 1st year master’s students, respectively. In the case of the bachelor’s study level, for each of the two analyzed disciplines there were approximately 90 students involved, while for the master’s study level the group consisted of around 20 students. The evaluation of the outcomes of project-based learning combined with group work and self-assessment as well as peer-assessment, was performed by comparing the overall educational performances of the student generations before and after applying the method, for each of the three analysed disciplines. In terms of engagement evaluation, the conclusions were drawn from collecting oral feedback from students, and from the observations of teachers involved in the three disciplines. Bachelor’s Study Level (Discipline 1). For Discipline 1, in the academic year 2022– 2023, the creation of a mini-project with PowerPoint support of the received topic was introduced as a mandatory laboratory activity, in addition to the laboratory portfolio and the individual activity for each laboratory meeting. In the previous academic years, from 2019 up until 2022, in addition to the portfolio with test sheets, the students had to redo an experimental test report with the same theme. However, this resulted in many copied works from other peers. With the new approach, the mini-project was based on two main objectives: a) deepening the given topic in the field of the discipline, from a scientific point of view, using individual topics for each team of students; b) public defense of a presentation on a certain topic, to enhance soft skills like public speaking, summarizing of a certain work, and presenting.

36

I. Duma et al.

In order to achieve the first objective, the students were presented with the topic and the teammate with whom they are elaborating the mini-project. Pairing students in alphabetical order and not letting them chose their peer had the purpose of encouraging collaboration among the whole group of students, regardless of their previously-made preferences in terms of colleagues, thus taking them out of the comfort zone. From the content point of view, each team composed of two students had to elaborate a test report describing the testing procedure of a road vehicle subsystem, starting with the aim and objectives of the discussed test, up to the test method, presentation of the conclusions and interpretation of the obtained results. The general requirements are depicted in Fig. 1. Additionally, students were provided with the necessary bibliography for documentation, together with citation requirements and measures of avoiding non-validated references from a scientific point of view.

Fig. 1. Instructions for carrying out the mini-project for Discipline 1.

The second objective of the mini-projects consisted of students getting used to presenting their work in front of an audience, while being able to describe a scientific topic. Given that in the past years it has been noticed – especially when defending the bachelor’s and master’s theses – that most students have not fully succeeded in developing a technical vocabulary, while some students could not manage their nervousness properly. Therefore, the presentation of their mini-projects, which had a maximum defense time of 4 min for the two teammates, would contribute to the development of soft skills such as public speaking, presentation time management, nervousness management and summarizing. Presentation rules were imposed, such as appropriate attire, the interdiction to fully read the text from the slides, as well as encouragement for self-confidence was made. The presentation must not be interrupted and questions from the audience would be answered afterwards, as described in Fig. 2. Based on the evaluation of the mini-projects, it was observed that all students managed to finish and upload their work on time. From the content point of view, all teams – except for one – managed to reach all the important imposed points. The team that did not manage to fulfill the requirements motivated the fact that they could not organize and mobilize themselves, because of different schedules. In terms of references used for the

Empowering Students in Technical Higher Education

37

specialty literature analysis, it was noticed that only one team used a single bibliographic reference which was available online, while all the other teams did a thorough analysis of the literature.

Fig. 2. Instructions for presenting the mini-project for Discipline 1.

The grades awarded for the mini-projects varied from 6.50 to 10 (out of maximum 10), taking into account the content quality, the evaluation provided by the teammate and the percentage of involvement in the elaboration of this work, as well as their presentation mode. The average grade was 8.47 (out of 10). In terms of involving female students in the study field and their applicative work, it was noticed that the average grades obtained by them were 8% higher than those obtained by male students (Table 1). Usually the grades obtained by female students were between 9.5 and 10 in each academic year, with exceptions (below grade 8) due to the fact that those students were either pregnant or already in motherhood, or they would simultaneously follow two university study programs. It was also noticed that students employed full-time during studies would have lower academic performance than their peers, mostly due to the lack of time to be invested for studying or doing homework. The final laboratory grades from Discipline 1, in which the mini-project counted for 40% of the final mark in the academic year 2022–2023, increased compared to the previous year, by 6.6%. Regarding the evaluation of the teammate, most of the students described that the work was split equally inside the team, therefore the marks awarded by them varied from 8.50 to 10 (Table 1). The feedback from students, after presenting the mini-projects and finding out their grading, was a positive one based on the fact that they were pleasantly surprised by the collaboration with their colleagues with whom they did not previously had a close professional relationship. Students also appreciated the exercise of presenting their scientific work in front of an audience, while seeing the usefulness of this evaluation method for preparing them for the defense of their bachelor’s theses. They also found some ways to manage their nervousness when speaking in public, such as a thorough documentation of the work and the rehearsal of their speech. Bachelor’s Study Level (Discipline 2). In this case, the project activity involved the simulation of an internal combustion engine based on a project guide, in teams of 3–4

38

I. Duma et al. Table 1. Average grades obtained by students for the laboratory activity in Discipline 1.

Academic year

Subgroup 1

Subgroup 2

Subgroup 3

Subgroup 4

Total number of female students

Average grade of female students

Average grade of all students

2019–2020

8.08

8.65

8.32

8.28

3

9.03

8.33

2020–2021

8.37

8.37

8.07

−

5

8.91

8.27

2021–2022

8.26

8.27

8.49

7.08

3

9.00

8.02

2022–2023

8.73

8.75

8.90

−

6

8.95

8.79

Table 2. Average grades obtained by students for the laboratory activity in Discipline 2. Academic year

Subgroup 1

Subgroup 2

Subgroup 3

Subgroup 4

Total number of female students

Average grade of female students

Average grade of all students

2019–2020

8.80

8.00

8.50

8.70

3

8.50

8.33

2020–2021

6.41

7.60

7.16

−

5

8.45

8.27

2021–2022

8.16

7.67

6.61

7.20

3

7.35

8.02

2022–2023

8.33

6.52

7.96

6

8.96

7.24

students (mixed in terms of gender). However, for the bachelor’s program of Automotive engineering, the number of female students is still low, despite the further development opportunities offered equitably in terms of gender. Each project topic was established by mutual agreement with the teaching staff. Although the project was done as a team, each team member had to go through a minimum of work, according to the project guidelines, with the aim of familiarizing themselves with the software program they had to use. In addition to the project guidelines, the students had access to experimental results and to a project template with an imposed format. The structure and requirements of the project were designed in such a way as to stimulate students to be creative, to document thoroughly, to use data processing programs, to make presentations, as well as to apply the previously acquired theoretical bases and familiarize themselves with the simulation software programs in the field. In addition, students had to work in a prescribed paper format, which provided them with knowledge and skills that will enable them to develop their bachelor’s and master’s theses in a smoother manner. The project had to be structured into several mandatory chapters, such as Introduction, State of the art, Simulation model and working methodology, Results and discussions, Conclusions, and References. For each of these chapters, students received a set of minimal instructions, along with the evaluation grid.

Empowering Students in Technical Higher Education

39

To carry out the project, students were encouraged to share their tasks within the team and to have regular discussions/meetings regarding their progress and possible encountered difficulties. At the same time, students were encouraged to identify solutions to the encountered difficulties, before addressing to the teaching staff. At the end of the semester, the teams had to present their project in front of an audience formed by their colleagues. The presentation should be carried out with the contribution from each team member, followed by a short session of questions and discussions, as well as suggestions for improving the presentation. The project grading involved several contributions, as follows: – The grade awarded for the presentation, which represents the average of all grades given by those who participated in the presentation (students and teaching staff) on three criteria: general aspect of the presentation, the presentation mode and the content; – The grade awarded for the in extenso project, according to the evaluation grid, which includes both formatting aspects (imposed format and attention to details), and content; – Team member evaluation which included both self-evaluation of one’s own contribution, as well as the contribution of the other members. To encourage critical evaluation and objectivity, the results of this evaluation were only available to the teaching staff. For grading, the self-assessment component is of great importance, as it is used to differentiate between the final grades of all team members. When analysing the results, it was found that students who have not succeeded in finalizing the project encountered issues in splitting the tasks within the team and did not assume responsibility for failure. Those students who have managed to efficiently organize their work, finished and submitted the project in time, with a moderate amount of effort (depending on previous knowledge and involvement). The self-assessment highlights individual involvement of each team member with respect to the one of their peers and can be used for correction purposes of the final grades. According to the feedback received from students, they appreciated the importance of the imposed formatting and the guidelines, for carrying out their bachelor’s theses. Furthermore, they emphasized the utility of team work for reaching the proposed learning outcomes, as well as receiving feedback for their presentations regarding those aspects that could be improved. It was also noticed that students managed to enhance their critical thinking and identifying alternative solutions where needed, as well as scientific research skills based on the international specialty literature. Nonetheless, it was observed that female students did not only manage to involve and was included in each project teams, but they would often take the lead role within the team as due to their organizational skills. In terms of academic performances, the final grades awarded in the past 4 academic years per each subgroup of students are depicted in Table 2. The above-described method was used starting with academic year 2020–2021, while before the projects were carried out individually by students.

40

I. Duma et al.

Master’s Study Level (Discipline 3). For Discipline 3, master’s students had to prepare projects in teams of 4–6 students (academic years 2020–2021 and 2021–2022) and in teams of 13 students (academic year 2022–2023), respectively. The project implies use of MATLAB for implementing something practical, using the V model of development, where the input comes from the client (teacher plus members of other teams). Among the project topics, there were: lateral parking, vehicle wireless pressure communication to cloud, overtaking availability system, backward parking of a truck, control of an omniwheel vehicle, or control of two Porsche rear spoilers. The final grade consists of the working prototype and a report that follows the V model distributed: 1 ex officio point, 1 point for the requirements documentation, 1 point for the architecture documentation, 1 point for the development, 2 points for the code generation and configuration, 1 point for the testing documentation, 1 point for the implementation and product list documentation and finally 2 points for the working concept, summing up to 10 points max grade. For the first two generations of students, a classic mode of grading was applied, meaning that at the end of the semester, a general presentation was made, where each student defended what they have done to help the team, a final team report was made and the grading followed the previously presented method. For the latter generation (2022 – ‘23), before the examination the students were asked to also self-assess their work, as well as to grade their colleagues work (from their own team). Because the project is applicative, the engagement of the students is high, especially since they work with their colleagues but also with the tools from the University and under the supervision of the teacher, but in the last generation there was an increase of engagement also because the projects were quite similar and there was a rivalry between the teams. The evolution of the average grades (presented in Fig. 3) are: 9.82 for the 2020– 21 generation, 9.68 for the 2021–22 generation and 9.41 for the 2022–23 generation (where students evaluation was 70% and teacher evaluation was 30%), underlining that the students have a more critical view of their working colleagues. Unfortunately, because the masters and the side of automotive engineering is not that attractive (yet) to female students, there are 1/23 female/total students in the 1st generation, 1/27 in the 2nd generation and 1/25 in the 3rd generation. Here as well, it was noticed that female students would take the lead within each project team, distributing the work load among team members and following-up with the team progress. In terms of content, female students contributed equally in teams, being responsible with programming and working the mechanical and electrical sub-team. In one case, the female student’s important role was underlined during a meeting where she was missing and her colleagues did not manage to properly start the prototype.

Empowering Students in Technical Higher Education

Average grade per academic year

10

10

41

10

9.82 9.68 9.41

2020-2021

2021-2022

2022-2023

Average grade of all students Average grades for female students

Fig. 3. The evolution of grades per academic year for Discipline 3, with accent on female students.

3 Conclusions and Discussions Overall, the teamwork activities implemented were a good opportunity for students to get to know each other better from a psycho-social perspective, as well as to collaborate for various technical tasks with their colleagues. Moreover, teamwork contributed to the development of interpersonal skills, while presenting their work definitely had an impact on their technical vocabulary and public speaking skills, as well as nervousness management. Nonetheless, self-assessment and assessing their colleagues as part of the activity evaluation had an important role on developing their critical thinking. It was noticed that the trends in academic performance is similar from one discipline to another (Fig. 4) and, in addition, it is important to note that these variations might also be influenced by each generation, as well as by the pandemic outbreak in 2020 which led to full or partial online teaching and learning activities, at least for a limited period of time. When it comes to involving female students in technical activities and teamwork, it was observed that, owing to their organizational skills, which enabled them to split the workload within each group and to follow-up on the progress, they have the tendency to take up leading roles, as well as receiving support from their peers. However, further research should be conducted to better understand the perception of female students in technical study fields in terms of engagement and empowerment, as well as to understand the barriers of accessing technical higher education.

42

I. Duma et al.

Discipline 1 Discipline 2 Discipline 3

2022-2023

2021-2022

2020-2021

2019-2020

Average grades for female students Average grade of all students Average grades for female students Average grade of all students Average grades for female students Average grade of all students 5

6

7

8

9

10

Fig. 4. The evolution of grades per academic year for all Disciplines, with accent on female students.

References 1. 2. 3. 4. 5. 6. 7. 8.

9. 10. 11.

European Higher Education Area: Bologna Declaration (1999) European Higher Education Area: London Communique (2007) European Higher Education Area: Sorbonne Joint Declaration (1998) European Higher Education Area: Yerevan Communique (2015) Alvarez-Huerta, P., et al.: Student engagement and creative confidence beliefs in higher education Author. Think Skills Creat. 40 (2021) Boss, S., Krauss, J.: Reinventing Project-Based Learning: Your Field Guide to Real-World Projects in the Digital Age. International Society for Technology in Education Guo, P., et al.: A review of project-based learning in higher education: Student outcomes and measures. Int. J. Educ. Res. 102, (2020) He, Y., Tao, Y.: Unity of knowing and acting: an empirical study on a curriculum approach to developing students’ translation technological thinking competence. Transl. Technol. Teach. Views Visions 16(3), 348–366 (2022) Kahn, P.E.: Theorising student engagement in higher education. Br. Educ. Res. J. (2013) Kahu, E.R.: Framing student engagement in higher education. Stud. High. Educ. 38(5), 758– 773 (2013) Poort, I., et al.: Does the group matter? effects of trust, cultural diversity, and group formation on engagement in group work in higher education. High. Educ. Res. Dev. 511–526 (2020)

Real World Experiences

Parameterization and Modeling of Structural Designs for the Transformation of a Smart City Anastasia Gasidou, Dimitrios Kotsifakos(B) , and Christos Douligeris Department of Informatics, University of Piraeus, Piraeus, Greece {kotsifakos,cdoulig}@unipi.gr

Abstract. As societies are rapidly transforming and technology is evolving and bringing constant change, contemporary trends such as digital technologies and process automation are emerging and evolving. These new trends are changing how we live as well as the wider economy. The scientific questions of this paper focus on the design and modeling of the role of smart cities, in combination with the functions of the active citizen-user in creating online cultural applications. The digital environment engineer and digital transformation designers, in combination with the functions of the active citizen-user in the creation of online cultural applications, are also analyzed. Keywords: Parameterization · Smart Cities · Structural Designs - Modeling

1 Introduction There is an urgent need to study and document the formulation of the necessary parameters for an overall process of transformation or conversion of a city (or region) into a “smart” one. Digital technologies [1] and their application in the wider economy [2] make interactive the expansion of design and modeling studies about the role of smart cities [3], especially with the functions of the active citizen-user in creating online cultural applications. To formulate the basic designs and to capture the structural modeling of the transformation into a ‘smart city - smart area’, we have considered the technical conditions of digitization of objects, the processes of digitization of the spatial boundaries of the city or area, and the living conditions that describe it [4]. A key premise for the formulation of this design was the intention to strengthen an online collaborative model, in which priority is given to strengthening the community participatory processes. The role of the digital environment engineer and digital transformation designers, in combination with the functions of the active citizen-user in the creation of online cultural applications. By prioritizing the citizen-user experience we determine both the development model of the emerging “smart city” and the success of an overall democratic state participation. A “democratic state” as an environment refers to the political and governance system of a country or region where democracy is the prevailing form of government. These axes also define the ways of exploiting the user-citizen experience © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 45–56, 2024. https://doi.org/10.1007/978-3-031-56075-0_5

46

A. Gasidou et al.

so that the transformations are functional and attractive and ensure the realization of a democratic society. Emphasis is placed on the participatory model and on eliminating the risks of covered alteration of the democratic function of the civil society’s social partners [5]. Public bodies, political parties, associations, unions, trade unions, municipalities, and communities can strengthen their participatory processes with the appropriate tools and set their conditions in the technical mediation of digital or digitized information. The major focus is on the quality improvement of an urban city. Design solutions and modeling also suggest the range of opportunities that will be realized if smart city planners use the combinations of technologies and information produced today in information-intensive societies. In addition to techniques or technologies, these models are also permeated by multiple ethical codes. To be used fairly and to function effectively, the increase in participation in data access should be planned and the collective discussion on its use should be organized on a massive participatory scale. Big data should become “open data” to profoundly impact urban life. Accelerating urban development is a complex endeavor that requires the collaboration and engagement of multiple stakeholders. Government authorities and urban planners provide the regulatory framework and strategic guidance, while community organizations ensure local perspectives and needs are incorporated. Private developers and investors bring expertise and resources for implementation, while academic institutions offer research-backed insights. Infrastructure providers guarantee the necessary foundational support, and environmental experts promote sustainability. Financial entities and technology specialists facilitate funding and innovation, and civic leaders garner public support. Collaboration with transportation experts, health officials, and cultural representatives ensures comprehensive, safe, and culturally enriched urban spaces. In uniting this array of stakeholders, urban development can be accelerated with a holistic approach that yields thriving, inclusive, and forward-looking cities. All of the above will be studied through the prism of the citizen-user experience [6]. The four basic scientific questions that surface are: • Which issues need to be addressed as a priority for parameterization and design of structural designs for the transformation of a smart city? • How should reforms be undertaken in the plan’s parameterization and design? • Who should be engaged to accelerate urban development? • What approach should be taken to achieve urban rejuvenation and transformation? The specific article, in addition to the present Introduction, is divided into four subsequent chapters. The second chapter entitled “The Basis of a Different Ethical, Social and Political Ethics for the Parameterization and Design of Structural Modeling during the Transformation of a Smart City” clearly presents the limits of today’s social and strategic transformations, precisely setting the goals for a new perspective for the economy and society. This part puts the burden on the connection of this project with cybernetics, internet technologies, applied telecommunications technologies, and finally on the conceptual restoration of contested social terms, such as participatory collectivity, equality, justice, and freedom. The third chapter «Structural Modeling of the Design» uses the conceptual approach of the Unified Modeling Language™ -UML® and the use of metaverse tools to capture the models. The fourth chapter entitled “Fields of Customization” presents a part of the possible terms of how an “intelligent” design

Parameterization and Modeling of Structural Designs

47

could be formed that would serve the “common” area of a city. The article closes with the “Conclusions” chapter, in which the minimum conditions for democratic participation of citizens, the perspectives but also the difficulties for the coupling of parameterization and the design of structural modeling for the transformation of a smart city with the vision of social emancipation are recorded. And multi-participation.

2 The Parameterization and Design of Structural Modeling During the Transformation of a Smart City A very important reason why an effective holistic “smart” transformation of a city is necessary and important has to do with our relationship with the planet. The environmental destruction – especially but not only through climate change – that comes from the pursuit of growth, threatens to cause a catastrophe, the signs of which are already visible in every city. Environmental destruction and disasters resulting from climate change and depletion of natural resources are not natural phenomena but result from the existing prevailing socio-economic model. These ills hit more quickly and more intensely, the most bankrupt places, which have fewer resources to mitigate the disaster. The prospect of a total collapse of the social fabric of cities is dire and present, as already seen by forced migrations, present social disorganization, and active hotbeds of war everywhere in the world [7]. Beyond these factors, there is a potentially more serious question: We now know that increasing wealth, when measured in ordinary monetary terms, has little real effect on people’s sense of well-being, if their basic needs are adequately met. Pursuing “development” in these terms, to realize life’s goals and desires, “economies” and “markets” are chasing a chimera, since growth indicators may at times “prosper” but the degree of satisfaction with their daily experience as citizens remains very low, as their lives remain intensified and stubbornly unchanged. Indeed, it is now known that to the extent that the dominant development model increases inequality, which it does, it is the main factor in declining health and increasing crime and social ills, as opposed to what might be happening in a more equal and fair society [8]. Realizing that the urban planning of the inhabited areas was essentially invented as a technical instrument and as a spatial methodology for the organization of space, without giving particular importance to social problems and at the same time realizing in the passage of the 20th century that this approach and the accompanying specialization of the abilities transferred to developing and emerging countries in the 21st today, we should consider the transformation of cities into “smart” ones from a dual perspective: – to challenge conceptual and methodological principles when applied to different social contexts than at ones for which they were originally intended, and – to invent the methods to restore a new urban design, so that these interventions can fight poverty, exclusion, and segregation while promoting a more sustainable and inclusive urban development. From this perspective, we need to reexamine Cybernetics, Web Technologies, Telecommunications Theory, and Artificial Intelligence (AI) (Fig. 1). These four areas must be linked in the design of a smart city through the creation and exploitation

48

A. Gasidou et al.

of advanced digital infrastructures. Reexamining them offers fresh insights into the symbiotic relationship among these domains. As Web Technologies become more sophisticated, the principles of Cybernetics can guide the design of adaptive and selfregulating systems, bolstered by the capabilities of Artificial Intelligence. Concurrently, the advancements in Telecommunications Theory enable the seamless exchange of data between interconnected devices, amplifying the potential for real-time decision-making in AI-driven applications.

Fig. 1. A design of a smart city.

The goal of a smart city is to improve citizens’ quality of life, enhance sustainability, and optimize the performance of city services by using technology and data. Cybernetics plays a critical role in smart city planning, as it is responsible for creating and implementing the policies and regulations that govern technology use and data protection [9]. Cybernetics also contributes to the creation of strategies for the development and implementation of digital services that will serve citizens [10]. Web technologies with sensors, cameras, wireless connections, cloud computing, and ubiquitous management of smart cities big data form the basis of smart cities [11]. These technologies collect data from the city’s environment and infrastructure and turn it into information that can be analyzed for decision-making [12]. Collecting and analyzing this data allows the government and citizens to better understand the city’s needs and take steps to improve its operation [13]. Telecommunication also contributes to the design of a smart city, as it provides the basic principles for data transfer and communication between devices and systems within the city context [14]. From the basic concepts of telecommunication, such as communication networks, communication protocols, and data transmission, the communication infrastructures necessary for the operation of smart cities are developed and implemented. AI is critical to achieving the goals of a smart city [15]. Since it can be used to analyze data, automate processes, and make intelligent decisions [16]. For example, AI can be used to control traffic, manage waste, detect accidents, and provide tailored services to citizens [17].

Parameterization and Modeling of Structural Designs

49

Overall, the connection of these aspects - cybernetics, internet technology, telecommunications, and artificial intelligence - enables the creation of a smart city, where technology and data are used to improve the performance and sustainability of the city, as well as to enhance the life experience of citizens.

3 Structural Modeling of the Design Package diagrams are structural diagrams used to show the organization and arrangement of various model elements in the form of packages [19].

Fig. 2. Basic Package Diagram for a “Smart City”.

If we want to capture in Package Diagrams (Fig. 2) the correlations and generalizations of the categories of a “smart conversion” for a city or for a region we must include eight basic categories. The Basic Package Diagrams for a “Smart City” that could be used as a metamodel [20] are: • • • • • • • •

Intelligent management of energy resources Smart transfers Smart health and wellness service delivery systems Smart insurance services and immediate intervention in natural disasters Smart training structures Smart buildings Smart inner governance and smart democratic control Smart economy and trading.

These Basic Package Diagrams for a “Smart City” represent a structured and comprehensive approach to shaping modern urban environments. Each component within these diagrams addresses a fundamental aspect of urban living in the digital age. From sustainable energy management to efficient transportation, accessible healthcare, and

50

A. Gasidou et al.

disaster preparedness, these elements cater to the citizens’ well-being and safety. Furthermore, the inclusion of smart training structures, technologically advanced buildings, and innovative governance models highlights the commitment to fostering education, infrastructure development, and citizen participation. The incorporation of a smart economy and trading underscores the importance of economic growth and adaptability in an interconnected world. By encompassing these diverse dimensions, the diagrams lay the groundwork for a holistic, resilient, and forward-looking urban transformation. The next step of our article refers to specific examples and fields of customization for the organization of smart city planning.

4 Fields of Customization APIs play a vital role in facilitating integration, interoperability, and the development of powerful, interconnected applications. Following a set of rules and protocols, the Application Programming Interface (API) facilitates the exchange of requests and responses between users and API servers. When a user or application enters a request, the API server processes it, performs the required actions, and produces a structured response. This response is then sent back to the user or application, which can use the data or take further action based on the information received. More specifically, in the modern era, digital networking platforms have become intertwined with digital culture, thus shaping, and reflecting the rules, behaviors, and values of our online world. These platforms have revolutionized the way people connect, communicate, and collaborate, leading to the emergence of a distinct digital culture characterized by hyperconnectivity, information sharing, participatory engagement, and digital activism. These platforms have revolutionized the way people connect, communicate, and collaborate, leading to the emergence of a distinct digital culture characterized by hyperconnectivity, information sharing, participatory engagement, and digital activism. Digital networking platforms have revolutionized collaboration, connecting individuals and institutions around the world. That is, they have promoted hyperconnectivity, transcending geographical boundaries and fostering a sense of global interconnectedness. Through platforms such as social media, individuals can create links with other individuals from around the world, leading to the confusion of physical and virtual social interactions. This excessive connectivity has reshaped digital culture, creating an environment where constant communication and information sharing are the norm. The deployment of 5G technology in smart cities (Fig. 3) has the potential to truly revolutionize urban environments, offering unprecedented connectivity, data transfer speeds, and reduced delays. As the next generation of wireless communication, 5G networks can handle massive amounts of data enabling the seamless integration of various smart city applications and services. With its ultra-high bandwidth and low latency, 5G opens the door to a wide range of transformative possibilities such as autonomous vehicles, smart grids, remote healthcare, and immersive augmented and virtual reality experiences. These features have the potential to significantly improve efficiency, sustainability, and quality of life in smart cities. With the ability to connect a vast number of sensors, devices, and infrastructure elements, 5G facilitates the exchange of data in

Parameterization and Modeling of Structural Designs

51

Fig. 3. Deployment of 5G technology in smart city.

real-time, enabling intelligent decision-making and adaptability in urban environments. These functions have the potential to significantly improve the efficiency, sustainability, and quality of life in smart cities. With the ability to connect a vast number of sensors, devices, and infrastructure elements, 5G facilitates the exchange of data in real-time, enabling intelligent decision-making and adaptability in urban environments. This connectivity extends to critical sectors such as transport, energy, healthcare, and public safety. As an example, 5G-enabled smart traffic management systems can optimize traffic flow, reduce congestion, and enhance safety through real-time data exchange between vehicles, infrastructure, and traffic management centers. In addition, 5G networks facilitate the implementation of smart energy grids, enabling efficient energy distribution, load balancing, and the integration of renewable energy sources, thus contributing to sustainability goals. In essence, the widespread adoption of 5G technology in smart cities promises huge benefits for transforming urban environments, improving services, and creating a more connected and efficient society. The availability of real-time digital transport journey updates in smart cities has many advantages, such as reducing journey times, improving transport efficiency, improving the commuter experience, and making more frequent use of sustainable transport options. As an example, Fig. 4 illustrates a citizen-user of a smart city who receives a notification from his mobile device about the arrival of his bus. The inclusion of “digital transport journey updates” aligns with the increasing importance of efficient and informed transportation systems in urban areas. This example was likely chosen due to its ability to enhance citizens’ daily lives by providing real-time information and optimizing commuting experiences. The deployment of 5G technology is revolutionizing bus applications for citizens, enabling faster and more reliable connectivity that enhances real-time tracking, payment systems, and information dissemination, ultimately improving the overall public transportation experience. Mobile phone applications play a key role in informing citizens about bus arrival times in smart cities. Leveraging real-time data integration and analytics, these apps provide users with up-to-date information on bus schedules and estimated arrival times. By integrating with the city’s transport system and using sensors and GPS

52

A. Gasidou et al.

Fig. 4. Mobile application notification of bus arrival.

devices on the buses, the apps receive accurate location data. When a user selects a desired route or bus stop, the app records their preferences and continuously monitors the bus route. As the estimated arrival time aligns with the user’s request, the app sends a notification to the user’s mobile device, immediately informing them of the impending arrival. This seamless integration between the transport system, data processing, and mobile applications enables citizens to plan their journeys more efficiently and make the most of the smart city infrastructure. Smart grids play a key role in the management of renewable energy sources, implementing various strategies and technologies to ensure the efficient integration and use of these clean energy sources. Figure 5 illustrates the smart grid management of renewable energy sources like wind, solar, and thermal power, paving the way for a sustainable and eco-friendly future. In smart cities, the management of these energy sources is based on the implementation of smart grids. Smart grids enable the efficient and effective use of these renewable energy sources. Through real-time monitoring and control, grid operators can collect data on energy production from wind turbines, solar panels, and thermal energy systems. This information allows them to optimize the integration of renewable energy sources into the grid, balancing supply, and demand in real time. Smart grids also facilitate the seamless flow of electricity and the efficient management of distributed energy resources. Last, but not least tool for the Parameterization and Modeling of Structural Designs for the Transformation of a Smart City is the Metaverse (Fig. 6). Metaverse can provide a flexible, interactive, and connected environment that helps in the development and design of a smart city [20]. From virtual representation to data analysis, communication, and education, Metaverse can help improve a city’s quality of life and efficiency. Metaverse is a term that describes a virtual environment where people can interact with each other and with the digital world [21]. When combined with the smart city concept, Metaverse can provide many benefits and help design a more efficient, sustainable, and intelligent city [22]. Metaverse can create a virtual representation of the smart city, including information about the city’s infrastructure, transportation, energy, and other features (Fig. 7). This virtual representation can be used to analyze, plan, and optimize the city [23].

Parameterization and Modeling of Structural Designs

53

Fig. 5. Smart renewable energy management.

Fig. 6. Metaverse Customization of a Smart City v1.

Metaverse can act as a data collection platform for the smart city. By using sensors, IoT devices, and other connected devices, data can be collected on traffic, air quality, energy consumption, and other aspects of the city. This data can be analyzed to provide effective solutions to improve the city. Virtual reality and augmented reality can be used to present information about the city and its services in real-time. Residents can use these technologies to communicate with smart city systems, schedule transportation, find information about events, and interact with other residents. The Metaverse can create a space for social collaboration and communication among residents of a smart city. Residents can meet in virtual environments, exchange ideas, collaborate on projects, and participate in communities to improve the city. Metaverse can provide educational opportunities for smart city residents. Through virtual environments and interactive applications, residents can learn about the services, technologies, and best practices used in the smart city.

54

A. Gasidou et al.

Fig. 7. Metaverse Customization of a Smart City v2.

5 Conclusions In this article, we delve into the imperative need for defining parameters that underpin the comprehensive process of transforming a city or region into a “smart” entity. We address the central inquiry regarding the roles of digital environment engineers, digital transformation designers, and active citizen-users in shaping areas of smart cities. These explorations highlight not only the technical aspect but also the social dynamics involved. Our investigation underscores that a well-structured smart city design must be deeply rooted in principles of equity, solidarity, inclusivity, and productive collaboration. A strategic approach to urban revitalization, intertwined with participatory engagement, forms the crux of awakening urban centers. This strategy orchestrates a synergy among the public, private, and social sectors to propel sustainable development, tap into cultural potential, and kindle human resources, all while leveraging the richness of European culture. We assert that effective design and modeling transcend mere technical proficiency. They intricately blend form and content to resonate with individuals and societal clusters. Notably, within the realm of cultural applications, where visual content predominates, the significance of reliable design cannot be overstated. This underscores the necessity of employing user-citizen experience studies to ensure success. Our recommendations for political and social parameters that govern the Modeling of Structural Designs for the Transformation of a Smart City are as follows: • Codetermination in the construction of the proposed plan for the transformation of a Smart City, and full transparency about the constructive algorithm. • Public control over the selection and the pace of technological reorganization of production processes. • Harnessing the profit motive in the name of satisfying social needs. • Production is subordinated to the well-being of society and not only for profit. • Reduction in necessary labor time must be distributed equally across society. • Less workload time for each means freer time for all to cultivate individual and collective human capabilities.

Parameterization and Modeling of Structural Designs

55

The impending phase pertains to conceiving a strategic plan to bridge the gap between workers, unions, and the interplay of intellectual and manual labor. This initiative aims to forge a path of resilience and optimism for the future. Acknowledgment. This work has been partly supported by the University of Piraeus Research Center.

References 1. Dolgui, A., Ivanov, D.: 5G in digital supply chain and operations management: fostering flexibility, end-to-end connectivity, and real-time visibility through internet-of-everything. Int. J. Prod. Res. 60(2), 442–451 (2022) 2. Ebel, P., Bretschneider, U., Leimeister, J.M.: Leveraging virtual business model innovation: a framework for designing business model development tools. Inf. Syst. J. 26(5), 519–550 (2016) 3. Gasco-Hernandez, M., Nasi, G., Cucciniello, M., Hiedemann, A.M.: The role of organizational capacity to foster digital transformation in local governments: the case of three European smart cities. Urban Gov. 2(2), 236−246 (2022) 4. Komninos, N., Kakderi, C., Collado, A., Papadaki, I., Panori, A.: Digital transformation of city ecosystems: platforms shaping engagement and externalities across vertical markets. In: Sustainable Smart City Transitions, pp. 91–112. Routledge (2022) 5. Nicolas-Agustin, A., Jiménez-Jiménez, D., Maeso-Fernandez, F.: The role of human resource practices in the implementation of digital transformation. Int. J. Manpow. 43(2), 395–410 (2022) 6. Marginson, S.: What drives global science? the four competing narratives. Stud. High. Educ. 47(8), 1566–1584 (2022) 7. Raimi, M.O., et al.: Leaving no one behind: impact of soil pollution on biodiversity in the global south: a global call for action. In: Chibueze Izah, S. (ed.) Biodiversity in Africa: Potentials, Threats and Conservation. Sustainable Development and Biodiversity, vol. 29. Springer, Singapore, pp. 205–237 (2022). https://doi.org/10.1007/978-981-19-3326-4_8 8. Bolay, J.C.: Urban Planning Against Poverty: How to Think and Do Better Cities in the Global South, p. 214. Springer Nature (2020) 9. Zandbergen, D., Uitermark, J.: In search of the smart citizen: republican and cybernetic citizenship in the smart city. Urban Stud. 57(8), 1733–1748 (2020) 10. Rose, G.: Existing sociality in a smart city: the social as sociological, neoliberal and cybernetic. City 24(3–4), 512–529 (2020) 11. Iatrellis, O., Panagiotakopoulos, T., Gerogiannis, V.C., Fitsilis, P., Kameas, A.: Cloud computing and semantic web technologies for ubiquitous management of smart cities-related competencies. Educ. Inf. Technol. 26, 2143–2164 (2021) 12. Silva, B.N., Khan, M., Han, K.: Towards sustainable smart cities: a review of trends, architectures, components, and open challenges in smart cities. Sustain. Cities Soc. 38, 697–713 (2018) 13. López-Quiles, J.M., Rodríguez Bolívar, M.P.: Smart technologies for smart governments: A review of technological tools in smart cities. Smart Technol. Smart Governments Transparency Effi. Organ. Issues, 1–18 (2018) 14. Amin, F., Choi, G.S.: Hotspot analysis using the cyber-physical-social system for a smart city. IEEE Access 8 (2020)

56

A. Gasidou et al.

15. Batty, M.: Artificial intelligence and smart cities. Environ. Plann. B Urban Analytics City Sci. 45(1), 3–6 (2018) 16. Allam, Z., Dhunny, Z.A.: On big data, artificial intelligence, and smart cities. Cities 89, 80–91 (2019) 17. Ullah, Z., Al-Turjman, F., Mostarda, L., Gagliardi, R.: Applications of artificial intelligence and machine learning in smart cities. Comput. Commun. 154, 313–323 (2020) 18. Döller, V., Karagiannis, D.: Formalizing conceptual modeling methods with MetaMorph. In: Augusto, A., Gill, A., Nurcan, S., Reinhartz-Berger, I., Schmidt, R., Zdravkovic, J. (eds.) BPMDS/EMMSAD -2021. LNBIP, vol. 421, pp. 245–261. Springer, Cham (2021). https:// doi.org/10.1007/978-3-030-79186-5_16 19. Karagiannis, D., Buchmann, R.A., Utz, W.: The OMiLAB Digital Innovation environment: Agile conceptual models to bridge business value with digital and physical twins for productservice systems development. Comput. Ind. 138 (2022) 20. Allam, Z., Sharifi, A., Bibri, S.E., Jones, D.S., Krogstie, J.: The metaverse as a virtual form of smart cities: Opportunities and challenges for environmental, economic, and social sustainability in urban futures. Smart Cities 5(3), 771–801 (2022) 21. Kusuma, A.T., Supangkat, S.H.: Metaverse fundamental technologies for smart city: A literature review. In: 2022 International Conference on ICT for Smart Society (ICISS), pp. 1–7. IEEE (2022) 22. Bibri, S.E.: The social shaping of the metaverse as an alternative to the imaginaries of datadriven smart Cities: a study in science, technology, and society. Smart Cities 5(3), 832–874 (2022) 23. Kuru, K.: MetaOmniCity: towards immersive urban metaverse cyberspaces using smart city digital twins. IEEE Access (2023)

Smart m-Observation and Study of Orthodox Art Desislava Paneva-Marinova1(B) , Detelin Luchev1 , Maxim Goynov1 , Emanuela Mitreva1 , Radoslav Pavlov1 , and Dušan Tati´c2 1 Institute of Mathematics and Informatics at the Bulgarian Academy of Sciences, Sofia,

Bulgaria {dessi,radko}@cc.bas.bg, [email protected] 2 Mathematical Institute of the Serbian Academy of Sciences and Arts, Belgrade, Serbia

Abstract. This paper presents a progressive web application for Bulgarian iconographical art that provides specific and complex features for knowledge discovery, dynamic categorization, filtering, content analysis, shared content usage, etc. The main goal of the application is to make the learning process of students interactive and full-fledged through m-observation and to provide an easier and more effective way of researching and comparing iconographic objects. Keywords: Mobile Learning · Progressive Web Application · Orthodox Art

1 Introduction Orthodox art represents an era with great effect on spiritual life and strive for culture. So it is of utmost importance that it is accessible digitally for the purpose of education, culture and observation. Studying and understanding Orthodox artworks is not easy and requires different tools and methods, especially when they will be used for e- or m-learning. Orthodox art resources, metadata, how they can be accessed, searched and selected and displayed in a content management platform are extremely important for an innovative and ubiquitous learning process. The development of digital technologies is changing the way students learn and teachers present learning material. Contemporary education in the humanities and social sciences is increasingly supported by specialized digital libraries, virtual museums, virtual reconstructions of historical sites, etc., accessible via either desktop or mobile devices. Recent studies show that most students are positive about such a change and mostly use mobile technologies, despite some hardware or software limitations or the inability of the students themselves to use such devices properly [1, 2]. For example, small screen size and quality present challenges for user interface visualization, categorization and presentation of materials, which is especially important when comparing virtual historical objects such as relics or paintings from different periods. In order to enable deeper exploration of historical content, specialized functions like search options, content categorization and filtering have to be easily accessible to students. On the other side, the ability to easily access any content and communicate with anyone regardless of their © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 57–62, 2024. https://doi.org/10.1007/978-3-031-56075-0_6

58

D. Paneva-Marinova et al.

location and physical abilities is something that facilitates collaboration and learning [3, 4]. Mobile technologies increase the engagement of students and their feeling of belonging to a group [5]. Mobile technologies provide flexibility at a low cost and learning across platforms, devices and locations [6]. In combination with virtual reality (VR) and augmented reality (AR) solutions, m-learning may help students to better acquire historical knowledge. Using VR solutions, they can virtually walk through museum exhibitions or visit historical ancient places [7]. Also, by using AR students can access multimedia storytelling about monuments near them or see a 3D reconstruction of the parts that are missing in real-time at the exact location. The purpose of this paper is to present a progressive web application (PWA) for Bulgarian iconographical art, targeting improved learner m-experiences and providing specific and complex features for knowledge discovery, dynamic categorization, filtering, content analysis, shared content usage, etc. Effective art study requires the students to be engaged actively in the observation and analysis phase in more interactive and meaningful ways so the main goal is to identify and provide technological solutions for smart mobile observation and study of Orthodox Art. In Sect. 2 we focus on the used methodology and technology and in Sect. 3 we discuss the achieved results – the software platform, its functionality, current usage scenarios, impact and testing. The conclusion summarizes the achieved results.

2 Approach 2.1 Used Methodology The development of the application follows the research and development method starting with the stages of identifying m-learning context, data collection, and software and hardware analysis. Problem identification is carried out by using observations and interviews with target groups (lecturers and students at art schools and universities) for using mobile applications in the education process as a whole or for specific learning scenarios. For the software analysis stage, the used dataset is based on the Digital Library “Virtual Encyclopedia of Bulgarian Iconography” (BIDL, https://bidl.math.bas.bg/en), which current version is developed as an infrastructure component of the Bulgarian National Interdisciplinary Research e-Infrastructure for Resources and Technologies in Favor of the Bulgarian Language and Cultural Heritage. That stage identifies how the target dataset and services can be customized in the mobile version for the needs of education and to accomplish improved learner m-experiences of art students. In the hardware analysis phase, different types of mobile devices are tested to see how they can be used for specific educational goals. In the next stages, we go through product design and development, software testing and validation, and product evaluation and revision. The product design and development stages include creating detailed specifications of all the required features and services and their realization. The last stages are software testing and validation, product evaluation and revision with users to get to the final stable product.

Smart m-Observation and Study of Orthodox Art

59

2.2 Used Technologies The BIDL platform is based on some of the most powerful technologies used for web development - a load-balanced Node JS-based back-end server behind NGINX web server, No-SQL database management system - MongoDB, Vue and Bootstrap as frontend frameworks and SPHINX search for the full-text index search capabilities. The platform is implemented using a three-layered architecture (database, application, and presentation layers). Each layer is divided into two parts – core and custom. This allows the platform to be widely reused for different purposes. Following the currently developed technological web-based infrastructure, a transformation into a progressive web application is a reasonable continuation of our strategy to create a closer relationship between technologies and users. Progressive web applications as a technology enhance the described infrastructure, providing many features common for modern native mobile applications [8]. PWA is not only a web application, it is a new philosophy for building such. It defines some specific patterns and good practices, application interfaces (APIs), and other features. The most significant features that a PWA has in comparison to a traditional web application are the ability to use the device’s storage, periphery (USB devices attached), camera, and geolocation. PWA can send push notifications and they are installable and updatable just like a native mobile application. PWA is cross-platform. This means the same codebase can be used on Android, iOS, Windows, Linux, etc.

3 Actual Outcomes 3.1 BIDL m-Application The main objective of BIDL platform is to store and manage different types of digitized copies of Orthodox artworks, including text, graphics, video, or other media objects as well as the relevant metadata, based on ontology [9]. The Orthodox iconographic objects presented originate from the end of the XII to the beginning of the XX centuries and the majority of them belong to significant Bulgarian schools of iconography such as iconographic schools in Bansko, Tryavna, Samokov, Strandzha, as well as icons from Veliko Tarnovo, Rila Monastery, Arbanasi, etc. churches, monasteries, and private collections. The objects are grouped into thematic collections according to different descriptive criteria (classes in the descriptive ontology), incl. Title, artist, period (in years and centuries), school, iconographic technique, base material, location, etc. The application offers dynamically extensible schemas for the meta-description of the target dataset allowing dynamic updates of the metadata for sustainable enrichment. These features and those offered for full data access, enhanced searching, dynamic content categorization, filtering, and customized grouping make the platform a versatile learning tool used in art schools and universities during their laboratory and outdoor learning activities. Figure 1 depicts a thematic grouping of iconographic objects by schools and resulting content. In addition, the PWA infrastructure allows the implementation of a proactive mobservation environment, giving the users the ability to be notified when new content related to their interests is available, or when other user-related events are triggered. The

60

D. Paneva-Marinova et al.

access of PWA to the user device’s camera, geolocation, and push notifications gives us the ability to implement specific features for on-place analysis and learning. The PWA is able to identify the user’s location, match it with relevant content from the BIDL metadata database (icons relative geolocation), and send push notifications to the user if specific content is found. The camera can be used for identifying specific image content and finding matches in the BIDL content database. Augmented reality technology is used in the created application to recognize the icons in visitors’ surrounding and to provide information about them. When the camera of the mobile device is directed towards an icon, AR software compares the image captured by the camera with previously prepared images of icons stored in the application database. When the icon is recognized, a virtual button for more information about the icon is displayed as an overlay. The button overlay is positioned in real time and tracks the icon in 3D space while the icon is in the field of view of the mobile device camera. The interaction with the button enables the loading of educational content from a purposely prepared m-learning platform.

Fig. 1. Thematic grouping of iconographic objects by schools and resulting content.

3.2 Educational Usage and Impact The realization of the application is forced due to a shift to online education of students. The m-observation and study were done with specific churches, monasteries, museums, regions, iconographic schools, etc. The functionality of the mobile application allowed one to carry out tasks and projects related to art critic analysis of the chronological development of iconography. Moreover, comparing iconographic objects in one or more iconographic techniques and evaluating the quality of their execution; periodization of

Smart m-Observation and Study of Orthodox Art

61

basic iconographic techniques were used in the best Bulgarian examples of iconography; comparing selected iconographic objects in terms of clothing, gesture, proportions of characters and objects, presence of another character or symbol, backgrounds, other elements in the iconography of the image, etc. The ability to share results provides conditions for completing tasks in a group. At the same time, even during an individual visit to a certain church or museum, the m-observation makes it possible to use the database for additional and more detailed information about the observed and studied objects of Orthodox art, as well as to use already achieved results on assigned tasks and projects. The application was provided to art students that study iconography to see whether the developed functionality is useful for their work. During the testing period the students shared that most of them (more than 80%) were satisfied with the features of the application. They were able to make observations on different iconographic objects without the need of being present in the specific place or if present it increased their productivity when they needed to observe a specific object and compared it to other objects of the same period, technique, base material or other features. They also pointed out that having all of the information for example for a specific period and object in the palm of their hand sped up their research and learning process. They discovered that even if seeing the object in person, they could observe the object in more details (zoom in through the application) and compare it with the actual display and read up about it in the application and to find some other research papers on the topic. The application would be helpful not just for art students that are learning about the domain, but for visitors of churches, monasteries, museums, etc. Providing an easy way of observing and comparing objects makes the experience of visiting memory institutions more fulfilling. The specific and complex features for knowledge discovery, dynamic categorization, filtering and content analysis can provide a fulfilling experience to any visitor - from the art student that needs to research the objects to the ordinary visitor that would like to just learn more about the domain. We should however admit that young people would be more inclined to use this application as they are more comfortable with mobiles and mobile applications.

4 Conclusions In this paper the symbiosis between mobile application and Orthodox art and culture has been studied. The digital representation allows a rich educational process in a live environment with access to everything at any moment. With this application it is possible to learn with more focus on the practical side - such as observing, analysing and studying target artefacts and iconographic content (iconographic schools, authors, art periods, etc.); finding or verifying features and influences; making new art or learning projects, documentaries, performance, exhibitions, interactive virtual gaming and gamification, storytelling, studying, etc. The content-dependent use of digital objects for different learning purposes is applicable in real life and stimulates creative thinking, learning-bydoing and learning-by-authoring, the force of any successful and lasting learning.

62

D. Paneva-Marinova et al.

Acknowledgements. This research work was carried out and is supported partly by the joint research project “Development of Software Tools and Multimedia Technologies for Digital Presentation, Preservation and Management of Cultural Heritage” between the Institute of Mathematics and Informatics, Bulgarian Academy of Sciences and the Mathematical Institute of the Serbian Academy of Sciences and Arts (2023–2025) and CLaDA-BG, the Bulgarian National Interdisciplinary Research e-Infrastructure for Resources and Technologies in favor of the Bulgarian Language and Cultural Heritage, part of the EU infrastructures CLARIN and DARIAH, Grant number DO01–167/28.07.2022, https://clada-bg.eu/bg/. It is also supported by the program “Increasing Research Capacity in the Field of Mathematical Sciences” (PICOM, DO1–67/05.05.2022), financed by the Bulgarian Ministry of Education and Science.

References 1. Guruge, D., Paudel, K., Kadel, R., Aziz, S., Karagiannidis, V. Analysing student expectation on mobile technologies to enhance student learning experience. In: Proceedings of the 2021 2nd International Conference on Education Development and Studies (ICEDS 2021), pp. 45–49. ACM, New York, NY, USA (2021). https://doi.org/10.1145/3459043.3459050 2. Poong, Y.S., Yamaguchi, S.Y., Takada, J.I.: Analysing mobile learning acceptance in the world heritage town of Luang Prabang, Lao PDR. In: Mobile Learning in Higher Education in the Asia-Pacific Region, pp. 191–211. Springer, Singapore (2017). https://doi.org/10.1007/978981-10-4944-6_10 3. Garg, S., Prasad, Y., Kumar, V.: Mobile-based active learning can enhance engagement in classroom. In: Proceedings of the 14th International Conference on Education Technology and Computers (ICETC 2022), pp. 212–218. ACM, New York, NY, USA (2023) 4. Lian, Y.: Smart education: education reform in the age of intelligence. In: Proceedings of the 2021 5th International Conference on Education and E-Learning (ICEEL 2021), pp. 41–45. ACM, New York, NY, USA (2022). https://doi.org/10.1145/3502434.3502478 5. Pearce, A.: Learning with mobile technologies: their potential to enhance well-being, collaboration and social enrichment among diverse students. In: Proceedings of the 13th International Conference on Education Technology and Computers (ICETC 2021), pp. 230–236. ACM, New York, NY, USA (2022). https://doi.org/10.1145/3498765.3498801 6. Wang, X.: Mobile learning in Chinese higher education: student perspectives, advantages and challenges. In: Proceedings of the 14th International Conference on Education Technology and Computers (ICETC 2022), pp. 207–211. ACM, New York, NY, USA (2023) 7. Argyriou, L., Economou, D., Bouki, V. 360-degree interactive video application for cultural heritage education. In: 3rd Annual International Conference of the Immersive Learning Research Network, pp. 297–304. Verlag der Technischen Universität Graz (2017) 8. Sharma, S., Bhardwaj, A.: Progressive web apps (PWA). J. Emerg. Technol. Innov. Res. 8(7), e696−e706 (2021). https://www.jetir.org/view?paper=JETIR2107594 9. Pavlova-Draganova, L., Paneva-Marinova, D., Pavlov, R., Goynov, M.: On the wider accessibility of the valuable phenomena of orthodox iconography through digital library. In: Proceedings of the 3rd International Conference dedicated on Digital Heritage (EuroMed 2010), 8–13 November 2010, Lymassol, pp. 173–178. Archaeolingua (2010)

Interactive Chatbot for Improving the Text Classification Data Quality Doaa S. Elzanfaly1 , Nada Amr Mohamed2 , and Nermin Abdelhakim Othman1,2(B) 1 Department of Information Systems, Faculty of Computers and Artificial Intelligence,

Helwan University, Cairo, Egypt [email protected] 2 Department of Information Systems, Faculty of Informatics and Computer Science, British University in Egypt, Cairo, Egypt

Abstract. The pandemic is affecting the global community in many ways. In most developing countries, there is a limitation in the detection facilities, which affect many suspected cases. This paper proposes a chatbot framework to assist and provide guide for the suspected/infected patients with COVID-19. Conversational software agents activated by natural language processing is known as chatbot, are an excellent example of such machine. Our COVID Bot is based on an integrated model between the rule-based model and the class classification model, having the rule-based model integrated with the MongoDB NoSQL database. Chatbot, using Natural Language Processing (NLP) and data mining techniques to assist patients by providing immediate answers for their questions. It also acts as a novel communication mean for impaired people for sharing knowledge and information, through conversing with them. Based on the literature review, this paper compared our methods with three classical classification algorithms: random forest, gradient boosting, and multi-layer perceptron (MLP). Experimental results show that our proposed chatbot greatly improves the classification performance, with IE-Net as 94.80%, 92.79% as recall, 92.97% as precision and 94.93% as AUC for distinguishing COVID-19 cases from non-COVID-19 cases (with only common clinical diagnose data). Keywords: COVID 19 · Pandemic · Coronavirus · Chatbot · Natural Language Processing · Rule-Based Model · Deep Learning · Dialog System

1 Introduction COVID-19 is a worldwide pandemic that affected all the people around the world. People all around the world are seeking information about COVID-19 symptoms, how to deal with it in case they are infected, and many other questions. Chatbot is a program for artificial intelligence that can develop or simulating a communication with people and interact as a person with them. Chatbot is essential as it can communicate with individuals and machines and can respond to any inquiries using natural language processing. Remarkably, chatbot is used © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 63–77, 2024. https://doi.org/10.1007/978-3-031-56075-0_7

64

D. S. Elzanfaly et al.

nowadays in several aspects for several purposes such as: social, medical, education and business fields. As demonstrated in [1], about 55% of the users seeks an instant answer from the chatbot as they ask a question. Chatbots have many features that are beneficial; as it could help people asking urgent questions, patients that cannot reach to a doctor and wants someone to analyse their symptoms, students that need to solve their educational inquiries and disabled people who need a new way of communication and interaction. Therefore, chatbot should be expanded and used in several fields to improve the ways of interaction between humans and modern technology. Therefore, the goal of this paper is to develop a COVID-19 chatbot by using natural language processing, data mining techniques and integrate the NLP with NoSQL databases. The proposed COVID-19 chatbot allows communication with the user through chatting, it can respond to all questions related to COVID-19 pandemic such as: its symptoms, tests available, precautions need to be taken, etc. The COVID-19 chatbot is developed using two models to understand the user’s input accurately. The first model is a rule-based model, and the second is a classification model that also incorporates text vectorization techniques. The rule-based approach is chosen for the COVID-19 chatbot as it provides a reliable response as the system’s results are based on rules. The output replies are predictable and are implying meaning that cannot be ambiguous. Furthermore, data mining techniques are utilised to improve the bot’s response if the rule-based model is unable to identify the correct response. Additionally, the core intention for connecting the rule-based model to MongoDB as it’s the best recommended database among the other NoSQL databases, because it stores data in a document format, and it works well with text and document data. MongoDB as well doesn’t use a schema, which makes working with data much easier and faster and contributed to the chatbot’s efficiency. In this paper we are concerned with vectorization. Which is converting a text document into vector representations [6]. It is essential in NLP systems to translate textual materials into understandable numerical representations (matrix representation), so that it can allow machines to interpret them [7]. There are various text vectorization techniques such as: frequency vectorizer and TF-IDF vectorizer. In our work we have made a comparative study between the classical classification models. There are several text classifiers like Support Vector Machine, K-nearest Neighbour, Multi-layer Perceptron, types of Naive Bayes classifier as Multinomial, Bernoulli, Complement and Categorical. This study shows the strength and weakness of each classifier, then the best classifier is used within out proposed model. In this paper, the COVID-19 chatbot is developed using two approaches: • First, the rule-based model that uses internally a dictionary lookup method (levenshtein distance) to compare the user’s input with data stored in MongoDB. • Second, using data mining techniques including text vectorization to extract features of data questions and represent them in a numerical matrix form so the models can deal with it. The vectorized data is sent to the classification models to see which model gives better result to used. The rest of the paper is organized as follows: Sect. 2 presents related work. Section 3 illustrate the proposed framework. Section 4 discuss the experimental results. Section 5 present conclusion and future work.

Interactive Chatbot for Improving the Text Classification

65

2 Literature Review 2.1 Chatbot Definition Chatbot is a computer program designed to simulate conversation with human users, especially over the internet. It is a new form of customer interaction powered by artificial intelligence using chat interface. Chatbot is based on artificial intelligence techniques that provide the user with a support, as its able to understand the natural language, and emotion. As an example, it makes it easy for the users to get answers for their queries in a suitable way without wasting time wither on a phone or waiting for an email which therefore will makes the phone calls and mail’s number decline. Researchers have published many chatbot models in recent years. The reason for that is the improvement and development of artificial intelligence that have contributed a great deal in building chatbots in various domains like social and educational domains. This study provides an overview on the recent literature of the chatbot applications used in different fields. But it focuses on COVID-19 field, describing it’s the strength and challenges. 2.2 Chatbot Types In our work on chatbots, we noticed three major types of chatbots namely, Rule-based, Retrieval based model, Generative based model. • Rule-based Model: a bot answers questions upon a predesigned set of rules. The rules defined can scale from simple to complex. The chatbots using this model can answer a simple and limited number of questions. • Retrieval based Model: this model uses a limiting condition (heuristic) and selects a response from a set of already defined responses. The bot makes its selection of the best response based on conversation context. They are easy to build when compared to generative models. • Generative based Model: this model generates a response based on the current and the previous experience. Bots designed with this model is highly advanced and require high computational models and a huge amount of data to be trained. 2.3 Education and Knowledge Transfer System In the learning field, [14] propose the Tutor-bot chatbot which supports students in their courses by answering their questions and issues. It checks the student’s questions and provide respond by using natural language processing techniques. Tutor-bot architecture consists of front-end interface, back-office, knowledge base module and e-learning bot module which is core engine in the system. Latent Dirichlet allocation (LDA) algorithm is used to manage and process the user query and extract keywords. An experimental test has made to evaluate the performance of the bot, the results was obtained as the following: 71.13% correct responses, about 16.04% was responses doesn’t fit the student requirements due to not recognizing the actual user needs and 12.83% was wrong suggestions. The students were satisfied with

66

D. S. Elzanfaly et al.

the chatbot as it was user friendly and easy to use. But the proportion of wrong responses is not small which is considered its drawback. Xiao-Ice is a social chatbots [12], it is developed to communicate and response with people through conversations and understand people’s emotions. They consider both intelligent quotient (IQ) and emotional quotient (EQ) in the design of the system. The main purpose of developing Xiao-Ice is to make an emotional connection with users, as the users can talk to the bot about his interests anytime. The architecture of XiaoIce is composed of three layers which are: user experience that is used to process inputs from users and answers from the bot, conversation engine that consists of dialogue manager, and data layer that contains datasets that stores human’s data [13]. 2.4 Emergency Response- COVID-19 Many research papers focused on building a chatbot for the COVID-19 cases to assist and reassure the suspected patients. In [19], they made the outline of the Penn medicine (academic medical centre) chatbot working together with verily, google cloud and Quantiphi company. They analyse the patient’s questions to respond accurately to the questions and symptoms. A FAQ bot is created by making a list of the frequently questions asked by the patient, analysing them, and then sorting them in groups by using a database of COVID-19. An internal search tool was developed to search the FAQ database and show both the questions and answers. The bot was tested by gathering 500 feedback submissions in three days to identify the problems and the questions that the bot couldn’t answer them. The accuracy of the bot was 75%. Moreover, they provided the bot with the functionality to triage symptomatic patients to the required level of treatment. They developed an algorithm that reflects what experienced doctors do instead of training the bot to learn all the patient symptoms. In [20] Doctor Bot is implemented using retrieval-based technique and natural language processing. They used TF-IDF vectorizer to accurately extract the features and return a reasonable response according to cosine similarity index. An AI-based chatbot-based workflow was introduced by the University of California, San Francisco Health. Within the first two month, it performed over 270,000 screenings. It affected positively the physical distancing, stopped potentially infectious individuals from accessing the facility, and provided valuable live data for decision making personnel [37]. 2.5 Comparative Analysis In [15], the researcher focused on developing a conversation chatbot that can comprehend the natural language processing, so that the chatbot can respond in the manner that the user expects. This paper compared several Classification techniques as Naive Bayes, linear regression model, Naïve Bayes model and linear regression model. The output a confusion matrix was as the following Naive Bayes as 63%, regression model as 72%. That showed that the regression model was the higher accuracy model. Although the accuracy is good, errors could happen because they have limitations in the size of training data as it is small.

Interactive Chatbot for Improving the Text Classification

67

The Xiao-Ice chatbot proposed by [12] and [13] has a great performance as it can determine people’s emotions and providing them with support and encouragement. But despite current developments in social chatbots like Xiao-Ice, the basic structure of people’s thoughts as often expressed in human’s interaction, is still not completely understood. As it is difficult to develop a chatbot that can completely comprehend people’s needs [13]. As demonstrated in [2, 16, 17] and [18], all these chatbots are designed to support the healthcare and medical field. The healthcare chatbot shown in [2] and [17] don’t use a data store to save the users input which is considered a drawback as the user may use the bot again to ask in the same domain. The chatbot seen in [16] has better performance than two previous bots because it has more advanced features like storing the user’s information in a database and can recommend a suitable doctor to each case. The chatbot in study [18] has the best performance among all the other healthcare bots as it tried various data mining models, then they used an ensemble classifier which is an advanced approach compared to other bots that combine more than one classical model. The accuracy of the COVID-19 chatbot [19], in the first time is 61% which is not so efficient as the bot failed in answering the questions that are not included in the data. In the second time, after adding more data, the chatbot becomes more efficient as the accuracy is 75%. The COVID-19 chatbot (Doctor-Bot) accomplished the goal and tried to respond with the most accurate response, but it is still limited by the dataset that was given [20].

3 Proposed Chatbot Framework Our proposed COVID Bot is a chatbot regarding the medical field, that responds to any inquiry related to COVID19 pandemic. The user can communicate and interact with the bot by text-to-text chatting through a Graphical User Interface (GUI). 3.1 Chatbot Framework The COVID Bot is implemented using the rule-based model and classical classification model. When a question is received from the user, the bot responds to the user’s input by using either the rule-based model or the classical classification model, based on a threshold value. The Threshold is a predetermined cutoff for recognizing an evaluation and adjusting the model. As shown in Fig. 1. The chatbot receives the user’s question, it looks up for possible responses and determines how probable each one is. If the threshold value of the rulebased model is greater than the value of classical classification model, then the bot will return the response of the rule-based model; otherwise, it will return the classification model response. Threshold helps the system to respond more accurately and enhance its performance [23]. 3.2 Chatbot Model This section will discuss the chatbot model that is based on the rule-based model and the classification model, and how the data is used in each model.

68

D. S. Elzanfaly et al.

Rule-Based Model NoSQL databases is used within this model, they are non-relational databases suitable for coping with huge storage of data, handling and querying them [8]. As relational databases have constraints and limitations with dealing with huge data. The main characteristics of NoSQL databases are strong consistency, high availability and partition tolerance, NoSQL loses the consistency to improve availability and partition tolerance [9]. NoSQL databases are classified into four models which are: key values stores, column family stores, document databases and graph databases. In this paper we are concerned with the document databases as they are built to handle and store large data or documents [9]. Moreover, it organizes the stored information as a collection of documents, so it is easy to add any new field to the database [10], in this paper, MongoDB is going to be used to store the data. As stated in [11], the features of MongoDB are: “documentoriented storage, replication, high availability, auto sharing, rich queries and fast changes in place”.

Fig. 1. Chatbot Flowchart

Interactive Chatbot for Improving the Text Classification

69

Model Steps: 1. First, the COVID-19 dataset is read, then the data is saved in a list in a form of question and answer that will be sent later to the MongoDB. 2. Then some pre-processing is applied on the data questions, which includes converting upper case characters to lower case, tokenizing the sentence into words to extract keywords from the data questions, removal of stop words list so that they can’t be counted as keywords, and removal of punctuation. 3. After that, a connection is opened with MongoDB database to store and hold all the data in a collection. The data is saved in the form of dictionary that contains the data questions as a key, and the value as a list of pairs which are: keywords of the questions and answers to the questions. When the chatbot receives any input, the model searches within the database to find the closest match. 4. When chatbot receives the user’s question, it extracts keywords and removes any stop word or punctuation. 5. The model then compares the input question and keywords extracted with the questions and keywords stored in the MongoDB to return closest response. As an example, if an input question “what is symptoms of COVID-19?” comes to the bot, the bot will compare with both the question and the keywords. The model compares the input question and keywords by using a dictionary lookup method called edit distance or levenshtein distance that compares between two strings. The levenshtein distance is a string metric that is used to compare two strings. It is the smallest number of character modifications such as: insertions, replacements, or deletions, necessary to convert one word to another [24]. Example on that, the edit distance between the two words “Horse” and “ROS” is 3. As “H” character is replaced with “R”, remove middle “R” character, and final operation is removing “E” character. The algorithm finds the minimum operation needed to transform one word to another. At last, it gets the edit distance between two strings where the greater the edit distance, the greater the difference between the strings are. The model compares with the keywords; there are two keywords extracted which are “COVID19” and “symptoms”, while the other words are removed due to their existence in the stop words list. The model takes the first keyword “symptoms” and starts searching in the whole MongoDB and compares the word with each keyword of every question in the database until finding the closet match to the input keyword. Then the second input keyword is taken, and the process is repeated until all input keywords find a match, and total score of all the keywords is returned. Then it compares with the question, the model takes the input question and starts searching in all the database as well until finding the question that is the closest match and achieves highest score among all the other data questions. At the end, if the score of the response is greater than the threshold, then the response is returned to the user. Data Mining Model A datamining model used is shown in Fig. 2 as a general flowchart, that shows the model steps, as well as how each model attempted functions, which will be described in detail. First, the COVID-19 dataset file is read, then data question and answers each are

70

D. S. Elzanfaly et al.

saved in different lists. Then we apply the two methods of text vectorization on the data questions. The text vectorization methods are the frequency vectorizer (TF) and term frequency inverse document frequency (TF-IDF). Both vectorizers are considered as a bag of words techniques that converts the data questions into vectors that is represented in a form of numerical matrix to describe the features of the data. TF vectorizer focuses on counting/seeing frequency of occurrence of each word that exists in each phrase [6]. It takes data questions, tokenize the sentence into words, remove words included in the stop words list, convert characters into lower case. Then, the all the words are put in an arranged list. After that, it starts counting the frequency of occurrence of each word relative to the sentence and represents it in a numerical matrix representation. TF-IDF vectorizer considers two things, the term frequency of word and its importance relative to the document. It takes data question, tokenize the sentence into words and put these words arranged in a list, remove words included in the stop words list, convert characters into lower case. After that, the TF-IDF counts the term frequency of each word and its importance relative to the document. The TF-IDF is calculated using Eq. (1): [25]   N (1) tfidfi,j = tfi,j X log dfi The TF-IDF vectorizer usually outperforms the TF because it holds two information: the term frequency and its importance relative to the document. Despite that, some classification models perform better when using the TF vectorizer while learning. Therefore, we applied the two vectorizers to discover which one performs better with this specific model. After the data questions are vectorized, various classical classification models are applied to analyze which one achieves better accuracy and performance, to be used in the chatbot. The classification models used are Support Vector Machine, K-nearest Neighbour, Multi-layer Perceptron, types of Naive Bayes classifier which are: Multinomial, Bernoulli, Complement and Categorical.

4 Results and Discussion 4.1 Dataset In this paper we worked on two datasets which contains COVID-19 questions and answers, including its definition, symptoms, ways of protection, how it transmits, tests required, vaccine, etc. Both datasets are stored in a.csv file format. The first dataset has 1612 questions and their answers [19]. While the second dataset has 86 questions and their answers [20]. Then they are merged by collecting all the data in one file. Then, the data is handled by removing the merged spaces, questions that have a missing answer are excluded. As the dataset is cleaned and doesn’t need to have much pre-processing on it. After that, to form the dataset, all the questions and their corresponding answers were manually gathered from multiple sources as mentioned above and saved in a text file that contains 1418 questions and answers concerning COVID-19 pandemic. In addition to that, a general English dataset is added, which includes greetings data that will be used in the chatbot. The greetings data contains 56 lines of data [21]. All these data are then saved in a collection inside the MongoDB.

Interactive Chatbot for Improving the Text Classification

71

Fig. 2. General flowchart for data mining model

Preparing test data is the initial stage in determining the accuracy of the COVID-19 chatbot. Thus, two test data files are created manually, the first of which contains 134 test questions that are distinct from the data on which the data bot was trained. The second file provides the test questions’ answers which are 134 answers as well. Consequently, this data is used inside a test function to calculate the bot accuracy. The test function work on the two test files, after reading them, it calls the function that returns the bot answer, passing to it: the test data questions, threshold of rule-based model and threshold of classification model (have a separate function that calculates them). Next, the function adds all the returned bot answers to an empty list and puts the actual answers that exist in test file in another list. Finally, the two lists are compared. The accuracy is calculated using a metric from sklearn library which is called “accuracy score” [22]. Precision, recall and f1-score metrics are calculated using sklearn library as well [23]. They are used to evaluate the reliability of predictions. The threshold function assigns an initial value for both the rule-based model threshold and the classification model threshold. Next, it starts looping on the two values, then it calls the test function defined above, and checks / validates the accuracy after each iteration. When the accuracy increases, it is updated, and the optimum threshold for two models is updated as well. At the end, the threshold for both models and accuracy of the chatbot are printed to see at which threshold the bot has highest accuracy. 4.2 Classifications Models Comparison Each model of the 7 classical classification models mentioned above is tried twice, once with TF vectorizer and once with TF-IDF vectorizer, to see which model performs better

72

D. S. Elzanfaly et al.

according to which vectorizer. Consequently, Naïve Bayes classifiers like Multinomial NB, Complement NB, Bernoulli NB, and Categorical NB gave better performance and higher training accuracy when TF vectorizer was applied. On the other hand, SVM, KNN and multi-layer perceptron models showed better performance with the TF-IDF vectorizer. Table 1 shows the accuracy of the training of the classifiers. As seen in table 2 while testing the model, the Multinomial NB has the highest accuracy, which results in 64.9%. Therefore, it will be used while implementing our proposed COVID-19 chatbot. Naïve Bayes: it is a probabilistic classifier that rely on Bayes theorem which calculates the conditional probability. For example, a probabilistic result of classifier P (c | d) is the conditional probability that document “d” relates to class “C”. Each word inside a document is assigned to it a probability depending on the number of times it appears in that document. In addition to that, Naïve Bayes could understand an evaluating set of classified texts and comparing all the data by creating a list of words and how many times each word appears. As a result, the highest probability frequency of word appearances can categorise incoming documents into the appropriate category [26]. Multinomial NB classifier is implemented while developing the COVID-19 chatbot. Table 1. Accuracy of training classifiers Model

Vectorizer

Training Accuracy

Multinomial NB

TF

97.2%

Complement NB

TF

96.5%

Bernoulli NB

TF

97.6%

Categorical NB

TF

97.6%

SVM

TF-IDF

98.2%

KNN

TF-IDF

51.2%

Multi-layer perceptron

TF-IDF

98.2%

Multinomial NB: this classifier is based on the term / word frequency idea, that refers to the number of times a word appears in a document [27]. Likewise, the classifier reveals two things: if a word appears in a document and how frequently it appears in that document. The aspect of Multinomial NB makes it a good option for document categorization because a word might be crucial. In addition to that, the term frequency is beneficial in determining whether a word is valuable in the evaluation [27]. The Multinomial NB is calculated through the following equations: • P(c) = Number documents with that class/Total number of documents (2) [27] • P (w | c) = count (w, c) + 1/count(c) + |V| (3) [27], Equation3 can be used to determine the conditional probability of a word belonging to a particular class. Besides, 1 and |V| are smoothing constants that are added to try to prevent computation errors when the term doesn’t appear in the document; Laplace Smoothing is the name for this concept.

Interactive Chatbot for Improving the Text Classification

73

• P(p|n) ∝ P(p) 1≤ k ≤ n P(t k |p) (4) [27], this equation computes the prior and likelihoods for a new test document. Finally, the model will choose which class for this specific document bases on highest probability predicted.

80% 70% 60% 50% 40% 30% 20% 10% 0% Rule-based Multinomial Combining model NB model two models Fig. 3. Accuracy of the models

The classifier chosen chose the Multinomial Naïve Bayes, it works as follows: • When an input question is received from the user, the term frequency vectorizer (TF) is applied on the input question to extract features. • The vectorized input question (features) is given to the Multinomial NB classifier to predict the closest response and its probability, then return them to the user. The answer is returned based on its probability as the maximum probability answer among all the other answers is taken and returned to the user

80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Precision

Recall

F1-score

Fig. 4. Chatbot model accuracy metrics

74

D. S. Elzanfaly et al.

4.3 Discussions In this section we are going to show our model accuracy, in compare with the other models. As shown in Fig. 3, the accuracy when implementing the rule-based model only was 58% at threshold 0.75. Accordingly, data mining techniques like text vectorization methods, classical model classifiers are used to enhance the accuracy of the chatbot as much as possible. So, when testing the accuracy of the classifiers, the Multinomial NB classifier results the highest accuracy which is 64.9%. Therefore, the COVID-19 chatbot accuracy is tested after merging the two models together. And the highest accuracy resulted is 72.4% at rule-based threshold 0.72, and at classical threshold 0.5. Furthermore, precision, recall and f1-score are calculated as shown in Fig. 4 and resulted in 72.4% for all. The percentages for the three-accuracy metrics were all the same. This indicates that the model is stable and operating in its average mode. Table 2. Accuracy of testing classifiers Model

Vectorizer

Testing Accuracy

Multinomial NB

TF

64.9%

Complement NB

TF

62.7%

Bernoulli NB

TF

60.5%

Categorical NB

TF

61.2%

SVM

TF-IDF

58.2%

KNN

TF-IDF

51.5%

Multi-layer perceptron

TF-IDF

61.9%

As the model’s ability to prevent misclassifying negative instances as positive is 72.4% for precision. In addition, 72.4% of the model’s competence is to get all positive examples (recall). The F1-score is the same percentage as it is the average of both precision and recall. Consequently, the precision equation’s false positive (FP) and recall equation’s false negative (FN) are the same. Based on the chatbot testing results, the accuracy of the chatbot before using the classification model is inefficient, and its performance is poor. It is obvious that when the classification model is integrated with the rulebased model, the accuracy increases by 14.4%, reaching 72.4% instead of 58%. In addition, the precision, recall, and f1-score findings are all identical to the accuracy of 72.4%. This signifies that the model is in its average mode and is regarded stable.

5 Conclusion Our work presents the idea of building a chatbot that replies to concerns regarding the COVID-19 epidemic. The chatbot interacts with the user by conversing through a GUI interface and retrieves an answer based on the data saved. The chatbot is built using two

Interactive Chatbot for Improving the Text Classification

75

approaches: a rule-based model and a classical classification model. Depending on the threshold, the bot will answer with one of the two models, as it aids in delivering the most confident answer. In addition to that, the rule-based model is connected to MongoDB NoSQL database, as just a few papers have attempted to link a machine learning model to a NoSQL database. In addition, multiple classifiers are tested to discover which one performs the best. Text vectorization approaches, such as TF and TF-IDF vectorizers, are also employed with classifiers. The chatbot implemented has a good level of accuracy (72.4%) and can be called efficient. It answers most questions about the COVID-19 epidemic and is flexible enough to allow users to interact and communicate with it. Despite this, our work in applied on small database, applying chatbot proposed on large database will allow error possibility. Also, if the entered information is unclear or the keyword comparison seems to be off, the chatbot could respond incorrectly. Because COVID-19 is a new disease with regular news updates, the data may need to be updated later. To overcome this in the future, two approaches can be taken. The first is to expand the dataset, as the model’s performance improves as the dataset grows. Another option is to shift the chatbot from a retrieval based to a generative-based model, in which the chatbot generates responses based on the user’s input.

References 1. Finances online (2020). https://financesonline.com/chatbot-statistics/ 2. Ayanouz, S., Abdelhakim, B.A., Benhmed, M.: A Smart Chatbot Architecture based NLP and Machine learning for health care assistance. In: Proceedings of the 3rd International Conference on Networking, Information Systems Security 24 April 2020 3. Mnasri, M.: Recent advances in conversational NLP : Towards the standardization of Chatbot building (2019) 4. Peng, Z., Ma, X.: A survey on construction and enhancement methods in service chatbots design (2019) 5. Chizhik, A., Zherebtsova, Y.: Challenges of building an intelligent Chatbot. In: International Conference “Internet and Modern Society (IMS-2020) 6. Kozhevnikov, V., Pankratova, E.: Research of the text data vectorization and classification algorithms of machine learning. Int. Sci. J. 85(05) (2020) 7. Kumari, A., Shashi, M.: Vectorization of text documents for identifying unifiable news articles. Int. J. Adv. Comput. Sci. Appl. 10 (2019) 8. Oussous, A., Benjelloun, F.Z., Lahcen, A.A., Belfkih, S.: Comparison and Classification of NoSQL Databases for Big Data. In: International conference on Big Data, Cloud and Applications, Tetuan, Morocco (2015) 9. Moniruzzaman, A.B.M., Hossain, S.A.: NoSQL database: new era of databases for big data analytics - classification, characteristics and comparison. Int. J. Datab. Theory Appl. 6 (2013) 10. Tauro, C.J.M., Patil, B.R., Prashanth, K.R.: A comparative analysis of different nosql databases on data model, query model and replication model. In: International Conference on Emerging Research in Computing, Information, Communication and Applications, Bangalore, India (2013) 11. Krishnan, H., Elayidom, M.S., Santhanakrishnan, T.: MongoDB – a comparison with NoSQL databases. Int. J. Sci. Eng. Res. 7(5), 5 (2016) 12. Zhou, L., Li, D., Gao, J., Shum, H.Y.: The design and implementation of Xiaoice, an empathetic social chatbot. Comput. Linguist. 1 (2019)

76

D. S. Elzanfaly et al.

13. Shum, H.Y., He, X., Li, D.: From Eliza to XiaoIce: challenges and opportunities with social chatbots (2018) 14. Colace, F., Santo, M.D., Lombardi, M., Pascale, F., Pietrosanto, A., Lemma, S.: Chatbot for e-learning: a case of study. Int. J. Mech. Eng. Robot. Res. 7(5), 528–533 (2018) 15. Awangga, R.M., Setyawan, M.Y.H., Efendi, S.R.: Comparison of multinomial naive bayes algorithm and logistic regression for intent classification in chatbot. In: 2018 International Conference on Applied Engineering (ICAE), October 2018 16. Kumar, S.A., Krishna, C.V., Reddy, P.N., Reddy, B.R.K., Jacob, I.J.: Self-diagnosing health care chatbot using machine learning. Int. J. Adv. Sci. Technol. 29(5) (2020) 17. Kalla, D., Samiuddin, V., Malware, E.: Chatbot for medical treatment using NLTK Lib. IOSR J. Comput. Eng. (IOSR-JCE) 22 (2020) 18. Bali, M., Mohanty, S., Chatterjee, S., Sarma, M., Puravanka, R.:“Diabot: a predictive medical chatbot using ensemble learning. Int. J. Recent Technol. Eng. 8(2) (2019) 19. Herriman, M., Meer, E., Rosin, R., Lee, M.V., Washington, M.P.V., Volpp, K.G.: Asked and answered: building a chatbot to address COVID-19-related concerns. NEJM Catalyst (2020) 20. Thukrul, J., Srivastava, A., Thakkar, G.: DoctorBot - an informative and interactive Chatbot for COVID-19. Int. Res. J. Eng. Technol. (IRJET) 07(07) (2020) 21. Guti´errez, B.J., Zeng, J., Zhang, D., Zhang, P., Su, Y.: Document classification for COVID19 literature. In: Findings of the Association for Computational Linguistics: EMNLP 2020 (2020) 22. Wei, J., Huang, C., Vosoughi, S., Wei, J.: What Are People Asking About COVID-19? (2020). https://www.aclweb.org/anthology/2020.nlpCOVID19-acl.8/ 23. How do chatbots work? An architecture of chatbots, 12 December 2019. https://rakebots. com/blog/howdo-chatbots-work-an-architecture-of-chatbots/ 24. Babar, N.: The Levenshtein Distance Algorithm, 2 October 2018. https://dzone.com/articles/ thelevenshtein-algorithm-1 25. Le, J.: 1 May 2018. https://towardsdatascience.com/the-4-recommendation-engines-that-canpredict-yourmovie-tastes-109dc4e10c52 26. Ting, S., Ip, W., Tsang, A.H.: Is Naïve Bayes a good classifier for document classification?. Int. J. Softw. Eng. Appl. 5 (2011) 27. Singh, G., Kumar, B., Gaur, L., Tyagi, A.: Comparison between multinomial and bernoulli Naïve Bayes for text classification. In: International Conference on Automation, Computational and Technology Management (ICACTM) (2019) 28. Oprea, S., Bâra, A.: Setting the time-of-use tariff rates with NoSQL and machine learning to a sustainable environment. IEEE Access 8, 10 (2020) 29. Sanyal, S., Hazra, S., Ghosh, N., Adhikary, S.: Resume parser with natural language processing. Int. J. Eng. Sci. Comput. 7(2) (2017) 30. Barres, I.G., Milla, S., Cebrián, A., Fan, H., Millet, J.: Detecting weak signals of the future: a system implementation based on text mining and natural language processing. Sustainability (2020) 31. Haldar, R., Mukhopadhyay, D.: Levenshtein Distance Technique in Dictionary Lookup Methods: An Improved Approach (2011) 32. https://chatterbot.readthedocs.io/en/stable/faq.html. Accessed July 2021 33. Pupale, R.: Support Vector Machines(SVM) — An Overview, 16 June 2018. https://toward sdatascience.com/https-medium-com-pupalerushikesh-svm-f4b42800e989 34. Subramanian, D.: A Simple Introduction to K-Nearest Neighbors Algorithm, 8 June 2019. https://towardsdatascience.com/a-simple-introduction-to-k-nearest-neighbors-alg orithm-b3519ed98e 35. Nicholson, C.: Multilayer Perceptrons (MLP). https://wiki.pathmind.com/multilayer-percep tron. Accessed July 2021

Interactive Chatbot for Improving the Text Classification

77

36. Complement Naive Bayes (CNB) Algorithm, 5 September 2020. https://www.geeksforgeeks. org/complement-naive-bayes-cnb-algorithm/ 37. Judson, T.J., et al.: Implementation of a digital chatbot to screen health system employees during the covid-19 pandemic. J. Am. Med. Inform. Assoc. 27(9), 1450–1455 (2020)

Cultivating Computational Thinking in Early Years Through Board Games. The Cthink.it Approach Tharrenos Bratitsis1(B) , Maria Tsapara1 , Kiriaki Melliou1 , Leonard Busuttil2 Diane Vassallo2 , James Callus2 , Gonçalo Meireles3 , Iro Koliakou4 , Nabil Tarraf Kojok5 , and Sofia Sousa6

,

1 University of Western Macedonia, Macedonia, Greece

[email protected]

2 University of Malta, Msida, Malta

{Leonard.Busuttil,diane.vassallo,jcall01}@um.edu.mt 3 Advancis - Business Services, Lda, Matosinhos, Portugal [email protected] 4 Anatolia College, Thessaloniki, Greece [email protected] 5 Chiswick House School, San Gwann, ˙ Malta [email protected] 6 Projeto Scholé, Lda, Matosinhos, Portugal

Abstract. Computational Thinking (CT) was brought up originally in the 1950s as the notion of using structured or algorithmic thinking. It refers to a specific set of mental skills used to formulate solutions to problems by converting given inputs to produce appropriate outputs while following detailed algorithms to perform this process. CT is broadly defined as a combination of thinking processes which include abstraction, algorithm design, decomposition, sequencing, pattern recognition and data representation. The prominence of CT in education has been steadily increasing. Lately, the corresponding research is focusing also on early years (starting from the age of 4). Towards this direction and in combination with the blooming of Game-based and gamified learning approaches, this paper discusses the design of a board game addressed to children 4–8 years old, for cultivating CT skills. A detailed roadmap on the design of the game, including a systematic literature review is presented in this paper. Keywords: Computational Thinking · Board games · Early Childhood Education

1 Introduction Computational Thinking (CT) refers to a specific set of mental skills used to formulate solutions to problems by converting given inputs to produce appropriate outputs while following detailed algorithms to perform this process [1]. Essentially, CT is broadly defined as a combination of thinking processes which include abstraction, algorithm design, decomposition, pattern recognition and data representation [2]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 78–89, 2024. https://doi.org/10.1007/978-3-031-56075-0_8

Cultivating Computational Thinking in Early Years

79

CT has emerged in education related discussions as the “new literacy of the 21st century [2], as it is tightly connected to technology, a fundamental element of everyday life [3, 4]. CT proficiency seems to provide a systematic and efficient scaffold when dealing with complex problems [5]. Thus, it has been characterized as a “universally applicable attitude and skill set” [2] which should be conquered en from a young age. Regarding school integration, many scientists agree that early childhood is the ideal initiation point [6]. Early childhood education (ECE) is considered a critical stage for the development of skills pertaining to the cognitive, social and emotional domains. It is a perfect time to begin CT instruction as it is the stage where children transition between concrete and abstract processes, thus, providing active experiences that stimulate their thought processes seems to bear the potential to support this transition. Through CT and programming, children become aware of the automation process and the fact that it is a clearly defined set of instructions that drive a computer. This has the potential to lead children to begin solving problems in real world through systemic steps and ultimately become designers of technology rather than just passive users [7]. This paper is supported by an ERASMUS + funded project called CTHINK.IT, which aims at introducing CT to ECE through a board game. A systematic literature review on the definitions and approaches of CT, but also board game use in ECE was carried out, along with an examination of the state of the art in three EU countries, regarding the different approaches to CT in ECE. The paper is structured as follows: initially a conceptualization of CT, followed by the systematic literature review on CT in early years and the state of the art in three European countries regarding CT in ECE. Then, the idea of the board game which is designed for the needs of the project is presented, in correlation with findings from a literature review in bard-game use in ECE, before the concluding discussion.

2 Computational Thinking Although CT has been gaining growing attention, it still lacks a single, universally accepted definition. The existing definitions emphasise different aspects and components, making the concept of CT elusive and at times difficult to comprehend in its entirety. Haseski et al. [8] found 59 different definitions for CT, that borrow from a range of concepts that include problem-solving, technology, thinking, and individual and social qualities. Others think that CT is founded upon Constructionism which involves learning by constructing something tangible using digital tools [9]. Wing’s [2] definition of CT seems to comply with this idea, proposing CT as a set of processes, namely abstraction, algorithmic thinking and decomposition in problem-solving and decision-making. Resnick [10] emphasises the creative and expressive aspects of using digital tools and technologies in designing and creating processes, establishing his approach on Logo. Barr and Stepenson [11] see CT focusing on problem-solving and decision-making through digital tools (e.g. modelling, simulation, data analysis). In this case, following Papert’s [9] approach in Mindstorms, digital technologies are treated as learning medium allowing experimentation, exploration and discovery of new concepts and ideas. Although programming is considered as pivotal for developing CT [12], Voogt et al. [13] indicated that CT goes beyond programming. Wing [2] first considered CT as a

80

T. Bratitsis et al.

broader notion which correlates with system’s design, problem solving and understanding human behavior via computer science concepts, only to refine the idea later by also considering the thought processes necessary to frame and solve problems in ways understandable by computers [14]. Just like language proficiency n aids communication, Lu and Fletcher [5] define CT proficiency as means to “systematically, correctly and efficiently process information and tasks” when dealing with intricate problems. Over the years, various elements have been proposed as CT constituents, following Wing’s [2] initial proposal. Thus, other constituents have been proposed, such as implementation and evaluation [15], generalization [16] and algorithmic thinking [17]. Based on the different interpretations, a set of the seven “big ideas” of computing have emerged [3] that seek to summarise the characteristics of CT. Concluding, all attempts to define CT in order to integrate it in education, according to Voogt et al. [13] should attempt to adopt a view of considering similarities and relationships surrounding the CT discussion, which will hopefully guide in defining the importance of CT and thus inform integration efforts.

3 Literature Review 3.1 Methodology A systematic review was conducted to identify relevant research articles in ACM digital library, IEEE Xplore, and SCOPUS, in the field of CT. The search string included a combination of keywords related to the study’s research topic, which was focused on the use of unplugged and plugged activities to promote CT skills in young children. The data extraction procedure was designed to extract relevant information from the included studies in a consistent and systematic manner. A standardized data extraction form was developed to collect information on the following variables: Study characteristics, Participants, Intervention, Outcomes, Study quality. The data extraction form was pilot-tested on a sample of included studies to ensure that it captured all relevant information and was easy to use. Two reviewers independently extracted data from each included study, and any discrepancies were resolved through discussion and consensus. Data were entered into a database for analysis. 3.2 Findings on CT Teaching Overall, the findings revealed different approaches to CT in the early years. The first is the utilization of unplugged approaches. These involve the use of tangible or nondigital materials such as games, puzzles, crafts, and manipulatives to teach CT concepts without the use of a computer or digital device. Out of 18 papers this category, only 4 deal exclusively with unplugged implementations of CT. The majority (14 papers) deal with a combination of robotics, coding and unplugged activities. Examining the papers, a connection between CT and STEM is revealed. As programming was considered for many years as a core skill for mathematicians, scientists and engineers, Computer Science teaching borrowed from methodologies already in use in STEM disciplines [18], possibly explaining the link between STEM and CT.

Cultivating Computational Thinking in Early Years

81

Storjak et al. [19] incorporated unplugged programming and hands-on robot play into informal STEM workshops for young children. They relied on pen and paper to introduce a Pacman unplugged CT activity for introducing core programming concepts to students. Then robots were utilized to test the acquired knowledge. Although STEM focused, these activities have been shown to promote CT skills, among others. Sung et al. [20] investigated the successful effects of different levels of embodied instruction on a wide range of soft-skills that include problem-solving, debugging, programming proficiency, as well as skills such as self-determination and self-efficacy. They first used LEGO WeDo and unplugged embodied activities in the planning stages, where the students were required to perform whole body movements to mimic their robots to physically experience what the robot was doing. They then moved on to using robots and coding on Scratch Jr. to apply these thinking and problem-solving skills to test and debug their designs. Lee et al. [21] utilized tabletop board games (including the computer science board game On the Brink) in the first 2 weeks of their 8-week unit, before moving on to the Scratch programming environment. Metin [22] used a coding board within the context of a story which was told to the students through pictures, shapes and other signs. Coding activities with the Cubetto robot followed, revealing a significant change in children’s practices. In this case the story contextualization provided a direct connection with the children’s interests and provided concrete, meaningful experiences. Gartig-Daugs et al. [23] support this claim through their findings when they combined ideas from Froebel and with computer science to explain technical and theoretical computational concepts to children. Other papers make use of CT to achieve outcomes beyond CT and programming, following a STEM framework. Ioannou and Bratitsis [24] tackled the conceptual understanding of the notion of speed in Kindergarten using unplugged race-based physical activities and robot programming. Canbeldek and Isikoglu [25], showed how their Coding and Robotics Education Program positively affected the cognitive development skills, language development and the creativity of children. Barragán-Sanchez et al., [26] used CT interventions to modify problematic behaviours and found that combining unplugged and robot-based CT activities contributed to the collaborative problem resolution, by stimulating students’ participation and creativity. Luo et al. [27] combined mathematics and CT integrated activities, whereas Perez-Matin et al. [28] utilized programming for improving sequencing skills and plotting a route. Furthermore, Somuncu and Aslan [29] relied on programming to improve mathematical reasoning of children. Considering the target group, creativity, play and playful practice were common themes in the examined papers. Noma et al. [30] utilized a model railway toy and Meyer-Baron et al. [31] were inspired by escape rooms in their study. Also, Critten et al. [32] designed guided play activities to enhance problem solving and logical thinking in a classroom, before proceeding to incorporating robots. In all cases, as aforementioned, a story running through the unplugged activities was a fundamental element. Following this path, Yang et al. [33] studied the effects of digital storytelling to bridge the gap between programming and CT, concluding that storytelling activities were more effective than unplugged approaches in providing inclusive and sustainable CT learning experiences. Also, Benetti and Mazzini [34] and Perez-Marin et al. [28], also used stories to provide a context that served as a motivational drive for coding experiences.

82

T. Bratitsis et al.

All the considered studies suggest that cultivating CT in ECE is a feasible and worthwhile initiative. Many of the studies treated elements of CT as described previously, such as pattern recognition, sequencing, algorithmic design and decomposition [26, 34]. Other papers utilized coding in ECE. Some involved programming tangible objects. This review identified 12 papers on the use of coding in ECE. Of these papers, 6 dealt with open-ended coding and programming environments (e.g. ScratchJr or Viscuit), 4 dealt with programming with tangible objects, and 2 dealt with coding games. Flannery et al. [35] utilized ScratchJr in K-2 grades, whereas Portelance and Bers [36] incorporated a modified version of the ScratchJr “animated genres” curriculum in Grade 2. The same tool was utilized by Kyza et al. [37] for a six-hour intervention as part of a four-day summer club to foster a disciplinary perspective on CT. These studies too relied on storytelling to contextualize their approach. Viscuit’s [38] distinguishing feature is that it uses only images and layouts to express programming without any letters or text. Also, tangible programming toys have been utilized for CT teaching, such as Osmo Coding Awbie [39], but also coding games, like Kobable and Daisy [40] or game-like activities from the Code.org platform [41]. The creative problem-solving approach afforded by games is also taken up by Kim [42], who explores how games facilitate creative problem-solving in young children, through hands-on robot activities and educational programming language (EPL) activities. Another approach for incorporating CT competencies into ECE is using robotics. Including some studies already mentioned in this section, a total of 51 such papers were identified. Of them, 35 exclusively focus on robotics implementations of CT, 11 combine robotics with unplugged activities, 3 combine robotics with programming, and only 2 explore the combination of all three methods. Examining the 11 papers that combine robotics with unplugged activities revealed that the latter focus on nurturing CT skills, such as algorithmic thinking, abstraction, decomposition, pattern recognition, and logical reasoning. On the other hand, robotics activities aim to provide hands-on experiences with coding and robotics that support the development of CT skills such as sequencing, debugging, conditional logic, and iteration. This is demonstrated by several papers, including [21, 23, 43]. In summary, several studies investigate the effectiveness of different educational interventions involving programming and robotics, such as hands-on robot activities, social robot toolkits, screen-based programming environments, and tangible robotic kits. Other studies focus on the design and development of educational robotics kits and devices suitable for young children, as well as on the evaluation of their interaction with these technologies. Some studies also explore how teachers can support and promote young children’s programming and CT skills. Finally, some studies also highlight the importance of cultural responsiveness and environmental awareness in ECE. Other studies initiate with unplugged activities to teach basic programming, before shifting to robot manipulation appropriate for ECE, such as Beebot [30], Codey Rocky and Thymio [20] or the Code & Go Robot Mouse [27]. The 35 papers that exclusively focus on robotics identify several themes, including curricula and educational activities design, aimed at enhancing learning outcomes. For instance, Bers et al. [19] developed a curriculum with KIBO robots, Coding as Another

Cultivating Computational Thinking in Early Years

83

Language (CAL). Pinto et al. [44] replicated the KIBO robot curriculum previously developed in an academic context, using traditional stories for children as a theme. González & Muñoz-Repiso [45] developed a framework for designing and integrating robotics learning activities to foster programming skills and CT in ECE with BeeBot based activities. Silvis et al. [46] investigate how children develop technical knowledge in tangible programming environments with robots like Botley and Cubetto. Flood et al. [47] focus on tangible programming toys that support body syntonicity, where children use knowledge of their own bodies to solve problems computationally, and how teachers respond to preschoolers’ body syntonic problem solving. Samuelsson [48] introduces programming for children by using storytelling and programmable robots. Nacher et al. [49] evaluated the usability of a tangible-mediated robot in collaborative kindergarten instruction using tangible sticks to represent the robot’s actions (move forward, stop, turn left, and turn right). This study applied the CT principles of decomposition, abstraction, algorithm design, and debugging. Other used robotic kits were Matalab, RoboTito, SoRo, Cubelets, iRobiQ, TACTOPI and others. Some incorporate the design of robots and processes for promoting CT. For example, KIBO utilizes tangible programming blocks, enabling children to engage in foundational programming skills without the need for a computer or screen [50]. Ultimately, unplugged approaches are seen to provide some sort of embodied cognition that allows the physical enactment of the CT concepts, which seems to be very effective for young children in understanding these abstract concepts. Overall, the unplugged approach is portrayed as a good strategy to start with when introducing CT implementations for very young students. Unplugged activities are seen to have the potential to establish a foundation for educational methods that are based on problem-solving and constructive learning, which involves the learner’s active engagement with the environment on both a physical as well as a mental level in order to construct and scaffold their understanding. Advancing, robotic devices and/or programming tools and games are widely utilized for applying and further developing CT attributes. 3.3 Examining Three European Countries All three countries participating in the CTHINK.IT project adopt a different approach as to how CT is currently being treated. In Malta, CT shares a common terminology with Digital Literacy and this is further elaborated in the Learning Outcomes Framework where the CT learning outcomes are clearly defined. In Portugal, CT is not mentioned in The Curriculum Guidelines for Pre-School Education with the exception of concepts related to pattern recognition which are implemented in songs, sounds, shapes and other activities. The scenario in Greece is completely different since in 2020, the Greek Ministry of Education, Research and Religious Affairs implemented a new curriculum for primary education that includes courses on coding, CT, and robotics. It was clear through a detailed desk research and a curriculum examination that CT is most prominent in Greece, where recent curricular updates have embraced CT and thus put forward the instantiation of the integration process of CT into classroom practice. In Malta, things are well underway, and a good number of initiatives promote the integration of CT in education, despite the National Curriculum Framework not directly targeting CT as part of its vision. In Portugal, the situation with the integration of CT seems to take

84

T. Bratitsis et al.

on a more decentralised diffusion, given that the National Curriculum Framework also does not include CT as part of its general outlook for education. However, despite the different progress rates in these countries, it was very clear that CT has been garnering unwavering support and its importance recognised throughout. 3.4 Board Games in ECE Additionally, the design and use of board games in ECE was examined. Although not in the scope of this paper to present it in detail, an additional literature review was conducted. Searches were conducted through Scopus, Eric, Google Scholar, using various combinations of keywords related to the study’s research topic. The eligibility criteria in this case also, included: Study design, Interventions, Age of the participants, Outcomes, Publication date and Language. Overall, 13 studies satisfied the criteria. According to the findings, board games are used as an educational tool in ECE in order to cultivate 21st century skills (collaboration, communication, creativity, critical thinking), social skills (sharing, following directions, be patient, respect others, listening to others, wait for their turn), cognitive skills (including skills such as pre-reading, language, vocabulary, and numeracy) and educational knowledge. According to the researchers, board games increase students’ motivation, while their learning through the game process was more entertaining and pleasant [51]. They help to achieve the objectives of curriculum frameworks, by fostering critical thinking and problem solving. The possibilities for students to make use of lessons they have learnt can be offered by playing board games. Some of the studied games were utilized for cultivating mathematical and, at extension, CT skills [52, 53]. A game that provides complete information encourages open-ended, strategic, predictive, abstract, and creative thinking [54]. Based on the literature review there are studies focused on teaching CT through board games. These games aim to introduce students to concepts like abstraction, algorithms, decomposition, evaluation and patterns. Other board games focus on language [55, 56], behavior [57] or environmental and sustainability issues [58], incorporating realistic problems to be dealt with. Specifically focusing on CT and ICT, Bratitsis et al. [59], designed a board game to teach internet related issues to Kindergarteners. Machuqueiro & Piedade [60] analyzed 11 studies between 2011 and 2021 and stressed that board games can be used as unplugged activities, promoting the development of CT and suggest that game mechanics, typical of Eurogames (also known as Modern games), can also reveal the potential to promote CT. Ching et al. [61], argue that “… these board games do not include any electronic physical agents to enact the codes or commands. Programming manipulatives are available to express codes, usually in the format of cards”, referring to CT focused board games. Tsarava et al. [62] developed three board games – Crabs & Turtles: A Series of Computational Adventures. Apostolellis et al. [63] designed the RabBit EscApe board game to investigate its effectiveness in enhancing children’s problemsolving skills while playing with peers. Scirea and Valente [64], reviewed eight board games and discussed “which CT concepts they integrate and how they are expressed through the game-mechanics”, stating that the mechanics that seemed to foster CT concepts were “action queues, simultaneous actions, board modularity, cooperation, and

Cultivating Computational Thinking in Early Years

85

resource management”. Gresse von Wangenheim, et al. [65] created “SplashCode” to teach basic concepts of computation about algorithms and programming. Overall, this literature review enlightened the consortium’s perspective on board game design, including game mechanics, elements, formats and components.

4 Discussion The 4-step process of the previous section allowed to fully grasp current trends in both CT approaches implemented in ECE on an international level, and how board games are utilized in educational settings. The main outcome is a mapping of the trends in CT research with learning outcomes which are inline with the current curricula and the children’s abilities. A main conclusion is that despite the rich and diverse state of the art of CT, it is still in a very early stage of development and further research is needed to better understand a number of questions that still remain. These include gaps in the literature about an agreed way forward about how to best integrate CT, what learning outcomes a curriculum should focus on and which ways are most effective in presenting CT activities to younger children, amongst others. Likewise, the literature review on board game utilization in education provided useful information on the types of activities incorporated in such games, especially for young ages. As the intention was to design a game which can be played in common classrooms, special attention was paid to unplugged activities which can be connected to game mechanics. Up to this point, the game concept, the core storyline and the fundamental mechanics have been designed. Activities are being designed and the first set will be completed by September 2023. The game treats six attributes of CT in 4 levels: abstraction, algorithm design, decomposition, pattern recognition, sequencing and data representation. The storyline of the game comprises four levels. The concept is about a lost toy during a museum visit. The goal is to identify the toy (Level 1) and the owner (Level 2). In Level 1 mainly pattern recognition, abstraction and data representation are treated. The main idea to be tested is that the classroom will be divided into groups who will be assigned a part of the toy. They will randomly choose a card with two images to compare and identify one or more differences (based on the age level). This will distinguish the designated part of the toy and by putting all the cards in a specific order, the toy will be revealed. This also highlights the fact that different actions lead to different outcomes. In Level 2, the toy owner will be revealed via a matching game similar to the famous “Guess Who” game. In this case, also pattern recognition and abstraction are treated, along with data representation. Preliminary tests indicated that children like this approach as it allows them to become attached to the character and engaged in the storyline, increasing the motivation to move forward to the next levels. Then the player tries to track the house of the toy owner in the town’s map (Level 3) in order to return the toy. For this reason, the players need to build algorithms with a predefined number of moving cards. Their goal is to reach squares marked with a question mark in order to get hints about the house (e.g. color of the front door, the roof, etc.). The initial approach is that a moving grid will allow a specific number of cards to be used for moving around the city grid. This way, algoritminc thinking will be treated as there are various ways to move around and the optimal algorithm always depends on the set criteria (e.g. shortest or fasted path, with a set number of turns, etc.).

86

T. Bratitsis et al.

In the house, the player may strive through the 6 available rooms (Level 4) to meet the owner while exercising CT attributes through activities designed explicitly for this purpose. The latter are still being designed at the moment this paper is written, but they will be founded on the existing literature. For example, significant work exists through the international Bebras competition. While in the rooms, based on the activities, rewards will be given in order to complete a certification of CT by collecting a minimum amount of credits/rewards (stars, tokens or something similar will be chosen) per room (and thus per CT attribute). This approach will allow the design of different activity sets for different age levels, also addressing the appropriateness of difficulty level. Augmentation will be incorporated in some of the activities who can also be implemented in an experiential manner in the classroom. The next steps include training workshops with teachers who will pilot study the game in their classrooms. The feedback will allow the refinement of the game which will be freely released in 4 languages (English, Greek, Maltese and Portuguese). Acknowledgements. This work is funded by the Eramus+ programme, key action 2, Partnerships for cooperation (Proj No: 2022–1-MT01-KA220-SCH-000086903).

References 1. Denning, P.J.: The profession of IT beyond computational thinking. Commun. ACM 52(6), 28–30 (2009) 2. Wing, J.M.: Computational thinking. Commun. ACM 49(3), 33–35 (2006) 3. Grover, S., Pea, R.: Computational thinking in K-12: a review of the state of the field. Educ. Res. 42(1) (2013) 4. Mishra, P., et al.: Rethinking Technology & Creativity in the 21st Century. TechTrends 57(3) (2013) 5. Lu, J.J., Fletcher, G.H.L.: Thinking about computational thinking. SIGCSE Bull. Inroads, 41(1) (2009) 6. Manches, A., Plowman, L.: Computing education in children’s early years: a call for debate. Br. J. Edu. Technol. 48(1), 191–201 (2017) 7. Wyeth, P., Wyeth, G.: Robot building for preschoolers. RoboCup 2007: Robot Soccer World Cup XI 11, 124–135 (2008) 8. Haseski, H.I., Ilic, U., Tugtekin, U.: Defining a new 21st century skill-computational thinking: concepts and trends. Int. Educ. Stud. 11(4), 29–42 (2018) 9. Papert, S.: Mindstorms children, computers, and powerful ideas (second edition). In BasicBooks, vol. 1 (1980) 10. Resnick, M.: Lifelong Kindergarten. The MIT Press (2017) 11. Barr, V., Stephenson, C.: Bringing computational thinking to K-12: what is involved and what is the role of the computer science education community? ACM Inroads 2(1), 48–54 (2011) 12. Council, N.R.: Committee for the Workshops on Computational Thinking: Report of a Workshop on the Scope and Nature of Computational Thinking. National Academy, Washington, DC (2010) 13. Voogt, J., Fisser, P., Good, J., Mishra, P., Yadav, A.: Computational thinking in compulsory education: towards an agenda for research and practice. Educ. Inf. Technol. 20(4) (2015) 14. Wing, J.M.: Computational Thinking: What and Why? The link - The Magaizne of the Varnegie Mellon University School of Computer Science (2010)

Cultivating Computational Thinking in Early Years

87

15. Kalelioglu, F., Gulbahar, Y., Kukul, V.: A framework for computational thinking based on a systematic research review. Baltic J. Mod. Comput. 4(3) (2016) 16. Angeli, C., et al.: A K-6 computational thinking curriculum framework: implications for teacher knowledge. Educ. Technol. Soc. 19(3) (2016) 17. Van Borkulo, S., Chytas, C., Drijvers, P., Barendsen, E., Tolboom, J.: Computational thinking in the mathematics classroom: fostering algorithmic thinking and generalization skills using dynamic mathematics software. In: The 16th Workshop in Primary and Secondary Computing Education, pp. 1−9 (2021) 18. Bers, M.U., Govind, M., Relkin, E.: Coding as another language: computational thinking, robotics and literacy in first and second grade. In: Computational Thinking in PreK-5: Empirical Evidence for Integration and Future Directions, pp. 30–38. Association for Computing Machinery (2022) 19. Storjak, I., Pushkar, L., Jagust, T., Krzic, A.S.: First steps into STEM for young pupils through informal workshops. In: 2020 IEEE Frontiers in Education Conference, pp. 1–5 (2020) 20. Sung, W., Ahn, J., Black, J.B.: Elementary students’ performance and perceptions of robot coding and debugging: embodied approach in practice. J. Res. Child. Educ. 36(4), 681–696 (2022) 21. Lee, V.R., Poole, F., Clarke-Midura, J., Recker, M., Rasmussen, M.: Introducing coding through tabletop board games and their digital instantiations across elementary classrooms and school libraries. In: Proceedings of the 51st ACM Technical Symposium on Computer Science Education, pp. 787–793 (2020) 22. Metin, S.: Activity-based unplugged coding during the preschool period. Int. J. Technol. Des. Educ. 32(1), 149–165 (2020) 23. Gartig-Daugs, A., Weitz, K., Wolking, M., Schmid, U.: Computer science experimenters kit for use in preschool and primary school. In: Proceedings of the 11th Workshop in Primary and Secondary Computing Education (2016) 24. Ioannou, M., Bratitsis, T.: Teaching the notion of speed in kindergarten using the sphero SPRK robot. In: 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp. 311–312 (2017) 25. Canbeldek, M., Isikoglu, N.: Exploring the effects of productive children: coding and robotics education program in early childhood education. Educ. Inf. Technol. 28(3), 3359–3379 (2023) 26. Barragán-Sánchez, R., Romero-Tena, R., García-López, M.: Educational robotics to address behavioral problems in early childhood. Educ. Sci. 13(1) (2023) 27. Luo, F., Israel, M., Gane, B.: Elementary computational thinking instruction and assessment: a learning trajectory perspective. ACM Trans. Comput. Educ. (TOCE) 22(2), 1–26 (2022) 28. Pérez-Marin, D., Hijón-Neira, R., Pizarro, C.: Coding in early years education: which factors influence the skills of sequencing and plotting a route, and to what extent? Int. J. Early Years Educ. 30(4), 969–985 (2022) 29. Somuncu, B., Aslan, D.: Effect of coding activities on preschool children’s mathematical reasoning skills. Educ. Inf. Technol. 27(1), 877–890 (2021) 30. Noma, H., Sasamoto, H., Itoh, Y., Kitamura, Y., Kishino, F., Tetsutani, N.: Computer learning system for pre-school-age children based on a haptized model railway. In: 1st Conference on Creating, Connecting and Collaborating Through Computing, pp. 118–119 (2003) 31. Meyer-Baron, L., Stieger, R., Waldvogel, B., Schumann, N., Ruedisueli, C., Mayr, F.: Observing computational thinking skills of kindergarten children. In: Proceedings of the 17th Workshop in Primary and Secondary Computing Education, pp. 1–4 (2022) 32. Critten, V., Hagon, H., Messer, D.: Can pre-school children learn programming and coding through guided play activities? a case study in computational thinking. Early Childhood Educ. J. 50(6), 969–981 (2022)

88

T. Bratitsis et al.

33. Yang, W., Ng, D.T.K., Su, J.: The impact of story-inspired programming on preschool children’s computational thinking: a multi-group experiment. Thinking Skills Creativity, 47 (2023) 34. Benetti, E., Mazzini, G.: Coding training proposal for kindergarten. In: 2020 International Conference on Software, Telecommunications and Computer Networks, pp. 1–5 (2020) 35. Flannery, L.P., Silverman, B., Kazakoff, E.R., Bers, M.U., Bontá, P., Resnick, M.: Designing ScratchJr: support for early childhood learning through computer programming. In: 12th International Conference on Interaction Design and Children, pp. 1–10 (2013) 36. Portelance, D.J., Bers, M.U.: Code and tell: Assessing young children’s learning of computational thinking using peer video interviews with ScratchJr. In: Proceedings of IDC 2015: The 14th International Conference on Interaction Design and Children, pp. 271–274 (2015) 37. Kyza, E.A., Georgiou, Y., Agesilaou, A., Souropetsis, M.: A cross-sectional study investigating primary school children’s coding practices and computational thinking using scratchJr. J. Educ. Comput. Res. 60(1), 220–257 (2022) 38. Watanabe, T., et al.: How Can Children Express Their Intentions Through Coding?-Analysis of Viscuit Programs in Kindergarten (2019). https://doi.org/10.1145/3304221 39. Liu, Y.Y., Iversen, O.S.: Computational thinking through tangible play: understanding social dialogues in children’s learning. In: Proceedings of Interaction Design and Children, IDC 2022, pp. 596–603 (2022) 40. Lennon, M., Pila, S., Flynn, R., Wartella, E.A.: Young children’s social and independent behavior during play with a coding app: digital game features matter in a 1:1 child to tablet setting. Comput. Educ. 190 (2022) 41. Montuori, C., Ronconi, L., Vardanega, T., Arfé, B.: Exploring gender differences in coding at the beginning of primary school. Front. Psychol. 13 (2022) 42. Kim, S. U.: A Comparative study on the effects of hands-on robot and EPL programming activities on creative problem-solving ability in children. In: ACM International Conference Proceeding Series, pp. 49–53 (2020) 43. Sung, J.: Assessing young Korean children’s computational thinking: a validation study of two measurements. Educ. Inf. Technol. 1–29 (2022) 44. Pinto, M., Santos, M., Fernandes, M.: (Traditional stories and the integration of programming and robotics with the kibo robot. Rev. Prisma Soc. 38 (2022) 45. González, Y.A.C., Muñoz-Repiso, A.G.-V.: A robotics-based approach to foster programming skills and computational thinking. In: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality (2018) 46. Silvis, D., Clarke-Midura, J., Shumway, J.F., Lee, V.R., Mullen, S.: Children caring for robots: expanding computational thinking frameworks to include a technological ethic of care. Int. J. Child-Comput. Interact. 33, 100491 (2022) 47. Flood, V.J., Wang, X.C., Sheridan, M.: Embodied responsive teaching for supporting computational thinking in early childhood. In: Proceedings of International Conference of the Learning Sciences, ICLS, pp. 855–862 (2022) 48. Samuelsson, R.: Cultural heritage through educational robots: using a Ukrainian folk tale with a programmable robot in early childhood education. In: 24th International Conference on Human-Computer Interaction, HCII 2022, Part I, pp. 336–343 (2022) 49. Nacher, V., Garcia-Sanjuan, F., Jaen, J.: Evaluating the usability of a tangible-mediated robot for kindergarten children instruction. In: 16th IEEE International Conference on Advanced Learning Technologies (ICALT), pp. 130–132 (2016) 50. Sullivan, A., Elkin, M., Bers, M. U.: KIBO robot demo. In: Proceedings of the 14th International Conference on Interaction Design and Children (2015) 51. Selvi, M., Ço¸san, A.: The effect of using educational games in teaching kingdoms of living things. Univ. J. Educ. Res. 6, 2019–2028 (2018)

Cultivating Computational Thinking in Early Years

89

52. Stebler, R., Vogt, F., Wolf, I., Hauser, B., Rechsteiner, K.: Play-based mathematics in kindergarten. a video analysis of children’s mathematical behaviour while playing a board game in small groups. J. Math Didakt 34, 149–175 (2013) 53. Vlassis, J., et al.: Developing arithmetic skills in kindergarten through a game-based approach: a major issue for learners and a challenge for teachers. Int. J. Early Years Educ. 31, 1–16 (2022) 54. Chen, K., Chi, H.H.: Novice young board-game players’ experience about computational thinking. Interact. Learn. Environ. 30, 1–13 (2020) 55. Hassinger-Das, B., Ridge, K., Parker, A., Michnick, Golinkoff, R., Hirsh-Pasek, K., Dickinson, K.: Building vocabulary knowledge in preschoolers through shared book reading and gameplay. Mind Brain Education 10, 71–80 (2016) 56. Sorsana, C., Guizard, N., Trognon, A.: Preschool children’s conversational skills for explaining game rules: communicative guidance strategies the power of board games for multidomain learning 89 as a function of type of relationship and gender. Eur. J. Psychol. Educ. 28, 1453–1475 (2013) 57. Mattsson, P., Laike, T.: Young children’s learning about lighting and turn-off behavior in preschool environments. Energy Build. 268, 112193 (2022) 58. Tsapara, M., Bratitsis, T.: A board game for sustainable development education: kindergarten students as game designers. In: Auer, M.E., Tsiatsos, T. (eds.) IMCL 2021. LNNS, vol. 411, pp. 1072–1084. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96296-8_98 59. Bratitsis, T., Kontovounisiou, A., Kiriazoglou, M.: A board game proposal for teaching informatics related topics in early childhood education. In: Conference 2021, SHSHS Web of Conferences, vol. 102, p. 01001 (2021) 60. Machuqueiro, F., Piedade, J.: Development of computational thinking using board games: a systematic literature review based on empirical studies. Prisma Soc. 5–36 (2022) 61. Ching, Y.H., Hsu, Y.C., Baldwin, S.: Developing computational thinking with educational technologies for young learners. TechTrends 62(6), 563–573 (2018) 62. Tsarava, K., Moeller, K., Ninaus, M.: Board games for training computational thinking. In: Gentile, M., Allegra, M., Söbke, H. (eds.) GALA 2018. LNCS, vol. 11385, pp. 90–100. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11548-7_9 63. Apostolellis, P., Stewart, M., Frisina, C., Kafura, D.: RaBit EscApe: a board game for computational thinking. In: ACM International Conference Proceeding Series, pp. 349–352 (2014) 64. Scirea, M., Valente, A.: Board games and computational thinking: how to identify games with potential to support CT in the classroom. In: Proceedings of the 15th International Conference on the Foundations of Digital Games. Association for Computing Machinery (2020) 65. Gresse von Wangenheim, C., et al.: SplashCode -A Board Game for Learning an Understanding of Algorithms in Middle School (2019). https://doi.org/10.31235/osf.io/2qbnp.

Board Game Design by Children as an Assessment Mechanism in Kindergarten. A Case Study About Disability and Vulnerability Maria Tsapara and Tharrenos Bratitsis(B) University of Western Macedonia, Kozani, Greece [email protected]

Abstract. This paper presents the idea of a technology-enhanced board game, designed by students of a Greek kindergarten. The board game implementation emerged as an idea by the students, after their participation in a Skills Lab Project related to awareness about disability and vulnerability. Through this project students were able to understand that each of them brings unique experiences and ideas in the classroom and cultivate their empathy towards individuals with disabilities. The students had prior knowledge on board game design. In the classroom, board games are prepared as a part of a project, in which the children participate as game designers on their own initiative, depending on their interests. The goal of the study was to research whether the process of designing a board game by kindergarten students, based on the engineering design process, could also function as a tool for assessing and valuing the knowledge acquired by them during their participation in the Skills Lab project. Keywords: board game · kindergarten · engineering design process · inclusive education

1 Introduction Board games can be used in the educational process as a learning tool [1–3] while creating an attractive and authentic learning environment [4], providing an educational approach which is transdisciplinary, allowing students to work on various disciplines, developing and utilizing multiple skills [5–7]. Using board games in Early Childhood Education (ECE) cultivates students’ social skills [8–10], 21st century skills and cognitive skills [11–13], while helping them to increase their motivation and achieve the objectives of the curriculum framework. According to Wendell [14], children’s nature is related to the concepts of design and construction as a result of their curiosity, while playing and designing board games based on their interests and learning contents of the curriculum [15], helps them improve their learning experience. Additionally, other skills are cultivated such as following rules, decision making, problem solving and collaboration [16]. Educational games and board games as well, can also be used as inclusive educational tools, by providing equal learning © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 90–105, 2024. https://doi.org/10.1007/978-3-031-56075-0_9

Board Game Design by Children as an Assessment Mechanism

91

opportunities [17] and ensuring the participation of all students, removing any barriers to learning [18, 19]. For producing a Board Game based on previous knowledge related to their awareness on disability and vulnerability and their empathy towards individuals with disabilities the framework of the engineering design process was followed. The board game designing was grounded on the 5-step model of the Engineering Design Process (EDP) proposed by the EIE project [20] of the Museum of Science of Boston. Borrowing this methodology, 21 students of a mixed public kindergarten class in Greece, designed a board game following the five main steps that included tasks to be completed in order to create the structure of the game and the game prototype. The five steps are: 1) Ask: involves prior knowledge of the students in game designing, choosing the audience that the game was addressed to, define the goals and game rules, select the theme, consider why it is important to design a board game for this theme and try to build a game around it. 2) Imagine: Despite the fact that the name of this step is also imported from EDP, it does not involve brainstorming and coming up with ideas for solving a problem but is focused on board game ideas. 3) Plan: Students gather information related to game elements, select game mechanics and components. They also set rules and identify required materials. 4) Create: it involves the development of a game prototype, testing the game mechanics. 5) Improve: it involves playtesting the game, reflection, receiving feedback from players, and redesigning parts of the game (if needed). The game idea and design, including the board, the game mechanics and resources are described hereinafter. The paper is structured as follows: A brief theoretical background supporting board games as an educational tool in kindergarten, inclusive education, inclusive game design and enhanced board games through QR Technologies are highlighted. Then, the game design framework is described in detail, as explained in the previous paragraph. Since the game has been tested only from the creators (kindergarten students), the paper concludes with an evaluation of the game by the designers (the kindergarten students) and a discussion which also focuses on future research, utilizing this board game.

2 Theoretical Background 2.1 Board Games as a Tool for Education in Kindergarten As in board games complex knowledge can be combined with hands-on practice, they are considered as an ideal teaching tool. Furthermore, students’ creativity, strategy skills, imagination, but also understanding of a topic can be cultivated [3, 21]. Tsai et al. [22] state that board games encourage student-centered learning by allowing students’ initiative to discover and share knowledge with peers, highlighting interactivity as one of their attributes. The teacher is required to clarify concepts, lead discussions and provide necessary information prior to, during and after playing the game. Chien-Sing & Kian-Wei [23] argue that board game playing facilitates children’s understanding of their learning context. In all board game players are encouraged to: define goals, apply patience, consider alternative moves, and anticipate their outcomes, enhance relationships, and learn based on their experience [16].

92

M. Tsapara and T. Bratitsis

Additionally, students’ earning experience may be improved through skill development, such as rule following, problem solving, critical thinking, creating and groupwork. Board games have specific rules are played on a board by placing several tangible items (e.g. pawns or cards) on it. Players act in turns and apply some predefined mechanism for moving around the board, such as die rolling or by utilizing a spinner. At the same time, they interact with one another, partnering up or playing as opponents. Board games utilization within education can facilitate the cultivation of students’ socio-cognitive skills, enhance their motivation for learning, improve skills of collaboration, abridge competitiveness, allow knowledge acquisition through memory enhancement, while being an enjoyable activity during which they spend time as a group and build up communication, develop problem solving and decision making skills [13, 24–27]. 2.2 Inclusive Education and Inclusive Game Design (IGD) Inclusive education primarily aims at changing school structures for ensuring that equal educational opportunities are provided, while creating an environment in which every child is supported and given individual attention, enhancing their self-esteem, focusing on every students’ equal and quality participation in the intra-school processes, by addressing exclusion conditions [28, 29]. Furthermore, when inclusive activities are implemented in ECE team-building is enhanced as children with or without disabilities are offered a sense belonging and meaning-making, by improving the relationships among them, aiming at reshaping school culture to achieve sustainable change which is fundamental for social equity and critical research [30]. Utilizing game design as a teaching tool for the inclusive design concepts can be more efficient, compared to traditional methods [31]. Board games can act as means for overcoming barriers which players may face, ensuring game accessibility [32]. Westin, Brusk and Engström [33] studied inclusive game design which allowing designers to create games that are accessible to everyone. Their study aimed at “identifying activities that constitute the biggest obstacles to realizing sustainable design processes for inclusive game design, from the perspectives of the game industry as well as from disabled people”. According to their findings it is crucial to diversify game content during the design process, raise awareness regarding IGD and apply a participatory or co-design approach in which disabled people may be involved the earliest possible in the process, while technology may facilitate this direction. 2.3 Enhanced Board Games Through QR Technologies in Education Digital tools are integrated into the educational process for improving teaching. Mehendale et al. [34] suggested that QR Code use in a classroom may cultivate motivation, communication, creativity, critical thinking and collaboration (4 C’s). According to Trentin & Repetto [35] it is a very important to create learning material and achieve educational objectives as part of the learning process. Law and So [36] showed that QR codes may be utilized within mobile learning approaches for supporting various teaching environments and cognitive objects [37]. Collaborative learning can be enhanced by using QR codes, by also creating and authentic, effective and motivating environment for leaning, while students are mobilized to be engaged in learning pathways [38]. Integrating QR

Board Game Design by Children as an Assessment Mechanism

93

codes into the board games further facilitates mobile learning, as “learning can take place anywhere and anytime”, expanding the number and availability of both resources and materials to students. QR code integration in board games makes interaction tangible and enhances engagement, which at extend makes learning more effective and efficient [37, 39].

3 Game Design Framework A total of 21 students of a Greek public kindergarten participated in the study. Their age was between 4 and 6 years and they lived in an urban district. They had prior knowledge on the process of board games design. In the classroom, board games are prepared as a part of a project, in which the children participate as game designers on their own initiative, depending on their interests, while the topics emerge or are chosen by them [40]. The students had an idea to create a board game inspired by a Skills Lab Project that was implemented in the classroom and the story of the “Yellow Bird” written by Olga De Dios. Children worked together as a team, interacted with each other but also collaborated within the working groups in order to achieve a common goal, that of inclusive education. During the board game design process they tried to think out of the box, develop a good set of rules, while being creative they give it a professional look [8]. The data collection was based on the method of direct observation and a questionnaire, while the design of the action was based on the approach of the engineering design process. Children’s reactions, idea sharing, negotiation and overall engagement (including motivation and pleasure reception) were qualitative factors of evaluation. The teacher, one of the authors of this paper, took notes in a researcher’s journal during and after the intervention. 3.1 Design a Board Game Based on Engineering Design Process The Kindergarten students’ game design was grounded on the 5 step model of the Engineering Design Process (EDP) proposed by the EIE project [20] of the Museum of Science of Boston in 2016. Borrowing this methodology, designed a board game following the five main steps that included tasks to be completed in order to create the structure of the game and the game prototype. The five steps are the following: 1) Ask, 2) Imagine, 3) Plan, 4) Create and 5) Improve. Step 1: - Ask. It all started with a question from the students “Can we design a board game?” Students had prior knowledge in game design as in the previous school year they had become familiar with the process of designing board games. They chose the audience that the game was addressed to. “Identifying the game’s audience is an important step to ensure that game complexity is appropriate to maintain students’ involvement with the game and the context embedded” [41]. During a brainstorming in the plenary they express their ideas, discuss and come to a decision. Their intention was to create a board game for preschool children, but at the same time they also expressed the wish that all children could play this game (referring to hildren with vision impairment/blindness or hearing impairment, etc.).

94

M. Tsapara and T. Bratitsis

Fig. 1. The game idea

They defined the goals and game rules, while selecting the theme it was an easy decision for them, as the board game they were trying to build was inspired from the story of the “Yellow Bird” and the activities that were implemented during the Skills Lab project. The “Yellow Bird” is a story about the value of being different, sharing, caring and helping each other, setting your ideas for the sake of humanity and overcoming physical disabilities. The character of the story became the character of the board game while students created a story of implementation for their game. Step 2: - Imagine. Children discussed board game design. They had time to work together, listen to each other’s ideas. They were encouraged to express their ideas and suggestions. Through brainstorming, the children’s suggestions were recorded by using the digital whiteboard jamboard, while their ideas included information about the mechanics, the components and the type of the board game.

Fig. 2. Brainstorming for the board game design

Step 3: - Plan. Students gather information related to the core elements of board games. According to Beltrami [42], board games are built upon three core elements: Mechanics, Theme and Components (Fig. 4). Game mechanics are constructs of rules which guide the game and provide a structure by establishing what can be expected from the players in order to win [43–46]. Fabricatore [47] stresses that through the game mechanics players have the opportunity to interact and conduct gameplay activities while in the literature have been identified many board game mechanics (e.g. 195 are listed at https://boardgamegeek.com/bro wse/boardgamemechanic). Students as board game designers decided to integrate the following mechanics [46].

Board Game Design by Children as an Assessment Mechanism

95

• Action. These are action rules, describing the way and timing of a player’s interaction with the state of the game, in a direct or indirect manner. • Action Programming. This mechanic refers to every player’s need for secretly choosing his/her next set of actions, and apply this choice to their following game turns. • Movement. This mechanic happens when components move around the play area during the game; this movement may be dictated by rules or chosen by players. • Collection. A game where players need to collect something, a component, some points, various resources etc. • Card Draft. This mechanic applies when picking cards out of one pool or more. • Co-operative play. Players work together to beat the game. The game is won either by players reaching an objective which has been predetermined, or all players lose the game because they have not reached that target before a certain event ends the game. • Dice Rolling. A mechanic that requires players to roll any kind of dice • Goal. It refers to the objectives of the players during the game, which they attempt to achieve. • Ruleset. It is the set of rules, usually of an abstract nature, which control the game, specify what players can do or not, describe the behavior of various game components, and several other aspects of the game. • Mission. A game system that can be applied to a variety of different missions, starting resources and positions. These variable conditions can be assembled into a broader narrative or campaign, or they can be entirely disconnected from one another. • Partnerships. This regards the payers ability to form groups within the game or undo them as well. • Lose a turn. A player who “Loses a Turn” must wait for the next round or the next time his/her turn arises. • Card. This relates to the use of cards within the game. Although this concept is rather basic, cards constitute a very versatile component which also has great expression in a game. • Questions and Answers. Players ask and answer questions in a manner constrained by rules. • Traps and helps. They are board game mechanics and more specifically that can reward or hinder the players in the course of the game. Designing the Elements of the Game. Students used for their tokens, pictures depicting themselves. With the help of their teachers they took photographs and then printed their tokens. All the children participated actively according to the role they had taken, having the support of the teacher - researcher. Working in teams they designed cards that would help or trap players. “Traps” and “helps”, are board game mechanics and more specifically that can reward or hinder the players in the course of the game. Children in the plenary but also in small groups, propose the “traps” and “helps” they want to include in the game, while then they create relevant cards and visualize them through painting. It is worth mentioning the children’s proposal to record their voices so that if a visually impaired or blind person plays the game, they can hear the contents of the card. During the board game design they recorded

96

M. Tsapara and T. Bratitsis

their voices, while using embedded QR codes which would lead to the audio (Fig. 3). The children’s suggestions throughout the game design were a result of the prior knowledge they had acquired as board game designers throughout the year.

Fig. 3. Game cards

Creating the Rules. Kindergarten students as the first step towards making their board game created a set of rules that direct what players are and are not allowed to do, as well as the objective of the game and conditions for victory. They decided that the board game would be a Cop-op game. This type of board game involves all players who work together to achieve a common goal. The game is won either by players reaching an objective which has been predetermined, or all players lose the game because they have not reached that target before a certain event ends the game. The general goal was that the players will collaborate in order to help the inventor STEAMY Tsiou to collect 10 tools for its inventions. They have to navigate their tokens around the board, complete “missions” and earn tools. They must be careful because along the way there are “traps” and “helps”. In order to win all the players have to collaborate and accomplish every mission to gain a tool (in each mission). Step 4: - Create. In this step students developed the game prototype and then tested the game core elements. The game was designed in a way that it would allow a maximum of five players/teams at a time. The progression details were as follows: a) players are allowed to roll the dice, b) move a number of squares equal to the roll, c) pick up a card (help or trap) if the square gives them this choice, and d) claim the tool by completing the corresponding mission. It is very important to mention that during the design of the board game each child undertook a role, such as illustrator, photographer, etc. They also set the goal of the game. Players or teams had to gather 10 tools in order to win. Every tool was connected with a mission. As aforementioned, the main topic chosen was diversity. Overall 12 missions were created. Missions included a knowledge quiz, puzzle and memory games (analog and digital), activities in braille and sign language. For the design of the missions, students formed teams. Each of them shared ideas towards the common goal and collaborated. They made decisions regarding the mission content, chose materials or ICT tool to work with, created QR codes to embed, which would lead to the various content such as quizzes and puzzles, Using QR Codes and by connecting them to digital and other resources with learning materials, students are provided with enriched and motivating learning experiences [48, 49] and also conditions are created for a more accessible game through the elimination of every obstacle.

Board Game Design by Children as an Assessment Mechanism

97

After examining various tools, students made the following choices: • • • •

jigsaw planet, for the creation of a digital puzzle (Mission 2) learning apps, for the creation of the digital memory game (Mission 4) genially, for the creation of a quiz (Mission 12) www.prosvasimo.iep.edu.gr/el/ using the online dictionary of concepts in sign language, from the platform of Institute of Educational Policy • audacity, for the recorded elements of the board game • qr code generator, for the creation of the qr codes Students were rather familiarized with the tools they selected. Figure 4 depicts the material for the missions.

Fig. 4. The missions

Designing the Board of the Game. Students were familiarized with the design of a board game as they had already done similar activities during the same school year. For creating the board, students worked in pairs and co-designed the board they would like for the game to have. Each group then presented its proposal in a plenary session (Fig. 5). As they wanted to consider all the proposals, they decided to make a board that would have elements from all the boards presented. Furthermore they express their ideas in the plenary for “STEAMY Tsiou”, tools and the divided in groups they illustrate them (Fig. 6). Kamii and De Clack [50] argue that a game board could be very simple or even too complicated as an artwork. It can be a 2 or 3 dimensional board, while the track could also be in any shape. They also collaborated in order to create the box of the game where one of the groups proposed to decorate the box and make the use of Braille by “writing” the word GAME. The teacher - researcher had a supportive and mediating role while students were designing the board of the game.

98

M. Tsapara and T. Bratitsis

Fig. 5. Proposed boards presented in a plenary session.

Fig. 6. The collaborative board and box of the game

Step 5: - Improve. In the final step students as game designer’s playtest the game and reflect by giving feedback also as players [51]. Students intended to test both the board and other game elements. They re-tested the missions and other, to see if everything worked correctly. They tested the QR codes, an internet connection; at least one mobile device and a QR code reader was necessary for this procedure (Fig. 7). This board game was designed to incorporate QR codes in a creative way while still contributing to students learning [52] and multimedia material.

Fig. 7. Testing the qr codes and the digital resources

Board Game Design by Children as an Assessment Mechanism

99

They found errors, collaborated with each other and proposed solutions, while recording their ideas for any changes to the game (e.g. they suggested finding symbols in order to depict (Fig. 8).

Fig. 8. Testing the elements of the game - make suggestions

While playing the board game they also tried to play the game “getting in other students’ shoes”, wondering if other children who might have some difficulty could play the game. The game is completed when all the players have achieved the ten tools. Reflecting on the game rules, it was clear that different strategies can be followed. Therefore, there are many ways to approach the game and it is the authors’ impression that this will motivate the children to play the game repeatedly in order to find the best strategy, collaborate and acquire the tools in order to win [13].

4 Evaluation The authors developed a questionnaire for students (as designers and players) to obtain feedback. Furthermore data collection was also based on direct observation complementing the findings. The evaluation process helped both authors and the designers (students) to improve the board game in the future. All 21 students answered with the help of their teacher. The results have been selected as follows by analyzing the response. The questions were divided into 3 categories: 1) Game Design Process, 2) Testing the Game and 3) Game’s improvements. The answers to the questions 1 and 2 point out that all the children enjoy the board game design process while expressing their ideas and making suggestions. As they worked in teams they contributed (question 3) at different points in the game (e.g. in missions, board, box, qr codes, cards, recorded messages, ruleset, tokens, etc.). Approximately more than half of the students stated that there was nothing they didn’t like while designing the game while one third pointed that they wanted to contribute more (question 4). In question 5 it appeared that the collaboration between the groups helped all the children to be actively involved in the board game design process and to get help from their classmates where necessary. Students in the majority liked (question 6) the mission of the game, the qr codes and the use of the Braille and sign language while one third pointed to the character of the game (STEAMY Tsiou). Regarding questions 7 and 8, with 100% students quoted that

100

M. Tsapara and T. Bratitsis

Fig. 9. Students’ suggestions for differentiating the content of the game

Board Game Design by Children as an Assessment Mechanism

101

they would like to play again and they liked to collaborate with their schoolmates in order to win the game (Fig. 10).

Fig. 10. Quantitative results for questions 7 and 8

During testing the game students had the opportunity to playtest not only the board game but also the missions, the qr codes, the card and the board separately. The answers to the questions 9 and 10 point out that kids liked most missions created while one third agreed that they liked missions 1 and 5 the least (Fig. 11).

Fig. 11. Quantitative results for questions 9 and 10

In question 11 “Was there anything that made it difficult for you while testing the game?” the majority of the students did not find anything that made it difficult for them while some said “we have to improve the game in order to be accessible to every child”. All participants enjoyed the 5th step of the EDP (improve). Regarding question 12, a portion of 90,4% the vast majority suggested improvements for the elements of the board game, in order in order to differentiate the content of the game so that it is more accessible to all children.

5 Discussion and Further Plans Through this teaching practice we expected to see if the design of an educational board game could work as a tool to assess the knowledge acquired by children during the Skills Lab project. Students as game designers, while being creative, have to consider various game elements and rules to promote meaningful design changes. When designing games, students learn to determine the knowledge and skills needed for their games

102

M. Tsapara and T. Bratitsis

and recognize game genres, rules and mechanisms for them to adopt and modify [51]. Students created a board game that can be played by 2 to 4 players. The players are not required to be familiar with concepts such as diversity, disability, etc. The board game contained a board, dice, tokens and was a collaborative one. Players had to complete missions that include mini games that students created. Every mission had an embedded QR code, after students’ proposal in order to help visually impaired students to play the game. The children as game designers during the board game design process were thinking about all players’ needs, so they tried to differentiate the board game in order to be as inclusive and accessible as possible [53]. The children’s participation in the whole process of creating and designing a board game helped them to transfer the knowledge gained through the Skills Lab. At each stage of the planning they had the opportunity to become more creative, to collaborate with their classmates for a common goal. During the creation of the missions they expressed their ideas but also their concerns. They actually communicated with the members of their team, jointly deciding on the final result. During the process of testing the game they were able to cultivate their critical thinking but also to find themselves in problem solving situations especially in the cases where they identified some failures of the material. The game itself and the involvement of the children formed an attractive learning framework which motivated all children to participate creatively in this process. Board game-based teaching offers new insights into and approaches to teaching and also can help children to cultivate 21st century skills, as well as gain self-esteem, learn to listen to the others, be patient, express their ideas, make mistakes, and try to find solutions. In the questionnaires that were conducted in order to evaluate various aspects of the game, students seemed to like their game. They not only enjoyed playing the game but also were pleased to design an inclusive board game. They make suggestions in order to improve the game prototype. Answering the question “Would you like to play the game again?” the majority of them wanted to play the game again. Students also noticed as an advantage of their board game, that they could interact with the missions independently of the game. Taking into account that nowadays children at the age of 4–5 are already internet and smart device users [13], students claimed that the creation, the use of QR Codes and mobile devices made the board game more accessible to every child with no exclusion. As for the future, the authors intend to conduct a small scale survey with teachers during the next school year in order to examine the appropriateness of the board game and its contents, test the board game to another classroom and evaluate game prototype with disabled players. Moreover, printing the board game in a more professional version in order for kindergarten students to have the opportunity to play the final design of the board game created after the playtest. Other ideas include the enrichment of the missions created by the children and the addition of other elements of augmented reality.

References 1. Bayliss, J.D.: The effects of games in CS1-3. J. Game Dev. 2(2), 7–17 (2007) 2. Yoon, B., Rodriguez, L., Faselis, C.J., Liappis, A.P.: Using a board game to reinforce learning. J. Contin. Educ. Nurs. 45(3), 110–111 (2014) 3. Treher, E.: Learning with board games. tools for learning and retention. Learn. Key Inc (2011)

Board Game Design by Children as an Assessment Mechanism

103

4. Ali, R., et al.: A qualitative study on the effect of game board strategy on the students’ motivation of learning statistics courses. J. Acad. 7(Special Issue 1), 142–145 (2019) 5. Strode, A.: A board game as a teaching tool in a pedagogically psychological perspective. In: Edulearn 19 Proceedings, pp. 5437–5444 (2019) 6. Cojocariu, V.M., Ioana, B.: Teaching the relevance of game-based learning to preschool and primary teachers. Procedia-Soc. Behav. Sci. 142 (2014) 7. Anderson, O.B., Anderson, N.M., Taylor, A.T.: New Territories in Adult Education: GameBased Learning for Adult Education (2009). http://www.adulterc.org/Proceedings/2009/pro ceedings/anderson_etal.pdf. Accessed 30 Apr 2023 8. Mostowfi, S., Mamaghanib, N.K., Khorramarb, M.: Designing playful learning by using educational board game for children in the age range of 7–12: a case study: recycling and waste separation education board game. Int. J. Environ. Sci. Educ. 11(12), 5453–5476 (2016) 9. Tsapara, M., Bratitsis, T.: A board game for sustainable development education: kindergarten students as game designers. In: Auer, M.E., Tsiatsos, T. (eds.) IMCL 2021. LNNS, vol. 411, pp. 1072–1084. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96296-8_98 10. Eriksson, M., Kenward, B., Poom, L., Stenber, G.: The behavioral effects of cooperative and competitive board games in preschoolers. Scand. J. Psychol. 62(3), 355–364 (2021) 11. Anzman-Frasca, S., et al.: Evaluating a board game designed to promote young children’s delay of gratification. Front. Psychol. 11, 581025 (2020) 12. Ambarita, M., Rahayu, W., Supena, A.: Number sense ability: the application of game with rules and emotional intelligence. Proceedings of the 4th International Conference on Learning Innovation and Quality Education, pp. 1–7 (2020) 13. Bratitsis, T., Kontovounisiou, A., Kiriazoglou, M.: A board game proposal for teaching informatics related topics in early childhood education. In: SHSHS Web of Conferences, vol. 102, p. 01001 (2021) 14. Wendell, K.B.: The Theoretical and Empirical Basis for Design-Based Science Instruction for Children. Tufts University, Qualifying Paper (2008) 15. Kuo, H.C., Weng, T.L, Chang, C.C., Chang, C.Y.: Designing our own board games in the playful space: improving high school student’s citizenship competencies and creativity through game-based learning. Sustainability, 15(4), 2968. MDPI (2023) 16. Zirawaga, S., Adeleye, O., Tinovimbanashe, M.: Gaming in education: using games as a support tool to teach history. J. Educ. Pract. 8(5), 55–64 (2017) 17. Mehta, A., Bond, J., Sankar, C.S.: Developing an inclusive education game using a design science research gestalt method. AIS Trans. Hum.-Comput. Interact. 14(4), 523–547 (2022) 18. Nieminen, J.H.: Assessment for Inclusion: rethinking inclusive assessment in higher education. Teaching in Higher Education (2022) 19. Ainscow, M.: Understanding the development of an inclusive education system. Electron. J. Res. Educ. Psychol. 3(3), 5–20 (2005) 20. EiE. The engineering design process (2016). www.eie.org/overview/engineeringdesign-pro cess 21. Uribe, V.R., Cobos, S.: Board Games as a Tool for Teaching Basic Sustainability Concepts to Design Students (2014) 22. Tsai, J.C., Liu, S.Y., Chang, C.Y., Chen, C.Y.: Using a board game to teach about sustainable development. Sustainability 13, 4942 (2021). https://doi.org/10.3390/su13094942 23. Chien-Sing, L., Kian-Wei, L.: The effectiveness of object-oriented-QR monopoly in enhancing ice-breaking and education UX: a preliminary study. In: Rodrigo, M.M.T., et al. (Eds.) (2021). Proceedings of the 29th International Conference on Computers in Education. Asia-Pacific Society for Computers in Education (2021) 24. Bayeck, R.Y.: Examining board gameplay and learning: a multidisciplinary review of recent research. Simul. Gaming 51(4), 411–431 (2020)

104

M. Tsapara and T. Bratitsis

25. Akmal, H., Coulton, P.: Research Through Board Game Design (2019) 26. Barbara, J.: Measuring user experience in multiplayer board games. Games Cult. 12(7–8), 623–649 (2017). https://doi.org/10.1177/1555412015593419 27. Zagal, J.P., Rick, J., His, I.: Collaborative games: lessons learned from board games. Simul. Gaming 37(1), 24–40 (2006) 28. Booth, T., Ainscow, M.: Index for inclusion: Developing learning and participation in schools (3rd ed.). Bristol: Centre for Studies on Inclusive Education (CSIE) (2011) 29. Felder, F.: Celebrating diversity in education and the special case of disability. Educ. Rev. 73(2), 137–152 (2021) 30. Kairiene, B., Sprindziunas, A.: Social equality as groundwork for sustainable schooling: the free lunch issue. J. Teach. Educ. Sustain. 18(1), 127–139 (2016) 31. Özer, P., Çatak, G.: Using board games as a method for improving awareness and empathy in inclusive design: PUDCAD game case study. In: Cordan, Ö., Dinçay, D.A., Yurdakul Toker, Ç., Öksüz, E.B., Semizo˘glu, S. (eds.) Game + Design Education. Springer Series in Design and Innovation, vol. 13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-650605_11 32. Filho, F., Mirza-Babaei, P., Kapralos, B., Mendonça Junior, G.: Let’s play together: adaptation guidelines of board games for players with visual impairment. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–15. (2019) 33. Westin, T., Engström, H., Brusk, J.: Towards Sustainable Inclusive Game Design Processes (2020) 34. Mehendale, D., Masurekar, R., Nemade, S., Shivthare, S.: To study the use of QR code in the classroom to enhance motivation, communication, collaboration and critical thinking. Int. J. Innov. Res. Comput. Commun. Eng. 5(4), 6987–6993 (2017) 35. Trentin, G., Repetto, M.: Using Network and Mobile Technology to Bridge Formal and Informal Learning. Woodhead/Chandos Publishing Limited, Cambridge (2013) 36. Law, C., So, C.S.: QR codes in education. J. Educ. Technol. Dev. Exchange 3(1), 85–100 (2010) 37. Rikala, J., Kankaanranta, M.: The use of quick response codes in the classroom. In:11th World Conference on Mobile and Contextual Learning (2012) 38. Mousa, A.A., El-salam, M. A.: Employing QR code as an effective educational tool for quick access to sources of kindergarten concepts. world academy of science, engineering and technology. Int. J. Educ. Pedagogical Sci. 10, 2367–2370 (2016) 39. Ramsden, A.: The use of QR codes in Education: a getting started guide for academics. University of Bath (2008) 40. Katz, L.G., Chard, S.C.: Engaging Children’s Minds: The Project Approach. Ablex, Norwood, NJ (2000) 41. Cardinot, A., McCauley, V., Fairfield, J.: Designing physics board games: a practical guide for educators. Phys. Educ. 57, 035006. https://doi.org/10.1088/1361-6552/ac4ac4 (2022) 42. Beltrami, D.: A board game design process: A game is a system. A board game design process: A game is a system (2020). https://uxdesign.cc/a-board-game-design-process-a-game-is-asystem-5469dfa4536. Accessed 20 Apr 2023 43. Essop, H., Kekana, M., Sethole, M., Ahrens, E., Lovric, G.: A board game to enhance teaching and learning in the radiographic technique module: a pilot study. Int. J. Innov. Educ. Res. 6, 124–140 (2018). https://doi.org/10.31686/ijier.Vol6.Iss10.1173 44. Sicart, M.: Defining game mechanics. game studies. Int. J. Comput. Game Res. 8 (2008) 45. Robson, K., Plangger, K., Kietzmann, J.H., McCarthy, I., Pitt, L.: Is it all a game? understanding the principles of gamification. Bus. Horiz. 58(4), 411–420 (2015) 46. Kritz, J., Mangeli, E., Xexéo, G.: Building an ontology of board game mechanics based on the boardgamegeek database and the MDA framework SBC. In: Proceedings of SBGames (2017)

Board Game Design by Children as an Assessment Mechanism

105

47. Fabricatore, C.: Gameplay and Game Mechanics Design: A Key to Quality in Video Games. OECD (2007). https://www.oecd.org/education/ceri/39414829.pdf 48. Lin, H.C.K., Lin, Y.H., Wang, T.H, Su, L.K., Huang, Y.M.: Effects of incorporating augmented reality into a board game for high school students’ learning motivation and acceptance in health education. Sustainability 13, 3333 (2021) 49. Rouillard, J.: Contextual QR codes. CCGI 2008: The 3 rd International. Multiconference on Computing in Global Information Technology. Athens, Greece: IEEE (2008) 50. Kamii, C., De Clark, G.: Children Reinvent Arithmetics. Patakis publications, Athens (1995). (in greek) 51. Beaumie, K., Reyhaneh, B.: Students as game designers: transdisciplinary approach to steam education. ATA J. 45, 45–52 (2017) 52. Puentedura, R.R.: Transformation, Technology, and Education (2006) 53. Lauren, T., Rodier, N., Oriola, G., Hachet, M., Jouffrais, C.: Inclusive adaptation of existing board games for gamers with and without visual impairments using a spatial augmented reality framework for touch detection and audio feedback. In: Proceedings of the ACM on Human-Computer Interaction, vol. 5 (ISS) (2021). ff10.1145/3488550ff. ffhal-03405986

Progressive Healthcare Pedagogy: An Application Merging ChatGPT and AI-Video Technologies for Gamified and Cost-Effective Scenario-Based Learning Matthew Pears , Cherry Poussa , and Stathis Th. Konstantinidis(B) University of Nottingham, Nottingham NG72QL, UK {Matthew.Pears,Cherry.Poussa, Stathis.Konstantinidis}@Nottingham.ac.uk

Abstract. Healthcare education faces numerous challenges in meeting the expanding needs of students while providing personalized learning experiences. Artificial Intelligence (AI) technologies, specifically Large Language Models (LLMs), have emerged as promising solutions to address these challenges. However, the gap between technological advancements and practical implementation remains a significant bottleneck in AI integration. This paper presents an exploration of the practical implementation of AI in healthcare education, focusing on user-friendly, controllable, and transparent AI tools. The study reviews existing literature on AI in healthcare education, emphasizing the potential of LLMs but also addressing challenges, such as bias and fairness. A methodology section describes a serious game-based workshop that leveraged AI tools including ChatGPT-4 to simulate dynamic healthcare scenarios and foster user engagement. Results demonstrate the efficacy and adaptability of AI-driven applications in healthcare education, highlighting their potential as cost-effective learning resources. The paper discusses the implications of AI implementation, including its capacity to transform traditional educational methods, promote curiosity, and foster trust. Ultimately, this paper aims to inspire foster innovation and inform best practices for the practical integration of AI in healthcare education, bridging the gap between theoretical complexity and real-world application. Keywords: Gamification · ChatGPT · Healthcare Education · Large Language Models · Interpersonal Skills · Chatbots

1 Introduction Healthcare education faces a myriad of challenges due to expanding needs, costs, and complexities. The rapid evolution of medical knowledge necessitates a dynamic and adaptable learning environment, a demand that divisions in traditional educational methods often struggle to meet [1, 2]. Furthermore, the need for personalized learning experiences, which cater to the unique strengths and weaknesses of individual students, is becoming increasingly apparent. Medical and surgical data and information are complex and multifaceted, requiring high-level cognitive and reasoning skills to effectively © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 106–113, 2024. https://doi.org/10.1007/978-3-031-56075-0_10

Progressive Healthcare Pedagogy: An Application Merging ChatGPT

107

transfer this knowledge to students. Traditionally, this has been the domain of human educators, trainers, and other roles. However, the practical, hands-on experience that is crucial to healthcare education is often difficult to provide at scale, especially in remote learning scenarios. The potential of Artificial Intelligence to address these challenges has been explored in several studies. Large Language Models (LLMs), a subset of AI, have shown promise in a variety of applications relevant to healthcare education. Large Language Models (LLMs) are AI systems trained on extensive datasets, capable of understanding and generating human-like text, and performing tasks such as information retrieval and reasoned thought. [2]. These capabilities can be harnessed to provide personalized learning materials, up-to-date information, simulated practical experiences, and facilitate remote learning. However, the application of AI in healthcare education is not without its challenges. Issues such as bias in machine learning models and the need for fairness have been highlighted [3]. Despite the promising capabilities of AI and LLMs, the rapid development of these technologies has led to a new set of challenges. Stakeholders in healthcare education, ranging from educators and administrators to students and policymakers, face the daunting task of navigating this complex and rapidly evolving landscape [4–6]. The most popular and powerful large language model available to consumers, named ChatGPT [7] in medical education has shown its potential as a valuable tool for students. A comprehensive literature review of 118 peer-reviewed papers identified Large Language Models (LLMs) like ChatGPT as having the potential to automate the laborintensive process of generating and analyzing textual content, which can greatly benefit students [8]. However, the review also emphasized the importance of addressing practical and ethical challenges associated with LLM-based innovations in healthcare education. While text-based tools like GPT [7] and image generation tools such as Stable Diffusion [9] offer promising solutions to the limitations of traditional educational methods, their practical demonstrations remain a significant challenge. This challenge is compounded by several factors including the apprehension of staff and students towards these new technologies, often stemming from a lack of understanding and control over these tools [10–12]. Our exploratory initiative aims to bridge this gap between the theoretical complexity of AI and the practical needs of healthcare education. We believe that the key to overcoming these challenges lies in creating user-friendly, controllable, and transparent AI tools that can be easily integrated into the existing educational framework. By demystifying AI and making it more accessible, we can alleviate the apprehension of staff and students and encourage their active engagement with these tools. Finally, our exploratory initiative aims to leverage the insights gained from the development and implementation of our AI application to inform best practice recommendations for the use of AI in healthcare education. By sharing our experiences and lessons learned, we hope to contribute to the ongoing discourse on this topic and provide valuable guidance for future endeavors in this field. We synergized different tools to create a GPT-based, dynamic and interactive healthcare-themed serious game, within a digital learning workshop. This design was aimed at demystifying AI and fostering trust in the model’s data management, enhancing the experience for both online and face-to-face participants through a high-quality visual and audio interface that improved communication style.

108

M. Pears et al.

2 Methodology 2.1 Design and Participants Our exploratory initiative was designed around a serious game, aimed at fostering nontechnical skills, also known as soft skills, such as decision-making, communication, and efficient information gathering. The game was implemented in a preliminary workshop attended by 20 learners, providing a diverse data set for analysis. This group consisted of individuals (academics, learning technologists, postgraduate students) with varying levels of exposure to healthcare scenarios, thereby ensuring a broad spectrum of user experiences. 2.2 Tools and Techniques The serious game was built on Python’s Flask framework, leveraging the capabilities of the GPT-4 model, and made decisions in different scenarios. A unique scoring algorithm analyzed learner performance, with GPT-4 providing detailed feedback, enhancing participants’ understanding of the application’s usability (Fig. 1). The application also incorporated a modified rolling memory for the AI character, ensuring contextual consistency and enhancing the realism of the simulated scenarios. This allowed 16k token usage to feed the model patient history information but a continued discourse that had much longer decay of information. Neural network text-to-speech (TTS) was used to add natural language audio to the GPT text responses.

Fig. 1. Abstract Architecture of the proposed system.

2.3 Gameplay and Interaction The application revolved around a patient whose comprehensive fictional history was created and stored in a text file that was accessible to the AI model and added to each API call. This response was then intercepted by Stable Diffusion (via D-ID [13]), a technology that modifies an image to match the text, creating a dynamic visual representation of the model’s response. The data had hierarchical access that allowed progression only if users retrieve certain information, showing the control of data by the model (Fig. 2).

Progressive Healthcare Pedagogy: An Application Merging ChatGPT

109

Fig. 2. Figure 1: Screenshots of the AI-driven healthcare education application, displaying the user interface with text-based interaction and the representation of the character using D-ID hosting technology.

2.4 Procedure and Evaluation The time slot was allocated for the session whereby five minutes was informing participants of how to use the application and the website to go to. A brief video was showed demonstrating the goals and instructions - also included on the web page. Participants divided into 5 teams and had 10 min to interact with the virtual doctor and make note of all the patient information they could extract. Once this time was up, they sent their results via email to the lead researcher, who then added each set of responses into the separate preconfigured ChatGPT scoring model, to allow participants to see how they have performed against others. The unique scoring system allowed for the semi-autonomous evaluation of learner performance. The GPT-4 model, guided by a specific scoring prompt created by the research team, provided comprehensive feedback on performances during workshops or online sessions. This approach facilitated an indepth understanding of the application’s compatibility with a diverse range of users, each possessing varying levels of technological understanding and unique interaction styles with AI. After the workshop, we gathered feedback and conducted quantitative and qualitative analyses of learner scores to evaluate the application’s effectiveness and identify user experiences. To assess information gathering proficiency, we utilized a six-criteria scoring system, examining quantity, diversity, accuracy, completeness, relevance, depth, creativity, specificity, scope, and consideration of chronological sequences (Table 1). Group performance was determined using weighted criteria rated on a 1–10 scale, aided by a crafted prompt for analyzing responses.

3 Results This serious game, with albeit whimsical patient history, was designed as an experiential demonstration rather than a training tool and illuminated the intricate controls and capabilities within AI models. We believe it achieved showcasing how information

110

M. Pears et al. Table 1. Evaluation Criteria for the users’ response to the scenario.

Criterion

Weight Description of the criterion

Quantity of Responses

10%

Evaluate the number and diversity of questions the group asked. Are they exploring different facets of the patient’s life, or are they focused on just one or two areas?

Accuracy and Completion of Information 30%

Is it accurate based on what we know about the patient? Did they manage to collect a significant portion of his history, or did they miss out on important areas?

Relevance to Medical Perspective

25%

Determine how medically relevant the group’s questions and findings are. Remember, the main goal is to understand the patient’s health status, so questions that lean towards this should be given more weight

Creativity and Depth of Investigation

15%

Gauge how innovative the group was in their approach and how deeply they delved into the patient’s history. Did they connect the dots in unexpected but effective ways?

Specificity and Scope

10%

Measure the specificity of the group’s questions and the scope of their investigation. Did they ask precise questions and cover a wide range of areas in Jonathan’s life?

Chronological Investigation

10%

Observe if the group considered the chronological sequence of events in the patient’s life in their investigation

can be strategically manipulated and regulated, in a fun and engaging way. While the scoring results, varying across the groups due to differing strategies and strengths in the information-gathering task, aren’t necessarily indicative as this measure has yet to me validated, they served to demonstrate the diverse functionalities of AI, underscoring its potential utility for the participants’ unique case needs. The performances ranged from Group 1’s lower score of 6.2, resulting from a somewhat narrow focus to Group 5’s high score of 7.9 (Table 2), achieved through a balance of all evaluation criteria and a comprehensive, systematic investigation. ChatGPT provided individual feedback for each criterion as can be seen in Fig. 3. These results exemplify the differences AI can help to identify, based on our full control of its elements, to form trust of information, and navigate in current problems, offering a transparent and well-regulated solution.

Progressive Healthcare Pedagogy: An Application Merging ChatGPT

111

Table 2. Groups scoring results based on specific narrative inserted by ChatGPT. Criteria

Weight

Group 1

Group 2

Group 3

Group 4

Group 5

Quantity of Responses

10%

7

5

8

9

8

Accuracy and Completion of Information

30%

6

7

8

7

8

Relevance to Medical Perspective

25%

6

9

7

8

9

Creativity and Depth of Investigation

15%

5

7

7

8

7

Specificity and Scope

10%

5

8

8

8

8

Chronological Investigation

10%

4

6

5

5

6

5.7

7.3

7.3

7.5

7.9

Final Scores

Fig. 3. Example of participants’ answer (left) and ChatGPT feedback (right).

The analysis of group responses showed enhanced soft skills. Participants’ ‘critical thinking’ were displayed as they linked diverse aspects of the patient’s life and considered reasons beyond given information, like why he ceased medication. They applied a “cautious approach” to sensitive topics, using open-ended questions to pursue a thorough understanding of his health. ‘Communication and collaboration’ skills improved, with clear articulation of their inquiries promoting productive exchanges. Cultural sensitivity was evident as participants respected the virtual doctor’s limitations, adeptly handling social interactions. Problem-solving skills developed through connections between Jonathan’s personal background with subtle clues demonstrating adept analysis. Some

112

M. Pears et al.

groups showcased resilience amid humour and light-hearted teasing, while patience was clear, persisting despite the virtual doctor’s limited information.

4 Discussion and Conclusion This workshop utilized a serious game designed not as an explicit training tool, but rather as an experiential environment to shed light on the intricate processes and controls inherent in our AI models. Through this simulated setting, participants were able to gain insights into the models’ capabilities in managing and regulating information flow and at the same time practice to enhance their soft skills. The evaluation and scoring methodology served to demonstrate how these models can be manipulated to fit specific tasks, ranging from information retrieval to restriction. Although the resulting scores were inherently subjective due to the nature of the game, they nonetheless provided participants with a tangible, systematic approach to evaluate performance in, what many students may find to be, a complex, multi-faceted task. The workshop illuminated the potential of such AI-driven serious games to serve as teaching and training tools once validated by subject matter experts. The experience served as a template that can be further developed, customized, and implemented to meet specific training needs in a diverse range of contexts. The modularity of our application extends beyond context and characters. The Large Language Model that powers the application can also be modified, trained, or fine-tuned to adapt to different healthcare topics and generate appropriate responses. This flexibility not only enhances the tool’s effectiveness but also its future scalability. As AI technologies evolve and improve, this application can be easily updated to harness these advancements and offer a continually evolving and improving learning resource. From a practical perspective, the use of AI in healthcare education has often been limited by high costs and complex implementation processes. However, the model we developed demonstrates that a powerful and impactful learning resource can be created with cost-effective AI technologies. The use of open-source technologies, such as the GPT-4 model and the Flask web framework, allowed us to build a dynamic and interactive tool without incurring substantial development costs. We used D-ID as a placeholder for the free Stable Diffusion version that is to be added in the next update [14]. This approach presents a viable path for the wider adoption of AI in healthcare education, particularly for institutions and educators who may have previously been deterred by the perceived cost barriers associated with AI technologies. Our exploration also elucidates the potential of AI as a catalyst for behavioural change in healthcare education. The interactive and experiential nature of the application served to break down barriers, dispel misconceptions, and foster a more positive and receptive attitude towards AI among participants [10]. Through their active engagement in the serious game, participants were able to witness first-hand the capabilities of AI, understand its potential benefits, and build trust in AI technologies. Our exploratory initiative serves as a significant steppingstone towards the practical implementation of AI in healthcare education, showcasing the potential of AI as a transformative and accessible tool for enhancing the learning experience of healthcare students. This exploration emphasizes the importance

Progressive Healthcare Pedagogy: An Application Merging ChatGPT

113

of adopting an integrative approach that combines technological innovation, pedagogical strategies, user engagement, and policy considerations. Transparency and controllability are crucial aspects, allowing users to understand the AI’s decision-making process and influence outcomes to build trust and confidence. Ethical considerations, such as addressing bias and fairness, are essential for responsible AI implementation. Ongoing research and development are necessary to overcome limitations like false text generation and inconsistency in responses. By continuously refining AI applications, we can maximize their potential impact on healthcare education, inspiring further innovation, and advancement in this dynamic field.

References 1. Chandra, M., Kumar, K., et al.: Digital technologies, healthcare, and Covid-19: insights from developing and emerging nations. Health Technol. (Berl) 12(2), 547–568 (2022) 2. Ahn, S.: The impending impacts of large language models on medical education. Korean J. Med. Educ. 35, 103 (2023) 3. Norori, N., Hu, Q., Aellen, F.M., et al.: Addressing bias in big data and AI for health care: a call for open science. Patterns 2, 100347 (2021) 4. Lekadir, K., Osuala, R., Gallin, C., et al.: FUTURE-AI: guiding principles and consensus recommendations for trustworthy artificial intelligence in medical imaging. arXiv preprint arXiv:2109.09658 (2021) 5. Langer, M., Oster, D., Speith, T., et al.: What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artif. Intell. 296, 103473 (2021) 6. Sallam, M.: ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare 11(6), 887 (2023) 7. ChatGPT. https://openai.com/chatgpt. Accessed 16 June 2023 8. Garg, R.K., Urs, V.L., et al.: Exploring the role of Chat GPT in patient care (diagnosis and Treatment) and medical research: a systematic review. In: BMJ medRxiv Preprints (2023) 9. Stable Diffusion—Stability AI. https://stability.ai/stablediffusion. Accessed 16 June 2023 10. Mousavi Baigi, S.F., Sarbaz, M., et al.: Attitudes, knowledge, and skills towards artificial intelligence among healthcare students: a systematic review. Health Sci. Rep. 6(3), e1138 (2023) 11. Jiang, L., Wu, Z., Xu, X., et al.: Opportunities and challenges of artificial intelligence in the medical field: current application, emerging problems, and problem-solving strategies. J. Int. Med. Res. 49(3), 3000605211000157 (2021) 12. Lomis, K., Jeffries, P., Palatta, A., et al.: Artificial intelligence for health professions educators. In: NAM Perspectives (2021). https://doi.org/10.31478/202109a 13. D-ID|The #1 Choice for AI Generated Video Creation Platform. https://www.d-id.com/. Accessed 16 June 2023 14. Zhang, W., Cun, X., Wang, et al.: SadTalker: learning realistic 3D motion coefficients for stylized audio-driven single image talking face animation. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8652-8661. Vancouver, BC, Canada (2023). https://doi.org/10.1109/CVPR52729.2023.00836

A Project Overview: The Implementation of a Native Android App for Mobile Signal Detection and a PHP Laravel Web-Based Platform for Real-Time Monitoring and Analysis of Wireless Communication Networks for Educational Purposes Georgios Roussos1,2(B)

and Petros Nicopolitidis1

1 Aristotle University of Thessaloniki, Thessaloniki, Greece

{roussosg,petros}@csd.auth.gr 2 Aegean University, Mytilene, Greece

Abstract. This paper presents an in-depth examination of the creation and deployment of an Android application and a companion PHP Laravel web-based platform, both engineered to extract, process, and scrutinize real-time mobile signal parameters. Capitalizing on the ubiquity and open-source features of the Android system, the application acquires nuanced mobile data and interacts seamlessly with the web platform for comprehensive analysis. The platform, designed to be an accessible educational resource, serves a broad demographic, ranging from inquisitive children to professional researchers, fostering a more profound comprehension of wireless communication networks. Despite initial success, the study acknowledges the need for ongoing enhancements to adapt to the continually evolving Android ecosystem. The article systematically articulates the methodology encompassing project planning, design, development, testing, and data collection. It underscores pivotal research inquiries, the selection of software tools, and the type of data handled. The project spotlights the relentless pursuits in technological research, signifying its considerable potential to contribute to the domain of wireless network analysis. Keywords: Wireless Communication Networks · Mobile Applications · Android OS · Real-time Signal Detection

1 Introduction Wireless communication networks have come a long way over the past few decades. Advances in technology have improved data transmission speed and quality, taking us from 1st generation to the current 5th generation (5G) networks [1, 2] in this smart digital world. At the same time, the wide use of Android OS devices, which hold a significant share of the market [19], gives us a great platform for exploring these networks. This © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 114–125, 2024. https://doi.org/10.1007/978-3-031-56075-0_11

A Project Overview: The Implementation of a Native Android App

115

project takes an educational approach to help a range of users, from everyday individuals to researchers and even children, understand wireless communication networks better. It includes a straightforward Android app that lets users access important information about their mobile signals, such as signal strength and network type. Additionally, a web-based platform collects this data for analysis, providing useful insights into network performance, usage trends, and connectivity issues. 1.1 Purpose This initiative presents an Android application and a PHP Laravel web-based platform designed to explore mobile signal parameters. The Android application under the name 5G Signalyzer, identifies signal parameters such as Mobile Operator, Device Model, MCC, MNC, TAC, PCI, ECI, EARFCN, BW, RSRP, RSRQ, TA, BAND from user smartphones [1, 3]. The app communicates with the Laravel web platform via APIs, enabling real-time data collection, analysis, and visualization. Users voluntarily contribute data, creating a rich database for in-depth examination of wireless networks. The article elucidates the app’s features for users of all backgrounds (ranging from curious children and tech-savvy adults to researchers or developers), clarifying the range of data it can process, its implications, and its contribution to the web panel. Meanwhile, for developers, the article covers the app’s design, the choice of Android as a platform, the deployed development tools, and the implementation logic, enhancing understanding of the app’s benefits and its specific implementation strategy. 1.2 Research Questions (RQs) Prior to initiating and throughout the execution of this project, various pertinent research questions (RQs) were posed and explored. This article highlights some of the most interesting ones as follows: • What are the primary features of this application and what benefits does it provide to the end user? • Why the Android OS was chosen as platform instead of iOS? • What assortment of software development tools and resources have been required to design and develop the Android mobile App? • What technologies were employed in developing the web server application responsible for collecting data from the mobile app? • What specific data are collected from users’ smartphones and how are they processed? • How could the application be further developed and valuable for research purposes in an ever-growing digital world?

2 Background From the inception of first-generation (1G) systems in 1981, wireless communication networks have evolved remarkably. The 1G systems offered rudimentary voice services, which were further augmented by second-generation (2G) networks a decade later, introducing basic low-speed data services. The transformational third-generation

116

G. Roussos and P. Nicopolitidis

(3G) systems came into existence in the early 2000s, significantly expanding the range of high-speed data and voice services [1, 2]. Currently, we are in the era of fifth-generation (5G) networks, which herald a paradigm shift with unprecedented capacity, diminished latency, and an array of sophisticated services [11, 12]. In parallel to this progression, the Android OS, since its introduction in 2008, has grown exponentially. For this study, Android OS was chosen due to its vast global user base and open-source nature, which encourages customization and flexibility. Its extensive library support and compatibility with diverse hardware facilitate the development of an app capable of real-time data collection across numerous device models, making it an ideal choice [3, 5]. In research, Android OS open-source nature and customizability have positioned it as an invaluable platform for numerous research initiatives [3, 6–8, 11, 19]. For instance, in research, it has catalyzed the development of applications that calculate power consumption for video streaming in wireless environments and measure network Key Performance Indicators such as received power, latency, and throughput and many other applies [7, 9]. Hence, Android OS has proven to be a robust tool for in-depth network analysis. However, despite these advancements, in research remains a noticeable gap in the development of an Android application specifically geared towards measuring mobile wireless signal parameters in real-time. To address this void, the current project aims at an educational approach, cultivating a deeper understanding of wireless communication networks [8]. The initiative features an easy-to-use Android app that allows users to retrieve crucial mobile signal data like signal strength and network type [11]. In addition, a complementary web-based platform collates, analyzes, and visualizes this data, providing insights into network performance, user behavior trends, and connectivity issues. As such, the project presents a fresh trajectory in the field, harnessing the capabilities of Android OS for comprehensive network analysis.

3 Methodology In this chapter, an overview is provided of the systematic approach to the development and testing of the Android app using the waterflow model [13, 14]. The methodology unfolds in five key phases: – Project Planning: This initial phase involves requirement gathering and meticulously documenting project goals, ensuring a clear understanding of the project’s scope and laying the groundwork for all subsequent development stages. – Design: During this stage, decisions are made regarding the architectural framework and design principles for the Android app and the web-based admin panel. – Development: This phase encapsulates the coding and construction of the app, using Java for the Android platform and Laravel for server-side development, alongside the strategic deployment of databases and secure RESTful APIs. – Testing: The fourth phase sees the application undergo thorough testing across various devices to validate functionality and performance. – Maintenance: after the initial release of the software, a continual process of modifications, improvements, error corrections, and refinements is often necessary. This maintenance phase ensures that the software not only functions smoothly but also

A Project Overview: The Implementation of a Native Android App

117

evolves to meet changing user needs, comply with new technological standards, and address unforeseen challenges. 3.1 Project Planning The application’s function and design aim to provide users with a seamless experience while accessing and understanding mobile wireless signal parameters. To achieve this, the application follows a carefully designed flow of processes and functions, represented through a flowchart (see Fig. 1).

Fig. 1. Android Project Flowchart – Capabilities, Processes, Permissions

– Installation and Permission Request: Upon installing and opening the application, the user is requested to grant certain permissions, such as phone status, exact location,

118

G. Roussos and P. Nicopolitidis

and device antenna usage. This is crucial for the functionality of the application. If the user denies these permissions, they’re continuously prompted until they accept. – Login/Signup Process: Following the permission acceptance, the user can either login or sign up. During signup, user information is cross-checked with the server database to avoid duplication. If the user already exists, an alert message is displayed. If not, the new user gets registered successfully. – Login and User Interface: Upon successful login, the application accesses system data (phone status, exact location, antenna information), which is then presented to the user through three sections: 1) Overview 2) Stats and 3) Profile. The user also gets an option to choose whether their data should be recorded in the database. – Profile Settings and Options: Under the profile section, the user can view their data, logout from the application, or choose to delete their profile and data from the system entirely. By establishing clear targets and objectives during the project planning phase, this forms the fundamental groundwork for the subsequent steps of designing, developing, and testing. It serves as a robust roadmap guiding each stage of the project, ensuring consistency and focus towards achieving the outlined goals. 3.2 Design To align the application with contemporary design standards and ensure it adequately adapts to the diverse dimensions, architectures, and technologies of modern devices, robust graphic design tools were employed. The design process utilized Adobe Photoshop and Adobe XD (see Fig. 2) for detailed visual creation and user experience design, while Material.io, Google’s open-source design system, was employed to imbue the interface with modern, responsive, and intuitive elements. This strategic blend of tools was instrumental in creating an application that is not only functionally efficient but also aesthetically appealing and user-friendly. For the web-based admin panel, the MVC (Model-View-Controller) architecture, a feature inherent in Laravel, was employed to manage data, user interface, and server-side control efficiently [15, 16]. 3.3 Development During the development phase, essential tools and resources were carefully chosen to align with the project’s requirements. Android Studio, with its comprehensive suite of tools, was chosen as the Integrated Development Environment (IDE), and Java, renowned for its robustness and abundant open-source library support, was selected as the development language for the Android app (see Fig. 3). In the context of development, the ongoing challenge is to ensure support for devices running Android 7.0 and newer versions. Libraries from the Android Platform APIs were leveraged in the main Java code, specifically to retrieve signal information and parameters from the user’s device (see Table 1). These libraries facilitated efficient and accurate data collection, furthering the application’s functionality and performance.

A Project Overview: The Implementation of a Native Android App

119

Fig. 2. GUI design using Adobe Photoshop and Adobe XD and Material.io

Fig. 3. Android Studio as IDE for application development

Also, Laravel [16] was used for server-side development, a PHP framework known for its robust security features and efficient ORM.

120

G. Roussos and P. Nicopolitidis Table 1. Libraries for signal information and parameters retrieval from the user’s device 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.

Package com.example.signalyzer.ui.dashboard; import android.location.Address; import android.location.Geocoder; import android.location.Location; import android.location.LocationListener; import android.location.LocationManager; import android.net.ConnectivityManager; import android.net.NetworkInfo; import android.net.wifi.WifiInfo; import android.net.wifi.WifiManager; import android.preference.PreferenceManager; import android.provider.Settings; import android.telephony.CellInfo; import android.telephony.CellInfoGsm; import android.telephony.CellInfoLte; import android.telephony.CellInfoWcdma; import android.telephony.CellInfoNr; import android.telephony.NeighboringCellInfo; import android.telephony.PhoneStateListener; import android.telephony.SignalStrength; import android.telephony.TelephonyManager; import android.telephony.gsm.GsmCellLocation; import android.telephony.CellIdentity;

Fig. 4. Database structure – User profile

A Project Overview: The Implementation of a Native Android App

121

Lastly, MySQL (see Fig. 4) was chosen as the database1 for both applications, for its reliability and efficiency [17]. The Communication between the Android app and the server was facilitated through secure RESTful APIs [16, 18]. The following APIs, which are sets of rules and protocols that allow different software entities to communicate with each other. Here’s a brief description of each endpoint and its purpose used: • POST addUser: This endpoint is responsible for adding a new user to the system. By receiving user details through a POST request, it stores the new user’s information. • POST signIn: This API endpoint manages user authentication. When a user attempts to log in, this endpoint validates the provided credentials. • GET getUsers: This endpoint retrieves the list of all users, usually for administrative purposes or to provide an overview. • GET getUsersByYear: This endpoint allows the retrieval of users filtered by a specific year. This can be useful for analyzing user data over time. • GET getSignalDataByAllTime: Retrieves all signal data, possibly for comprehensive analysis or reporting. • GET getSignalDataByHour: This endpoint fetches signal data based on an hourly breakdown, which may be useful for real-time analysis or performance monitoring. • GET signOUT: This is used to log a user out of the system, terminating their active session. • GET getUserById: Retrieves specific user details based on the provided user ID, allowing for individual user analysis or profile viewing. • PUT updateUser: This endpoint is used to update existing user information. It accepts the new user data and updates the corresponding record. • DEL deleteUser: Used to delete a user from the system, this endpoint removes the specified user record. These APIs, taken together, form the core interaction model for managing users, authentication, and signal data within the application. They enable structured communication between different components of the system, such as the Android application and the server, and ensure that data can be effectively retrieved, stored, and manipulated. 3.4 Testing This stage of this methodology was an in-depth testing phase, examining the Android app’s functionality and performance both in Android Studio’s built-in simulator and across various devices from Samsung, Xiaomi, Huawei, and OnePlus. Through extensive testing, the app proved its reliability across different devices and its readiness to provide crucial mobile network information to users. The app showcases details about communication between the user’s device and nearby mobile antennas, supporting GSM, UMTS, LTE, and NR technologies with varying compatibility based on the Android version. GPS activation is required for specific functionalities. 1 Same database for Android App and Server-Side admin panel.

122

G. Roussos and P. Nicopolitidis

The application comprehensively displays and gather information pertinent to various aspects of mobile network functionality concerning to Mobile Operator, Network type, Signal Level, Power, Voice Network, Data Network, Country, City, Signal Duration, Device Model, MCC, MNC, TAC, PCI, ECI, EARFCN, BW, RSRP, RSRQ and TA and other user profile data (see Fig. 5).

Fig. 5. Android App screens in a real testing environment (mobile device)

For example, as mentioned, a critical parameter known as RSRP2 is utilized to determine the signal strength. As the accompanying image illustrates, beyond the mere categorization of signal type, RSRP provides nuanced insights into the quality of connection. This value, always measured in negative dBm, reveals the strength of the LTE reference signals across both Broadband and Narrowband spectrums [1, 12]. According to next figure (see Fig. 6), the closer the RSRP value is to zero, the stronger the signal, thus serving as a precise metric for evaluating whether the detected LTE signal by a smartphone is deemed good or bad. Lastly, the web-based platform underwent evaluation within a live web server environment, providing valuable insights into the app’s performance and usage under realworld conditions (see Fig. 7). All user and phone data are secured stored in the database (see Fig. 8).

2 RSRP - a number that identifies the received signal strength received in an LTE cellular network.

A Project Overview: The Implementation of a Native Android App

Fig. 6. RSRP – Signal Strength

Fig. 7. The PHP Laravel Web-based Platform for Real-time Monitoring and Analysis

Fig. 8. Phone Statistics stored in database

123

124

G. Roussos and P. Nicopolitidis

3.5 Maintenance Regarding maintenance, the development of the Android app and web admin panel marks just the beginning of a complex, ongoing process. As technology advances and security settings become increasingly stringent and aligned with the operating system, there arises a continual need for maintenance. This ensures that the application remains compatible with current technological trends and complies with ever-changing requirements, such as support for new network protocols, new devices, new OS e.g. from Android 11 to 12 and then to 13. Additionally, the integration of secured APIs and the utilization of new and improved cryptographic techniques are crucial for data safety [8, 16]. These advancements require constant vigilance, updates, and adaptability to ensure that the app remains functional, secure, and aligned with the latest industry standards and user expectations [17, 18].

4 Results, Conclusions and Recommendations The system, currently in its testing phase, has demonstrated promising preliminary results. The Android app successfully captures the required signal data in real-time, and the web-based admin panel efficiently processes and manages this data, providing a platform for insightful analysis. In the full implementation, we anticipate this project to provide an accessible and practical tool for researchers, students, and industry professionals interested in real-time monitoring and analysis of wireless communication networks. The tool could potentially help identify patterns, discrepancies, or areas of improvement within these networks. Based on the evidence collected thus far, the integration of an Android application with a PHP Laravel web-based admin panel can indeed facilitate real-time data collection and analysis in wireless communication networks. This system could pave the way for more hands-on research in this domain, promoting a deeper understanding of the practical aspects of mobile network signals and their management. While the conclusions at this stage are preliminary, we anticipate that the final outcomes will reinforce these initial findings, providing a valuable contribution to the current understanding of wireless communication networks. Our recommendation, therefore, is for further research and development in this sphere, especially given the continually evolving nature of mobile network technology.

References 1. Papadimitriou, G.I., Pomportsis, A.S., Nicopolitidis, P., Obaidat, M.S.: Wireless Networks. Wiley, Hoboken (2003) 2. Obaidat, M.S., Nicopolitidis, P.: Smart Cities and Homes: Key Enabling Technologies. Morgan Kaufmann (2016) 3. Lauridsen, M., Rodriguez, I., Mikkelsen, L.M., Gimenez, L.C., Mogensen, P.: Verification of 3G and 4G received power measurements in a crowdsourcing Android app. In: 2016 IEEE Wireless Communications and Networking Conference, Doha, Qatar, pp. 1–6 (2016) 4. Zeqiri, R., Idrizi, F., Halimi, H.: Comparison of algorithms and technologies 2G, 3G, 4G and 5G. In: 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), Ankara, Turkey, pp. 1–4 (2019)

A Project Overview: The Implementation of a Native Android App

125

5. Ferreira, D., Kostakos, V., Beresford, A.R., Lindqvist, J., Dey, A.K.: Securacy: an empirical investigation of Android applications’ network usage, privacy and security. In: Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks, pp. 1–11 (2015) 6. Zhao, N., Wu, M., Chen, J.: Android-based mobile educational platform for speech signal processing. Int. J. Electr. Eng. Educ. 54(1), 3–16 (2017) 7. Zou, L., Javed, A., Muntean, G.-M.: Smart mobile device power consumption measurement for video streaming in wireless environments: WiFi vs. LTE. In: 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Cagliari, Italy, pp. 1–6 (2017) 8. Nicopolitidis, P., Misra, S., Yang, L.T., Zeigler, B., Ning, Z. (eds.): Advances in Computing, Informatics, Networking and Cybersecurity: A Book Honoring Professor Mohammad S. Obaidat’s Significant Scientific Contributions, vol. 289. Springer, Cham (2022). https://doi. org/10.1007/978-3-030-87049-2 9. Han, S., Kang, T., Seo, J.: Smartphone application to estimate distances from LTE base stations based on received signal strength measurements. In: 2019 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), JeJu, Korea (South), pp. 1–3 (2019) 10. Liu, B., Lyu, X., Fan, W.: Analysis of 5G signal for radar application. J. Phys. Conf. Ser. 2356(1), 012027 (2022) 11. Liotou, E., Passas, N., Merakos, L.: The emergence of experience packages in the 5G era. In: IEEE 5G Tech Focus, vol. 1, no. 3 (2017) 12. Salih, A.A., Zeebaree, S.R., Abdulraheem, A.S., Zebari, R.R., Sadeeq, M.A., Ahmed, O.M.: Evolution of mobile wireless communication to 5G revolution. Tech. Rep. Kansai Univ. 62(5), 2139–2151 (2020) 13. Balaji, S., Murugaiyan, M.S.: Waterfall vs. V-model vs. agile: a comparative study on SDLC. Int. J. Inf. Technol. Bus. Manag. 2(1), 26–30 (2012) 14. Alshamrani, A., Bahattab, A.: A comparison between three SDLC models waterfall model, spiral model, and Incremental/Iterative model. Int. J. Comput. Sci. Issues (IJCSI) 12(1), 106 (2015) 15. Bean, M.: Laravel 5 Essentials. Packt Publishing Ltd. (2015) 16. Chen, X., Ji, Z., Fan, Y., Zhan, Y.: Restful API architecture based on Laravel framework. J. Phys. Conf. Ser. 910, 012016 (2017) 17. Laaziri, M., Benmoussa, K., Khoulji, S., Kerkeb, M.L.: A comparative study of PHP frameworks performance. Procedia Manuf. 32, 864–871 (2019) 18. Bagwan, M.K., Ghule, P.S.: A modern review on Laravel-PHP framework. IRE J. 2(12), 1–3 (2019) 19. Roussos, G., Aliprantis, J., Alexandridis, G., Caridakis, G.: Augmented reality in primary education: adopting the new normal in learning by easily using AR-based Android applications. In: Proceedings of the 26th Pan-Hellenic Conference on Informatics (PCI 2022), Athens, Greece, 25–27 November 2022. ACM, New York (2022)

Identifying the Most Mobile Content Sections Within a Course of Biosensors from the Last Decades Cristian Ravariu1(B)

, Gabriel Dima1 , Musala Sarada2 , Avireni Srinivasulu3 , and Bhargav Appasani4

1 Facultatea de Electronica, Universitatea Nationala de Stiinta si Tehnologie Politehnica

Bucuresti, Street Splaiul Independentei 313, 006042 Bucharest, Romania [email protected] 2 Department of Electronics and Communication Engineering, Vignan’s Foundation for Science Technology and Research deemed to University, Guntur, AP, India 3 Department of Electronics and Communication Engineering, Mohan Babu University, Tirupati, AP, India [email protected] 4 Department of Electronics Engineering, Kalinga Institute of Industrial Technology, Bhubaneswar, India [email protected]

Abstract. Biosensors have experienced a fulminant development in the last decades. Therefore, the content of a Biosensors and Electronic Bio-devices course held at the Master of a technical faculty in Romania had to keep up with the state of the art. The chapters with the most mobile content will be highlighted in this study. Motivation of this study starts from the accelerated mobile content in the last two decades, on all directions: receptors, analytes, transducer, technology, co-integration techniques. The revealed goals have to be connected to the actual research fields of the biosensors, transistor-based. Collaboration among institutions seem to be the next solution for overdosed courses, due to the coexistence of high development and poor financing of these interdisciplinary fields, like biosensors and electronic devices. Keywords: Biosensors · Bioelectronics · Mobile Content

1 Introduction The unprecedented development of biosensors has pushed their manufacture in clean rooms [1], in the case of using transducers from micro-nano electronics. On the other hand, the nowadays electronics is dominated by the MOS transistor reaching the 2 nm technological node, besides to all its related nano-transistors [2–4]. The main advantages are: the integrated biosensors have small size from few millimeters up to ten nano-meters, can work with very small amounts of biological samples and have a long lifetime. Any biosensors course must contain sections regarding the internal sub-components and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 126–133, 2024. https://doi.org/10.1007/978-3-031-56075-0_12

Identifying the Most Mobile Content Sections Within a Course of Biosensors

127

general detection principles, such as the class of analytes, the types of specific receptors and the variety of transducers. Therefore, this article will address the mobile content from the last two decades of a course entitled “Biosensors and Nano-Electronics of Cells” (BioNEC), held at the Micro-Nanoelectronics Master (MNM), from the Faculty of Electronics in Bucharest, Romania. Although Enzyme-Field Effect Transistors (EnFETs) are still one of the most widespread representatives [5], new types of receptors are taking the place of enzymes: the antibodies, aptamers and natural receptors [6]. The latter pave the way of label-free and direct measurements. Until 2015 the chapter about the Nano-Electronics of Cells (NEC) seemed to belong rather to a medicine faculty than an electronics faculty. Since 2018, the NEC part founds a serious demand for electronics and IT engineers, through few important international job portals, such as LinkedIn, Research-gate or Talent. The posted jobs required designers capable to model the currents through the ionic nano-channels in cell membranes, task that was associated with the modeling of currents through nano-transistors of various types, having the same length, as the trans-membrane channels (e.g. 10–40 nm as common lengths in MOS devices or nano-devices). Similar applications like this can be extended to a huge number of cells for the drugs development, providing many jobs [7–9]. Students already have received these new changes and have presented them in their annual reports at MNM program. This is one of the many examples of adapting content to the requirements of the labor market, which will be discussed in this article.

2 Content of the Course 2.1 Short History In 2005, a proposal for the BioNEC course was highlighted at the ICL Conference [10], while two months later it also received approval to be taught at the MNM program from Polytechnic University of Bucharest, Romania, where it has been continuously taught till present. Over the years, some chapters have cemented themselves, like physics of liquids and main transistors with biomaterials, while others have had to modify their content, like new receptors, new analytes, aptasensors [11] or NEC part [12]. 2.2 Methodology The methodology to emphasize those chapters with the most mobile content are the statistical data from the biosensor’s literature, and the annual statistics from the own studies of students to justify the change in teaching practice. Students must familiarize themselves with the types of Biomaterials Field Effect Transistors (BioFET) currently available, with the receptor and transducer elements, as well as with their manufacturing technology. Completely new sub-sections have been inserted over the years to cover: a separate lecture about all analyte classes, a separate lecture on viruses and their detection techniques, and the axiomatic definition of the laws that govern the living matter.

128

C. Ravariu et al.

2.3 List of the Main Chapters The main chapters of the initial BioNEC course from 2005 [10] are represented in Fig. 1, left column. From 10 to 10 years, the chapters are represented in Fig. 1, next columns, taking into account the following abbreviations: Introduction in Bioscience (IB), Elements of Liquid Physics (ELP), Electro-Chemistry for Sensors (ECS), Biosensors Generalities (BG), Receptors (R), Transducers (T), BioFET, NEC. In 2014 some new preparatory chapters were added: Specific Technologies for Biosensors (STB) closely related to BG, Preparatory chapter for nano-electronics of cells (Pnec) speaking about the logic and axioms inside the living matter. But the maximum dynamic of the course content occurred in the last 5 years.

Fig. 1. A figure caption is always placed below the illustration.

There are few flag journals of biosensors: Biosensors and Bioelectronics Elsevier, Biosensors MDPI and Sensors. They promote novelties in the field of receptor elements, such as the use of new enzymes [13] or the ion channels, especially those that activate the G protein for the detection of a wide class of neurotransmitters [14]. But the technologies are still complicated and expensive. Alternatively, aptamer receptors created from nucleic acids have been recently developed and have the advantage of direct and simple detection based on the lock-key principle [15]. Obviously, the specific technologies for biosensors are annually reported and represent 90% of the published papers. But in a course, like BioNEC, limited to 3 h per week, STB was sacrificed, due to its excessively dynamic domain, useless to electronics students from the point of view of chemical functionalization/optimization agents. However, the literature indicate an increased interest for co-integration of microelectronic devices and biosensors [16]. Micro-nano-devices, Microfluidics or Micro-Electro-Mechanical Systems (MEMS) offered promising applications in several biodetection techniques, such as blood chemistries, enzyme sensors,

Identifying the Most Mobile Content Sections Within a Course of Biosensors

129

immunoassays, nucleic-acid amplification tests, viral detection [13–16]. For instance, a glucose oxidase enzymatic membrane can be integrated in the Gate space of a modified MOS transistor, Fig. 2a. The students must learn to design a technological flow, based on their knowledge of Microelectronic technologies and brief STB chapter, having an explicit example in [5, 17]. They have to reach to the final target: the translation curve, usually by simulation or modeling [18] and practically by experiment (e.g. the glucose detection curve, Fig. 2b).

Fig. 2. (a) The practical set-up for a drop of glucose on the sensitive area of an En-FET biosensor - design and produced device; (b) the recorded calibration curve, expressed by the dependence of the output electrical transduced signal (Drain current) on the input biological signal (Glucose concentration); experimental points were picked up and adapted from curves given in [17].

Hence, in 2023 some major changing must be addressed: STB must be diminished, ECS must move its content from Electro-Chemistry toward Elements of Genetics (EG) [14, 15], ELP must add new sub-sections about Electro-Phoresis and Electro-Osmosis (EP-EO), a separate chapter about Analytes (A) and an applicative section about Proteins Viruses Bacteria Detection Experiments (PVBDE) must be added, while the BioNEC curricula becomes very crowding. The optimal solution is to split the BioNEC discipline in two separate disciplines: Integrated Biosensors, starting from the enhanced ELP with EP-EO, up to BioFET transistors and a second discipline - Nano-electronics of Cell, starting from IB up to NEC, including Proteins Viruses Bacteria Detection Experiments (PVBDE) and Organ On Chip (OC) [19], a content that was considered by the students important for the next future, through their annual reports.

3 The Dynamic Content over Time 3.1 Results Over the years the number of hours allocated to course and application suffered changes. Hence, we will continue to refer to the percentage weight that each chapter had in relation to the entire content of the BioNEC discipline, Fig. 3a. The student impact is measured

130

C. Ravariu et al.

by the percentage numbers of homework freely chosen by students at the end of the year, from the total number of homework proposed at the beginning of the year, averaged over last 10 years, Fig. 3b. For instance, Fig. 3a can estimate minimum/maximum volume allocated to a chapter inside the entire discipline. Section T about transducers had the maximum weight in teaching over entire course with 21%, in 2005, reaching in 2023 the weight of 10%, due to the transfer of nanodevices study to other courses. The project topics chosen by the students in 2005 showed a surprising interest for the NEC section with 19%, but the maximum interest of 32% was directed to the general knowledge from IB section, where they had the freedom to choose any subject from a wider field of biosciences. In 2014, a transient dynamic can be seen, while in 2023, students not only have chosen the BioFET as their top section, with 20%, but also they have highlighted new fields of interest themselves, such as Proteins Viruses Bacteria Detection Experiments (PVBDE) with 19%, and Organs On Chip (OC) with 2%, sections that have to be included in the next courses.

Fig. 3. (a) The percentage weight that each chapter had in relation to the entire content of the course; (b) the students impact is measured by the percentage numbers of homework freely chosen by students from all course topic.

3.2 International Connections and Future Predictions So far, a much expanded horizon of the development of biosensors has been highlighted. Unfortunately, the expansion of an condensed course also requires a lot of material resources. In order to be able to split the BioNEC course in two new separate courses, it requires additional governmental funds and approvals. There is not always the financial availability for multiplying new courses, rather the opposite happens. Therefore, an auspicious solution would be the expansion through collaborations with other institutions, with industrial actors, and especially international connections. Collaboration with foreign partners can result in organizing activities for benefits of both parties. For instance,

Identifying the Most Mobile Content Sections Within a Course of Biosensors

131

bilateral Erasmus agreements, or IEEE events, or Research projects will allow students from two or more countries to attend mini-conferences, special student sessions, summer schools, intensive courses, workshops, with the role of supplementing those knowledge that they did not have time, during the hours accredited by their usual programs. We are almost convinced that every professors, if they were asked, would find many sections from their expertise area, which they would like to teach their students, but the allocated time is no longer enough. The procedure of international collaborations is also useful for students, and especially for any learners of any age, because they themselves have the freedom to choose which extra-courses or specializations they need. Figure 4 presents the main fields required by a Biosensors and Bioelectronic mobile content, in connections with adjacent topics.

Fig. 4. Interconnections among Biosensors, Bioelectronics and Nanoelectronic of cell - NEC and adjacent topics.

4 Conclusions New branches in bioelectronics are developing rapidly. Biosensors and Bioelectronics courses must be permanently upgraded with the latest trends, replacing useless chapters of outdated technologies or classical electrochemistry, with the new challenges about

132

C. Ravariu et al.

integration of biosensors at the nanometric scale, bringing the chips with transistors as close as possible, up to physical contact, to cells, viruses or organelles. The paper presented a brief review over a discipline called “Biosensors and NanoElectronics of Cells” held at the Micro-Nanoelectronics Master from the Faculty of Electronics in Bucharest, Romania. The last years of advances in Biosensors, overcrowded this discipline, so that a split in Integrated Biosensors and Nanoelectronics of Cell, as two separate discipline would be necessary. An alternative solution to expand and permanently upgrade this kind of discipline may be the international and inter-partnership collaboration. Occasional summer schools, workshops, IEEE events sometimes under regional Chapters auspices, can fill the gaps in some curricula, being useful for students, learners of any age, till the large public.

References 1. Mirzaei, M., Sawan, M.: Microelectronics-based biosensors dedicated to the detection of neurotransmitters: a review. Sensors 14, 17981–18008 (2014) 2. Stern, E., et al.: Label-free immunodetection with CMOS-compatible semiconducting nanowires. Nature 445, 519–522 (2007) 3. Ravariu, C., Rusu, A., Udrea, F., Ravariu, F.: Simulation results of some diamond on insulator nano-MISFETs. Diam. Relat. Mater. 15(2), 777–782 (2006) 4. Ravariu, C., Banes, V., Enescu, A., Vasile, R.: Mobile models for biosensors with diffusion layer through enzyme receptor. In: Auer, M.E., Tsiatsos, T. (eds.) IMCL 2021. LNNS, vol. 411, pp. 962–969. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96296-8_87 5. Ravariu, C., Manea, E., Babarada, F.: Masks and metallic electrodes compounds for silicon biosensor integration. J. Alloys Compd. 697(3), 72–79 (2017) 6. Dalirirad, S., Steckl, A.J.: Lateral flow assay using aptamer-based sensing for on-site detection of dopamine in urine. Anal. Biochem. 596, 113637–113642 (2020) 7. Ravariu, C., Srinivasulu, A., Appasani, B.: Viral invasion flow-chart for pathogens with replication target in a host cell. In: Zeshan, F., Ahmad, A. (eds.) Recent Advancements in Smart Remote Patient Monitoring, Wearable Devices, and Diagnostics Systems, Chapter 2, pp. 33–53. IGI Global Publisher, Hershey (2023) 8. Neuvoo. https://ch.talent.com/w/rnd-templateWhitepage/?id=2d2d64cd69e6&utm_med ium=email&source=neuvoo-email&context=email&user_id=43b8f8a33d2344597637f383 59f9a867&refid=70c3c39a2740. Accessed 07 June 2023 9. Svoboda, J., Passmore, C.: The strategies of modeling in biology education. Sci. Educ. 22, 119–142 (2013) 10. Ravariu, C., Rusu, A., Ravariu, F., Babarada, F.: Bio-electrical-engineering: a strategic course with an European opening. In: Auer M. (ed.) 8th International Interactive Computer Learning Conference, ICL2005, Villach, Austria, pp. 1–8 (2005) 11. Liu, S., et al.: A novel label-free electrochemical aptasensor based on graphene–polyaniline composite film for dopamine determination. Biosens. Bioelectron. 36(1), 186–191 (2012) 12. Ravariu, C., Sevcenco, A., Auer, M., Ionescu-Tirgoviste, C., Ravariu, F., Babarada, F.: The BioNEC platform. In: Auer, M. (eds.) International Conference on Internet Computer Learning ICL, Villach, Austria, pp. 1–4 (2008) 13. Pimpilova, M., Kamarska, K., Dimcheva, N.: Biosensing dopamine and L-epinephrine with Laccase (Trametes pubescens) immobilized on a gold modified electrode. Biosensors 12, 719 (2022). https://doi.org/10.3390/bios12090719

Identifying the Most Mobile Content Sections Within a Course of Biosensors

133

14. Park, S.J., Song, H.S., Kwon, O.S., et al.: Human dopamine receptor nanovesicles for gatepotential modulators in high-performance field-effect transistor biosensors. Sci. Rep. 4, 4342 (2014). https://doi.org/10.1038/srep04342 15. Chen, X., et al.: Ultratrace antibiotic sensing using aptamer/graphene-based field-effect transistors. Biosens. Bioelectron. 126, 664–671 (2019). https://doi.org/10.1016/j.bios.2018. 11.034 16. Tsutsumi, J., Turner, A.P.F., Mak, W.C.: Precise and rapid solvent-assisted geometric protein self-patterning with submicron spatial resolution for scalable fabrication of microelectronic biosensors. Biosens. Bioelectron. 177, 112968 (2021). https://doi.org/10.1016/j.bios.2021. 112968 17. Ravariu, C., Parvulescu, C.C., Manea, E., Tucureanu, V.: Optimized technologies for cointegration of MOS transistor and glucose oxidase enzyme on a SI-wafer. Biosensors 11(12), 497 (2021). https://doi.org/10.3390/bios11120497 18. Ravariu, C., Srinivasulu, A., Mihaiescu, D.E., Musala, S.: Generalized analytical model for enzymatic BioFET transistors. Biosensors 12(7), 474 (2022). https://doi.org/10.3390/bios12 070474 19. Low, L.A., Mummery, C., Berridge, B.R., Austin, C.P., Tagle, D.A.: Organs-on-chips: into the next decade. Nat. Rev. Drug Discov. 20, 345–361 (2021)

Work-in-Progress: SYNERGIA, Towards an Online Communication and Collaboration Interactivity Hippokratis Apostolidis(B) , Spyridon Armatas, George Tsantikis, and Thrasyvoulos Tsiatsos Aristotle University of Thessaloniki, Thessaloniki, Greece {aposti,sarmatas,gtsantiki,tsiatsos}@csd.auth.gr

Abstract. During the period of pandemic, teleconference platforms and various online tools became critical options for working or educational environments to continue their activities. Nowadays, in many cases there is a mixed environment (online and onsite) taking advantage of the development of various online platforms. This situation revealed the need of the development of dynamic and immersive online communication tools. Thus, the field of teleconference tools is a very fast-growing area. The most known teleconference tools provide basic collaboration features such as voice and video communication screen sharing and text chat. The proposed application tries to provide advanced technological support to various distant collaboration activities, especially in the area of e-learning. It focusses on interactive communication features that can significantly reduce the distance between the persons participating in a web conference. There is a considerable gap between web conference and web collaboration. This article is proposing a web conference tool which is focused to support online collaboration and interactivity trying to contribute to an attempt to bridge this gap. Keywords: web conference · web collaboration · e-learning · interactive communication

1 Introduction Web conferencing expanded in many work and educational environments due to the COVID-19 pandemic. In many companies the employees were forced to work from home, utilizing the online communications. Besides, the physical presence was prohibited in all kinds of schools and universities. Thus, they continued their lessons utilizing web conference tools. Nowdays, 16% of companies worldwide operate with 100% remote work [1] and 71% of workers are doing their job from their home at least part of the time each week [2]. Moreover, the number of distance learning activities is increasing worldwide, and, in many cases, mixed approaches are adopted combining remote online and on-campus learning [3]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 134–140, 2024. https://doi.org/10.1007/978-3-031-56075-0_13

Work-in-Progress: SYNERGIA, Towards an Online Communication

135

However, technology does not create learning or working processes by itself, it simply supports the educational process or the work to be more efficient. For this reason, the critical point for any technological feature is the human being to whom it is addressed. Moreover, we could state that the future belongs to systems focusing to promote the communication, the interactivity, and the collaboration between their users. There is a significant difference between web conference and web collaboration. Web conferencing allow people from different locations to meet online in real-time. This kind of technology supports mainly online meetings and discussions than working together on a specific task. On the other hand, web collaboration involves teams or people from different locations working together, sharing resources supporting multiple editing and other collaborative features [4]. The aforementioned difference between web conference and web collaboration and the extended use of various web conference platforms in education and working environments revealed several critical issues. Mainly, in educational activities, there is a problematic condition when a teacher is speaking, and the students have closed their cameras and their microphones. In this case, there is no interaction between the instructor and his/her students or between the students themselves. The teacher does not know whether his/her students are even pay any attention to him/her. From the students’ point of view, in many cases they are experiencing virtual learning fatigue or e-learning fatigue [5]. They are feeling that the whole procedure is impersonal to try to conduct conversations or lectures while sitting at home desks looking at a screen. Thus, many attendees are turning their cameras off, and their engagement is significantly decreased or eliminated. Furthermore, this problem exists in working environments too. For example, in the case where a meeting is taking place and someone has turned his/her camera and microphone off, nobody knows if this person is active and is watching the meeting. The above mentioned cases show that in most online web conference platforms there is a lack of adequate interactive tools. This can be explained, because these tools are mainly focused on an efficient telecommunication functioning establishment. Therefore, this research work is proposing “Synergia”, to provide significant features promoting the online interactive working, learning and collaboration. In the following section a literature review is presented. Then, the sections describing the research motivation and the system design and implementation are following. Finally, the conclusions and the future steps are presented.

2 Related Work A web conference is a method of placing calls or holding meetings for multiple people over the Internet. It’s an umbrella term including a wide range of digital conferencing systems and solutions, from video conferencing to screen sharing. It also supports various devices, such as smartphones, laptops, and voice over IP (VoIP) telephones [6]. One of the key technologies that stands behind most modern video conferencing services is Web Real-Time Communication (WebRTC). This is a free and open-source peer to peer (P2P) real time communication utilizing application programming interfaces (APIs) developed jointly by Mozilla, Apple, Microsoft, and Google. It gives browsers the ability to broadcast high-quality voice and data audio in real-time. In addition, P2P

136

H. Apostolidis et al.

enables user’s data to be encrypted, safe and it is able to bypass all the problems associated with plugins [7]. Thus, WebRTC technology is widespread, since it provides end-to-end encrypted P2P communication with audio-visual content and data being transmitted directly from peer to peer. This article is presenting a communication and collaboration tool, which is developed utilizing WebRTC architecture and its APIs. Some of the most popular digital online communication platforms are the following, [8] (Table 1). Table 1. Most popular web communication platforms and their collaboration-communication features Title

Collaboration – Communication Features

Zoom & Zoom Pro • Preferred tool for many to make videoconference due to its well-balanced features • Share screen • Raise hand • Active speaker annotation Google Meet

• • • • • •

Share screen Share digital presentations documents, spreadsheets, or other files, Raise hand Active speaker annotation It includes several reactions for interactivity as like, dislike, smile etc. It includes whiteboard with basic features (drawing and inserting a text box)

Google Classroom • Same as Google Meet • It is part of “Google for Education”, and it is targeted to teachers and students Microsoft-Teams

• It provides share screen, file sharing and whiteboard with basic features (drawing and inserting text box) • It provides connectivity with many social and other communication tools

Cisco Webex

• • • • • •

Jitsi meet

• Screen sharing • Recording and streaming • Dashboard screen with participants’ face expression display during the meeting • In the same dashboard participants’ speech duration is displayed • Raise hand • Active speaker annotation

Screen sharing Interactive whiteboard Recording transcriptions File sharing, Polling, Connectivity with social platforms

Work-in-Progress: SYNERGIA, Towards an Online Communication

137

Almost all of the aforementioned video conference tools provide a limited number of features to support basic needs of communication and interaction between their attendees. Jitsi meet, seems to consider in more detail with the interactive communication providing historical data about participants’ face expressions and speech duration during the meeting as factors to qualify the collaboration. However, as mentioned in the previous section there is a significant difference between web conference and web collaboration [4]. Moreover, remote collaboration along with the remote communication is a critical factor for team building, well – being and improved productivity outputs for working and learning environments [9]. Camilleri and Camilleri [10] highlights the importance of providing appropriate facilitating conditions to improve perceptions and attitudes toward interactive conferencing software. Moreover, focusing on e-learning, virtual learning environments should be designed in such a way to entice students to engage in collaborative approaches [10, 11]. Besides, in the same article the authors support that COVID-19 has accelerated the educators’ engagement with remote learning technologies [10, 12, 13]. Thus, many instructors adopted synchronous learning methodologies [10, 14], as they were expected to interact with their students in virtual sessions, in real time [10, 15]. However, this research gathered data through a structured questionnaire among 777 students in tertiary education and the results showed that the use of synchronous video conferencing could continue in the foreseeable future as they can easily be used in blended learning approaches. However, a critical factor for their extended use is to promote students’ engagement through advanced educational facilitating conditions provided by interactivity and collaboration tools [10]. In addition, another research work is stating that the adoption of web-conferencing can be promoted by supporting online instructors in the following ways [16]. • Web-conferencing will provide the appropriate features in order to be utilized as online instructional tool. • Web-conferencing will provide the appropriate affordances to support constructively the social and teaching presence in online learning environments. • Overcome barriers of using web-conferencing in online instruction. Thus, considering the aforementioned research works, we can state that there is a critical need for distant collaboration in work and in educational environments. The most known web conferencing tools are providing basic collaboration features. This support is getting insufficient as time passes and the need for more advanced distant collaboration is increasing. The proposed application is focusing on interactive collaboration trying to utilize modern technology features to this direction.

3 Research Motivation The literature review revealed that in most online web conference platforms there is a lack of adequate interactive and collaborative tools. This can be explained, because these tools are mainly focused on an efficient telecommunication functioning establishment. Therefore, this research work is proposing “Synergia”, to provide significant features promoting the online interactive working, learning and collaboration.

138

H. Apostolidis et al.

4 System Design, Architecture and Implementation This article presents a web communication and collaboration tool called “Synergia” (Fig. 1). This system is developed utilizing pure JavaScript and HTML5 as programming languages and is deployed under Node.js web platform. The architecture, design and implementation of the proposed system is based on Web Real Time Communication (WebRTC) technology [17].

Fig. 1. The main screen of Synergia. Enable – Disable a) microphone, b) camera, c) display on camera window, d) share screen, e) draw whiteboard, f) collaborative editing, g) invite teammates, h) raise hand, i) face expression recognition (display corresponding emoji), k) chat, l) display participants, m) leave the conference room.

Synergia initially was based to the open-source web conference tool “quickmeet” [18] and on the open-source web chat tool called “node-chat-app” [19]. Therefore, after combining the basic functionality of the aforementioned tools in one platform, our main effort was focused to further develop various features promoting the web communication and collaboration supported by interactivity. To this direction, we added the following features to promote the collaboration, • Whiteboard providing rich text multi-editing. Every user is aware of the person who is currently editing the text. This development was based on open-source JavaScript library for rich text editor called Quill [20]. • Detection and indication of the user who is speaking, his/her speaking time period. This information is stored to a data base. If a user is speaking a lot, a message is displayed encouraging him/her to allow others to speak too. On the other hand, for users who are rarely speaking a message is displayed encouraging them to participate more actively. This development was based on the open-source JavaScript library called hark [21].

Work-in-Progress: SYNERGIA, Towards an Online Communication

139

• There is a menu option displaying in a pop-up window the amount of time speaking and the variation of face emotions for every participant during a session using Synergia. • In addition, user text messages exchanged between participants are stored to a data base for further processing. Furthermore, we added the following features providing interactivity, • Face emotion expressions recognition showed by relevant emojis displayed on every user camera window. The development of user face expression detection was based on a well-known open-source JavaScript library called face-api.js [22]. • Hand tracking recognising gestures i.e. when a user makes a raises hand gesture, the raise hand symbol is displayed on the user camera window shown by all the participants. This development was based on an open-source library Mediapipe [23]. • In addition, every user can choose whether he/she wants to be displayed by the camera window or not without closing the camera. Thus, if a user has chosen “no camera display” his/her camera window is not displaying him/her, but his/her face emotion expression and his/her gestures are displayed. Therefore, the user is not displayed without spoiling the interaction. The user can choose to stop the function of face emotions expression recognition or the hand tracking for him/herself. • Moreover, we added in chat operation the “user is typing” and “user online presence” recognition showed with specific indications revealed to all the participants.

5 Conclusion and Future Steps The proposed system “Synergia” is aiming to support the online collaboration and interactivity in working and training environments. Our main effort was focused to provide interactive feedbacks to support better communication between participants and self-regulation. Our future steps will focus to implement another feature providing a collaborative software development environment and carry out an evaluation activity in order to assess the proposed system usability, the perceived interactivity provided by the proposed web communication tool, and the quality of collaboration provided by the proposed tool.

References 1. Underwriter Editors. https://underwriter.gr. Accessed 12 Mar 2023 2. Bemer. https://www.megameeting.com/news/benefits-video-conferencing-post-covid-era. Accessed 6 Apr 2023 3. Garment, V.: https://www.parallels.com/blogs/ras/what-is-a-blended-learning-approach/. Accessed 02 Apr 2023 4. Eztalks. https://eztalks.com/video-conference/7-best-cloud-collaboration-tools.html. Accessed 10 May 2023 5. Reed, H.C.: E-learning fatigue and the cognitive, educational, and emotional impacts on communication sciences and disorders students during COVID-19. Perspect. ASHA Spec. Interest Groups, 1885–1902 (2022). https://doi.org/10.1044/2022_PERSP-22-00049

140

H. Apostolidis et al.

6. Fusion Connect. https://www.fusionconnect.com/blog/web-conferencing. Accessed 20 May 2023 7. Suciu, G., Stefanescu, S., Beceanu, C., Ceaparu, M.: WebRTC role in real-time communication and video conferencing. In: 2020 Global Internet of Things Summit (GIoTS), vol. 202, pp. 1–6 (2020). Journal of information technology and its utilization, Volume 4, Issue 1, EISSN 2654–802X 13 8. Dash, S., Samadder, S., Srivastava, A., Meena, R., Ranjan, P.: Review of online teaching platforms in the current period of COVID-19. Indian J. Surg. 18, 512–517 (2021). https://doi. org/10.1007/s12262-021-02962-4 9. Forbes. https://www.forbes.com/sites/forbestechcouncil/2022/08/15/addressing-collabora tion-and-team-building-in-remote-work-environments/?sh=a13486020e28. Accessed 15 June 2023 10. Camilleri, M.A., Camilleri, A.C.: Remote learning via video conferencing technologies: implications for research and practice. Technol. Soc. 68, 101881 (2022). ISSN 0160-791X. https://doi.org/10.1016/j.techsoc.2022.101881 11. Camilleri, M.A., Camilleri, A.C.: The students’ acceptance and use of their university’s virtual learning environment. In: Proceedings of the 2020 11th International Conference on E-Education, E-Business, E-Management, and ELearning, Osaka, Japan, pp. 48–53. ACM (2020) 12. Aguilar, S.J.: Guidelines and tools for promoting digital equity. Inf. Learn. Sci. 121(5/6), 285–299 (2020) 13. Al Murshidi, G.: Videotaped teaching and learning methodology–an experiential learning and action research approach. J. Int. Educ. Bus (2020) 14. Szymkowiak, A., Melovic, B., Dabic, M., Jeganathan, K., Kundi, G.S.: Information technology and Gen Z: the role of teachers, the internet, and technology in the education of young people. Technol. Soc. 65, 101565 (2021) 15. EUA: Covid-19 and Universities European University Association, Brussels, Belgium (2020) 16. Sun, Y., Beriswill, J., Allen, M.: Adopting web conferencing in online teaching: a perspective from logistic regression. Int. J. Distance Educ. Technol. 20(1), 2022 (2022) 17. WevRTC. https://webrtc.github.io/webrtc-org/architecture/. Accessed 10 May 2021 18. QuickMeet. https://github.com/i-aryan/quickmeet. Accessed 6 May 2021 19. Note-Chat-App. https://github.com/ashikmahmud1/node-chat-app. Accessed 10 May 2021 20. Quill. https://quilljs.com/. Accessed 10 May 2023 21. Otalk/Hark. https://github.com/otalk/hark. Accessed 26 Apr 2023 22. Face-api.js. https://justadudewhohacks.github.io/face-api.js/docs/index.html. Accessed 21 Nov 2021 23. Mediapipe. https://developers.google.com/mediapipe. Accessed 20 Dec 2022

The Development of Interdisciplinary Digital Learning Platform to Advance Digital Learning Strategic Framework Fei Geng(B) and Daniel D’Souza W Booth School of Engineering Practice and Technology, McMaster University, Hamilton, ON L8S 4L8, Canada [email protected]

Abstract. The field of biotechnology and biomanufacturing is experiencing rapid advancements, necessitating the provision of quality learning materials for students to keep pace with the evolving industry. Digital learning platforms have emerged as indispensable tools, offering convenience and flexibility for learners. However, the abundance of unverified digital resources presents a significant challenge, contributing to the dissemination of inaccurate information and exacerbating the current misinformation pandemic. Moreover, the integration of AI tools, such as ChatGPT, adds complexity as students often lack a comprehensive understanding of these tools’ functioning and limitations. To address these issues, this research paper proposes the development of a verified digital learning platform catering specifically to biotechnology and biomanufacturing education. The platform ensures the reliability and accuracy of learning materials by curating and validating content from reputable sources, creating a reliable resource hub for self-learning and skill development. In addition, the paper aims to cultivate critical thinking skills among students, enabling them to effectively evaluate and utilize AI tools like ChatGPT in their learning journey. Equipping students with the ability to discern the reliability of information from AI tools is crucial to fostering a discerning approach towards digital resources. Keywords: Biotechnology and biomanufacturing education · Digital learning platform · ChatGPT · Critical thinking skills · Skill development

1 Introduction 1.1 Rationale The field of biotechnology and biomanufacturing is evolving rapidly, and interdisciplinary knowledge plays a crucial role in its advancement. To keep up with the demands of this field, students require access to quality learning materials that facilitate selflearning and skill development. Digital learning platforms have emerged as essential tools in education, providing convenience and flexibility for students. However, the abundance of unverified digital resources poses a significant challenge, leading to the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 141–150, 2024. https://doi.org/10.1007/978-3-031-56075-0_14

142

F. Geng and D. D’Souza

dissemination of inaccurate information and contributing to the current misinformation pandemic. Furthermore, the utilization of AI tools, such as ChatGPT, adds another layer of complexity, as students often lack a comprehensive understanding of these tools’ functioning and limitations. The goals of the program also align with the industry needs as the program seeks to bridge the collaboration between academia, industry, and government. By offering a space for all of these entities to come together to train the future leaders of the biomanufacturing industry, the collaboration fosters innovation, growth, and trust that is essential to catapult the biomanufacturing industry. The program also addresses a changing paradigm in the way education is offered by giving students the opportunity to gain hands-on experience while still part of the program. In a very hands-on field such as biomanufacturing, it is important that such experiential skills are developed in tandem with the foundational learning that students gain as part of the program, as experiential learning and theoretical learning can complement each other well. Moreover, by involving industry partners in the development and execution of the Master’s program, this relationship offers the industry partners the opportunity to play a direct role in shaping the learning of students. This enables them to ensure students are learning the skills and competencies they require. At the same time, this partnership also benefits students and the program as they are able to leverage the very individuals to ensure their learning and program is relevant to the current needs of the biomanufacturing industry. Specific skills and competencies that the program aims to develop in students include technical background in biomanufacturing, bioprocessing, and biopharmaceutical theory with experiments in areas including biocatalysis, cGMP manufacturing, and regulatory affairs. Students also engage in program-long projects conducted in a simulated cGMP environment, which provides students with insights into areas such as advanced cell therapeutics, novel vaccine development and the standardized processed involved in biomanufacturing. In addition to the biomanufacturing specific competencies and skills, students also develop professional skills in effective oral, electronic, and written communications. The availability of unverified digital learning resources and the potential inaccuracies associated with AI tools hinder effective and reliable learning experiences in biotechnology and biomanufacturing education. Students are at risk of receiving misleading or incorrect information, which can undermine their knowledge acquisition and critical thinking skills. The absence of a robust and verified digital learning platform compounds this problem, as students struggle to navigate through the vast amount of available information. As a result, there is a pressing need to develop an innovative platform that not only provides easy access to quality learning materials but also fosters critical thinking and open-ended thinking skills. 1.2 The Targeted Program of Digital Learning Platform The digital learning platform aims to enhance the learning experience and outcomes in the biotechnology undergraduate program and Biomanufacturing Master’s program in Faculty of Engineering at McMaster University. This program is designed to equip students with the interdisciplinary knowledge and skills required to succeed in the biotechnology industry. The undergraduate program typically enrolls approximately 50 students per year, and Biomanufacturing Master’s program enrolls approximately 30 students per

The Development of Interdisciplinary Digital Learning Platform

143

year. Both the programs are delivered through a combination of lectures, laboratory sessions, and hands-on projects, facilitated by a team of dedicated teaching staff, including professors, industry experts, and lab technicians. By addressing the challenges of unverified digital resources and the limitations of AI tools in biotechnology and biomanufacturing education, this research has significant implications for student learning. The development of a verified digital learning platform ensures that students have access to reliable and accurate learning materials, enhancing their comprehension and understanding of interdisciplinary concepts. By promoting critical thinking skills, students gain the ability to critically evaluate and validate information from AI tools, thus fostering informed decision-making and mitigating the risks associated with misinformation. Moreover, fostering open-ended thinking skills equips students with the necessary attributes to excel in their work in Biotechnology Undergraduate Program or in the Biomanufacturing Master’s Program, enabling them to tackle complex problems, adapt to new technologies, and contribute to the field of biotechnology. 1.3 Priority Areas in the Development of Digital Learning Platform The rationale for this research stems from the rapid advancements in technology and the increasing importance of interdisciplinary knowledge in the field of biotechnology. As the field evolves, it becomes crucial for students to develop critical thinking and open-ended thinking skills to tackle complex challenges and adapt to new technologies. However, the abundance of unverified digital learning resources and the limitations of AI tools, such as ChatGPT, pose a significant challenge to student learning. The research seeks to address this need by developing a verified digital learning platform that promotes critical thinking and open-ended thinking skills, providing students with reliable and accurate learning materials to enhance their understanding and application of biotechnology concepts. This research aligns with current research in the field of teaching and learning, particularly in the areas of digital education, self-directed learning, and interdisciplinary approaches. Relevant research indicates that: 1) Digital learning platforms can improve access to resources, foster engagement, and enhance students’ learning experiences [1]. 2) Self-directed learning empowers students to take ownership of their education, develop critical thinking skills, and cultivate lifelong learning habits [2–5]. 3) Interdisciplinary approaches can promote deeper understanding, creativity, and innovation by encouraging students to connect knowledge across different disciplines [6, 7].

2 Implementation of Digital Learning Platform 2.1 Purpose and Research Questions The purpose of this research is to enhance the Biotechnology Undergraduate Program and Biomanufacturing Master’s Program through the development and implementation of a verified digital learning platform that promotes critical thinking and open-ended thinking

144

F. Geng and D. D’Souza

skills. The research question guiding this research is: How does the integration of a verified digital learning platform impact student learning outcomes and the development of critical thinking and open-ended thinking skills in the undergraduate and graduate program? This research aligns with current research in the field of teaching and learning, particularly in the areas of technology-enhanced learning, digital learning platforms, and the development of critical thinking skills. Research studies on the effectiveness of digital learning platforms in enhancing student learning outcomes and the importance of critical thinking in biotechnology and biomanufacturing education provide a foundation for this research. 2.2 Research Design The research design for this research involves a mixed-methods approach. The research is conducted in several phases:

Fig. 1. The development of the digital learning platform in Biomanufacturing and Biotechnology Curriculum. The platform development of digital learning is achieved via three phases platform design & development, quality control mechanism, and collaboration & engagement.

Phase 1: Platform Development and Content Curation

The Development of Interdisciplinary Digital Learning Platform

145

During the first phase, the research team collaborates to develop the verified digital learning platform (Fig. 1) This involves curating high-quality learning materials, designing interactive exercises and assignments, and incorporating features that promote critical thinking and open-ended thinking skills. Phase 2: Pilot Testing and Evaluation In this phase, a pilot implementation of the digital learning platform (Fig. 1) is conducted with a subset of students in the 6 courses in Biotechnology Undergraduate Program and 3 courses in Biomanufacturing Graduate Program (Table 1). Data is collected through surveys, interviews, and assessments to evaluate the impact of the platform on student learning outcomes and the development of critical thinking skills. Phase 3: Collaboration and Engagement Based on the findings from the pilot testing phase, the digital learning platform will be applied for collaboration and engagement (Fig. 1). Feedback from students and faculty is incorporated to improve usability and address any identified challenges. The platform is then fully implemented in the Biomanufacturing program. Table 1. The pilot implementation of digital learning platform in the courses from Biotechnology Undergraduate Program and Biomanufacturing Graduate Program Course

Why this course is being selected How the courses benefit from a digital platform

Biotechnology Undergraduate Program Courses BIOTECH 2CB3: Cell Biology

Cell biology is often considered the fundamental course in student’s academic biotechnology journeys, as an understanding of cell-related principles is key for further education in biotechnology

With cell biology being a field that is constantly evolving, the platform can ensure instructors and students have access to a platform where they can share and receive the latest understanding in this field

BIOTECH 2OC3: Organic Chemistry

Organic chemistry is often considered one of the most challenging courses for undergraduate students, yet a solid understanding is important for understanding atomic level interactions in biologically important materials

The variety of resources in organic chemistry provided by the platform can ensure students have access to learning materials that best suit their learning styles

BIOTECH 3B03: Industrial Biotechnology

This course examines the applications of biotechnology technologies and principles in an industrial setting

The platform can provide a means for industry partners to share information they feel is important for students to know (continued)

146

F. Geng and D. D’Souza Table 1. (continued)

Course

Why this course is being selected How the courses benefit from a digital platform

Biotechnology Undergraduate Program Courses BIOTECH 3IV3: Immunology and Virology

Given the need from the federal level to prepare for the next pandemic and given the complex field of immunology, it is important that students have a solid understanding in this subject

The variety of resources in immunology and virology provided by the platform can ensure students have access to learning materials that best suit their learning styles

BIOTECH 4BL3: Biomaterials and Biocompatibility

This is a relatively newer field that is gaining traction for its importance when designing products destined for the clinical or research domains

The platform allows for the sharing of the latest discoveries and findings in this field, and provide students with examples of research undergoing in these fields

BIOTECH 4BS3: Biotechnology Regulations

The regulatory field is far and wide, yet it is important for students to be familiar with the appropriate regulations such that they can work in compliance with regulations

The platform allows students to access a variety of resources offered by industry, governing bodies, and governments that pertain to regulations in a biotechnology context

Biomanufacturing Graduate Program Courses SEP 767 – Multivariate Statistical Methods for Big Data Analysis and Process Improvement

This course has a large emphasis on mathematical principles and coding in programming languages

The platform can provide students with a variety of learning materials and demonstrations to ensure they develop a solid grasp of important statistical methods

SEP 744 – Biomanufacturing

This course serves as the flagship course for the program, introducing students to the big ideas regarding biomanufacturing

The platform allows instructors and industry partners to share with students important concepts and advancements in the biomanufacturing field

SEP 764 – Current Good Manufacturing Practice Upstream Operations

This course involves teaching current biomanufacturing principles related to the acquisition of materials

The platform allows instructors and industry partners to share with students important concepts and advancements in the upstream operations

2.3 Assessment Methods of Digital Learning Platform The evidence to assess and evaluate the impact of this research on student learning and teaching include:

The Development of Interdisciplinary Digital Learning Platform

147

2.3.1 Quantitative Data As shown in Fig. 2, pre- and post-implementation surveys provide quantitative measures of students’ perceptions of engagement, satisfaction, and self-directed learning skills. Academic performance analysis provides quantitative evidence of any changes or improvements in students’ grades and assessment outcomes [8, 9]. 2.3.2 Qualitative Data As shown in Fig. 2, focus groups, interviews, and document analysis yield qualitative data on students’ experiences, faculty perspectives, and emerging themes related to engagement, interdisciplinary learning, and platform usability [10, 11].

Fig. 2. Assessment and evaluation on the implementation of digital learning platform in biomanufacturing and biotechnology curriculum. The methodology includes both quantitative methods and qualitative methods.

2.4 Evaluation Plan The evaluation plan involves the following steps: Step 1: Data collection: Conduct surveys, interviews, focus groups, and document analysis to gather both qualitative and quantitative data. Step 2: Data analysis: Analyze the collected data using appropriate qualitative and quantitative analysis techniques to identify patterns, themes, and trends. Step 3: Interpretation and synthesis: Interpret the findings, compare pre- and postimplementation data, and synthesize the qualitative and quantitative evidence to provide a comprehensive understanding of the research’s impact. Step 4: Evaluation report: Prepare an evaluation report that summarizes the findings, highlights the impact of the digital learning platform on student learning, and provides recommendations for further improvement and future implementation.

148

F. Geng and D. D’Souza

3 Results and Benefits 3.1 Enhanced Learning Experience for Biotechnology Students The development of a verified digital learning platform provides students with reliable and accurate learning materials, improving their comprehension and understanding of interdisciplinary biotechnology concepts. Interactive features and collaborative opportunities foster engagement and knowledge-sharing among students, enhancing their learning experience. 3.2 Improved Critical Thinking and Open-Ended Thinking Skills By promoting critical thinking skills, students develop the ability to analyze information, evaluate sources, and make informed decisions. Open-ended thinking skills, including creative problem-solving and adaptability, are cultivated, enabling students to apply their knowledge effectively in Biotechnology Undergraduate Program or in the Biomanufacturing Master’s Program. 3.3 Application of Knowledge in Biotechnology Undergraduate Program and the Biomanufacturing Master’s Program Equipped with critical thinking and open-ended thinking skills, students are prepared to tackle complex problems, adapt to rapidly evolving technologies, and contribute to advancements in the field of biotechnology. The application of their knowledge in Biotechnology Program or in the Biomanufacturing master’s program will be enhanced, leading to better career prospects and contributions to student success. 3.4 The Applications of Digital Learning Framework in Biomanufacturing and Biotechnology Education The success of this research relies on the collaboration and involvement of real-world applications. As shown in Fig. 3, we have targeted the following areas of application during the development of digital learning framework: 1) Training Programs (Institutional Administration): The institutional administration, including department heads and program coordinators, play a vital role in supporting and facilitating the implementation of this research. They are informed of the research’s progress, including milestones, evaluation results, and any necessary resources or support required. 2) Industrial Partners Contributions: Collaboration with industry partners is crucial for ensuring the relevance and applicability of the learning materials. Industry experts can provide insights into the skills and knowledge required in the biotechnology industry, helping to shape the content and assignments on the digital learning platform. 3) Research Projects and Intellectual Property (Faculty and Teaching Staff): The teaching staff in the Biotechnology Undergraduate Program and Biomanufacturing Master’s Program are instrumental in integrating the verified digital learning platform

The Development of Interdisciplinary Digital Learning Platform

149

into the curriculum. They provide valuable input regarding the selection and curation of learning materials and contribute their expertise to the development of interactive exercises and assignments that promote critical thinking and open-ended thinking skills. 4) Job placement (students): The students in the Biotechnology Undergraduate Program and Biomanufacturing Master’s Program are the primary beneficiaries of this research. Their feedback and engagement are sought during the pilot testing phase to evaluate the effectiveness of the platform. Additionally, students have the opportunity to provide input and suggestions for improvement, ensuring that the platform meets their needs and enhances their learning experience. 5) Bridging the gap: The digital learning platform is designed to bridge the gap between students, faculty and staff members, industrial partners and institutions. For example, we revise the curriculum to reflect the increasing demand for critical thinking and open-ended thinking skills in the presence of AI based tools (e.g. ChatGPT).

Fig. 3. The applications of digital learning platform in Biomanufacturing and Biotechnology Curriculum (BTC) in the aspects of training program, industrial partner collaboration, reach projects, intellectual property development, and job placement.

4 Conclusion This research paper seeks to address the challenges posed by unverified digital learning resources and the limitations of AI tools in biotechnology and biomanufacturing education. By developing a verified digital learning platform and promoting critical thinking and open-ended thinking skills, students are equipped with the necessary tools to excel in their work in Biotechnology Undergraduate Program or in the Biomanufacturing Master’s Program. By implementing this research plan, we aim to gather comprehensive evidence that informs the impact of the interdisciplinary digital learning platform on student learning

150

F. Geng and D. D’Souza

outcomes, engagement, and self-directed learning skills. The findings will contribute to the improvement of teaching practices and the enhancement of student experiences in both the undergraduate and graduate programs.

References 1. Noor, U., Younas, M., Saleh Aldayel, H., Menhas, R., Qingyu, X.: Learning behavior, digital platforms for learning and its impact on university student’s motivations and knowledge development. Front. Psychol. 13, 933974 (2022) 2. Khodaei, S., Hasanvand, S., Gholami, M., Mokhayeri, Y., Amini, M.: The effect of the online flipped classroom on self-directed learning readiness and metacognitive awareness in nursing students during the COVID-19 pandemic. BMC Nurs. 21(1), 1–10 (2022) 3. Lai, Y., Saab, N., Admiraal, W.: University students’ use of mobile technology in self-directed language learning: using the integrative model of behavior prediction. Comput. Educ. 179, 104413 (2022) 4. Song, Y., Lee, Y., Lee, J.: Mediating effects of self-directed learning on the relationship between critical thinking and problem-solving in student nurses attending online classes: a cross-sectional descriptive study. Nurse Educ. Today 109, 105227 (2022) 5. Xodabande, I., Atai, M.R.: Using mobile applications for self-directed learning of academic vocabulary among university students. Open Learn. J. Open Distance e-Learn. 37(4), 330–347 (2022) 6. Chang, T.S., Wang, H.C., Haynes, A.M., Song, M.M., Lai, S.Y., Hsieh, S.H.: Enhancing student creativity through an interdisciplinary, project-oriented problem-based learning undergraduate curriculum. Think. Skills Creat. 46, 101173 (2022) 7. Chen, W.T., Fu, C.P., Chang, Y.D., Shiao, Y.C., Chen, P.Y., Wang, C.C.: Developing an innovative medical ethics and law curriculum—constructing a situation-based, interdisciplinary, court-based learning course: a mixed methods study. BMC Med. Educ. 22(1), 1–8 (2022) 8. Chang, M.F., Liao, M.L., Lue, J.H., Yeh, C.C.: The impact of asynchronous online anatomy teaching and smaller learning groups in the anatomy laboratory on medical students’ performance during the Covid-19 pandemic. Anat. Sci. Educ. 15(3), 476–492 (2022) 9. Winstone, N.E., Ajjawi, R., Dirkx, K., Boud, D.: Measuring what matters: the positioning of students in feedback processes within national student satisfaction surveys. Stud. High. Educ. 47(7), 1524–1536 (2022) 10. Khan, S., Kambris, M.E., Alfalahi, H.: Perspectives of University Students and Faculty on remote education experiences during COVID-19-a qualitative study. Educ. Inf. Technol., 1–29 (2022) 11. Morgan, H.: Conducting a qualitative document analysis. Qual. Rep. 27(1), 64–77 (2022)

Artificial Intelligence in Power Distribution Systems Razvan Alexandru Moise(B)

and Aurel Fratu

Transylvania University, 500036 Brasov, Romania {razvan.moise,fratu}@unitbv.ro, [email protected]

Abstract. The artificial intelligence (AI) is part of the modern Power Systems. It is used in protection and control of electrical lines and transformers with good results, in the future will be widely used for implementing the smart grid. Any research is getting closer to an autonomous system which can learn how to adapt in different situations during a normal or fault regime. Optimal functioning of electrical distribution networks is to maintain the voltage into acceptable limits in order not to affect the insulation of the electric lines. During nominal regime the voltage variation is not so big due to the several voltage regulation methods, but during a fault regime, usually during earth-faults can occur over-voltages that can affect the lines’ insulation. That is why earth-faults must be identified and eliminated as quick as possible. The article is researching the possibility to detect the earth-faults with fuzzy logic. The fuzzy logic is using artificial intelligence and allows the earth-faults detection even the input data is not so accurate because of the errors given by the instrument transformers. So, using several logic rules, the phase and the circuit with earth-fault can be successfully identified even the input data is not so precise. The expert fuzzy systems can be successfully used in earth-fault detection and also in detecting any kind of fault with superior results to classical method because using the data with errors it can obtain good results using the logic small, normal, big for the currents and voltages in the system can take decisions similar to human operator. The system using AI can cover wide areas of electrical networks, it can be used for disconnecting the other side of the line by communication between the protection relays. Keywords: Artificial Intelligence · Fuzzy Logic · Earth-Fault Detection · Power System

1 Introduction Power system problems concerning encoding of an unspecified non-linear function are appropriate for AI. AI can be particularly useful for problems which require quick results, like those in real time operation. This is because of their ability to quickly generate results after obtaining a set of inputs. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 151–159, 2024. https://doi.org/10.1007/978-3-031-56075-0_15

152

R. A. Moise and A. Fratu

The main cause of earth-faults in an overhead distribution line is the failure or puncture of the insulator. The insulators are used in overhead transmission lines to provide insulation between the live conductor and metallic towers that are already connected with the earth’s surface. The main cause of earth-faults in cable distribution networks is the damage of the insulation of the cables during works for underground utilities or the aging of the cable insulation. During a fault earth-faults regime the over-voltages can affect the lines insulation. That is why earth-faults must be identified and eliminated as quick as possible with the maintenance techniques using artificial intelligence. Modern maintenance techniques become a competitive major advantage thanks to the early line-faults detection, before the lines’ insulation needs a major repair. With remote monitoring technologies and on-site diagnosis on the maintenance using artificial intelligence can maximizing continuity electrical energy delivery and on reduces costs caused energy delivery shutdown. Integrating smart monitoring systems - using artificial intelligence - perform to tracking for real time of the condition of a distribution electrical network, and of the performances of its’ critical components. Preventive maintenance, suppose the combination of activities consisting of monitoring and regular inspections. Its’ goal is to avoid any as possible the replacement of line with fault - as the costly solution intervention corrective. This level of maintenance is the cost-efficient solution and the necessary to implement solution.

2 The Need for Artificial Intelligence in Power Systems Power system analysis by conventional techniques becomes more difficult because of: • Complex versatile and large amount of information which is used in calculation, diagnosis and learning • Increase the computational time period and accuracy due to the extensive and vast system data handling The modern power system operates close to the limits due to the continuous increasing energy consumption and the extension of currently existing electrical transmission networks and lines. This situation requires a less conservative power system operation and control operation which is possible only by continuously checking the system states in a much more detail manner than it was necessary. Sophisticated computer tools are now the primary tools in solving the difficult problems that arise in the areas of power system planning, operation, diagnosis and design. Among these computer tools, Artificial Intelligence has grown predominantly in recent years and has been applied to various areas of power systems. 2.1 State of the Art in Protecting the Power Distribution Systems From the end of the XIX century when the first European City, Timisoara was illuminated and the first power-station was developed in South Africa by Thomas Edison, the electricity supply was continuously improved.

Artificial Intelligence in Power Distribution Systems

153

Between 1950 and 1980, the protection of the electrical lines and transformers was first stated by electromagnetic relays, which were connected in a logical scheme to detect the overcurrent measured on the equipment. From the early 1970 the digital relays were developed, they had both hardware and software and they use high speed microprocessor technology. This technology is used also in present time for protecting the equipment in the energy grid. The digital relays are connected through instrument transformers to read the information of current and voltage from the primary circuit. In their functionality there is no learning, only a logic program that is using the information received on the analogic and digital inputs to generate outputs. That means the relay of a line is the only one that can control the bay circuit breaker, but is has no information about what is happening in the entire substation. This is leading to faulty trips or multiple trips due to an earth-fault. The improvement came from the earth-fault detection using the admittance and injection current method in the early 2000, for the medium voltage grids where the neutral point is treated by Petersen Coil. This system calculates the capacitive current of the network by injecting a current in the primary circuit of the Petersen Coil, which is turning back through each circuit connected to the busbar. The capacitive current is measured by the Holmgreen Filter (Homopolar current transformer) for each circuit. This system can function with errors when the measurement of the current is no so precise. That leads that in the case of a fault to the trip of one circuit which is no faulted or to no trip at all. 2.2 The Propose of AI in Protecting the Power Distribution Systems The present research is looking to improve this existing technology in fault detecting, using artificial intelligence in a way that the protection system could adapt to the new states of the power distribution systems by learning from the previous states and parameters in order to eliminate the inconvenience like measurement errors that lead to multiple line trip in cause of a fault. In the actual protection technology, the relay is tripping the line/transformer by measuring the current on the circuit, in many cases the current can rise due to a faulted circuit supplied from the same busbar. The research theory is looking to an answer to trip the line having the information from all the busbar circuits in order to detect with maximum precision the faulted line. The AI could be used in Smart Grid Technology because its adaptability to new states of the systems and the global information that is collecting about the power system.

3 Smart Grid Using Artificial Intelligence for Network Management The modern grids are using artificial intelligence for protection, signaling, network management. The smart grid network concept is a network that have information about the production, the transport and distribution and the consumers. Having all the information in one data base, from the SCADA System and from the Metering System, the system is aware of any change of the network parameters, so it can

154

R. A. Moise and A. Fratu

learn the state of the grid and the main information that is assuring the grid functionality [1]. 3.1 Smart Grid for Balancing and Optimizing the Power System By knowing the current, and the voltage in any network nodes, the system can take decisions to stop or to start generators or to store energy, in order to maintain the network power balance. This cand happen by simulating the power-flow based on the information of the network elements and based on the parameters read from the SCADA System. The artificial intelligence can be used for voltage regulation in the grid by actioning to the plot of the transformers or by passive equipment in the grid for reactive power variation. It can maintain the voltage stability by having the global information about the grid parameters [2, 3]. The Smart Grid can disconnect and connect lines for maintain the network parameters and the technical and non-technical losses. It can be used an algorithm to minimize the losses in a network by disconnect the lines and the transformers that are not loaded at a minimum capacity in order to have a good efficiency. 3.2 Smart Grid for Operating and Maintenance The system can detect faults and disconnect the faulted elements of the network, can do system automation in order to maintain the grid operability. In case of fault, based on the electrical data for the grid, can be detected the place of the fault in the grid. From the information in the SCADA System and in the protection relays, it can be done the preventive maintenance of the equipment by setting thresholds for the number of switching, the level of the fluids, the temperature and any other parameter that can be used to inform the system for maintenance.

4 Current Application of Artificial Intelligence in Power Distribution Systems Power system problems concerning encoding of an unspecified non-linear function are appropriate for artificial intelligence. Artificial Intelligence can be particularly useful for problems which require quick results, like those in real time operation. This is because of their ability to quickly generate results after obtaining a set of inputs. 4.1 How Fuzzy Logic Can Be Used in Power Systems The problems in transmission and distribution of electricity can be fed to the artificial Intelligence so that a suitable solution can be obtained. Given the constraints of a practical transmission and distribution system, the exact values of parameters can be determined. For example, the value of inductance, capacitance and resistance in a transmission line can be numerically calculated by the Artificial Intelligence taking in various factors like environmental factors, unbalancing conditions, and other possible problems. Also the values of resistance, capacitance and inductance of a transmission line can be given as inputs and a combined, normalized value of the parameters can be obtained. In this way skin effect and proximity effect can be reduced to a certain extent [4, 5].

Artificial Intelligence in Power Distribution Systems

155

4.2 Earth-Fault Detection Using Fuzzy Logic in Electrical Distribution Networks – Case Study The expert system will use the fuzzy logic in order to detect the faults on the electric lines. The inputs for the system will be the current on each phase, the homopolar current, the voltages on each phase and the homopolar voltage. In the fuzzy system, the variables can take values between 0 and 1 and using the rules we can detect the outputs using approximate values for inputs. For each input we set 3 levels: “L” – Low; “N” - Normal and “H” - High for approximation and building of the logical rules in the table below [6]. Table 1. The Inputs and Outputs of The Fuzzy System Rules

Input variables

Output Variables

I1

I2

I3

I0

V1

V2

V3

V0

1

N

N

N

N

N

N

N

N

No Fault

2

H

N

N

H

L

H

H

H

L1-G

3

N

H

N

H

H

L

H

H

L2-G

4

N

N

H

H

H

H

L

H

L3-G

5

H

H

N

H

L

L

H

H

L1,2-G

6

H

N

H

H

L

H

L

H

L1,3-G

7

N

H

H

H

H

L

L

H

L2,3-G

8

H

H

N

H

L

L

N

H

L1,2

9

H

N

H

H

L

N

L

H

L1,3

10

N

H

H

H

N

L

L

H

L2,3

11

H

H

H

H

L

L

L

L

L1,2,3

Where: N–Normal, L–Low, H–High, I1 –Phase L1 Current, I2 –Phase L2 Current, I3 –Phase L3 Current, I0 –Homopolar Current, V1 –Phase Voltage L1, V2 –Phase Voltage L2, V3 –Phase Voltage L3, V0 – Homopolar Voltage, L1-G–Eartfault L1, L2-G–Earth-fault L2, L3-G–Eartfault L3, L1,2-G–Earth-fault L1, L2, L1,3-G–Earth-fault L1, L3, L2,3-G–Earth-fault L2-L3, L1,2–Biphasic Fault L1-L2, L1,3–Biphasic Fault L1-L3, L2,3–Biphasic Fault L2, L3, L1,2,3–Triphasic Fault. From the Simulink library we used, from the Fuzzy Logic Toolbox, the Fuzzy Logic Controller. For designing the fuzzy system, we use Fuzzy Logic Designer App from Matlab. We declared 8 inputs and one output like in the below figure and the membership function was set by Mandani method (Fig. 1).

156

R. A. Moise and A. Fratu

Fig. 1. The Fuzzy System designed in the Fuzzy Logic Design App

The simulation of this system was done using the program Matlab/Simulink using the block diagram in the Fig. 2. There are 8 input variables: the current of each phase and the homopolar current; the phase voltage on each phase and the homopolar voltage, with 3 levels values. The system uses the values of the inputs at 3 time periods for regulation.

Fig. 2. The Fuzzy System for fault detection block diagram

The defining of the membership function for input variables is done with the Mamdani Method, each input will be defined on a certain interval: L – Low – [−0,5 0 0,5], N – Normal – [0 0,5 1], H – High – [0,5 1 1,5], like in Fig. 3. Also, we defined the membership for output variables. We have 10 outputs, for each possible fault. Using the simulation, we obtain each type of fault by calculating a probability using the inputs. The membership function for the outputs is represented in Fig. 4. As an example, if we try to simulate the earth-fault for L2, that means we will use rule 3 from Table 1 we will obtain the results in the below figure. Similar we can obtain results for each output based on the values of the inputs.

Artificial Intelligence in Power Distribution Systems

157

Fig. 3. The membership functions for inputs

Fig. 4. The membership functions for outputs

Fig. 5. Results for simulation earth-fault on phase L2

In Fig. 5 are represented the rules simulated in the program, the activated output is the one for which all the inputs have the values set in the rule. If we considered the rule for L2 fault to ground, that means: the current on phase L1 is normal, the current on phase L2 is high, the current on phase L3 is normal, the homopolar current of the circuit is high, the voltage on phase L1 is high, the voltage on phase L2 is low, the voltage on phase L3 is high and the homopolar voltage is high, then we have a fault to ground on phase L2.

158

R. A. Moise and A. Fratu

For activating a certain output all the inputs must have the value set in the rules. In the rule all inputs are connected with logical operator “AND”, so for one sets of inputs will be only one possible output. If we think logic, we can find out other rules (until 26 rules) based on the values of the inputs, but only the 10 rules are logical and also to the technical possibility that the input to have the initial value set. When no output is active, that means there is no fault on the circuit. This fuzzy system could be multiplied for each line connected on the substation busbar related to the faults. Also, the fuzzy system could be used also for setting protections related to the measured temperature of the equipment.

5 Conclusion The conventional methods in detecting faults and earth-faults, have good results as long as the input data is measured with very high accuracy. If the data is no accurate, the detection method can lead to bad functioning of the system because the relay will not have the exact information as in reality. Here we have the improvement with the fuzzy system which uses artificial intelligence and it has the capacity of learning the behavior of the electric system. In this article, the artificial intelligence was simulated for one electrical line, but it can be adapted for any substation for all the electric lines supplied from the busbar, with one fuzzy regulator for each line and using as inputs the same voltage, only the current inputs will be different and will be measured for each circuit. The fuzzy system using AI for detecting faults and earth-faults can be successfully used in substations, it doesn’t need accurate measurements because the output is related to some logic rules and is calculated based on the same percentage of incertitude as the inputs have. With a smaller rate of fault in detecting the earth-fault, the fuzzy system can save money because any earth-fault detected and eliminated in time protects the electric line insulation and ensures that the other consumers in the substation are supplied with electricity in parameters.

References 1. Wadhwa, C.L.: Electrical Power Systems, pp. 760–796. New Academic Science Limited, Kent (2012). ISBN: 978-1-906574-39-0 2. Schlabbach, J., Rofalski, K.-H.: Power System Engineering, pp. 263–284. Wiley-Vch Verlag GMBH & Co. KgaA, Weinheim (2008). ISBN 978-3-527-40759-0 3. Weedy, B.M., Cory, B.J., Jenkins, N., Ekanayake, J.B., Strbac, G.: Electric Power Systems, pp. 239–275, 5th edn. Wiley, Hoboken (2012). ISBN 978-0-470-682685 4. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 15, 116–132 (1985) 5. Zadeh, L.A.: Knowledge representation in fuzzy logic. IEEE Trans. Knowl. Data Eng. I(I) (1989) 6. Vasilievici, A., Colceriu, M., Stanescu, D.: Digital equipment used in earth compensated networks for automatic ASC tuning and selective earth fault detection

Artificial Intelligence in Power Distribution Systems

159

7. Srivani, S.G., Kumar, A., Patil, A.U., Praveen, G.: Fuzzy logic technique for smart grid fault detection 8. Keshtkar, A., Arzanpour, S.: An adaptive fuzzy logic for residential energy management in smart grid environments. Simon Fraser University (2016) 9. Bashoo, P.T., Jazaeri, M.: A new multifunctional protection system for reducing fault current, ferroresonance overvoltage, and voltage fluctuations in power networks. Int. Trans. Electr. Energy Syst. (2022) 10. Chatterjee, S., Saha Roy, B.K.: Fast identification of symmetrical/asymmetrical faults during power swing with dual use line relays (2020) ˙I., Akdemir, H., Kekezo˘glu, B., Erdinç, O., Paterakis, N.G.: Power 11. Kiliçkirana, H.C., Sengör, ¸ system protection with digital overcurrent relays: a review of nonstandard characteristics. Electr. Power Syst. Res. (2018) 12. Sahu, V.K., Pahariya, Y.: Power transformer protection based on fuzzy logic system (2020) 13. Heidary, A., Radamanesh, H., Naghibi, S.H., Samandarpour, S., Rouzbehi, K., Shariati, N.: Distribution system protection by coordinated fault current limiters. Inst. Eng. Technol. (2020) 14. Dapshima, B.A., Essa, Y.C., Chaturvedi, S.: Fault detection and protection of power transformer using fuzzy logic. Int. Jo. Res. Appl. Sci. Eng. Technol. (2023) 15. Sonone, S.R., Mohod, A.V., Sharma, M.O.: Fault identification in power system network by using rule based fuzzy logic. Int. J. Res. Appl. Sci. Eng. Technol. (2020)

Adaptation of Internet of Things Technology to the Management of Educational Institutions Abdelghani Ait Ben Braim1(B) , Mustapha Raoufi2 , and Mohammed Skouri1 1 Laboratory of Physics, High Energy and Astrophysics, Faculty of Science Semlalia,

Cadi Ayyad University, Marrakech, Morocco [email protected] 2 Laboratory of Materials Energy and Environment, Faculty of Sciences Semlalia, Cadi Ayyad University, Marrakech, Morocco

Abstract. This paper offers a concise overview of doctoral research that explores the integration of Internet of Things (IoT) technology into the world of education and the management of educational institutions. The main objective of this study is to leverage IoT to enhance the quality of teaching and learning experiences. Traditional education systems often struggle to captivate the attention of both teachers and students, which creates a dull teaching environment. The implementation of IoT technology promises to address these challenges by facilitating the rapid and engaging delivery of information. One of the key outcomes of this research is the creation of an entirely new educational environment that seamlessly integrates technology, making the classroom experience more relatable and enjoyable for students, and eliminating the sense of being transported to a bygone era during lectures. by taking education to the next level into a new era where technology seamlessly augments the teaching and learning processes, in doing so, it paves the way for a future where education seamlessly integrates with the technology that permeates our daily lives, transforming the classroom into a space where learning is not only effective but also enjoyable and aligned with the digital age. Keywords: Internet of Things · Education · Distance Education · E-learning · Management of Educational Institutions

1 Introduction and Motivation The Internet of Things technology is part of the fourth industrial revolution, after computers, the Internet, and mobile networks. The advancement of communications of most computer systems around us has given rise to several means of helping to understand interconnected devices such as sensors, artificial intelligence tools, and others [1].

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 160–167, 2024. https://doi.org/10.1007/978-3-031-56075-0_16

Adaptation of Internet of Things Technology to the Management

161

Educational institutions are using IoT technology in the field of education to improve the quality of teaching and to make educational and administrative tasks more efficient. As a result, several smart systems have been invented such as virtual classrooms and universities, smart libraries, interactive whiteboards, smart lighting, air conditioning systems, automatic heating, etc. In these systems, students remain permanently connected to their programs and professors. For example, they can send their students the timetable, the tasks to be completed, personal course suggestions, and academic topics such as various training opportunities [2–4]. However, this research project takes a unique perspective by focusing on the administrative aspect within the IoT framework, represented by deans and institution directors. In this context, the research seeks to achieve several objectives: • Interactive Educational Space: Establish an interactive electronic platform where teachers can effortlessly create and modify educational content, fostering a dynamic learning environment for students to engage with. • Smart Digital ID Cards: To enhance security measures within institutions, faculties, and laboratories, by the implementation of smart digital ID cards. These cards serve the dual purpose of enabling student and visitor registration while seamlessly integrating with an autonomous access control system, ensuring that only authorized individuals gain entry. • Data and Results Protection: prioritizes the protection of student and teacher data and results, recognizing the significance of data security in an IoT-driven educational landscape. • Intelligent Building Monitoring: explores intelligent building monitoring, encompassing systems such as heating, ventilation, air conditioning, lighting, and access controls. By integrating these systems, institutions can optimize resource usage, energy efficiency, and overall operational effectiveness. In summary, this research aims to create a more interactive and secure educational environment while leveraging the advantages of IoT technology for intelligent building management. These endeavors align with the broader goal of improving the educational experience and administrative efficiency within educational institutions.

2 Background Reflecting on the history of teaching methods, it becomes evident that the motivation behind the concept of integrating IoT into education stems from the evolution of distance learning. This evolution necessitated the use of various technological components such as the internet, smart devices, phones, computers, and intelligent platforms [5]. However, it’s important to note that the initial appearance of distance learning predated the widespread availability of the Internet and Computers. It originated with the concept of correspondence education, enabling students to study from home and submit coursework through postal mail. This early form of distance education has since transformed into its electronic form, commonly referred to as E-learning. For this transformative shift in education, credit is owed to Mr. Isaac Pitman, widely recognized as the father of distance learning [6, 7].

162

A. Ait Ben Braim et al.

Recently, due to the spread of the coronavirus, we have come to recognize the importance of distance education, particularly in Moroccan schools. Consequently, every school and university has taken proactive measures to provide distance education. In this context, each institution has employed various methods of distance education to get through the crisis. This experience has not only enabled us to weather the immediate challenges but has also paved the way for the discovery of innovative and effective approaches to elevate education to the next level, harnessing the potential of the internet and IoT [8]. So, we can consider this crisis as the catalyst that prompted researchers to explore not only the methods to deliver courses online but also to discover more effective methods of teaching in both online and face-to-face learning. IoT technology can potentially enhance teaching in both distance and traditional learning environments and can revolutionize the management of educational institutions to take education to the next level.

3 Goals and Results In my pursuit of a comprehensive understanding of the current educational environment, I undertook a series of immersive experiences that allowed me to gain profound insights into the prevailing teaching and management methodologies in Moroccan schools. This hands-on involvement in the educational sector offered a unique vantage point from which to identify the challenges and opportunities associated with the effective integration of IoT technology. 3.1 Description of Immersion in Educational Environment Teaching Engagement. My immersion journey began by actively engaging in the role of a primary grade teacher. To better comprehend the day-to-day realities faced by educators and students, I registered as a teacher for third-grade primary students. This initiative aimed to provide courses to students, particularly those who had missed significant portions of their curriculum due to the disruptions caused by the COVID-19 pandemic. This hands-on teaching experience allowed me to intimately connect with the teaching environment within public Moroccan schools. Private Support Participation. In addition to my teaching role, I actively participated in private support programs. This involvement was crucial for gaining a more profound understanding of the prevailing education methods and practices. By immersing myself in these support sessions, I aimed to gain a deeper understanding of the methods of current educational techniques. This exposure provided valuable insights into the existing teaching approaches and paved the way for envisioning how IoT technology could enhance the quality of education and its facilitation. Administrative Internship. To gain a holistic perspective on educational management, I undertook an internship as an assistant administrator within an association responsible for the management of training programs for elementary educators. In this capacity, my responsibilities include overseeing administrative tasks related to the training, such

Adaptation of Internet of Things Technology to the Management

163

as tracking attendance, managing document copies, and calculating payments for candidates based on their attendance records, etc. This administrative role has allowed me to observe the backend operations of educational institutions, contributing to a well-rounded understanding of the entire educational environment.

3.2 Immersion Objectives Depth Understanding of Current Educational Methods. Through my immersion in the educational environment as both a primary grade teacher and a participant in private support and administrative tasks, I have gained an intimate understanding of the existing teaching and management practices in Moroccan schools. This practical experience has provided unique insights into the challenges and opportunities for integrating IoT technology effectively. Real-World Application of IoT Technology. My involvement in teaching and administration has allowed me to witness firsthand the potential gaps and areas where IoT technology can be applied to enhance the quality of teaching and administrative processes. This practical experience sets my research apart as it is grounded in the real-world context of educational institutions. Holistic Perspective on IoT Integration. By participating in various facets of the educational system, from teaching to administration, I am uniquely positioned to offer a comprehensive view of the potential impacts and benefits of IoT technology. This holistic perspective ensures that my research considers the broader implications and practical challenges associated with IoT implementation in education.

3.3 Expected Achievements In the course of this research, I anticipate achieving the following outcomes: Insights into IoT Integration. A deep understanding of how IoT technology can be effectively integrated into educational settings, including identifying specific areas where it can enhance teaching and management practices. Identification of Challenges. The ability to identify and address the challenges and barriers to IoT adoption in education, such as infrastructure limitations, cost considerations, and privacy concerns. Development of Practical Solutions. The creation of practical solutions and models for implementing IoT technology in education, including the development of intelligent systems to support teaching and administrative tasks. Enhanced Educational Quality. The potential to improve the quality of education by leveraging IoT technology to facilitate faster and more engaging information delivery, benefiting both teachers and students.

164

A. Ait Ben Braim et al.

3.4 Evaluation Metrics To establish the level of success of the research results, I propose the following evaluation metrics: Student Performance. Assess the academic performance of students in the classes where IoT technology is integrated compared to those where traditional methods are employed. Improved student performance can be an indicator of the success of IoT implementation. Teacher Satisfaction. Conduct surveys and interviews with teachers to gauge their satisfaction with the IoT-based teaching tools and platforms. Higher satisfaction levels can indicate the effectiveness of these tools in making teaching more engaging and efficient. Administrative Efficiency. Evaluate the administrative efficiency of educational institutions by measuring the time and resources saved through IoT-based administrative solutions. Reductions in paperwork and manual tasks can be indicative of success. In sum, these immersive experiences have equipped me with firsthand knowledge of the challenges faced by teachers, students, and administrators within the Moroccan educational context. This intimate understanding of current practices serves as a foundation for envisioning and designing IoT-based solutions that can effectively address these challenges and contribute to the enhancement of educational quality and efficiency.

4 Research Plan The proposed research plan spans three years and is outlined below, encompassing various stages aimed at investigating the integration of IoT technology in education and the management of educational institutions. Additionally, existing material resources essential to the research are highlighted. 4.1 First Year • Literature Review: The initial year will be dedicated to conducting a comprehensive literature review. This step involves a thorough examination of existing academic sources, research papers, and relevant materials to gain insights into the IoT’s potential in the educational domain. • Design of Management Models: During this phase, the research will focus on conceptualizing management models that integrate IoT technology into educational settings. Furthermore, the groundwork for a test bench will be established, and a list of necessary equipment for its realization will be compiled.

Adaptation of Internet of Things Technology to the Management

165

4.2 Second Year • Test Bench Implementation: In the second year, the emphasis will be on the practical aspect of the research. The test bench, designed in the previous year, will be constructed and configured. This test bench will serve as a practical platform for assessing and validating IoT-based educational solutions. • Commencement of Intelligent Component Development: Simultaneously, the development of the intelligent components essential for the IoT-enabled educational environment will begin. This includes creating intelligent systems that enhance the teaching and learning process. Additionally, a platform for information management will be initiated. • Publication of Research Findings: As research milestones are achieved, findings and insights will be disseminated through publications in relevant academic journals or conferences. This step ensures that the research contributes to the wider academic community. 4.3 Third Year • Completion of Ongoing Work: The final year will involve concluding the work initiated in the preceding years, which includes finalizing the intelligent components, refining the IoT-based educational platform, and ensuring the test bench is fully operational. • Publication of Research Outcomes: Continuing the dissemination of research findings, this year will witness the publication of the cumulative work and outcomes achieved throughout the project. • Commencement of Manuscript Writing: The research will conclude with the commencement of manuscript writing. The research paper/article will encapsulate the entire study, including the theoretical framework, methodology, findings, and recommendations. 4.4 Existing Material Resources for Research • Workstation: A dedicated workstation will be utilized for various tasks, including software development, data analysis, and experimentation. • Microcontroller Cards and Various Sensor Components: Essential microcontroller cards and sensor components will be used in the development of IoT-based educational solutions and the test bench (Fig. 1). This research plan outlines a systematic approach to exploring the integration of IoT technology in education over three years, from initial exploration to practical implementation and dissemination of findings. It underscores the importance of a thorough literature review, practical testing, and knowledge dissemination in the progression of the research project.

166

A. Ait Ben Braim et al.

Fig. 1. Schema of the research plan.

5 Conclusion In conclusion, this research embarks on a transformative journey to explore the integration of IoT technology into the world of education and institutional management. Through my immersive experiences teaching, and engagement in administrative tasks. These experiences have not only highlighted the challenges but also illuminated the immense potential for IoT technology to revolutionize education. The coronavirus crisis has acted as a driving force compelling us not only to deliver courses online but also to search for more effective teaching methods in both virtual and physical classrooms. With a holistic perspective that considers the entire educational environment, this research sets the stage for a promising future where technology seamlessly augments education, making it engaging, effective, and accessible to all.

Adaptation of Internet of Things Technology to the Management

167

References 1. Malomane, R., Musonda, I., Okoro, C.S.: The opportunities and challenges associated with the implementation of fourth industrial revolution technologies to manage health and safety. Int. J. Environ. Res. Public Health 19(2), 846 (2022) 2. Zahedi, M.H., Dehghan, Z.: Effective E-learning utilizing Internet of Things. In: 2019 13th Iranian and 7th National Conference on e-Learning and e-Teaching (ICeLeT), Tehran, Iran, pp. 1–6 (2019) 3. Al-Emran, M., Malik, S.I., Al-Kabi, M.N.: A survey of Internet of Things (IoT) in education: opportunities and challenges. In: Hassanien, A., Bhatnagar, R., Khalifa, N., Taha, M. (eds.) Toward Social Internet of Things (SIoT): Enabling Technologies, Architectures and Applications. SCI, vol. 846, pp. 197–209. Springer, Cham (2020). https://doi.org/10.1007/978-3-03024513-9_12. 4. Zaatri, I., Kharki, K.E., Bendaoud, R., Berrada, K.: The use of open educational resources at Cadi Ayyad University: state of art review. In: 2019 7th International conference on ICT & Accessibility (ICTA), Hammamet, Tunisia (2019) 5. Harting, K., Erthal, M.J.: History of distance learning. Inf. Technol. Learn. Perform. J. 23(1), 35–44 (2005) 6. Archibald, D., Worsley, S.: The father of distance learning. TechTrends 63, 100–101 (2019) 7. Alzahrani, J.: Investigating role of interactivity in effectiveness of e-learning. Diss. Brunel University London (2015) 8. Jasim, N.A., AlRikabi, H.T.S., Farhan, M.S.: Internet of Things (IoT) application in the assessment of learning process. In: IOP Conference Series: Materials Science and Engineering, vol. 1184, no. 1. IOP Publishing (2021)

Motivation to Learn in Immersive Web Environments: Pilot Study Bárbara Cleto(B)

, Carlos Santos , and Mário Vairinhos

DigiMedia Research Center, University of Aveiro, Aveiro, Portugal {barbara.cleto,carlossantos,mariov}@ua.pt

Abstract. The article describes a pilot study conducted with elementary school students (8th grade) of a public school. The study was conducted in the classroom for three weeks (one time, fifty minutes, per week). The students conceptualized an online classroom, to learn a programmatic content, chosen by them. During the sessions, the students imagined, thought and designed the room, which they would later model and in which they had classes. However, due to some problems (which are addressed in the article), the students did not model the room, so in the use phase, it was necessary to use one, that was already created. Two surveys were applied per questionnaire. The results demonstrated the need to reformulate the schedule for the fieldwork that is intended to be developed in the future, as well as to readjust some of the instruments and data collection techniques built. The familiarity of students with the concept of Immersive Web Environments (IWE) is limited, although they recognize the educational potential of these environments. Most students have a consumer attitude, which is reflected in the difficulty in imagining school and the classroom in these contexts. Despite this and inspired by the examples that the teacher presented, the students drew creative scenarios of unconventional classrooms. The results demonstrate a positive impact on students’ motivation and indicate that if students had an active participation in creating the environment, motivation to learn could increase. Keywords: Immersive Web Environments · motivation · co-creation

1 Introduction This full paper addresses a pilot study on the perception of students after the first experience with Immersive Web Environments. Students evaluate their level of motivation after attending an English class at IWE. This study is part of a research project of a doctoral program on metacognition in IWE co-created and/or personalized by students and teachers. The results presented in this article resulting from the pilot project were obtained after the application to the students of two questionnaires. The proposed is based on the active participation of students in processes, led by them, of collaboration and co-creation of IWE, encouraging them as “prosumers” in the educational process. Alvin Toffler, in 1980, in his book “Third Wave”, used the term “prosumer” to refer to the combination of “producer” and “consumer”. The “prosumer” © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 168–179, 2024. https://doi.org/10.1007/978-3-031-56075-0_17

Motivation to Learn in Immersive Web Environments

169

[1] not only consumes products or content, but also plays an active role in their creation, personalization and sharing. Similarly, Don Tapscott and Anthony Williams, in “Wikinomics: How Mass Collaboration Changes Everything,” published in 2006, draw on the principle of consumers as producers of content, highlighting “mass collaboration” [2] and the importance of collaborative networks and platforms. They indicate that products should be designed in such a way as to allow users to create, remix and share them. Teachers should develop strategies that encourage students to actively participate in learning activities. Motivation influences the choices, efforts and persistence to achieve the objectives has a significant impact on the degree of involvement and success of students in the activities [3]. The ARCS model (Attention, Relevance, Confidence and Satisfaction) is a framework that aims to promote the motivation of students in the learning processes, that is, it intends that students participate and maintain interest and participation through the educational path [3]. In 2008, the model was reformulated and included a fifth element related to students’ self-regulation [4]. According to the author, motivation refers to the greatness and direction of behavior, the choices people make when avoiding or not the experiences, as well as the degree of effort they invest [3]. Regarding learning, motivation arises from personal interests, values and internal desires (intrinsic motivation) [3]. Which helps to understand what students are willing to do, not what they can do. Immersive web environments by providing students with opportunities for collaboration among themselves in a virtual space, and by allowing them to actively participate in the creation, co-creation, sharing and construction of knowledge, lead to resulting in learning experiences. These experiences, in addition to capturing and maintaining the attention of students, highlight the relevance of the content, and increase confidence in their skills, providing a feeling of satisfaction and accomplishment. This can have a positive impact on students’ motivation by offering a sense of ownership over what they have learned and allowing them to play a more active role in their learning process. IWE uses the WebXR standard, which enables the accessibility of immersive content accessible from web browsers, without the need for specific hardware or software. These environments allow interaction, in real time, not only with two-dimensional content, but also with three-dimensional elements [5] and between users, in the form of avatars. Because they are accessed through web pages and the compatibility between different mobile devices and Virtual Reality (VR) and augmented reality (AR) devices [6], it makes them a practical and economical alternative. In these environments it is possible to learn through practical experiences and realistic simulations, which enables the development of relevant content. Students actively participate in immersive experiences, exploring simulated environments and manipulating virtual objects. These practical activities complement the theoretical learning and capture your attention. These environments enable interactions with digital content and virtual objects in a more interactive way, enhancing satisfying learning experiences and increasing confidence in your skills. By exploring 3D environments, and facing practical situations, which allows students to apply the concepts learned, it leads them to reflect on the learning process [7]. They can virtually experiment with varied approaches without worrying about possible consequences, as they are safe environments, which allows

170

B. Cleto et al.

them to focus on tasks without fear [8]. Personalization is a feature of these environments, allowing students to learn at their own time, select their own learning paths [9], explore topics of specific interest, and interact with content according to their preferences. IWE provide instantaneous and continuous feedback, including through visual and haptic representations [10] on student performance. This constant feedback allows you to monitor your progress and identify areas that need improvement. Although in IWE, students can experience individual activities, these environments can be set up for students to collaborate on virtual activities, share knowledge, and work collaboratively on projects, which contributes to a sense of belonging, increasing levels of motivation, immersion, and presence [11]. IWE can enhance the learning experience, preparing students for the challenges and opportunities of the digital age. The studies consulted indicate that students who participate in virtual labs express greater enthusiasm and interest [12], resulting in positive learning outcomes [10]. The ADDIE model was also adopted, a framework used in instructional design that incorporates an iterative process complete with essential steps for the effective development of a course or program [13]. It consists of five phases, each with its own objective and function in the progression of pedagogical design: Analysis, Design, Development, Implementation and Evaluation [13]. Students begin by identifying what content they intend to learn in IWE, which corresponds to the “Analysis” stage of the model. The imagination and collaborative design phase of IWE can be incorporated into the “Design” and “Development” stages of the model. Meanwhile, the availability to students and observation of their interactions, occurs in the “Implementation” stage. The “ Evaluation” stage of the model monitors the impact on student motivation, and data on student engagement, performance and satisfaction is collected. These data allow us to analyze whether IWE has had a positive effect on students’ motivation for learning, as well as to identify opportunities for improvement for future implementations.

2 Pilot Study The accomplishment of this first study was very important, since it allowed the confrontation between what was initially thought and developed, with the existing limitations in real context. This study makes it possible to make some changes, before the implementation of the educational experience that results from the empirical study designed, and to better prepare the intervention. The procedures for compliance with legal and ethical issues were considered, about data collection, informed consent was obtained from all stakeholders, guaranteeing their anonymity [14, 15] and the confidentiality of the data collected. The parents authorized the participation of their students. The realization of this pilot project was authorized by the school management. 2.1 Context and Participants The initial objectives of the study were: i) to understand if the fact that students participate in the design and construction of the immersive classroom has an influence on the interest and motivation of students to learn the school contents, ii) to analyze any difficulties in

Motivation to Learn in Immersive Web Environments

171

the planning already thought, to implement the field study and iii) to verify if the data collection instruments were adequate. The tasks proposed to the students were: i) idealize the classroom ii) draw, using scenario paper the immersive classroom; iii) build the classroom by modeling it in 3D (Spoke by Mozilla [16]); iv) export the model to be included in one of the platforms (Frame VR [17], Spatial [18]) or directly in the Mozilla Hubs [19] platform and v) explore, in a classroom context, the room created to teach the programmatic content, chosen by them. The instruments and techniques of data collection used were: i) diary (students), ii) Multimodal Narrations (teacher), iii) photographic record of the sessions (teacher) and iv) surveys answered by the students (one right at the beginning, for the characterization of the participants and another for the evaluation of the experience, applied after the IWE exploration class). Eight sessions were planned, but only three were held. The sessions related to the modeling and creation of the classroom in an immersive environment were not held. The students only designed the spaces to develop their own activities and used the IWE for more specific activities, namely, to take a class in a foreign language (in this case English). It should be noted that only one of the classes performed the first activity (idealizing the space). A series of constraints prevented the accomplishment of the planned tasks and the fulfilment of the schedule. One of them was that the holidays were not included when the schedule was set; another had to do with the school’s computer equipment, which is old and the low speed of internet access; in addition, students only learn to model in 3D the following semester. In the first session, the students started by answering the initial questionnaire and the working groups were created. The teacher explained to the students, the project in which they were going to participate, showed some examples of immersive rooms1 and indicated the tasks to be performed. Still, at the end of this session, the students, in groups, began brainstorming about what content they wanted to learn, conducted some research and imagined what the space would be like. In the second session, the students designed, using scenario paper, the classroom they idealized. In the following sessions, students were to 3D model the space, then import it into their chosen IWE. However, as already mentioned, the students did not have the ability to perform the task autonomously. The teacher also could not give more classes, for the students to see tutorials to learn how to model, since there was a program to finish. It was decided between the teacher and the researcher that they would use a room already created. The teacher presented the proposal to the students, who agreed. In the third session, an English class was taught at IWE, and at the end, the students answered the second questionnaire. In the idealization phase of the online classroom, twenty-eight students from the eighth year of schooling participated. In the phase of using the online classroom, only ten students attended classes in this space (this activity was held at the end of the semester,

1 https://www.spatial.io/s/The-EDUMetaverse-6139cdc5a4dec8000189522e?share=885795745

4452849079.

172

B. Cleto et al.

and the class of one of the classes coincided with a holiday). The study took place during fifty-minute classes once a week. The students did not create the diary because, according to them, it required a considerable effort of writing, since only guidance was provided on the topics to be addressed. The teacher, in turn, also mentioned that she did not use the Multimodal Narrations, because it implied spending a lot of time to describe the activities in the classroom. Teacher and students expressed a preference for answering a questionnaire survey, which allows them to answer in a simpler, more direct and concise way. 2.2 Findings The first questionnaire consists of eleven questions, divided into two groups. The first group focuses on issues related to playing video games and the second with issues related to the knowledge and use of IWE in a school context. The second questionnaire, which seeks to understand whether the fact that students participate in the idealization of the immersive classroom has an influence on the interest and motivation to learn the school contents, consists of six questions. One of the students, twenty-eight, who participated in the first two sessions (drawing of the classroom for an IWE) answered the questionnaire. The first group of questions aimed to understand if the students played, what games they played and if they programmed them. This group consisted of five mandatory questions: one open-answer and four closed-answer questions (two of the questions with only two answer options (Yes/No) and two single-answer questions). Of the twenty-eight students who participated in the study, twenty-two (78.6%) answered “Yes” to the question “Do you play video games?”, while six (21.4%) answered “No”. When asked to indicate the games they usually play, most mentioned “Minecraft”, “Roblox”, “Fortnite”, “GTA”, “FIFA”, “Free Fire”, “Call of Duty”, only one mentioned “Board games” (without indicating which ones) and six indicated that they did not play. To the question “What type of graphs do you prefer”, twelve (42.9%) chose the 3D option, while ten (35.7%) selected the option “2D and 3D mixture”, three (10.7%) chose the answers “2D” the same number of those who answered “None”. To the question “Created/programmed a game?”, only six (21.4%), answered “Yes” (the remainder, 22, marked “No”). Asked on which platform they created/programmed a game (Scratch, Roblox, Minecraft or Other), three answered “Roblox”, one “Minecraft” and two indicated “GDevelop”. Most students play video games but do not create them and/or program them. There are few students who create games in Roblox or Minecraft, although playing in these environments. In this group of students, most assume a role of “player” and not of “creator”. The second group of questions in this first questionnaire, consisted of six questions, only one non-mandatory and contained three of open answer and three of multiple answer. It was intended to understand if the students were familiar with IWE. Although the teacher had initially presented the study, including to evaluate the interest of the students in participating, she did not address the concepts involved (option decided between the teacher and the researcher). These concepts, as well as the exploration of some IWE, only happened after the application of the first questionnaire.

Motivation to Learn in Immersive Web Environments

173

Students were asked if they knew about Immersive Web Environments. If so, they were asked to define them. Eighteen (64.3%) of the students answered “No”, four (14.3%) “Yes” and two (7.1%) “I do not know”, the rest, despite not giving an answer “Yes, No or I do not know”, said that “I only know VR glasses”, “I do not know how to explain, but it must be like VR, in which each one, in the form of an avatar can have actions”, “I know sites and applications that allow the user an immersive and fun experience” and another refers “you have a feeling of “being even” there.“ In the next question, “What IWE do you know?”, some examples were already indicated, and although it was a multiple answer question, it was found that the students only ticked one of the options presented. Thus, ten (35.7%) marked VR chat, six (21.4%) VR frame, two (7.1%) Mozilla Hubs and one (3.6%) AltSpace VR, nine (32.1%) chose the “None” option. Next, they were asked how and for what they access these environments. The results are presented in Table 1. Table 1. Answers to the questions “how and why do they access IWE”. Questions/options

Answers (28 students)

How do you access IWE? I don’t access

9 (32,1%)

Computer

4 (14,3%)

Mobile phone

4 (14,3%)

Mobile and Computer

7 (25,0%)

Mobile, Computer and Tablet

2 (7,1%)

VR Glasses

1 (3,6%)

Other (Xbox, Nintendo)

1 (3,6%)

What do you access these environments for? I don’t access

9 (32,1%)

Chat with friends, play

7 (25,0%)

Play

5 (17,9%)

Play, Create/develop games

2 (7,1%)

Chat with friends, Play, Create/build spaces

2 (7,1%)

Create/develop games, Create/build spaces

1 (3,6%)

Play, Create/develop games

1 (3,6%)

Schoolwork

1 (3,6%)

When asked about the viability of learning in these environments, they were asked to provide some examples. Eighteen (64.3%) answered “Yes”, while six (21.4%) reported “I don’t know”, two (7.1%) chose not to answer, the same number of those who indicated “No”. This was the only optional question. As for the examples, the students mentioned that they consider it possible to learn in this type of environment, indicating the Minecraft constructions as an example. They say that these environments and digital technologies

174

B. Cleto et al.

allow them to learn more and in a more playful way. They declare that they would like to learn to program in these environments, and that the fact that they are in contact with colleagues of other nationalities facilitates the learning of foreign languages. They specify that it would be interesting to learn English in such an environment. The students highlighted the importance of having teachers who are dedicated to teaching in these spaces. Finally, they were asked about how they imagine the school, more specifically the classroom, if the classes were taught in these environments, seven students (25%) indicate that they cannot imagine (answered, “I have no idea”). Most say it would be identical to their room, but the space would be larger, in which colleagues were viewed in “big spot” in 3D. They imagine the school changing its decoration daily and the white rooms, equipped with a lot of technology and many computers. Students report that the classes would be much more “cool”, more interesting, fun, immersive, motivating, and very optimized in all aspects (but do not indicate which ones). Students warn of caution to be taken with the graphics and models when building their IWE (they should be reduced), because of the performance limitations of the school’s computers. Two students are more creative and imagine the room in the middle of a forest. It was found that the students are not familiar with the concept of IWE, although they know some of these environments, mainly due to Virtual Reality. Students access it primarily through mobile devices and computer and to chat with friends and play games. There are a significant number of students who do not access these environments. Students claim that classes in these environments would be more motivating, interesting and engaging, providing an immersive experience. However, when asked to imagine the school and more specifically the classroom, there is a huge group that expresses difficulty in imagining it and most have difficulties to conceive something different from what is familiar to them, presenting an extension of their classrooms and emphasizing the technological aspects. Only a small number of students take a more creative approach. After the initial questionnaire, the teacher showed the students some illustrative examples of IWE used in an educational context. His goal was for students to explore these environments. However, due to technological limitations, namely restrictions on internet connection and obsolescence of computers, mainly because of the available RAM, the students observed the teacher browsing the different IWE, because she used her laptop, connecting it to the video projector. Based on the examples presented, some students were able to break free from traditional classroom settings and design more creative environments. However, it was found that some of the visual representations of these ideas are inspired by the scenarios that were shown as an example. One group was inspired by the “Cat coffee shop” and proposed a space, consisting of several rooms, each dedicated to a specific animal. Another group designed a room shaped like an aquarium (see Fig. 1). The base of the aquarium would serve as a meeting room, which allows you to see a reef of corals and fish. From this room, students could move to an educational games room or move by teleport to another room, in the form of a submarine, where they would have the possibility to explore the ocean bottom. Another group imagined that the room, for physical education classes, would be a beach, with the sand divided for the practice of various sports: volleyball, soccer, handball. At sea, students would have the opportunity to swim, surf, canoe… Areas (bungalows) have been integrated where students can gather. Another group proposed a

Motivation to Learn in Immersive Web Environments

175

scenario, inspired by a museum, for history classes, where the different rooms could refer to different ages and historical contexts. One group designed, for the music education classes, a music studio, consisting of several rooms equipped with a variety of musical instruments and a stage. The proposals presented by the students demonstrate a more creative approach. The environments designed by them offer experiences that make it possible to increase motivation and engagement for learning.

Fig. 1. Immersive classroom proposed by one of the groups.

The second questionnaire was applied after the use of the IWE as a classroom. A classroom was used to learn English, since in questionnaire one, students showed interest in learning foreign languages in these environments. Students used their mobile devices to access the immersive classroom. Only ten students answered this questionnaire. The questions were all mandatory, except for the three open-ended questions. Two of them were closed-answered, with questions with three answer options (Increased/Decreased/Equal) and Likert scales were used, with five indicators: “Strongly disagree” (1), “Disagree” (2), “Undecided” (3), “Agree” (4) and “Totally agree” (5). When asked how using the classroom affected their motivation to learn English, six (60%) students indicated that it increased, while the other four ticked the “Equal” option. Asked about the factors that most contributed to the increase in motivation, they highlighted the complete immersion in an environment in which everything is in English (objects, the information of the rooms,…), and the possibility of interacting and socializing, with friends, in English. The fact that there are several games to exercise vocabulary was indicated as an important factor. Some students mention the creativity of the room and the possibility of including new objects, continuing their personalization. The students did not identify factors that could have contributed to diminishing their motivation. The results reflect a positive impact on student motivation. We also wanted to explore the students’ perception of the experience of using IWE as a classroom to learn English. Was used a five-point Likert scale: Strongly disagree

176

B. Cleto et al.

(1) to Strongly Agree (5), where students reflect on whether these IWE are facilitators of learning, how satisfied they are with the experience and whether they are willing to continue exploring these environments. The opinions of the participants are indicated in Table 2. Table 2. Students’ perception of the use of IWE as a classroom. Affirmations (10 students)

Strongly disagree (1)

Disagree (2)

Undecided (3)

Agree (4)

Strongly agree (5)

The use of the room 0 facilitated the learning of the contents

0

4

4

2

The use of the room 0 motivated me to know more about the subject I learn in the discipline

0

4

2

4

I enjoyed taking classes in this environment

0

0

4

2

4

I enjoyed learning using this environment

0

0

4

2

4

I would like to continue these environments in other disciplines

0

0

4

2

4

These environments 0 facilitate content learning

0

4

4

2

The results show a diversity of opinions, verifying that there is a high number of undecideds as to the ease of learning, motivation and preference for this type of IWE. However, 60% of students partially or totally agree with the benefits of the virtual environment. Regarding the question “If you had built the room, it would have an impact on your motivation to learn English (increased, decreased or remained the same)”, seven (70%) of the students chose the “Increased” option, while three (30%) indicated that their motivation remained the same. In the last question, students were asked to include additional comments, but none of them left any remarks. The results suggest that if the students had actively participated in creating the environment, the motivation to learn English increased.

Motivation to Learn in Immersive Web Environments

177

3 Final Remarks The article addresses a pilot study involving 8th-graders. Twenty-eight students answered the first questionnaire and only ten students answered the second survey. In this research are presented, analyzed and discussed the results obtained from the analysis of the treatment of those collected in the three sessions held, weekly and in the classroom. This study made it possible to readjust and redefine some data collection instruments, namely the students’ diary and the multimodal narrations for teachers. According to the students, preparing a diary implies an extra effort to describe the process of developing the online room. The teacher also chose not to give a detailed description of the classroom activities as she was focused on supporting the students. Students and teachers preferred to fill out a questionnaire with targeted questions. Based on this data, a pattern of consumption and not of production is highlighted regarding the exploitation of IWE, as had already seen with video games. Once again, there is a passive nature of the students, and there is a considerable percentage who do not play and do not inhabit these environments. Much of the activities in these environments are social and entertainment activities. Students have difficulty imagining the school and classrooms different from those they know, being very anchored in the traditional teaching structure, to which they only add technology, but recognize potential in the classes taught at IWE. However, after observing the examples the teacher showed, the students developed creative drawings of the IWE. The results show that the use of these environments has a positive impact on the motivation of students. In addition, they show that if students had played an active role in designing these environments, it would have resulted in an increase in motivation to learn. Based on the answers obtained and the analysis of the data, the importance of promoting the active participation of students in the creation and development of IWE, but also of the educational contents that may be within these spaces, is highlighted. The sample is limited and unrepresentative, which has implications for the results presented, since these results are linked to a very restricted universe. The application of this methodology to other classes may confirm or refute the results obtained. It is important to know the perception of students to understand how they perceive and respond to the incorporation of IWE in their learning process. This understanding is fundamental to delineate a strategy that promotes the integration of IWE in an educational context. As a future work it is intended to repeat the study. The first questionnaire must be modified, to include a set of questions about 3D modeling (to know if students have this competence). In step 2, it is planned to create tutorials that exemplify the process of building IWE, to train teachers and students with the necessary skills to perform this task. Tutorials should show you in detail how to design and customize an immersive web environment. The collaboration of students with experience in 3D modeling, to support their colleagues, in the 3D construction of the room, is also being considered. Also, it is not excluded the possibility of involving other classes, in the phase in which students use the room, created at IWE, to have classes, accessing with virtual reality glasses. This would allow us to draw conclusions about whether the process of idealization, construction and use of IWE had an impact on the metacognition of students. That is, we seek to understand if there are significant differences between the students who conceived the space and those who only use it. To perform this evaluation, the metacognition scale

178

B. Cleto et al.

will be used, which will be applied before the idealization of the IWE and later after each step. Students who only use the classroom will also be applied the metacognition scale. This research work, in which this pilot project was integrated, aims to understand the impact that the active participation of students in the creation and use of immersive web environments influences metacognition. Acknowledgments. To the teacher and students who participated in this pilot study. To the parents and guardians and to the direction of the school grouping for having consented and approved the participation of the students in the study.

References 1. Toffler, A.: The Third Wave, vol. 484. Morrow, New York (1980) 2. Williams, A.D., Tapscott, D.: Wikinomics. Atlantic Books Ltd. ( 2011) 3. Keller, M.: Motivational design of instruction. In: Instructional Design Theories and Models: An Overview of Their Current Status, vol. 1, no. 1983, pp. 383–434 (1983) 4. Keller, J.M.: First principles of motivation to learn and e3-learning 29(2), 175–185 (2008). https://doi.org/10.1080/01587910802154970 5. Maclntyre, B., Smith, T.F.: Thoughts on the future of WebXR and the immersive web. In: Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, pp. 338–342. Institute of Electrical and Electronics Engineers Inc., July 2018. https://doi.org/10.1109/ISMAR-Adjunct.2018.00099 6. Rodríguez, F.C., Dal Peraro, M., Abriata, L.A.: Democratizing interactive, immersive experiences for science education with WebXR. Nat. Comput. Sci. 1(10), 631–632 (2021). https:// doi.org/10.1038/s43588-021-00142-8 7. Papanastasiou, G., Drigas, A., Skianis, C., Lytras, M., Papanastasiou, E.: Virtual and augmented reality effects on K-12, higher and tertiary education students’ twenty-first century skills. Virtual Real. 23(4), 425–436 (2019). https://doi.org/10.1007/S10055-018-0363-2/TAB LES/2 8. Hu-Au, E., Okita, S.: Exploring differences in student learning and behavior between reallife and virtual reality chemistry laboratories. J. Sci. Educ. Technol. 30(6), 862–876 (2021). https://doi.org/10.1007/S10956-021-09925-0/TABLES/5 9. Pringle, J.K., et al.: Extended reality (XR) virtual practical and educational eGaming to provide effective immersive environments for learning and teaching in forensic science. Sci. Justice 62(6), 696–707 (2022). https://doi.org/10.1016/J.SCIJUS.2022.04.004 10. Kavanagh, S., Kavanagh, S., Luxton-Reilly, A., Wuensche, B., Plimmer, B.: A systematic review of virtual reality in education. Themes Sci. Technol. Educ. 10(2), 85–119 (2017) 11. Boel, C., Rotsaert, T., Schellens, T., Valcke, M.: Six years after google cardboard: what has happened in the classroom? A scoping review of empirical research on the use of immersive virtual reality in secondary education. In: EDULEARN21 Proceedings, vol. 1, pp. 7504–7513, July 2021.https://doi.org/10.21125/EDULEARN.2021.1524 12. Winkelmann, K., Keeney-Kennicutt, W., Fowler, D., Lazo Macik, M., Perez Guarda, P., Joan Ahlborn, C.: Learning gains and attitudes of students performing chemistry experiments in an immersive virtual world 28(5), 620–634 (2020). https://doi.org/10.1080/10494820.2019. 1696844 13. Peterson, C., Peterson, C.: Bringing ADDIE to life: instructional design at its best. J. Educ. Multimedia Hypermedia 12(3), 227–241 (2003)

Motivation to Learn in Immersive Web Environments

179

14. Amado, J.: Manual de Investigação Qualitativa em Educação, 2a (2014). https://doi.org/10. 14195/978-989-26-0879-2 15. Creswell, J.: Qualitative Inquiry and Research Design (2017 edition) | Open Library (2007). https://openlibrary.org/books/OL28633749M/Qualitative_Inquiry_and_Res earch_Design. Accessed 06 June 2021 16. “Spoke by Mozilla”. https://hubs.mozilla.com/spoke/. Accessed 22 Nov 2021 17. “Frame - Immersive Meetings, Classes, Events”. https://learn.framevr.io/. Accessed 22 Nov 2021 18. “Spatial - Virtual Spaces That Bring Us Together”. https://spatial.io/. Accessed 22 Nov 2021 19. “Hubs - Private, virtual 3D worlds in your browser”. https://hubs.mozilla.com/. Accessed 22 Nov 2021

Music Recommendation Based on Face Emotion Recognition Pallavi Ramsaran and Leckraj Nagowah(B) University of Mauritius, Réduit, Mauritius [email protected], [email protected]

Abstract. The paper presents a mobile application, EmoTunes, that detects the face emotions of people and thereafter plays a suitable music. The emotion detection system works for any skin tone as the dataset introduced contains a mix of ethnicities. The dataset was collected by asking friends, family members and acquaintances selfies of themselves expressing the Ekman’s basic emotions. EmoTunes then used an ensemble learning with MobileNetV2 and ResNet50 to detect the 7 Ekman’s emotions: Angry, Disgust, Fear, Happy, Neutral, Sad and Surprise. The system can detect single user as well as multi-user emotions. It has 2 types of recommendation: Recommendation from server, where it fetches songs from the database, and Recommendation via YouTube, where it takes only a single emotion and fetches a YouTube song that best relates to the user’s preferences and liking. EmoTunes uses 2 methods of discretizing multi-user emotions: either by finding the most common emotion among them, or by finding the emotion of the face nearest to the camera. During real-time emotion detection, the system detects the user’s emotion for 15 s and finds the most common emotion expressed within those 15 s to play a music directly from the server. A total validation accuracy of 99.64% was deduced from the final model, making it one of the promising models when compared to recent research work. The system can be deployed to any part of the world as it adapts to each user’s liking using the YouTube feature. While humans are social creatures, they do not like to wait for services. EmoTunes can hence be deployed in waiting rooms such as for healthcare facilities and at the metro/bus stations, or even in buses and trains in Mauritius, or other parts of the world, to provide the users with some recomforting music based on their current emotions. Future considerations might include further working on the dataset to achieve a much bigger variation. Keywords: Face Emotion Recognition · CNN · Data Augmentation · Transfer Learning · Mobile Application

1 Introduction Facial emotions are perceived as a way to know the feelings of the individual, whether he is happy or sad. Face Emotion Recognition (FER) is not only an intuitive mode of knowing one’s mental state but can also provide recommendations, which makes it a technology that be curated to any field [1]. Many applications include using detected © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 180–191, 2024. https://doi.org/10.1007/978-3-031-56075-0_18

Music Recommendation Based on Face Emotion Recognition

181

emotion to provide personalized messages, personalized recommendations, predict a person’s reaction to movies and detecting students’ emotions to determine their attention level [2]. For the personalized recommendation part, many possibilities exist, including therapy sessions, movie recommendation and music recommendation. This paper focuses on music recommendation based on FER using the 7 basic emotions proposed by Ekman [3]. Approximately 280 million people from the world’s population suffer from major depression [4]. Moreover, stress, the epidemic of the 21st century, is one of the prevalent social diseases eating away 2 in 3 adults [5]. Music can promote cognitive wellness by calming the individual in times of distress and enhance his joy during his happiness in order to achieve peak positivity. Music stimulates various parts of the brain that have direct impact on the muscles, attention, memory, and emotion [6]. Hence, combining FER with associated music would extract the emotion of the user(s) and recommend songs that have soothing properties. The name EmoTunes has therefore been coined to describe our system. For the blind community, while there are voice assistants that directly take commands from the user, it would have been innovative to have a system in which the person does not have to take any actions at all. Moreover, in a day-to-day sense, it can be bothersome to keep searching for music that best fits our mood manually. Although there exist apps such as Spotify and YouTube which use Artificial Intelligence (AI) to prove content to your liking, your mood detection seems far from the context. For instance, while driving, it can be fatal to use your phone or car’s dashboard screen to skim through songs. In a social sense, during a gathering, it is difficult to try to find a song that everyone in the group likes. Comes to play, EmoTunes which not only provides unique music to its users but also real-time detection and automatic music player for the scenario of the blind person and the driver. And it provides an appropriate song for the most significant emotion from a group of people in the detection. The primary aim of this work is to use an emotion detection model via a mobile application to detect the facial emotion of people and play music based on the detected emotion. The remainder of this paper is organized as follows. In Sect. 2, a literature review on face emotion recognition has been presented. Section 3 highlights the architecture of our system EmoTunes. The system prototype and testing of the application are presented in Sect. 4. An evaluation of the system has been carried out in Sect. 5. Finally, Sect. 6 concludes the paper.

2 Literature Review In the attempt to ensure better comprehension during lectures, Lasri et al. [7] developed a student face emotion detection system using a simple CNN architecture with Keras. The famous FER2013 dataset was pre-processed by resizing and normalizing and then subjected to data augmentation with rotation, width shift, height shift, zoom, horizontal flipping. The dataset was split into 80:20 and the CNN architecture had 4 convolutional layers and 2 FC layers with ReLU. The last classification layer had Softmax as activation function. The system was trained with 106 epochs and gained an accuracy rate of 70%. Ahmed et al. [8] worked with the following datasets CK, CK+, FER2013, MUG, KDEF, AKDEF, KinFaceW-I, KinFaceW-II in order to merge them into one to prevent

182

P. Ramsaran and L. Nagowah

biased results. The dataset was cleaned and pre-processed by cropping, normalizing, converting to grayscale and resizing to 48 by 48. Then, the dataset was augmented with rotation, shear, zoom, horizontal flipping. The initial dataset contained a total of 12,040 images but with augmentation, it reached up to 36,120 images. Their custom CNN consisted of 4 convolutional layers with ReLU and a final layer with Softmax. They achieved a validation accuracy of 96.24%. Modi and Bohara [9] believed that the advancement in facial emotion recognition may be a means for the blind community to know how the person he is speaking to is feeling by use of a CNN block. They selected three main expressions—happy, neutral, and sad for classification. They took the FER2013 dataset and chose these 3 classes only. The images were resized, converted to grayscale, and transformed into data frames for training. They used 2 convolutional layers equipped with LeakyReLU while the last classification layer used ELU. The results obtained gained the highest accuracy of 82.5% at 35 epochs. However, they restricted the model to recognize only 3 emotions. An emotion-based music recommendation system was developed by Athavle et al. [10] that captures the user’s face using the webcam, passed it through a CNN model and outputted the emotion. For training, the FER2013 dataset was pre-processed and then trained on 4 convolutional layers and 2 fully connected layers. They received 95% training accuracy. Based on each emotion class, a playlist of Bollywood songs was created with about 100–150 songs per emotion. Pranav et al. [11] took a total of 2550 images of size 1920 by 2560 pixels using a phone with a 48MP camera. The images were taken with different subjects expressing these five emotions: Angry, Happy, Neutral, Sad, Surprise. The pictures were resized to 32 by 32 pixels. The model architecture consisted of 2 convolutional layers and a Softmax layer. An overall accuracy of 78.04% was noted. Meena and Mohbey [12] trained the VGG-19, Inception-v3, and XceptionNet models on the different datasets: CK+, JAFFE and FER2013 in order to detect positive, negative and neutral emotions. The images were resized to 124 by 124 and rescaled. The dataset was split into train and testing with the ratio 80:20. The best performing model was Inception-v3 with CK+ as the best dataset. Akhand et al. [13] stated that the usual shallow CNNs have limited learning schemes and hence extracted features poorly in images. Hence, they proposed a pipeline in which they used several models such as VGG-16, VGG-19, ResNet18, ResNet34, ResNet50, ResNet152, InceptionV3 and DenseNet161. The KDEF dataset was cleaned and preprocessed by cropping and resizing. Then, data augmentation was applied with a rotation degree of −10° to 10°, a shear factor of 1.1 and horizontal flip. The earlier layers of the pre-trained models were frozen, and the output layer was replaced by FER layers. The 10-fold cross validation was used, and the best performing model was DenseNet161, giving test accuracy of 98.78%. The research done by Zahara et al. [14] suggested a micro facial expression recognition system based on the FER2013 dataset using Xception. The project amassed an average accuracy of 65.97%. However, they analyzed the issues found as follows. The face-tilt images did not lead to decent results due to little diversity in viewpoint in the training dataset, the maximum distance for prediction between the user and the camera

Music Recommendation Based on Face Emotion Recognition

183

was 500 cm only and the face rotation up to 45, 135, 225 and 315° failed to classify emotion. Jain et al. [15] suggested a hybrid Convolutional-Recurrent Network (CNN-RNN) for a reinforced FER. The CNN part was primarily used for feature extraction. The RNN part takes these extracted features to identify the temporal dependencies. The JAFFE dataset was processed by adding lighting fluctuations and performing horizontal, vertical flipping and rotation. They noticed that the CNN-RNN method achieved about 94.46% overall accuracy. Dhankhar [16] proposed a FER system using VGG16, ResNet50 combined as an ensemble model. The models were trained on KDEF. The author connected the outputs from both models for a more robust system by concatenating one output to the other. The ensemble model scored the highest performance of 78.3%. Shaees et al. [17] proposed transfer learning on an AlexNet network for feature extraction and using SVM for classification. The AlexNet model was fine-tuned using two datasets, CK+ and NVIE. The features were then passed to the SVM multiclass classifier for final prediction. The overall accuracy was 99.3%. Wahyono et al. [18] introduced the possibility of using sensors installed in ubiquitous devices in order to carry out FER by proposing a web-based application that does FER using mobile technologies. The mobile device’s face detection sensor took a picture of the user’s face. The picture was then added to the database for classification. Image segmentation into 3 colors: blue, green, and red was performed using K-Means clustering. The colors were representative of Happy, Normal and Sad respectively. KNN was then used to match the closest histogram values with the emotional value. The overall resulted in 77.5% accuracy. Men and women have different expressions models, making classification inaccurate when the genders are not considered. Zhong et al. [19] came up with a conditional probability for gender by opting for different random forests. Their proposed work could function in a non-controllable environment as well as with a small training dataset. With the LFW dataset, this technique proved a significant improvement of 5% when compared with the work without gender condition. The overall accuracy obtained was 98.83%. The main objective of Flores-Juarez et al. [20] was to realize a Bayesian model that used supervised learning and analyzed data collected to predict emotions. The target emotion chosen were Happy and Sadness. Three parameters were considered for FER, Nose-Eyebrows, Nose-Eyelids and Nose-Lower Lip distances. An accuracy of 85% was observed. A few problems were found that may have decreased the accuracy of the model such as different lighting and different orientations of the head. It can be deduced from Table 1 that CNN was used by majority of researchers. With CNN, robust systems were implemented with high performance metrics. While the custom CNNs were giving accuracies ranging from 70–96%, Transfer Learning (TL) gave much higher accuracies as from 94% till 100%. TL has therefore been preferred over shallow CNNs as they are already trained to learn patterns and will hence achieve high accuracies in shorter time. While FER2013 dataset was the most used, it was also an unbalanced dataset. With further analysis using this dataset, the dataset was balanced by oversampling and applying data augmentation. However, the performance degraded

184

P. Ramsaran and L. Nagowah

sharply after balancing it. Hence, a new and custom dataset has been sought that will be better suited to the problem domain and better fit the Mauritian skin. Table 1. Comparison Analysis of Related Works Reference

Algorithm

Dataset

Classes

Data Augmentation

Accuracy

Precision

Recall

F1 score

[7]

Custom CNN

FER2013

7

Yes

70.0

68.7

68.0

68.3

[8]

Custom CNN

Combined

7

Yes

96.2

94.9

94.8

94.8

[9]

Custom CNN

FER2013

3

No

82.5

82.1

81.6

81.8

[10]

Custom CNN

FER2013

7

No

95.0

–

–

–

[11]

Custom CNN

Own

5

No

78.0

85.2

77.9

81.4

[12]

Transfer Learning

CK+

3

Yes

99.6

66.0

71.0

68.0

[13]

Transfer Learning

KDEF

7

Yes

98.8

94.4

98.8

96.5

[14]

Transfer Learning

FER2013

7

Yes

94.3

94.5

94.3

93.1

[15]

Hybrid CNN-RNN

JAFFE

7

Yes

94.5

93.3

93.0

93.1

[16]

Ensemble Learning

KDEF

7

No

78.3

79.4

78.2

78.8

[17]

CNN-SVM

CK+

7

No

99.3

98.5

98.3

98.4

[18]

K-Means & KNN

Own

-

No

77.5

–

–

–

[19]

Gender-based Random Forest

Combined

7

No

98.8

98.8

98.8

98.8

[20]

Naïve Bayes

Own

2

No

85.0

–

–

–

3 System Architecture The proposed system, EmoTunes is a music recommendation app that uses Deep Learning to detect faces’ emotions and recommends suitable songs. As shown in Fig. 1, the EmoTunes dataset has been pre-processed and split into 2, training and testing. Transfer Learning has been applied on the dataset. Based on the metrics found, the hyperparameters have been tuned and the model was trained again until an optimized model was found. Among the models, an ensembling technique has been applied on 2 of them. EmoTunes provided the user with 3 types of detection modes, namely upload photo from gallery, upload photo from direct selfie and real-time detection where the system detected the user’s emotions for 15 s and provided a single song for the most expressed emotion. Both single-user and multi-user emotion detection were perfectly supported by EmoTunes. A manually collected dataset has been used with a mix of ethnicities. The best performing model was converted to TFLITE. Using OpenCV, the faces were detected, and the sizes of the bounding boxes were stored. These sizes were representative of the

Music Recommendation Based on Face Emotion Recognition

185

distance of the user’s face from the camera. If the box was smaller, it meant that the user was more distant from the camera. The images were pre-processed and passed through a TensorFlow interpreter for classification. As a result, different faces with their respective distances and emotion labels were obtained.

Fig. 1. EmoTunes Architectural Diagram

The default music app works like Spotify in which music was fetched from a server along with its attributes like artist, title, track, and album art. It mapped a song to each face. If similar emotions were found, EmoTunes ensured to provide unique songs so as not to repeat itself. This was the case for both Upload and Realtime processing. Music selection was based on two different approaches. 25 songs were predefined for each of the 7 classes while the user could also add his/her desired music from YouTube. For negative emotions such as Angry, Fear and Sad, comforting music was preferred. For positive and neutral emotions, the aim was to enhance happiness of the person(s). Surprise was whenever we experienced unexpected, startling sounds or motions. While much could not be found as to what type of music would alleviate the shock, fast-paced music with startling notes was sought. There were no songs dedicated to ‘Disgust’ as much as there were for the other emotions. In an emotional context, disgust might be correlated to bitterness and disappointment. It might also relate to funky and comical music pieces. The other recommendation type that EmoTunes offered was the YouTube integration. YouTube has its own algorithm which is based on one’s frequency of use and videos

186

P. Ramsaran and L. Nagowah

watched recommending the user content of similar genre. The user could enter his/her preferred YouTube music playlists for each emotion and let its algorithm work its wonder. However, since it would have been opened using a WebView, several problems might occur. Hence, it is not recommended to use this feature for Realtime/Hands-free option. This feature required getting only 1 emotion. Since only one emotion was required to get one song in general, some processing had to done on the multi-user detection. If there were common emotions in the frame, the YouTube recommendation would provide a song based on the most common emotion. Otherwise, if there were unique emotions in the frame, the YouTube recommendation would provide a song based on the nearest face’s emotion. The confidence levels for each emotion were not considered when it came to group photos. The confidence level of one user might have biased the music recommendation, as it would have ignored the other users’ emotions.

4 Prototype Implementation and Testing The development of the proposed work was done using Python version 3.10.12, TensorFlow version 2.12.0 and Keras 2.12.0. The neural network was trained using Google Colab due to its free resources. The model was deployed on Android Studio Electric Eel 2022.1.1. The face detection library used was OpenCV 4.7.0. The engine used was Acer Spin 3 SP314-52 Core i5 with 8 GB RAM equipped with SSD. Webscraper.IO and Tab Saver were used for web-scraping. Prior to building the dataset, images were collected from friends, family and acquaintances. Then with web-scraping, more images were retrieved. This was done in order for the system to learn from the Mauritian heterogeneous population including Caucasian ones. The dataset images were cropped to fit the face images, resized to 224 by 224 and converted to grayscale. Part of the dataset is shown in Fig. 2.

Angry

Disgust

Fear

Happy

Fig. 2. EmoTunes dataset

Prior to training, the images were imported and converted into numpy arrays and normalized by dividing by 255 to get color values from 0 to 1 and split into 80:20 ratio

Music Recommendation Based on Face Emotion Recognition

187

for training and testing respectively. Data augmentation was applied in order to generate new images from the existing instances of the dataset. The techniques included a rotation range of 1, width and height shift of 0.2, brightness ranges from 0.4 to 1.5, shear and zoom range of 2 and horizontal flip. Figure 3 shows the distribution of EmoTunes dataset’s class distributions after augmentation. Before augmentation, the total number of images was 3603 as opposed to 4203, after augmentation.

Fig. 3. EmoTunes dataset after augmentation

Six State-Of-The-Art (SOTA) models ResNet-50, MobileNetV2, DenseNet201, Xception, InceptionV3 and VGG-16 were investigated. The earlier layers of these models were frozen and new layers were added for face emotion recognition. The output consisted of a dense layer with 7 classes representative of the 7 emotion classes and Softmax activation. The pre-trained models were trained using Adam optimizer at a learning rate of 0.0001 and Sparse Categorical CrossEntropy for 75 epochs at batch size 5. ModelCheckpoint and ReduceLROnPlateau were used as model callbacks. Table 2. Performance metrics of Transfer Learned models Model ResNet-50 MobileNetV2 DenseNet201 Xception InceptionV3 VGG-16

Accuracy 97.0 91.0 90.0 89.0 87.0 76.0

Precision 97.0 91.0 90.0 89.0 87.0 77.0

Recall 97.0 92.0 90.0 88.0 87.0 76.0

F1-score 97.0 91.0 90.0 89.0 87.0 76.0

The 2 models with the highest metrics were ResNet-50 and MobileNetV2, as shown in Table 2. Hence, these 2 models were subject to Ensemble Learning. The ResNet50 and MobileNetV2 models were combined into a single one. The ensemble model was then created by taking the inputs and outputting the averaged values. The Ensemble model was trained again with the same hyperparameters and the .pb checkpoints are converted into TFLITE for android deployment. The Ensemble model amassed a 99.64% accuracy

188

P. Ramsaran and L. Nagowah

on its test set. Figure 4 shows the results for detection where 3 subjects were expressing the happy emotion. The several unique songs from the server were recommended in the list shown. Any song clicked on will open a music player screen. For the YouTube recommendation part, since the majority is ‘Happy’, it proposed a happy song.

Fig. 4. Three subjects expressing ‘Happy’ and recommendation results

Fig. 5. Two subjects expressing different emotions and recommendation results.

Figure 5 shows the results for the detection where 2 users were expressing the Disgust and Fear emotions. Unique songs fetched from the server were presented in the list. Since the person expressing the Disgust emotion is nearer to camera (shown as distance factor), the YouTube recommender chose a song related to disgust.

Music Recommendation Based on Face Emotion Recognition

189

5 Evaluation An evaluation was carried out by comparing the proposed system with similar systems to understand its uniqueness and originality. Table 3 shows the comparison of EmoTunes with similar systems that used CNN. Table 3. Evaluation of EmoTunes Reference

Dataset

[7]

FER2013

7

Shallow CNN

70

[8]

Mix of datasets

7

Shallow CNN

96.2

[9]

FER2013

3

Shallow CNN

82.5

[10]

FER2013

7

Shallow CNN

95

[11]

Manually made 5

Shallow CNN

78

[12]

CK+

3

Transfer Learning

99.6

[13]

JAFFE

7

Transfer Learning

100

[14]

FER2013

7

Transfer Learning

94.3

[15]

JAFFE

CNN-RNN

94.5

[16]

KDEF

7

Ensemble Learning by concatenating 78.3

[17]

CK+

7

Transfer Learning & SVM

99.3

7

Ensemble Learning by averaging

99.6

EmoTunes Custom

Classes Technique

Accuracy (%)

It could be deduced that EmoTunes manually collected dataset performed better than that of [11], surpassing from 78% to 99.6% in accuracy. Although several authors used shallow CNNs, the EmoTunes dataset exceeds their accuracies with 99.6% compared to their maximum accuracy of 96.2%. Meena and Mohbey [12] achieved 99.6% accuracy but detect only 3 emotions, positive, neutral and negative. With TL, [13] developed a system with 100% accuracy on JAFFE dataset. However, this dataset contains instances of only 10 different subjects. EmoTunes contains 4203 faces images of about 16 different Mauritians as well as web-scraped images. Hence, when it comes to TL, EmoTunes does a much better job in providing robustness to varying skin color and face structure, at a higher accuracy. Finally, [16] used a concatenating technique to combine 2 TL models and gained a 78.3% accuracy on the KDEF dataset. EmoTunes, however, provides an averaging technique to combine the two best TL models to give a final accuracy of 99.64%.

6 Conclusion In this paper, several works related to face emotion detection were analyzed to understand the most common and reliable techniques and the key factors to create a classification model that would outperform all of them. The dataset used was custom-made and collected by asking friends and acquaintances selfies of themselves expressing emotions. A

190

P. Ramsaran and L. Nagowah

novel approach was taken where different models were transfer-trained for the Ekman’s 7 emotions. Then the two best performing models were combined to give a robust and less biased classification model. For flexibility purposes, the deployment platform used was Android, giving more users the ability to perform FER anywhere. The user was able to take a picture, upload a picture or simply perform real-time detection on his phone and EmoTunes would keep recommending songs via a custom-made server, similar to Spotify. The user could also play his/her favorite songs from YouTube. By using the real-time detection, EmoTunes would not only be helpful to the blind community but could also assist a driver by simply placing his phone on his car dashboard and having music recommended to him. EmoTunes could be deployable in metro stations where it would capture a user’s emotion and suggest music from the server, while he is waiting for his commute. It could also be an innovative idea to implement EmoTunes software in waiting rooms, gyms, or sports arenas where cameras are already deployed. Finally, EmoTunes can be used by any user if he wishes to get music directly without having to manually search for it. Though EmoTunes has been successfully implemented, it does have some limitations. While other researchers were able to balance the FER2013 dataset and get working results, we were not able to achieve same on Google Colab. Instead of directly taking songs from YouTube, users of EmoTunes have to perform a few clicks to access functionality. Sometimes EmoTunes gave inaccurate results for the ‘Disgust’ and ‘Sad’ emotions. It also worked less precisely on faces that are less expressive. As future works, the plan is to collect additional images from more Mauritians in order to attain good performance for Disgust and Sad. Instead of extracting features from convolutional feature maps, facial landmarks detection can be implemented as different pose variations might give different results. According to these pose variations, a better recognition system could be investigated. For the music recommendations, a KNN approach can be used that takes into the valence/arousal of the emotions detected. Lastly, more songs can be added to the in-built database and the music recommendation can be fully automated.

References 1. Song, Z.: Facial expression emotion recognition model integrating philosophy and machine learning theory. Front. Psychol. 12 (2021). https://www.frontiersin.org/articles/10.3389/ fpsyg.2021.759485/full. Accessed 6 Sept 2023 2. Vemou, K., Horvath, A.: Facial Emotion Recognition. European Data Protection Supervisor (2023). https://edps.europa.eu/data-protection/our-work/publications/techdispatch/techdispa tch-12021-facial-emotion-recognition. Accessed 6 Sept 2023 3. Ekman, P.: Are there basic emotions? Psychol. Rev. 99, 550–553 (1992) 4. WHO: Depressive disorder (depression) (2023). https://www.who.int/news-room/fact-sheets/ detail/depression. Accessed 6 Sept. 2023 5. APA: Stress in America 2020: A National Mental Health Crisis. American Psychological Association (2020). https://www.apa.org/news/press/releases/stress/2020/report-october. Accessed 6 Sept 2023 6. MD, B.: Why is music good for the brain? Harvard Health (2020). https://www.health.har vard.edu/blog/why-is-music-good-for-the-brain-2020100721062. Accessed 6 Sept 2023

Music Recommendation Based on Face Emotion Recognition

191

7. Lasri, I., Solh, A., Belkacemi, M.: Facial emotion recognition of students using convolutional neural network. In: 2019 Third International Conference on Intelligent Computing in Data Sciences (ICDS), Marrakech, Morocco, 28–30 October 2019, pp. 1–6. IEEE (2019) 8. Ahmed, T.U., Hossain, S., Hossain, M.S., ul Islam, R., Andersson, K.: Facial expression recognition using convolutional neural network with data augmentation. In: 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Spokane, 30 May 2019, pp. 336–341. IEEE (2019) 9. Modi, S., Bohara, M.H.: Facial emotion recognition using convolution neural network. In: 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 06–08 May 2021, pp. 1339–1344. IEEE (2021) 10. Athavle, M., Mudale, D., Shrivastav, U., Gupta, M.: Music recommendation based on face emotion recognition. J. Inform. Electr. Electron. Eng. (JIEEE) 2(2), 1–11 (2021) 11. Pranav, E., Kamal, S., Chandran, C.S., Supriya, M.H.: Facial emotion recognition using deep convolutional neural network. In: 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 06–07 March 2020, pp. 317–320. IEEE (2020) 12. Meena, G., Mohbey, K.K.: Sentiment analysis on images using different transfer learning models. Procedia Comput. Sci. 218, 1640–1649 (2023) 13. Akhand, M.H., Roy, S., Siddique, N., Kamal, M.S., Shimamura, T.: Facial emotion recognition using transfer learning in the deep CNN. Electronics 10(9), 1036 (2021). MDPI 14. Zahara, L., Musa, P., Wibowo, E.P., Karim, I., Musa, S.B.: The facial emotion recognition (FER-2013) dataset for prediction system of micro-expressions face using the convolutional neural network (CNN) algorithm based Raspberry Pi. In: 2020 Fifth International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesia, 03–04 November 2020, pp. 1–9. IEEE (2020) 15. Jain, N., Kumar, S., Kumar, A., Shamsolmoali, P., Zareapoor, M.: Hybrid deep neural networks for face emotion recognition. Pattern Recognit. Lett. 115, 101–106 (2018) 16. Dhankhar, P.: ResNet-50 and VGG-16 for recognizing facial emotions. Int. J. Innov. Eng. Technol. 13(4) (2019) 17. Shaees, S., Naeem, H., Arslan, M., Naeem, M.R., Ali, S.H., Aldabbas, H.: Facial emotion recognition using transfer learning. In: 2020 International Conference on Computing and Information Technology (ICCIT-1441), Tabuk, Saudi Arabia, 09–10 September 2020, pp. 1–5. IEEE (2020) 18. Wahyono, I.D., Saryono, D., Ashar, M., Asfani, K., Sunarti: Face emotional detection using computational intelligence based ubiquitous computing. In: 2019 International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia, 21–22 September, pp. 389–393. IEEE (2019) 19. Zhong, L., Liao, H., Xu, B., Lu, S., Wang, J.: Tied gender condition for facial expression recognition with deep random forest. J. Electron. Imaging 29(2) (2020) 20. Flores-Juarez, U.A., Álvarez-Cedillo, J.A., Álvarez-Sánchez, T.: Two facial emotion detection based on Naive Bayesian classifier. J. Theor. Appl. Inf. Technol. 99(23), 1–10 (2021)

Utilizing New Technologies for Children with Communication and Swallowing Disorders: A Systematic Review Eugenia I. Toki1,2(B)

, Soultana Papadopoulou1

, and Jenny Pange2

1 Department of Speech and Language Therapy, School of Health Sciences,

University of Ioannina, Ioannina, Greece {toki,papasoul}@uoi.gr 2 Laboratory of New Technologies and Distance Learning, School of Education, University of Ioannina (UoI), Ioannina, Greece [email protected]

Abstract. Communication and swallowing abilities have proven to be essential components of our daily lives. People with communication and swallowing disabilities find it challenging to maintain relationships with family, friends, and social acquaintances, as well as to achieve social and academic goals. Innovative technology application in Speech and Language Pathology (SLP) is gaining attention. Digital and mobile technologies in health are continuously evolving, staging new opportunities from telepractice to monitoring, assessment and intervention procedures. This study focuses on reporting on the technical approaches and advances intended to support the clinical procedures and management of communication and swallowing disorders, with the potential to alleviate accessibility issues and improve the quality of SLT services for the pediatric population. The results of the systematic review demonstrated mostly utilization of telepractice procedures, use of smart systems, social robots, machine learning, speech processing, virtual reality and AAC. Future research is necessary to gain deeper knowledge on the use of new clinical digital approaches in the health care of children with communication and swallowing disorders to (i) support the children’s and families’, and (ii) suggest a positive economic impact according to current digital trends. Keywords: communication and swallowing disorders · dysphagia · child · smart system · telepractice · social robots · machine learning · speech processing

1 Introduction 1.1 Clinical Background: Communication and Swallowing Disorders According to the American Speech-Language-Hearing Association (ASHA), a communication disorder is “an impairment in the ability to perceive, process, understand and produce concepts, verbal and non-verbal and graphic symbol systems” [1]. ASHA distinguishes four types of communication disorders: a) speech disorders that affect the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 192–204, 2024. https://doi.org/10.1007/978-3-031-56075-0_19

Utilizing New Technologies for Children

193

processing or production of language sounds (phonology, motor planning, and executive function); b) language disorders that affect the understanding or production of semantics, morphosyntax, or speech; c) hearing disorders; and d) central auditory processing disorders [2]. Swallowing is a multifaceted neuromuscular, sensory- motor, and cognitive process. Swallowing, as a physiological function, ensures adequate nutrition and hydration by performing three vital functions at the same time: a) airway protection, b) nourishment, and c) hydration, all while significantly contributing to the maintenance of the human body [3, 4]. The term “dysphagia” is defined as any difficulty or inability to handle and transmit saliva, food of any consistency (liquids, semi-liquids, solids), and drugs/medication from the mouth cavity, through the pharynx, the esophagus, and finally into the stomach [5]. Communication and swallowing problems are especially common in children with neurological diseases. Speech and language therapists (SLTs) face the difficulty of providing sustainable and flexible services to address these persons’ unique and changing requirements, with the goal of sustaining functional communication and swallowing disorders. Problem management leverages information technology and robotics to deliver services remotely and has the potential to remove accessibility issues and increase the quality of SLT services for people with neurological conditions. Furthermore, a new study shows the value of using serious games in speech therapy sessions and how biofeedback is delivered [6]. 1.2 Technology, Sensors and Wearables as Assets in Diagnostic Procedures Technology has been a dominating force in the lives of children and adults for decades [7]. Technological advancements in speech therapy have improved diagnostic and intervention methods, especially for children [7–9]. Touchscreen, mobile devices are used in autism spectrum disorders to make communication easier [10]. Serious games are amongst the preferable methods for children, and can be used for screening and diagnosing autism, dyslexia, apraxia, and stuttering, as well as for planning skill-building interventions [11–16]. Eye tracking can help clinicians and educators understand what grabs attention and what is neglected, especially in toddlers at risk of autism [17–20]. Wearable sensors can record biomarkers such as heart rate, temperature, respiration rate, electrodermal activity, movement, and posture, which can be useful in measuring complicated health outcomes [21–24] with the employment of artificial intelligence (AI) approaches, i.e. neural networks, deep learning, machine learning [21, 25–28]. Innovative assistive technology in speech therapy is gaining increasing scientific interest as there are not many published studies on innovative intervention methods for children with speech, language, and swallowing disorders. According to (UNICEF Global Report on Assistive Technology, 2022), children with disabilities, primarily before their integration into the labour market, need major or minor adjustments or replacements of their assistive products to thrive, but also in order to be functional in contexts such as school, social and cultural, sporting activities and community participation [29]. Speech disorders, are also associated with brain dysfunction that prevents messages from being sent to the articulators, resulting in the inability to perform accurate vocal pronunciation. In some circumstances, such as apraxia, message transmission is

194

E. I. Toki et al.

impossible due to impairment to the motor programming of the structures involved in speech, resulting in the person’s inability to make speech. Traditional therapies involve a therapist who does activities to remediate such issues in individual sessions. A recent study in UK of non-communicative patients in intensive care units identified significant requirements for support material, software, and optimization of the content of augmentative alternative communication (AAC). Important factors affecting the adoption of AAC in the ICU were highlighted and included the need for staff training and bedside assessment of patients [30]. The use of computer vision and machine learning techniques to detect the positions of articulators and other anatomical structures of swallowing using a robot that can complete a therapy session partially or autonomously by independently performing the various parts of the therapy session through multimodal interaction is successful, giving a social robot the ability to assist therapists in speech production, and swallowing rehabilitation exercises [31]. Moreover, the steps taken to successfully use technology to guide individuals with speech, language, communication, and swallowing abilities to social integration are described [32]. According to the researchers, this process involves (1) identifying the individual’s strongest mode(s) of communication and swallowing, (2) matching the individual’s strengths with his or her personal goals and preferences, (3) developing a way to identify personal goals for using technology, and (4) selecting and training in the use of technologies that will support the individual in achieving his or her goals. This study aims to contribute to the increasing research area of digital health care and particularly on the utilization of technological advances to the detection and management of children with communication and swallowing disorders.

2 Materials and Methods 2.1 Research Questions This systematic review’s research questions are: Q1: What is the geographic distribution of current research on technological advances in communication and swallowing disorders? Q2: What is the current utilization of technology? Q3: How is technology employed to prevent communication and swallowing disorders in pediatric populations? To answer the questions of this study, the used the following methodology. 2.2 Search Strategy - Inclusion Criteria The present systematic review utilized the Assessment of Multiple Systematic Reviews (AMSTAR) [34] framework and adhered to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) [35] guidelines. In order to be eligible for final consideration, the publications had to satisfy the following requirements: • type of publication: original articles and reviews

Utilizing New Technologies for Children

• • • •

195

language of publication: English dates of publication: last 5 years publication process: peer reviewed all articles should address the utilization of technology and applications for children with communication and swallowing disorders.

A thorough examination of the existing literature was conducted utilizing the Google Scholar and PubMed databases. The search query employed on Google Scholar was as follows: [(technology OR computer OR smart OR online Or internet OR app OR mobile OR “Artificial Intelligence” OR AI OR “assistive technology” OR robot OR “serious games” OR telepractice OR telework OR teleintervention OR teletherapy OR telediagnosis OR teleconsultation OR elearning) AND (“communication disorder” AND (Dysphagia OR swallowing)) AND (child OR pediatric OR toddler)) AND (speech and language (therapy OR pathology) -surgery -adult]. Furthermore, the search query was also adapted to meet the requirements of PubMed. 2.3 Article Selection The PRISMA Flow Diagram (Fig. 1) shows that the search was conducted on two search engines, Google Scholar and Pubmed, which initially yielded 197 articles. In addition, 5 articles were found from other sources. After removing duplicate articles (by EIT), the final dataset contained 161 articles. The authors (EIT & SLP) reviewed the titles and abstracts of all articles, with any conflicts resolved by the authors (EIT & JP). Ultimately, 119 articles were excluded, leaving 42 full-text articles to be retrieved. After consultation with EIT and JP, it was concluded that 18 of these articles did not meet the inclusion criteria and were thus considered irrelevant. In total, 24 articles were included and used for data extraction and methodological quality assessment, as they met the specified criteria for inclusion. The search is visualized in the PRISMA Flow Diagram [33] (Fig. 1). 2.4 Data Extraction To answer the research questions, it is crucial to extract relevant information from the 16 publications in the established corpus. The extraction process should focus on several key aspects, including the geographic location, activity sectors, objectives pursued, methodologies used, and obtained outcomes. This will ensure that the necessary data is gathered and analyzed to address the research questions effectively.

196

E. I. Toki et al.

Fig. 1. PRISMA flow diagram

3 Results and Discussion There were 24 articles included in this systematic review. The first area of interest was to identify the geographic location of the activity sectors referred in the articles. According to the results of this study we found 9 research articles from USA (37.5%), 4 from Europe (16.67%), 9 from Asia (37.5%) 1 from Australia (4.17%) and 1 from Africa (4.17%) as described in Table 1. Geographic location. A visualization of the geographic distribution is presented in Fig. 2. To answer “Q2: What is the current utilization of technology?” we examined elements regarding: • the technology used, • the publication reference, • the purpose of use and the number of publications on that technology. Table 2 displays the current utilization of technology.

Utilizing New Technologies for Children

197

Table 1. Geographic location Geographic location

Reference

N

America

[34–42]

9

Europe

[2, 43–45]

4

Asia

[46–54]

9

Australia

[55]

1

Africa

[56]

1

Fig. 2. Visualization of Geographic Distribution

To answer “Q3: How is technology employed to prevent communication and swallowing disorders in pediatric populations?” we present main elements outlining the aim, method and results of the included publications in this systematics review. SLPs find telepractice modal convenient to provide evaluations and therapy to children with communication and swallowing disorders, to consult with parents, and other clinicians, but also find important the training of all parties on the use of the technology [34, 37, 39, 40, 47, 49, 51]. The use of Information and Communication Technologies (ICT) during therapy can have a positive effect on intervention for children with communication disorders, resulting in the development of various online speech and language therapy (OST) systems. One of the advantages of digital technology is its appeal attractive to children, along with the ease with which it simplifies the process of matching therapists with clinical competence [2]. Most systems built primarily to assist SLPs use supervised machine learning methodologies that are either desktop-based or mobile-phone-based applications. The findings

198

E. I. Toki et al. Table 2. Current utilization of technology

Technology

Reference

Purpose Of Use

N

Telehealth technology [34, 37, 40, 43, 47, 49, 51, 56] Telepractice; Teletherapy - investigate the parental opinion about effectiveness of teletherapy among children with communication disorders; Parent teleconsultation for treatment of food selectivity; Online delivery of care due to covid-19

8

Online Clinic System

[48]

Parents to be in touch with the speech-language pathologist

1

Simulation

[36]

Simulation environment for SLP training

1

Smart Toys

[50]

Development of a smart toy for 1 additional attention and guidance to understand different types of social events and life activities including a prediction module with a checker component to alert in at the time of abnormal behavior

AI, Machine Learning [35, 50, 52–54]

Automated algorithmic procedure that classifies a DLD; Classification Approach to Receptive and Expressive Communication in Intellectual Disability; Comparisons of speech parameterization techniques for classification; Novel SLT model with machine learning algorithms to better handle longer sentences in lower resource consumption, better interpretability;

5

Speech Processing

STT; Automatic Speech Recognition and Real-Time Transcription;

2

[42, 53],

(continued)

Utilizing New Technologies for Children

199

Table 2. (continued) Technology

Reference

Purpose Of Use

N

Robots

[2, 41, 44, 45],

Robot-based new therapeutic strategies that use technologies meant primarily to assist SLPs adapt and apply supervised machine learning approaches; Robot intervention impact of cognition on motor learning and skill acquisition; Social robots for alleviate anxiety;

4

AAC

[38, 39, 46]

Training experiences of AAC practitioners to improve in practitioner training and ultimately improve services for individuals with complex communication needs

3

Virtual Reality

[55]

Acceptance, barriers, and enablers of virtual reality technology in communication rehabilitation

1

show that speech therapy systems can have a significant impact on childhood speaking [48]. Another recent study explored the contribution of technology in bridging the communication gap between deaf and hearing people. In recent years, research on sign language translation based on neural translation frameworks has attracted much attention. Sign language translation (SLT) is an important application for this bridging [54]. The study of Fox et al., investigated the accuracy and potential clinical utility of two rapid transcription methods for narrative language samples from school-aged children (7; 5–11; 10 [years; months]) with developmental language disorder. The transcription methods used in this study comprised real-time transcription performed by speechlanguage pathologists (SLPs) and trained transcribers (TTs), as well as Google Cloud Speech automatic voice recognition [42]. Social Robots (SRs) can alleviate anxiety among children during painful procedures. In a clinical trial with 109 children, interaction with the NAO SR reduced salivary cortisol levels more than interaction with a study nurse or waiting with parents, suggesting that using SRs in emergency rooms can help reduce stress and improve the emotional well-being of children and their families [44]. Likewise, the previous study can contribute towards the training of managing stressful episodes in rare syndromes presenting obstructive symptoms like cough caused by chocking, food regurgitation, painful swallowing due to anatomical and functional abnormalities [57].

200

E. I. Toki et al.

SRs have not been extensively studied and used in SLT, despite their potential [45]. Promising results from limited research on social robots for children and adolescents with communication and swallowing disorders (CD) have also been reported [45]. Using SRs to improve communication skills in children and adolescents, as well as interactive play scenarios in SLP are only included presenting relevant information by disorder type, age, treatment method, robot type, etc. The authors discuss the methodological, technical, and ethical limitations of using SRs in SLP clinical or home settings and the huge potential of conversational AI as a secondary assistive technology for speech, language, and swallowing interventions. New therapeutic strategies that use technologies meant primarily to assist SLPs adapt and apply supervised machine learning approaches, which can be robot-based or mobile phone and computer-based applications. Our findings show that speech therapy systems can improve children’s communication and swallowing significantly. Collaboration between computer programmers and SLPs can aid in the implementation of effective automated applications, allowing more children to benefit from sophisticated speech therapy services. It should be mentioned that children with physical limitations, as well as children who live in remote places with limited access to speech therapy treatments, benefit substantially from these services [2]. Early interventions for infants with neurodevelopmental disorders such as Cerebral Palsy (CP) using robotic assisted movement devices have shown promising results in restoring motor function skills. As a neurodevelopmental disorder, cerebral palsy (CP) includes a variety of neurological disorders that occur from infancy or early childhood and persist throughout an individual’s life. Therefore, an early and valid intervention with technological means such as robots contributes to prevention, education in life situations and the rehabilitation of impairments before they become generalized [41]. The use of technology in the management of speech and swallowing problems requires familiarity and a good knowledge of the applications of its individualized programs, as noted in a study that examined graduate speech-language pathology students’ perceived readiness and confidence to work with AAC users in relation to their educational experiences [38].

4 Conclusions This systematic review demonstrates promising results of technology-based practices such as telehealth, machine learning, speech processing, robots, AAC, and virtual reality towards communication, swallowing, and social interaction. Future interdisciplinary research is needed for effective digital clinical tools and individualized practices in communication and swallowing disorders.

References 1. American Speech-Language-Hearing: Speech Sound Disorders: Articulation and Phonology: Overview. www.asha.org/Practice-Portal/Clinical-Topics/Articulation-and-Phonology 2. Attwell, G.A., Bennin, K.E., Tekinerdogan, B.: A systematic review of online speech therapy systems for intervention in childhood speech communication disorders. Sensors 22, 9713 (2022). https://doi.org/10.3390/s22249713

Utilizing New Technologies for Children

201

3. Rosenbek, J.C., Jones, H.N.: Dysphagia in Movement Disorders. Plural Publishing (2008) 4. Shaker, R., Easterling, C., Belafsky, P.C., Postma, G.N.: Manual of Diagnostic and Therapeutic Techniques for Disorders of Deglutition. Springer, New York (2012). https://doi.org/10.1007/ 978-1-4614-3779-6 5. Paik, N.-J., Kim, W.-S.: Recovery of swallowing. Clin. Pathw. Stroke Rehabil. Evid. Based Clin. Pract. Recomm., 85–95 (2021) 6. Aires, P.M.A.: BioVisualSpeech: deployment of an interactive platform for speech therapy sessions with children (2022) 7. Toki, E.I., Zakopoulou, V., Pange, J.: Preschoolers’ learning disabilities assessment: new perspectives in computerized clinical tools. Sino-US Engl. Teach. 11, 401–410 (2014) 8. Liu, X., Toki, E.I., Pange, J.: The use of ICT in preschool education in Greece and China: a comparative study. Procedia Soc. Behav. Sci.a Soc. Behav. Sci. 112, 1167–1176 (2014). https://doi.org/10.1016/j.sbspro.2014.01.1281 9. Anzulewicz, A., Sobota, K., Delafield-Butt, J.T.: Toward the autism motor signature: gesture patterns during smart tablet gameplay identify children with autism. Sci. Rep. 6, 1–13 (2016) 10. Koumpouros, Y., Kafazis, T.: Wearables and mobile technologies in autism spectrum disorder interventions: a systematic literature review. Res. Autism Spectr. Disord. 66, 101405 (2019) 11. Lekka, A., Toki, E., Tsolakidis, C., Pange, J.: Literature review on educational games for learning statistics. In: 2017 IEEE Global Engineering Education Conference (EDUCON), Athens, Greece, pp. 844–847. IEEE (2017) 12. Mavridis, A., Tsiatsos, T.: Game-based assessment: investigating the impact on test anxiety and exam performance: game-based assessment. J. Comput. Assist. Learn. 33, 137–150 (2017). https://doi.org/10.1111/jcal.12170 13. Boletsis, C., McCallum, S.: The smartkuber case study: lessons learned from the development of an augmented reality serious game for cognitive screening. In: De Paolis, L., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10324, pp. 457–472. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60922-5_36 14. Chaldogeridis, A., Politopoulos, N., Tsiatsos, T.: Designing exergames for working memory training using MaKey MaKey. In: Auer, M., Tsiatsos, T. (eds.) IMCL 2017. AISC, vol. 725, pp. 635–643. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75175-7_62 15. Mavridis, A.: Impact of online flexible games on students’ attitude towards mathematics. Educ. Technol. Res. Dev. 65, 1451–1470 (2017) 16. Mavridis, A., Tsiatsos, T., Terzidou, T.: Designing and deploying 3D collaborative games in education. Int. J. Game Based Learn. 6, 43–57 (2016). https://doi.org/10.4018/IJGBL.201601 0104 17. Ashraf, H., Sodergren, M.H., Merali, N., Mylonas, G., Singh, H., Darzi, A.: Eye-tracking technology in medical education: a systematic review. Med. Teach. 40, 62–69 (2018). https:// doi.org/10.1080/0142159X.2017.1391373 18. Bacon, E.C., Moore, A., Lee, Q., Barnes, C.C., Courchesne, E., Pierce, K.: Identifying prognostic markers in autism spectrum disorder using eye tracking. Autism (2019). https://doi. org/10.1177/1362361319878578 19. Borys, M., Plechawska-Wójcik, M.: Eye-tracking metrics in perception and visual attention research. EJMT 3, 11–23 (2017) 20. Özdemir, S., Kök, ˙I., Akin Bülbül, I., Özdemir, S., Töret, Z.: An eye-tracking data based decision support system for autism risk assessment (2018) 21. Alcañiz Raya, M., et al.: Application of supervised machine learning for behavioral biomarkers of autism spectrum disorder based on electrodermal activity and virtual reality. Front. Hum. Neurosci. 14 (2020). https://doi.org/10.3389/fnhum.2020.00090 22. Byrom, B., McCarthy, M., Schueler, P., Muehlhausen, W.: Brain monitoring devices in neuroscience clinical research: the potential of remote monitoring using sensors, wearables, and mobile devices. Clin. Pharmacol. Ther. 104, 59–71 (2018). https://doi.org/10.1002/cpt.1077

202

E. I. Toki et al.

23. Ueberham, M., Schmidt, F., Schlink, U.: Advanced smartphone-based sensing with opensource task automation. Sensors 18, 2456 (2018) 24. Mallegni, N., et al.: Sensing devices for detecting and processing acoustic signals in healthcare. Biosensors 12, 835 (2022). https://doi.org/10.3390/bios12100835 25. Georgoulas, G., Georgopoulos, V.C., Stylios, C.D.: Speech sound classification and detection of articulation disorders with support vector machines and wavelets. In: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2199–2202 (2006) 26. Xu, D., Gilkerson, J., Richards, J., Yapanel, U., Gray, S.: Child vocalization composition as discriminant information for automatic autism detection. In: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2518–2522 (2009) 27. Georgopoulos, V.C.: Advanced time-frequency analysis and machine learning for pathological voice detection. In: 2020 12th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP), pp. 1–5 (2020) 28. Pahwa, A., Aggarwal, G., Sharma, A.: A machine learning approach for identification amp; diagnosing features of neurodevelopmental disorders using speech and spoken sentences. In: 2016 International Conference on Computing, Communication and Automation (ICCCA), pp. 377–382 (2016) 29. Global report on assistive technology | UNICEF. https://www.unicef.org/reports/global-rep ort-assistive-technology 30. Mobasheri, M.H., et al.: Communication aid requirements of intensive care unit patients with transient speech loss. Augment. Altern. Commun. 32, 261–271 (2016). https://doi.org/10. 1080/07434618.2016.1235610 31. Castillo, J.C., Álvarez-Fernández, D., Alonso-Martín, F., Marques-Villarroya, S., Salichs, M.A.: Social robotics in therapy of apraxia of speech. J. Healthc. Eng. 2018, 1–11 (2018). https://doi.org/10.1155/2018/7075290 32. McCall, D.: Steps to success with technology for individuals with aphasia. Semin. Speech Lang. 33, 234–242 (2012). https://doi.org/10.1055/s-0032-1320043 33. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Prisma Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Int. Med. 151, 264–269 (2009) 34. Feehan, A., et al.: Rapid transition to telepractice across the lifespan in speech-language pathology: Insight from a survey of clinicians in Canada. Int. J. Speech Lang. Pathol., 1–13 (2023). https://doi.org/10.1080/17549507.2023.2181220 35. Walters, C.E., Jr., et al.: Automated phenotyping tool for identifying developmental language disorder cases in health systems data (APT-DLD): a new research algorithm for deployment in large-scale electronic health record systems. J. Speech Lang. Hear. Res. 63, 3019–3035 (2020) 36. Peker, H., Rosa-Lugo, L.I.: Speech-language pathology graduate students’ questioning strategies for English learners in a simulation environment. Top. Lang. Disord. 41, 347–356 (2021) 37. Campbell, D.R., Goldstein, H.: Evolution of telehealth technology, evaluations, and therapy: effects of the COVID-19 pandemic on pediatric speech-language pathology services. Am. J. Speech Lang. Pathol. 31, 271–286 (2022). https://doi.org/10.1044/2021_AJSLP-21-00069 38. Barman, B.E., Dubasik, V.L., Brackenbury, T., Colcord, D.J.: Graduate students’ perceived preparedness and confidence to work with individuals who use augmentative and alternative communication. Am. J. Speech Lang. Pathol. 32, 1165–1181 (2023). https://doi.org/10.1044/ 2023_AJSLP-22-00282 39. Douglas, S.N., West, P., Kammes, R.: The training experiences of augmentative and alternative communication practitioners in one midwestern state. Perspect. ASHA Spec. Interest Groups 5, 219–230 (2020). https://doi.org/10.1044/2019_PERS-19-00053

Utilizing New Technologies for Children

203

40. Bloomfield, B.S., Fischer, A.J., Clark, R.R., Dove, M.B.: Treatment of food selectivity in a child with avoidant/restrictive food intake disorder through parent teleconsultation. Behav. Anal. Pract. 12, 33–43 (2019). https://doi.org/10.1007/s40617-018-0251-y 41. Chandrashekhar, R., Wang, H., Rippetoe, J., James, S.A., Fagg, A.H., Kolobe, T.H.A.: The impact of cognition on motor learning and skill acquisition using a robot intervention in infants with cerebral palsy. Front. Robot. AI 9, 805258 (2022). https://doi.org/10.3389/frobt. 2022.805258 42. Fox, C.B., Israelsen-Augenstein, M., Jones, S., Gillam, S.L.: An evaluation of expedited transcription methods for school-age children’s narrative language: automatic speech recognition and real-time transcription. J. Speech Lang. Hear. Res. 64, 3533–3548 (2021). https://doi.org/ 10.1044/2021_JSLHR-21-00096 43. Law, J., Dornstauder, M., Charlton, J., Gréaux, M.: Tele-practice for children and young people with communication disabilities: employing the COM-B model to review the intervention literature and inform guidance for practitioners. Int. J. Lang. Commun. Disord. 56, 415–434 (2021). https://doi.org/10.1111/1460-6984.12592 44. Rossi, S., et al.: Using the social robot NAO for emotional support to children at a pediatric emergency department: randomized clinical trial. J. Med. Internet Res. 24, e29656 (2022). https://doi.org/10.2196/29656 45. Georgieva-Tsaneva, G., et al.: Exploring the potential of social robots for speech and language therapy: a review and analysis of interactive scenarios. Machines 11, 693 (2023). https://doi. org/10.3390/machines11070693 46. Chawla, M., Panda, S.N., Khullar, V.: Assistive technologies for individuals with communication disorders. In: 2022 10th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, pp. 1–5. IEEE (2022) 47. Kuriakose, T., Jijo, P.M., Lavanya, H.S., Malthesh, H.R.: Teletherapy for children with communication disorders: a survey on parental opinion. IJSA 4, 01–04 (2023) 48. Sarfraz, M., Almutairi, M.S., Jasem, Z.A.: A speech clinic system for children with communication disorder. Int. J. Extreme Autom. Connect. Healthc. IJEACH 2, 138–160 (2020) 49. Singh, S.J., Kamarudin, K., Sharma, S.: The implementation of telepractice by Malaysian speech-language pathologists during the COVID-19 pandemic. Commun. Sci. Disord. 27, 239–250 (2022). https://doi.org/10.12963/csd.21848 50. Jadi, A.: Improving the communication for children with speech disorder using the smart toys (2019). https://doi.org/10.48550/ARXIV.1906.04251 51. Rahim, S.: Impact of Covid-19 on Speech Therapy 52. Aggarwal, G.: Social communication disorder outside intellectual disability? A review and classification approach to receptive and expressive communication in intellectual disability. Wirel. Pers. Commun. 127, 2841–2858 (2022). https://doi.org/10.1007/s11277-022-09899-1 53. Aggarwal, G., Singh, L.: Comparisons of speech parameterisation techniques for classification of intellectual disability using machine learning. In: Research Anthology on Physical and Intellectual Disabilities in an Inclusive Society, pp. 828–847. IGI Global (2022) 54. Zheng, J., et al.: An Improved sign language translation model with explainable adaptations for processing long sign sentences. Comput. Intell. Neurosci. 2020, 1–11 (2020). https://doi. org/10.1155/2020/8816125 55. Vaezipour, A., Aldridge, D., Koenig, S., Theodoros, D., Russell, T.: “It’s really exciting to think where it could go”: a mixed-method investigation of clinician acceptance, barriers and enablers of virtual reality technology in communication rehabilitation. Disabil. Rehabil. 44, 3946–3958 (2022). https://doi.org/10.1080/09638288.2021.1895333 56. Abiodun, O.O., Tinuola, F.R.: Building resilience into the care of autistic children in Lagos state: a social worker’s perspective

204

E. I. Toki et al.

57. Wilkinson, J.M., Codipilly, D.C., Wilfahrt, R.P.: Dysphagia: evaluation and collaborative management. Am. Fam. Physician 103, 97–106 (2021) 58. Díaz-Boladeras, M., et al.: Assessing pediatrics patients’ psychological states from biomedical signals in a cloud of social robots. In: Kyriacou, E., Christofides, S., Pattichis, C. (eds.) XIV Mediterranean Conference on Medical and Biological Engineering and Computing 2016. IFMBE Proceedings, vol. 57, pp. 1185–1190. Springer, Cham (2016). https://doi.org/10.1007/ 978-3-319-32703-7_230

Fostering a Co-creation Process for the Development of an Extended Reality Healthcare Education Resource Juliana Samson(B)

, Petros Lameras , Natasha Taylor , and Rosie Kneafsey

Coventry University, Priory Street, Coventry CV1 5FB, UK [email protected]

Abstract. The aim of this research is to create an immersive healthcare education resource using an extended reality (XR) platform. This platform leverages an existing software product, incorporating virtual patients with conversational capabilities driven by artificial intelligence (AI). The initial stage produced an early prototype focused on assessing an elderly virtual patient experiencing frailty. This scenario encompasses the hospital admission to post-discharge care at home, involving various healthcare professionals such as paramedics, emergency clinicians, diagnostic radiographers, geriatricians, physiotherapists, occupational therapists, nurses, operating department practitioners, dietitians, and social workers. The plan moving forward is to refine and expand this prototype through a cocreation with diverse stakeholders. The refinement process will include the introduction of updated scripts into the standard AI model. Furthermore, these scripts will be tested against a new hybrid model that combines generative AI. Ultimately, this resource will be co-designed to create a learning activity tailored for occupational therapy and physiotherapy students. This activity will undergo testing with a cohort of students, and the outcomes of this research are expected to inform the future development of interprofessional virtual simulated placements (VSPs). These placements will complement traditional clinical learning experiences, offering students an immersive environment to enhance their skills and knowledge in the healthcare field. Keywords: Extended Reality · Artificial Intelligence · Interprofessional Healthcare Education

1 Research Summary 1.1 Introduction Immersive technologies such as XR and conversational AI have the potential to engage healthcare students in simulations by creating an interactive environment. However, their use in augmenting the development of clinical skills remains a novel area of research and practice [1]. Further, virtual platforms are an ideal way to deliver interprofessional IMCL Doctoral Consortium 2023. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 205–212, 2024. https://doi.org/10.1007/978-3-031-56075-0_20

206

J. Samson et al.

education (IPE), as they can overcome the scheduling constraints involved with coordinating programmes across different locations and schedules [2]. The aim of this research is to design an IPE resource which utilises a virtual patient with conversational AI capabilities. The design of the resource will be conducted as co-creation research to involve collaboration between service users, academics, practice partners (clinician-educators), industry, health and technology students. The resource will be used by the university to augment practice placement capacity. As will be outlined in the following section, the development of this resource addresses research gaps and identified needs in healthcare education. 1.2 Background Practice placements are important activities in the training of healthcare students as they contextualise academic learning through case-based interaction. Traditionally, practice placements are conducted in the workplace under the supervision of senior clinicians who support students in their management of service users presenting for care. Learning objectives in practice placements align with standards that set out the required competencies and number of hours necessary for registration on graduation [3–5]. Thus, practice placements are essential to the development of practice-ready healthcare graduates. Simulation-based education is considered an alternative to traditional workplace training. Healthcare simulation is an approach that provides experiential learning through a scenario designed to imitate actual clinical events [6]. Compared with traditional practice placements, simulation can provide a safe space for students to learn from rare events and high-risk scenarios without mistakes causing harm to real persons [7]. The pandemic was a catalyst for the rapid innovation of simulation-based clinical education in the virtual space. Lockdowns drove the necessity for centres to move their workplace learning online in order for students to complete the necessary practice placement hours for graduation [8]. These virtual simulated placements (VSPs) were met with positive responses and will form part of a strategy to augment placement capacity moving forward [9]. Thus, VSPs could be considered state-of-the-art in technology-enhanced learning for clinical education. There is an urgent need to alleviate the pressure on the UK National Health Service (NHS) as a provider of in-person practice placements. This organisation faces unprecedented pressures post-pandemic, and in response, the government commissioned a Long Term Workforce Plan [10]. This ambitious plan aims to double the number of medical school places, increase nursing student intake by 92%, increase the proportion of apprenticeship routes from 7% to 22% and reduce the reliance on international recruitment from 25% to 9–10% by 2031/32. As cited in the report, one of the key challenges to reaching these targets is the availability of practice placements. The Nursing and Midwifery Council recognise the value of technology-enhanced learning to augment placement by doubling the allocation of hours approved for simulation, virtual, digital, and other contemporary learning approaches in their post-COVID recovery standards [11]. An ageing population presenting with multiple co-morbidities places additional and increasing pressures on NHS care services. Frailty in the elderly is a complex syndrome which increases the susceptibility to rapid decline, resulting in falls, disability, hospitalisations, nursing home admissions, and mortality [12]. The cost to the NHS is estimated

Fostering a Co-creation Process for the Development

207

at £5.8 billion [13], and this is set to rise sharply as the number of persons over the age of 85 is projected to increase by 55% by 2037 [14]. The NHS long-term Workforce Plan specifically mentions frailty as a current and increasing challenge, requiring integrated care pathways and clinicians with generalist skills to facilitate interdisciplinary, personcentred care [10]. The growing challenge of frailty not only places additional pressure on an already stretched NHS but it also requires the training of students in generalist skills. To address this training need, the Enhance Programme was launched by Health Education England in April 2022. The Enhance Programme is designed to equip clinicians with the generalist skills necessary for multidisciplinary working to manage complex conditions, such as frailty in the elderly, as a joined-up service [15]. IPE is a pedagogical approach to support students to learn with, from and about one another to develop generalist skills [16]. The purpose of IPE is to foster communication, teamwork, and leadership between different disciplines, to produce graduates capable of delivering quality care in complex presentations [17]. VSPs are ideally suited to IPE, as they allow synchronous activity from multiple remote locations and asynchronous activity to overcome the timetabling challenges of co-ordinating different programme schedules. Another theme that cuts across the Enhance Programme is the use of digital technologies to improve healthcare outcomes [15]. The NHS long Term Workforce plan also discusses the importance of embracing advances in technology, such as AI to free up clinicians’ time for providing more personalised care and to improve overall service efficiency [10]. Related to this is the need for education to raise the digital literacy of the existing and future workforce [18]. In summary, there is a clear need for the development of VSPs. The current climate for healthcare training providers requires the increased recruitment of an agile, local workforce skilled in technology, lifelong learning, and the ability to substitute skills across professions [19]. VSPs can augment practice placement, thus alleviating some pressure on placement capacity. Simulation delivers advantages over traditional placements in providing a safe space for experiential learning. There is also the potential to host IPE in virtual spaces to support the learning of generalist skills for managing complex presentations as a team. Finally, technology-enhanced learning aligns with the drive to support digital literacy in the workforce. 1.3 Goals and Results A scoping review of VSPs was conducted across healthcare education research to define the state-of-the-art and identify gaps for future research. This review is pending publication, but the protocol is registered [20]. Identified gaps as relevant to the planned research project include a lack of representation across allied health and an absence of IPE placements. Another gap identified in the scoping review was a lack of broad and extended stakeholder involvement. All VSPs in the 28 review papers were designed by university staff. Eight teams consulted practice partners, four consulted students, three developed needs assessments from student surveys, and none involved patient groups. The current project will include a wide range of stakeholders throughout the whole creation cycle, using co-creation and design thinking principles [21, 22].

208

J. Samson et al.

Although six of the VSPs included virtual patients in extended reality (XR), all used menu-based options with pre-set questions for the students to select when taking a history. The current project utilises existing commercial software (PCS Spark™ https://www. pcs.ai/spark/), with conversational AI capability. The intent behind using this software is to provide greater interaction and flexibility for the students when taking a history. The AI technology uses speech-to-text, a grammatical analyser, a neural network classifier and speech synthesis. One of the limitations of the current classification AI model is the lack of programmed response in instances where a student might ask questions outside of the pre-defined categories. Further advances in generative AI are in process by the software developers, and the plan is to test a hybrid model against the current model as part of this research. An IPE scenario was developed as an early prototype in consultation with the software creation team, university staff, students, and NHS practice partners. The scenario features a virtual patient named Sarah Potter. Sarah presents as an 80-year-old woman with frailty syndrome and can be interviewed by multiple professionals across two consecutive time points. The first conversation takes place on admission to hospital and the second on follow-up at home. Scripts from paramedics, emergency clinicians, diagnostic radiographers, geriatricians, physiotherapists, occupational therapists, nurses, operating department practitioners, dietetics, and social workers were entered into the system, and 68 user testing sessions took place to train the conversational AI using human reinforcement training. As VSPs are a new area of research, no standardised outcome measures for student evaluation were identified in the literature. Across all of the papers in the scoping review, surveys were bespoke, designed by the authors, and lacked any validation or reliability testing. Therefore, all the survey questions relating to student evaluation were collated from the 28 VSP papers, and the development of a standardised survey is planned. Figure 1 illustrates how the scoping review and early prototype inform the planned research stages. 1.4 Research Plan As the project currently stands, the scoping review is complete and has been submitted for publication. At a minimum, a pre-print will be ready in time for the IMCL conference. The early prototype is complete, and a poster has been accepted for the UK Association for Simulated Practice in Healthcare Conference on the 7th of November 2023. The generated test data is collated but needs to be analysed to inform the future AI testing phase. Focus groups with students to identify their learning needs with respect to VSPs are scheduled for November/December 2023. The data will inform empathy maps ahead of the co-creation. The focus groups will also refine the student evaluation survey questions for later use in testing the intervention. A pilot umbrella review search identified four reviews of qualitative research examining the lived experience of people with frailty. The themes in these reviews will be analysed to inform the empathy maps ahead of the co-creation. This test analysis will also inform the protocol for a full umbrella review ahead of qualitative interviews of people with frailty for data triangulation.

Fostering a Co-creation Process for the Development

209

Fig. 1. Research Plan

A pilot of the co-creation is scheduled for the 20th of October 2023, and the results of this process will be presented at this IMCL session for feedback before proceeding to the first co-creation workshop, scheduled for January 2023. First Co-creation. The first co-creation workshop focuses on designing or redesigning the simulation scenario and scripting as needed across all of the professions included in the existing early prototype. The co-creators will include volunteers across paramedicine, nursing, geriatric medicine, diagnostic radiography, operating department practitioners, dietetics, social work, occupational therapy, and physiotherapy from the following stakeholder groups: students, staff and practice partners. The team will also include industry partners, technology students, staff, and service users. The setting will be a hybrid space: i.e., a face-to-face space on campus and an online platform (Engageli https:// www.engageli.com/). The online session will be recorded, and whilst cameras can be used for rapport purposes, participants will be invited to switch off their cameras if this is preferred.

210

J. Samson et al.

Data Entry and Testing of the AI Models. Once the agreed scenario and scripting are finalised, the data will be entered into the current model and replicated for a model powered by hybrid AI technology. As illustrated in Fig. 2 below, half of the participants will be randomly allocated to converse with the classification AI model, and the other half will converse with the hybrid (classification and generative) AI model. The questions asked of one model will be repeated in the other so that the full set of questions are in each, and order effects are controlled for. Then, both sets of responses will undergo human reinforcement training.

Fig. 2. Study design to compare AI performance between the two models.

Second Co-creation. For the second co-creation session, volunteers from occupational therapy and physiotherapy courses across students, staff and practice partners will be included, with industry partners on hand to support. The aim of this second workshop is to refine the prototype and design it for use as an occupational therapy and physiotherapy IPE resource. This session will also be hybrid over the Engageli platform. Co-Design in XR with the Simulation Team. The overarching aim of this co-design phase is to design an IPE-VSP activity. This activity will use the PCS Spark™ resource that was built in the previous co-creation phase. The activity, hosted on a WordPress site, will be designed for IPE between Occupational Therapy and Physiotherapy students in

Fostering a Co-creation Process for the Development

211

lieu of traditional practice placement hours. This activity and its co-design will inform future research directions for the development of a full IPE-VSP. The university’s simulation team will process the prepared co-creation outputs in an XR space (Microsoft Workrooms) to foster remote working in a creative environment. Testing. The IPE-VSP activities will be tested with occupational therapy and physiotherapy students on an IPE placement, and a focus group will be included in the evaluation. Other measures will include student demographics from survey data captured pre-participation and Readiness for Interprofessional Learning (RIPLS) using a validated questionnaire pre- and post-participation [23]. Usability post activity will be measured by the System usability scale https://www.usability.gov/how-to-and-tools/met hods/system-usability-scale.html and student engagement will be measured by website metrics. Future Directions. Post facto arguments from this proof-of-concept research will be derived from reflecting on all the research phases to generate future recommendations. The outputs of this phase will include: A Process Toolkit. A set of templates and guides to assist researchers undertaking the project of building VSPs using the principles of design thinking and co-creation. A Strategy Report. Which considers the governance aspects of implementing an IPEVSP into the curriculum. A Research Proposal and Protocol. For the development of a full IPE-VSP underpinned by co-creation and tested in empirical research.

References 1. Immersive Healthcare Collaboration: The Role of Immersive Technology in Healthcare Training & Education: Three Principles for Progress. Centre for Immersive Technologies, Leeds, UK (2021) 2. Qiao, J., Xu, J., Li, L., Ouyang, Y.Q.: The integration of immersive virtual reality simulation in interprofessional education: a scoping review. Nurse Educ. Today 98, 104773 (2021). https:// doi.org/10.1016/j.nedt.2021.104773 3. General Medical Council: Promoting excellence: standards for medical education and training (2015). www.gmc-uk.org/education/standards.asp. Accessed 07 Feb 2022 4. Health and Care Professions Council: Your duties as an education provider (2017). www. hcpc-uk.org. Accessed 07 Apr 2022 5. Nursing and Midwifery Council: Standards for student supervision and assessment (2018). www.nmc.org.ukStandardsforstudentsupervisionandassessment1. Accessed 07 Apr 2022 6. Society for simulation in Healthcare. www.ssih.org/About-SSH/About-Simulation. Accessed 01 June 2022 7. Gaba, D.M.: The future vision of simulation in health care. Qual. Saf. Health Care 13(1), 2–10 (2004) 8. Goh, P.S., Sandars, J.: A vision of the use of technology in medical education after the COVID19 pandemic [version 1]. MedEdPublish 9(49) (2020). https://doi.org/10.15694/mep.2020. 000049.1

212

J. Samson et al.

9. St John-Matthews, J., Hobbs, C.: Helping to ensure an essential supply of Allied Health Professions (AHP) Practice Placements: challenges and solutions (2020). www.hee.nhs.uk/our-work/allied-health-professions/increase-capacity/practice-pla cements-challenges-solutions. Accessed 04 Apr 2022 10. National Health Service: Long Term Workforce Plan. NHS publications (2023). www.eng land.nhs.uk/publication/nhs-long-term-workforce-plan/. Accessed 04 July 2023 11. Nursing and Midwifery Council: Recovery standards (2022). https://www.nmc.org.uk/glo balassets/sitedocuments/education-standards/current-recovery-programme-standards.pdf. Accessed 04 July 2023 12. Dent, E., Kowal, P., Hoogendijk, E.O.: Frailty measurement in research and clinical practice: a review. Eur. J. Intern. Med. 31(3), 3–10 (2016) 13. Han, L., Clegg, A., Doran, T., Fraser, L.: The impact of frailty on healthcare resource use: a longitudinal analysis using the Clinical Practice Research Datalink in England. Age Ageing 48(5), 665–671 (2019) 14. Office for National Statistics: Population projections, January 2022. https://www.ons.gov.uk/ peoplepopulationandcommunity/populationandmigration/populationprojections/#datasets. Accessed 04 July 2023 15. Health Education England Enhance: Programme Handbook (2022a). https://www.hee.nhs.uk/ our-work/enhancing-generalist-skills/enhance-learning-resources/handbook. Accessed 20 July 2023 16. Barr, H., et al.: Interprofessional education guidelines 2017. Prepared for CAIPE (2017). www.caipe.org/resources/publications/caipe-publications/caipe-2017-interprofessionaleducation-guidelines-barr-h-ford-j-gray-r-helme-m-hutchings-m-low-h-machin-reeves-s. Accessed 10 Aug 2022 17. CAIPE: CAIPE Strategy 2022–2027. Centre for the Advancement of Interprofessional Education (2022). www.caipe.org/strategy#1. Accessed 15 Nov 2022 18. Health Education England: Harnessing digital technologies for workforce development, education and training: an overview (2022b). https://www.hee.nhs.uk/our-work/innovationdigital-transformation/harnessing-digital-technologies-workforce-development-educationtraining-overview. Accessed 05 Dec 2022 19. Anderson, M., et al.: Securing a sustainable and fit-for-purpose UK health and care workforce. Lancet 397(10288), 1992–2011 (2021) 20. Samson, J., Gilbey, M., Taylor, N., Kneafsey, R.: Virtual simulated placements in healthcare education: a scoping review protocol (2022). osf.io/ay5gh. https://doi.org/10.17605/OSF.IO/ AY5GH 21. Sanders, E.B.N., Sappers, P.J.: Convivial Toolbox: Generative Research at the Front end of Design. BIS (2008) 22. Plattner, H., Meinel, C., Leifer, L.: Design Thinking: Understand – Improve – Apply. Springer, Berlin (2011). https://doi.org/10.1007/978-3-642-13757-0 23. Parsell, G., Bligh, J.: The development of a questionnaire to assess the readiness of health care students for interprofessional learning (RIPLS). Med. Educ. 33, 95–100 (1999)

Mobile Health Care, Healthy Lifestyle and Training

Enhanced Web Platform for Optimizing Medical Fundraising for a Charitable Fund Nurkhan Issin, Azamat Salamat, Assanali Aidarkhan, and Mariza Tsakalerou(B) Nazarbayev University, Astana 010000, Kazakhstan [email protected]

Abstract. This study focuses on the development of an innovative digital charity platform, concentrating primarily on accumulating funds for children requiring medical treatments. Medical crowdfunding has emerged in recent years as a noteworthy form of online fundraising, offering a pivotal alternative financial avenue for high-cost projects, especially medical treatments for critically ill patients devoid of sufficient funds. The study investigates the expansive reach and potential of crowdfunding initiatives, exploring its historical significance and global impact, and highlighting its pertinence across various sectors. In essence, medical crowdfunding serves as an essential monetary conduit for individuals facing severe health predicaments, facilitating the procurement of financial aid from diverse sources, encompassing charitable funds, governmental aid, and private donors. Advances in mobile communication technologies have significantly expanded the reach of crowdfunding, employing online and mobile channels to captivate a more extensive audience. This research scrutinizes the driving factors behind contributors’ participation in medical crowdfunding, revealing the altruistic impulses stimulated by compelling, emotion-evoking narratives. It then integrates these factors in the design, implementation, and assessment of a dedicated medical crowdfunding platform. A real-world campaign, launched in collaboration with a charitable entity in Kazakhstan, serves as a testbed, to assess the platform’s effectiveness. The proposed platform underscores the vital role of technological innovation in eliciting emotional responses and altruistic contributions. Keywords: medical fundraising · interactive communication tools · social media platforms

1 Introduction The principal objective of this research is to develop an innovative digital charity platform designed to gather funds to facilitate children’s medical treatments. Recently, the online landscape has offered opportunities for various sectors, including medical crowdfunding (Hoefer 2012), which seeks to raise financial support for critically ill patients who have limited financial resources to cover their treatment expenses. Crowdfunding may substitute conventional financing for costly projects and initiatives. To attract investors, fund seekers provide a comprehensive project description, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 215–225, 2024. https://doi.org/10.1007/978-3-031-56075-0_21

216

N. Issin et al.

specify the amount, and set a deadline. It can foster new enterprises and introduce valuable products to the market. Besides accumulating funds, project originators can also assess demand or backing from potential supporters (Kuppuswamy and Bayus 2018). Among the earliest recorded instances of public crowdfunding initiatives, Joseph Pulitzer is notable for leveraging his newspaper to raise funds for the construction of the pedestal for the Statue of Liberty (Allison et al. 2017). Numerous independent artists, inventors, entrepreneurs, and public figures have achieved success by relying on crowdfunding for their ventures (Steinberg 2012). In 2008, crowdfunding played a significant role in Barack Obama’s election campaign, earning him the title of the “fundraising phenomenon” among journalists (Tran 2008). Medical crowdfunding offers a financial solution for economically disadvantaged individuals with health conditions, unable to bear significant treatment expenses. Alternative support includes charitable funds, whether public or private, government assistance, or backing of affluent benefactors (Selby 2023). In medical crowdfunding, individuals create public campaigns outlining their conditions and payment details to receive donations. These campaigns often include personal stories or media to evoke emotional responses and encourage donations. Donors typically contribute small amounts, which cumulatively meet the recipient’s financial needs. A parallel can be drawn between crowdfunding and the equilibrium of supply and demand in retail services. Just as retail services match supply to consumer demand, crowdfunding aligns suppliers (donors) with those in need (beneficiaries), allocating resources where most desired (Young and Scheinberg 2017). Medical crowdfunding differs from crowdfunding aimed at commercial or artistic projects in such a sense that contributors are primarily driven by their intrinsic motivations and genuine altruism, not by materialistic gains. Studies suggest that individuals with a propensity to assist others are more prone to participating in online medical crowdfunding campaigns (Hou et al. 2021). Eliciting emotional responses in prospective donors is typically used to trigger the propensity for making contributions. Since mobile communication technologies have developed, crowdfunding has shifted from its in-person nature to online and mobile platforms, increasing campaign audience. This paper includes the design, development, execution, and assessment of a specialized crowdfunding platform tailored for medical fundraising. It incorporates methods and techniques from existing literature to bolster campaign credibility and confidence. The platform’s efficacy was scrutinized through a real-world campaign initiated by a charitable organization in Kazakhstan. The Literature Review explores analogous platforms and alternative strategies for medical fundraising globally and within Kazakhstan. The Methodology section describes steps taken for the literature review and the tools and strategies used to create the platform, elaborating on technical aspects. Section 4 presents and discusses campaign results, labeled as Results and Discussions. Lastly, the Conclusion section encapsulates deductions and implications for fundraisers and platform administrators.

Enhanced Web Platform for Optimizing Medical Fundraising

217

2 Literature Review 2.1 Online Crowdfunding Around the World Fundraising initiatives and donation behaviors are strongly influenced by the cultural and socioeconomic factors in diverse nations (Snyder et al. 2016; Ren et al. 2020). This influence may be present in the form of varying nature of contributions, choices of payment methods and platforms, donation sizes, and the manner in which the fundseekers portray themselves (Snyder et al. 2017). The prevalent embrace of user-friendly online payment approaches, along with communal culture and limited social security provisions, has propelled the rise of online crowdfunding in China (Jin 2019; Huang et al. 2021; Xu and Wang 2020). Platforms like GoFundMe and YouCaring are favored where individuals utilize diverse tactics to enhance donations. For instance, in Canada, strategies encompass establishing personal ties with donors, emphasizing the severity of the patient’s condition, and illuminating the recipients’ societal contributions (Snyder et al. 2017). A study by Zhang et al. (2021) examined GoFundMe, an American crowdfunding platform, finding health-related campaigns tend to achieve better success. In India, the emergence of online crowdfunding platforms gained momentum during the COVID-19 pandemic, though they are still expanding due to challenges like limited Internet coverage, public trust issues, and a lack of regulatory policies (Bishnoi et al. 2022). The cultural and historical significance of charity is noteworthy in India, and neighboring Bangladesh (Suresh et al. 2020). 2.2 Charitable Fundraising in Kazakhstan In Kazakhstan, charity and kindness are culturally significant, manifesting in support for non-profit organizations, volunteer work, and general benevolence. Charitable organizations tend to assist people with poverty alleviation, healthcare, education, and environmental preservation, while religious institutions offer social services to aid the disadvantaged (Asyldin 2020; Muftyat 2021). However, instances of fraud have eroded donation trust, lowering Kazakhstan’s position in the World Philanthropy Index (Turan Times 2022). This section highlights three dominant charitable fundraising modes in Kazakhstan: social media-propelled, volunteer-led initiatives, and campaigns by charitable entities. The rapid expansion of internet and social media in Kazakhstan presents a distinct opportunity for charitable fundraising to reach wider audiences (Kemp 2022). Social media channels empower individuals to promote campaigns, connect with donors, and leverage content creators and influencers for wider outreach (Freeconference 2022; Berezhnoy 2018). Fundraising via volunteer work, wherein volunteers collect donations, often publicly, is also popular. In both methods, credibility and transparency concerns have arisen, highlighting the importance of transparency, accountability, and clear communication about fund usage (Khabar 24, 2022; Burtch B. 2014). Consequently, Kazakhstan has implemented regulatory measures for charitable activities (Boyarova and Sultangazy 2021). Local charitable organizations are quite important in organizing campaigns, raising funds, and informing the public about charity in general (Turan Times 2022). They

218

N. Issin et al.

engage in partnerships with corporations via corporate social responsibility initiatives which can take advantage of tax incentives by the government (Altynbayev 2022). Nevertheless, issues related to the availability of accurate information and public trust in such foundations present challenges that constrain charitable endeavors (Egov 2023; Ermekkyzy 2021). Incidents of fraudulent activities further decrease the level of public trust, underscoring the necessity for organizations to prioritize transparency, dissociate from such misconduct, and improve the confidence in the donors (Kaztag 2022). 2.3 The Matter of Trust in Digital Crowdfunding Within the realm of crowdfunding, trust emerges as the paramount factor and discrepancies in information sharing can give rise to trust-related concerns (Swan and Nolan 1985). When the information on a crowdfunding platform contains inaccuracies, it erodes the level of trust. To counteract this issue, the integration of elements aimed at promoting social responsibility can enhance the trustworthiness of the recipient (Gefen and Straub 2004). One of the goals of managers of charitable funds and platforms is to develop effective strategies and tools for trust generation. The donation behavior is strongly influenced by the specific design elements and the level of trust in the organization (Küchler et al. 2020). The responsibility of clarifying the crowdfunding concepts and explaining what the donations will be spent on, especially to the new users, lies with the campaign initiators (Ferreira et al. 2022). A human touch can be added by featuring recipient pictures and personal stories, building a connection between the recipient and the donor (Young and Scheinberg 2017). Incorporating an online chat support system enables users to receive swift responses to their inquiries, thereby enhancing trust. Recognized external organizations can assist in formal registration and in drawing investments. In addition, demonstrating such partnerships can help to earn trust (Ba et al. 2022; Hou et al. 2022). The platform’s reputation can also be improved by showcasing previous successful campaigns, daily site visits, and progress made in each campaign (Ferreira et al. 2022). The latest security protocols are also important in building trust (Strohmaier et al. 2019). Revealing all associated dangers to the users enhances transparency and instills trust in the platform. Notifications about upcoming events via an app or email and facilitating communication channels for donors to stay engaged and ask questions can increase donations (Kuchler et al. 2020).

3 Methodology 3.1 The Literature Review Process During this study, a systematic literature review was conducted in order to pinpoint the most important trends (Fig. 1). According to Lettieri et al. (2009), such a review represents a logical, clear, and replicable approach for analyzing the current topic. Concurrently, assembling and assimilating prior empirical evidence can aid in refining the descriptive and thematic understanding of the accumulated body of knowledge (Delbufalo 2012; Sivarajah et al. 2017).

Enhanced Web Platform for Optimizing Medical Fundraising

219

Fig. 1. Literature review process stages. Adapted from Aidarkhan et al. (2023).

Consequently, the objective of this literature review is to offer a synopsis of the existing state of crowdfunding within the healthcare domain. In the preliminary phase of the review, a search of the database was executed on Scopus, recognized as the most comprehensive database for peer-reviewed literature (Nath and Chowdhury 2021). The phrasing in the search queries amalgamated terms from both the crowdfunding and healthcare sectors. For example, most of the queries included strings like “crowdfunding,” “medical,” “trust*,” “online donations*,” “philanthropy*,” and “digital platform*”. 3.2 Development The architecture chosen for the application has two primary components: the client side and the server side (Fig. 2). The React.js framework was chosen for the development of the first part since it is the most popular JavaScript framework according to surveys (Stack Overflow 2022). The server side is an ASP.NET Web API application. It also uses the SQLite database management system. This database serves as a repository for information pertaining to the recipients, comments on the campaigns, and all donations.

Fig. 2. Application architecture. Adapted from Aidarkhan et al. (2023).

3.2.1 Client-Side Development The client side is tasked with presenting the information from the server-side to the end user. Thus, its responsibilities include constructing the user interface. The platform has 4 pages in total that are available to the end users on the client side: 1. The “Main” page (Fig. 3a) serves as the website’s initial landing page. 2. The “Patient” page (Fig. 3b) shows the details about the patient, the campaign, and the individual donations.

220

N. Issin et al.

3. The “Donation” page (Fig. 3c) features donation buttons that link to various banking apps and includes a survey for those who choose to contribute. 4. The “About” page (Fig. 3d) allows clients to access a description of the project and information about the team responsible for it.

Fig. 3. Web platform pages

3.2.2 Server-Side Development The Application Programming Interface (API) that was created for the server side houses the majority of the platform’s logic. Its responsibilities include interacting with data stored in the database and exchanging it with the client side. Besides its main function, it also performs some data manipulation, data validation, calculates the required values, and transforms data from one format to another. This also shifts some of the processing load from client devices to the server, culminating in a web platform that is more adeptly optimized for less powerful devices and guarantees quicker loading times. The data garnered by the server side is relayed to the client side in JSON format. This response is then accessed by the client side, which triggers a request during the page-loading process. This entire procedure is overseen by four principal controllers, each accountable for interfacing with data in separate areas: 1. The “Recipient” Controller is responsible for creating, retrieving, updating, and deleting information related to recipients.

Enhanced Web Platform for Optimizing Medical Fundraising

221

2. The “Transaction” Controller manages operations related to donations. 3. The “Comment” Controller oversees operations pertaining to comments made by donors to the campaigns. 4. The “Recipient File” Controller manages file operations, including handling patients’ images and documents.

4 Results and Discussions Table 1 and Fig. 4 contain the results obtained after conducting a test case with an actual campaign on the Qaiyrym.kz platform and their comparison with average numbers for one case by the Miloserdie Foundation. For the charitable fund, the average is among the similar cases, namely genetic research. The total funds amassed by the Qaiyrym.kz platform have exceeded the average amount garnered by Miloserdie by 233%, as shown in Fig. 4a. Meanwhile, the result of the platform is larger by 50% for the average individual contribution (Fig. 4b). Table 1. Donation amounts for the Qaiyrym platform and the Miloserdie fund. Result

Qaiyrym.kz

Miloserdie

Fundraising amount, KZT

300,382 ($665, e613)

91,445 (est.)

M individual donation, KZT

3,004

2,044

Fig. 4. a) Average Amount Collected Per Campaign: Qaiyrym vs Miloserdie. b) Average Size of an Individual Donation.

Considering the page visits statistics, users mostly viewed the main page; the patient page and the donation page come after it (Fig. 5a). An average user visited the patient page most often (6.66 times) (Fig. 5b). A possible explanation is that the visitors wanted to re-read the payment process description. The least visited page was the About page; the fewest views per page also may indicate that users did not perceive a need to visit it again. Regarding the videos, the one located on the main page received the most views, being watched 49 times, which equates to 10% of all main page visitors. Conversely, the video detailing the patient’s situation was viewed merely 20 times, constituting 4% of all page visitors.

222

N. Issin et al.

Fig. 5. a) Page Views for Each Individual Page. b) Number of Page Visits per User for Each Page. c) Percentage of Users Originating from Various Sources. d) Percentage of Users Who Made Donations from the Total Number of Visitors on Each Page and Donation Button Clicks.

The statistics of where the users came from suggest that 25% of them found the platform on social media channels while others landed on it directly (Fig. 5c). A small portion of users (2%) had interacted with the share button. About one out of every five website visitors ended up donating (Fig. 5d). Most users who clicked the ‘Donate’ button proceeded to make an actual donation. An intriguing statistic is that, on average, each user had to click the ‘Donate’ button 2.67 times. The unstructured interviews conducted to get feedback on the platform revealed problems with being redirected to the Internet banking apps and copying of the card number to the clipboard. Some other issues were the clarity of submitting words of support and encouragement, understanding the donation process, bank fees when the users’ bank was different from the beneficiary’s one, as well as delayed updates in the number of donations and collected sum on the platform. An online survey available to donors and other communication channels was employed to gather community feedback. In general, the assessments were quite positive; the majority of respondents graded their experiences as “Very Good”. From the survey, the most compelling factors that encouraged donors to make contributions were recommendations from peers and the user-friendly payment service. Many of the respondents expressed a strong inclination to recommend the platform to others. Some users recommended placing comprehensive reports on how the donations have benefited the campaign initiators and rearranging the user interface to make it more user-friendly and organized.

Enhanced Web Platform for Optimizing Medical Fundraising

223

5 Conclusion In summary, utilizing a web platform has enhanced the efficacy of medical fundraising campaigns, proving beneficial for children and families needing expensive treatments. Donors were provided an efficient avenue to support charitable medical campaigns. Implementing features such as payment security, progress tracking, and streamlined communication improved transparency and encouraged participation. Thorough review and implementation allowed the platform to meet specific requirements and address charity fund problems. Partnership with the fund significantly contributed to the platform launch’s success, further supported by favorable outcomes from the Minimum Viable Product (MVP) launch. The platform’s outcomes surpassed the average duration Miloserdie typically requires to meet financial needs for similar medical cases, amassing a total of 300,382 KZT from 100 donations within merely one week. These results attest to the platform’s effectiveness. Additionally, it seems the donor experience on the platform was positive, as the average contribution exceeded that made directly to the charity fund. Survey results suggest that recommendations from friends and family were the most influential success factor. Therefore, platform administrators and fundraisers should especially leverage the viral effect of social media to promote their campaigns. Usability and user-friendliness of the service, identified as a primary concern through metrics and interviews, also require attention from platform designers and managers to avoid deterring potential donors. In conclusion, the web platform has optimized the charity fund’s fundraising procedure and set the stage for future innovations in medical crowdfunding. The substantial impact on the lives of disadvantaged children and families, facilitated by generous donations, underscores its effectiveness and potential for broader inclusion in philanthropic activities.

References Aidarkhan, A., Salamat, A., Issin, N.: Development of a crowdfunding web-platform to improve the effectiveness of medical fundraising for a charity fund. Master’s thesis, Nazarbayev University. Nazarbayev University (2023). http://Nur.Nu.Edu.Kz/Handle/123456789/7209 Allison, T.H., Davis, B.C., Webb, J.W., Short, J.C.: Persuasion in crowdfunding: an elaboration likelihood model of crowdfunding performance. J. Bus. Ventur. 32(6), 707–725 (2017) Altynbayev, A.: Kakie Nalogovye Lgoty Predusmotreny Dl Too Pri Okazanii Blagotvoritelno Pomowi? (2022). https://Shorturl.At/Kswz8. Accessed 20 Feb 2023 Asyldin: If You Are Wide, You Will Not Be Less (2020). https://Asyldin.Kz/Article/View/Id/ 18157.Html. Accessed 18 Feb 2023 Ba, Z., Zhao, Y.C., Song, S., Zhu, Q.: Does the involvement of charities matter? Exploring the impact of charities’ reputation and social capital on medical crowdfunding performance. Inf. Process. Manag. 59(3), 102942 (2022) Berezhnoy, V.: How to Collect Money for Treatment in Social Networks? (2018). https://Med iglobus.Com/Ru/Tips-For-Patients-On-How-To-Collect-Money-For-Treatment/. Accessed 19 Feb 2023 Bishnoi, P., Shukla, S.K., Khanna, S., Sheetal, S., Singh, V.K., Banerjee, R.: Medical crowdfunding and COVID-19: awareness and perception of healthcare professionals and role of crowdfunding platforms in India. Int. J. Commun. Med. Public Health 9(4), 1884 (2022)

224

N. Issin et al.

Boyarova, V., Sultangazy, Z.: Suspicious Volunteers Activated in the Capital (2021). https://24.Kz/ Ru/News/Social/Item/483213-Podozritelnye-Volontery-Aktivizirovalis-V-Stolitse. Accessed 19 Feb 2023 Burtch, B.: Importance of Transparency (2014). https://Www.Thenonprofittimes.Com/Npt_Art icles/Importance-Transparency/. Accessed 19 Feb 2023 Delbufalo, E.: Outcomes of inter-organizational trust in supply chain relationships: a systematic literature review and a meta-analysis of the empirical evidence. Supply Chain Manag. Int. J. (2012) Egov: Charity Foundations of the Republic of Kazakhstan (2022). https://Egov.Kz/Cms/En/Art icles/Charity-Foundation. Accessed 18 Feb 2023 Ermekkyzy, D.: Urgent Need or Empty Hype: About Collective Charity and the Launch of the Tooba Application in Kazakhstan (2021). https://Shorturl.At/Buy68. Accessed 20 Feb 2023 Ferreira, V., Papaoikonomou, E., Terceno, A.: Unpeel the layers of trust! a comparative analysis of crowdfunding platforms and what they do to generate trust. Bus. Horiz. 65(1), 7–19 (2022) Freeconference: What Is Campaign Fundraising? (2022). https://Www.Freeconference.Com/En/ Blog/What-Is-Campaign-Fundraising/. Accessed 19 Feb 2023 Gefen, D., Straub, D.W.: Consumer trust in B2c e-commerce and the importance of social presence: experiments in e-products and e-services. Omega 32(6), 407–424 (2004) Hoefer, R.: From web site visitor to online contributor: three internet fundraising techniques for nonprofits. Soc. Work 57(4), 361–365 (2012) Hou, T., Hou, K., Wang, X., Luo, X.R.: Why i give money to unknown people? An investigation of online donation and forwarding intention. Electron. Commer. Res. Appl. 47, 101055 (2021) Hou, X., Wu, T., Chen, Z., Zhou, L.: Success factors of medical crowdfunding campaigns: systematic review. J. Med. Internet Res. 24(3), E30189 (2022) Huang, Z., Ouyang, J., Huang, X., Yang, Y., Lin, L.: Explaining donation behavior in medical crowdfunding in social media. SAGE Open 11(2), 21582440211014520 (2021) Jin, P.: Medical crowdfunding in china: empirics and ethics. J. Med. Ethics 45(8), 538–544 (2019) Kaztag: V Destvih Blagotvoritelnogo Fonda «Hareket» Usmotreli Priznaki Moxenniqestva (2022). https://Shorturl.At/Lmoyz. Accessed 20 Feb 2023 Kemp, S.: Digital 2022: Kazakhstan (2022). https://Shorturl.At/Rhsy1. Accessed 21 Feb 2023 Khabar 24: Online Charity in Kazakhstan: How It Works (2022). https://24.Kz/Ru/News/Soc ial/Item/561164-Online-Blagotvoritelnost-V-Kazakhstane-Kak-Eto-Rabotaet. Accessed 1 Feb 2023 Küchler, L., Hertel, G., Thielsch, M.T.: Are you willing to donate? Relationship between perceived website design, trust and donation decisions online. In: Proceedings of Mensch und Computer 2020, pp. 223–227 (2020) Kuppuswamy, V., Bayus, B.L.: Crowdfunding Creative Ideas: The Dynamics of Project Backers. The Economics of Crowdfunding: Startups, Portals and Investor Behavior, pp. 151–182 (2018) Lettieri, E., Masella, C., Radaelli, G.: Disaster management: findings from a systematic review. Disaster Prev. Manag. Int. J. (2009) Muftyat: A Charity Event Was Held in the New Mosque (2021). https://Www.Muftyat.Kz/ Kk/News/Qmdb/2021-05-06/36102-Zhana-Meshitte-Kaiyrymdylyk-Sharasy-Otti-Foto/. Accessed 18 Feb 2023 Nath, R.D., Chowdhury, M.A.F.: Shadow banking: a bibliometric and content analysis. Financ. Innov. 7(1), 1–29 (2021) Ren, J., Raghupathi, V., Raghupathi, W.: Understanding the dimensions of medical crowdfunding: a visual analytics approach. J. Med. Internet Res. 22(7), E18813 (2020) Selby, K.: 9 Government Programs and Nonprofits That Can Help With Medical Bills (2023). https://www.Asbestos.Com/Treatment/Expenses/Government-Nonprofit-Financ ial-Assistance/. Accessed 11 July 2023

Enhanced Web Platform for Optimizing Medical Fundraising

225

Sivarajah, U., Kamal, M.M., Irani, Z., Weerakkody, V.: Critical analysis of big data challenges and analytical methods. J. Bus. Res. 70, 263–286 (2017) Snyder, J., Mathers, A., Crooks, V.A.: Fund my treatment!: A call for ethics-focused social science research into the use of crowdfunding for medical care. Soc. Sci. Med. 169, 27–30 (2016) Snyder, J., Crooks, V.A., Mathers, A., Chow-White, P.: Appealing to the crowd: ethical justifications in Canadian medical crowdfunding campaigns. J. Med. Ethics 43(6), 364–367 (2017) Stack Overflow: Stack Overflow Developer Survey 2022 (N.D.). https://Survey.Stackoverflow.Co/ 2022/. Accessed 17 Apr 2023 Steinberg, D.: The Kickstarter Handbook: Real-Life Crowdfunding Success Stories. Quirk Books (2012) Strohmaier, D., Zeng, J., Hafeez, M.: Trust, distrust, and crowdfunding: a study on perceptions of institutional mechanisms. Telemat. Inform. 43, 101252 (2019) Suresh, K., Øyna, S., Munim, Z.H.: Crowdfunding prospects in new emerging markets: the cases of India and Bangladesh. Adv. Crowdfunding Res. Pract., 297–318 (2020) Swan, J.E., Nolan, J.J.: Gaining customer trust: a conceptual guide for the salesperson. J. Pers. Sell. Sales Manag. 5(2), 39–48 (1985) Thielsch, M.T., Blotenberg, I., Jaron, R.: User evaluation of websites: from first impression to recommendation. Interact. Comput. 26(1), 89–102 (2014) Tran: Us Elections: How Obama Became A Campaign Fundraising Phenomenon (2008). https:// www.Theguardian.Com/World/2008/Jul/17/Barackobama.Uselections2008. Accessed 19 Feb 2023 Turan Times: How Developed Is Charitable Activity in Kazakhstan? (2022). https://Turantimes. Kz/Obschestvo/38202-Naskolko-Razvita-Blagotvoritelnaja-Dejatelnost-V-Kazahstane.Html. Accessed 1 Feb 2023 Xu, K., Wang, X.: Kindhearted people, please save my family: narrative strategies for new media medical crowdfunding. Health Commun. 35(13), 1605–1613 (2020) Young, M.J., Scheinberg, E.: The rise of crowdfunding for medical care: promises and perils. JAMA 317(16), 1623–1624 (2017) Zhang, X., Lyu, H., Luo, J.: What contributes to a crowdfunding campaign’s success? Evidence and analyses from GoFundMe data. J. Soc. Comput. 2(2), 183–192 (2021)

A Brain-Computer Interface Application Based on P300 Evoked EEG Potentials for Enabling the Communication Between Users and Chat GPT Oana Andreea Rusanu(B) Product Design, Mechatronics and Environment Department, Transilvania University of Brasov, Brasov, Romania [email protected]

Abstract. The brain-computer interface or the BCI is a high-technology provided by the breakthroughs from the biomedical engineering research field. The BCI is aimed at supporting people with neuromotor disabilities by enabling the achievement of movement and communication tasks using only the mental decoded intentions. This paper presents the development and the experimentation of BCI application integrated with the Chat GPT assistant. Considering that this AI modern instrument is really engaging triggering the feeling of communicating with a real human being, then it will prove its usefulness even for the disabled users who need to be treated as normal persons still having unaffected emotions and cognitive abilities. The EEG signals are acquired from the GTEC Unicorn headset embedding eight sensors placed to the frontal, parietal, temporal, and occipital cerebral lobes. The current work also focuses on the implementation of a LabVIEW instrument providing the solution of calling specific Python functions able to achieve the data transfer between computer and the Chat GPT API. Therefore, the originality is determined by solving the challenge underlying the software development of a functional brain-computer interface by combining LabVIEW graphical programming environment, Python language and P300 Speller Unicorn platform. This way, the users firstly need to focus their attention and eyesight to the alphanumeric symbols displayed by the Speller board. The target is to obtain simultaneous real-time data transfer starting with the questions addressed by the P300 Speller board and ending with the answers provided by the Chat GPT. Keywords: Brain-Computer Interface · Chat GPT · LabVIEW Python Node

1 Introduction Breaking the boundaries of the human interaction either by natural language or improved mobile communication, the brain-computer interface (BCI) is a significant milestone of the biomedical engineering research field aiming at alleviating the difficulties of people with neuromotor disabilities to establish a normal life existence. Therefore, implementing and optimizing the artificial intelligence methods led to increased accuracy of the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 226–238, 2024. https://doi.org/10.1007/978-3-031-56075-0_22

A Brain-Computer Interface Application Based on P300 Evoked EEG

227

ability that brain-computer interface systems [1] reported during the detection process of specific brain patterns generated through external stimulation or self-regulated cognitive tasks. Thus, among the most frequently employed BCI communication methods is the P300 Speller board consisting of applying signal processing techniques and machine learning algorithms for deciphering the highest brain response transmitted with a delay of approximately 300 ms after the users focused their attention and eyesight on a single desired symbol displayed by an oddball paradigm triggering positive neuronal deflections based on showing randomly flashing items. The development of multipurpose brain-computer interfaces by achieving the detection of P300 event-related potentials is a well-known application that researchers from computer engineering field and neuroscience have deeply investigated since the year 1988 [2]. Even though unexpectedly high accuracy values were previously obtained by the cutting-edge BCIs [3] developed until the current moment (the year 2023), the actual challenge is still referring to decreasing the time interval necessary for users to select the correct symbol by mentally counting a lower number of flashes and identifying the corresponding layout (P300 Speller board) composed of multiple options expressed by the high contrast between the flashing and dark items. Moreover, an attractive and interactive way of learning how to use a P300 based braincomputer interface consists of leveraging the generative artificial intelligence of Chat GPT that can be also accessed by the OpenAI API. In addition, the user’s performance of getting accustomed to working with the P300 Speller board is enhanced by the stronger motivation of a real-time communication thanks to the AI generated responses in a human language style provided by the ChatGPT virtual assistant. This paper presents a novel application by combining two versatile communication modalities such as ChatGPT based on using generative artificial intelligence to enable human conversations and the P300 Speller boards based on using the GTEC Unicorn EEG headset to create written content only by thinking to selecting the target letter. The current research article includes the implementation of an original P300 based brain-computer interface system by integrating the software components developed in Python programming language and LabVIEW graphical programming environment. The OpenAI API is called by the related Python function so that it accomplishes the transfer of messages between the users and the ChatGPT. The UDP packets encompassing the P300 evoked response are acquired and analyzed by the LabVIEW instrument. The EEG signals are acquired and processed by the official GTEC Unicorn software tool. This paper describes not only the programming stages of the proposed braincomputer interface application necessary for overcoming the challenges encountered along the pathway followed from the starting point of acquiring the EEG data until the ending destination of communicating with ChatGPT, but also the experimental results provided by multiple user selections. A supplementary functionality of the proposed BCI is the Internet browsing and searching (Google, Facebook, YouTube) for the generated ChatGPT solution so that the users receive engaging visual feedback. The structure of the paper is the following: Sect. 1 is the Introduction, Sect. 2 reviews the current state of art of P300 based BCIs s, Sect. 3 presents the hardware EEG equipment, Sect. 4 comprises the entire programming stages of the proposed BCI application

228

O. A. Rusanu

integrated with ChatGPT, Sect. 5 describes the experimental results and discusses their implications, and Sect. 6 contains the conclusions.

2 Scientific Literature Review about the Brain-Computer Interfaces Aimed at Enabling Mind Communication Beyond the complex neuroscience applications of decoding the imagined speech detected even with the non-invasive electroencephalography-based technology [4] or the magnetic resonance imaging [5], the GTEC Medical Engineering Company from Austria demonstrated the possibility of creating a real-time communication system for people with neuromotor disabilities by integrating Chat GPT and the brain-computer interface during the annual event of GTEC BCI & Neurotechnology Spring School 2023 [6]. Regarding the BCI systems facilitating mind communication for disabled persons, the most frequently applied working principle was related to detecting and processing the EEG event related signals such as the P300 evoked potentials triggered across a rowcolumn board displaying text or pictures flashing symbols [7]. According to scientific literature [8], the performance of non-invasive BCI systems should be improved for allowing free communication between disabled human subjects.

3 The GTEC Unicorn Hybrid Black EEG Kit Figure 1 shows the EEG equipment used at the development of the proposed braincomputer interface aimed at enabling the communication for disabled users by integrating the P300 Speller Board with the ChatGPT functionality. Thus, Fig. 1 presents the GTEC Unicorn Hybrid Black headset embedding 8 sensors placed to the positions – Fz, Cz, P3, Pz, P4, PO7, Oz, PO8 - according to the 10–20 International System. The Unicorn headset communicates by a Bluetooth dongle with the Windows based computer. The GTEC Unicorn free available software provides developers and researchers access to monitoring and recording the raw EEG signals as well as the EEG rhythms (delta, theta, alpha, beta, gamma). Moreover, the standard Unicorn Suite Hybrid Black includes free code examples for EEG data acquisition based on.Net API and C API. Otherwise, the Python and Matlab Simulink toolkits aimed at accessing the Unicorn raw EEG data require the payment of a permanent license key. Moreover, the GTEC company released advanced software packages for leveraging the functionality of the Unicorn EEG headset, such as: the P300 Speller board, the Blondy Check neuromarketing application, and the Unity Interface to develop mind-controlled games. The GTEC Unicorn headset constituted the most suitable EEG equipment enabling the current research thanks to the following reasons: the software and hardware-based compatibility with detecting the P300 evoked potentials, the availability of the UDP data transfer, the rapid setup, and the high quality of the acquired EEG signals considering the technical specifications – 24 Bit resolution, sampling rate of 250 Hz for each channel, input sensitivity equal to ± 750 mV, and high signal-to-noise ratio.

A Brain-Computer Interface Application Based on P300 Evoked EEG

229

Fig. 1. The GTEC Unicorn Headset with 8 EEG sensors used at the development of the proposed P300 based brain-computer interface for ChatGPT communication.

4 The development of a Brain-Computer Interface Aimed at Communication Between Disabled Users and Chat GPT API 4.1 The General Programming Framework of the Proposed BCI Application Figure 2 presents the general programming framework of the proposed brain-computer interface system consisting of the following stages: the setup of the Unicorn headset embedding 8 sensors for the EEG data acquisition; running the P300 Unicorn Speller Board for selecting the desired symbol; running the LabVIEW application for composing the first part of the question; running the LabVIEW application to add the second part of the question; executing the Python function integrated into LabVIEW to call the OpenAI API to display the response generated by the ChatGPT; selecting a new symbol using the P300 Speller Board; performing the Internet Search (Google, YouTube, Facebook) for showing ChatGPT solution; repeating the previous steps by starting to create another question. As it was specified in the previous sections, the Unicorn – LabVIEW interaction is based on acquiring and processing the UDP based EEG data consisting of the P300 evoked potentials.

Fig. 2. The general programming framework of the proposed brain-computer interface system aimed at enabling communication between disabled users and ChatGPT.

230

O. A. Rusanu

4.2 Preliminary Setup for Enabling the OpenAI API to Use ChatGPT The OpenAI company was founded in the 2015 year and got involved into the research and development of artificial intelligence based generative models aimed at understanding and processing the natural language to deliver responses in a human style. The OpenAI API (Application Programming Interface) provides the developers with the advanced functionality of generative AI to access the desired models enabling both the transfer of messages between users and ChatGPT (generative pre-trained transformer) and creating or editing images. The ChatGPT users should create a free personal account based on a Gmail address. Excepting the trial period when the grant of $5.00 is available for three months, the OpenAI API developers need to pay a price calculated considering the number of used tokens (words) and the called AI model. Referring the example of selecting the GPT-3.5 Turbo model with 4K context [9], the usage of OpenAI API requires two fees: firstly, $0.0015 for each set of 1000 tokens necessary for the input request (questions addressed to ChatGPT) and secondly, $0.002 for each set of 1000 tokens contained by the output text (response received from the ChatGPT). 4.3 The Graphical User Interface of the proposed LabVIEW and Python based Brain-Computer Interface aimed at ChatGPT Communication Figure 3 displays the graphical user interface of the proposed brain-computer interface application based on the integration between LabVIEW and Unicorn P300 Speller software using the UDP protocol communication as well as Python and ChatGPT to call the OpenAI API. Considering the implementation based on LabVIEW Python nodes, it resulted in showing the user question and the ChatGPT response to the LabVIEW graphical user interface of the proposed BCI.

Fig. 3. The graphical user interface of the proposed brain-computer interface aimed at enabling the integration between LabVIEW – Python – Unicorn P300 – ChatGPT.

A Brain-Computer Interface Application Based on P300 Evoked EEG

231

Likewise, the users receive visual feedback determined by generating intuitive pictures corresponding to: main or fundamental selection (four options: Holiday, Virtual Reality, Music, or Book), secondary or particular selection (some examples from the three categories: holiday destinations – seaside, mountain; applications – discover first computer, ancient castles, study foreign languages; feelings: express hope, loneliness, and many more), and the final web resource (YouTube, Facebook, Google) aimed at searching for ChatGPT response into Internet. 4.4 The Python Script Aimed at Transfer of Messages between the Users of the Brain-Computer Interface and the ChatGPT OpenAI API According to the YouTube tutorial [10], in the first phase of accessing the ChatGPT functionality in Python programming language, it is necessary to install the OpenAI package by using the pip related command. The second phase consists in generating the secret API key from the user Gmail account created to the OpenAI platform. The third step refers to writing the Python code for calling the OpenAI API and using the desired model (for example: gpt-3.5-turbo) aimed at returning the output ChatGPT responses.

Fig. 4. The Python script based on calling the OpenAI API for addressing the users’ questions and returning the ChatGPT response in the Terminal of Visual Studio Code.

Figure 4 shows the Python sequence code based on the following structure: import of the openai package, uploading the API key in a text file, the block of the chat_gpt function aimed at addressing the users’ questions and returning the generative artificial intelligence based response, and a print function for checking the chat_gpt Python call by adding the input parameter consisting of the request: What question should I ask ChatGPT by using the OpenAI API?”. Moreover, the Fig. 4 includes the Terminal window aimed at displaying the entire response returned by ChatGPT including the examples of questions that the users may ask.

232

O. A. Rusanu

4.5 The LabVIEW Instrument Aimed at Achieving the Communication between Users and ChatGPT by Integrating the Python Functionality The integration between Python and LabVIEW was achieved by installing the corresponding versions of the two programming platforms and considering the same bitness. Thus, this research work involved the installation of Python version 3.9 (32-bit) as well as LabVIEW version 2022 Q3 (32-bit). According to [12], other compatible versions of Python and LabVIEW are also available for successfully running of the LabVIEW Python Node providing the functionality of calling and executing a function from a Python script. Likewise, it is necessary to edit the Environment System Variables (in Windows Operating System) by adding the Python shared libraries.

Fig. 5. The ChatGPT LabVIEW state consisting of calling the Python functionality.

Figure 5 presents the LabVIEW code sequence called the ChatGPT state consisting of the three fundamental nodes aimed at calling the Python function. The Python Node requires the following essential input parameters: the session in (reference determined by the previous LabVIEW Node – Open Python Session), the module path (the address where the Python script is stored), the function name (the Python code that should be executed), the input terminals necessary to pass the users’ questions to the Python function. The proposed brain-computer interface application requires the output data of the Python Node meaning the return value parameter that is further connected to the input data of the Concatenate Strings function. The type of the returned value is string, but the LabVIEW Python node supports different data types, such as: numeric, (multi-dimensional) arrays, clusters, and Boolean. 4.6 The Customized GTEC Unicorn P300 Speller Board Providing the Graphically Expressed Questions Addressed to ChatGPT Figure 6 presents graphical user interface designed in the official GTEC Unicorn P300 Speller board software application. Thus, the panel was customized with selected graphical symbols indicating the requests (the main selections, the secondary options, or the

A Brain-Computer Interface Application Based on P300 Evoked EEG

233

browsing web resource) that will be further transmitted to ChatGPT API by associated the chosen letter with the text input defined in the proposed LabVIEW based brain-computer interface. In addition, the GTEC Unicorn P300 Speller board application provides developers or researchers with the facility of changing the default settings: the flash mode (row/column), maximum number (it is equal to 16 for the current BCI application) of flashes of each item, waiting time (it is set to 5 s in the proposed BCI) after item selection, flash time (100 ms), dark time (75 ms) and output text. Regarding the efficient integration between LabVIEW and Unicorn by implementing a smooth UDP data transfer, it was important to edit the uploaded pictures so that their size did not exceed 5 KB and their dimension was set to maximum 50 × 50 pixels. Also, the contrast between grayscale pictures used as dark symbols and the colored images used as flashing symbols led to a strong P300 response.

Fig. 6. The view of the GTEC Unicorn P300 Speller customized with new graphical symbols to cover the requests addressed to ChatGPT using the proposed BCI.

4.7 The UDP Data Packages Transfer between the LabVIEW Instrument and the Unicorn P300 Speller Board The proposed LabVIEW based brain-computer interface application required the implementation of an original code sequence aimed at enabling the UDP data transfer between LabVIEW instrument and the official Unicorn P300 Speller board. Therefore, there were employed the basic functions – UDP Open, UDP Read, UDP Close – offered by the UDP LabVIEW toolkit. The UDP data transfer between LabVIEW and Unicorn software platforms is achieved if the same port (for example: 1000) is set for both. Also, a simple approach is to use the same computer for running both LabVIEW and Unicorn applications. In this case, the IP address is set to 127.0.0.1. A particular feature implemented into LabVIEW refers to the implementation of the subVI (function or subsequent virtual instrument) aimed at extracting the file name of the selected graphical symbol from the P300 Speller board. Therefore, the filename should contain a letter that is further used as a command in the LabVIEW state-machine algorithm aimed at associating the selected target picture with a specific request addressed to Chat GPT. The author of the current paper previously described more information and details about the acquisition and processing of the UDP data transmitted by Unicorn EEG software to the LabVIEW based BCI application in a recently published conference paper [11].

234

O. A. Rusanu

4.8 The LabVIEW State-Machine Paradigm Aimed at Managing the Creation of Questions Asked by ChatGPT based on the P300 Response - UDP Input After the acquisition and analysis of the UDP data embedding the response of the triggered P300 evoked EEG potentials, an original LabVIEW state-machine algorithm needs to be implemented to manage the formulation of different questions including multiple parts so that ChatGPT will be determined to return creative and engaging responses. Therefore, this paper presents the structure of the LabVIEW state-machine paradigm aimed at enabling the transition across primary and secondary sequences. Figure 7 shows a sequence of the LabVIEW state-machine algorithm including the following stages: the Init State for checking the extracted UDP data and deciding which is the next executed state considering four options – Holiday, Virtual Reality, Music, or Book. Thus, the users can firstly set the primary context of their request by referring to one of the four possibilities: a holiday destination, a virtual reality application, a musical hit, or a book. The users’ decision is indicated by the symbols they focused on in the customized Unicorn P300 Speller Board. The UDP data transfer between the Unicorn and LabVIEW applications produced the output value (H – Holiday, V – Virtual Reality, M – Music, and B – Book) that is checked by the Init state to send the command of executing the next related state. The activation of a certain previously mentioned primary states is confirmed by showing the related picture in LabVEW.

Fig. 7. The first sequence of the State-Machine paradigm showing the content of Init State and the primary selections (Holiday, Virtual Reality, Music, Book).

Further, according to Fig. 8, regardless of the selected primary option, the next LabVIEW code sequence is based on comparing the received extracted UDP data with the string values corresponding to the secondary options included in three categories. The first category is related to specific holiday destinations so that the question addressed to ChatGPT is completed with the second part of text resulting in the entire phrase: Let me know strictly the name of a holiday destination where I can – enjoy seaside (UDP =

A Brain-Computer Interface Application Based on P300 Evoked EEG

235

S) – enjoy beach (UDP = b) – enjoy mountain (UDP = m) – enjoy fresh air (UDP = f). The second category refers especially to the virtual reality-based application, resulting in composing the second part of the user’s request to ChatGPT: Let me know strictly the name of a virtual reality application that provides me with the opportunity to – discover the first released computer (UDP = C) – visit the ancient castles (UDP = A) – study foreign languages (UDP = L) – play adventure games (UDP = E). The third category comprises the feelings emphasized either by music or literature book so that it results in formulation of the ChatGPT question based on integrating the second part: Let me know strictly the name of a musical hit whose lyrics refer to (Music option) or Let me know strictly the name of a book or a novel which is about how to (Book option) – express hope feelings (UDP = P) – express love feelings (UDP = O) – express forgiveness feelings (UDP = F) – express loneliness feelings (UDP = I).

Fig. 8. The second sequence of the State-Machine paradigm showing the content of secondary selections (Seaside, Mountain, Ancient Castles, First Computer, etc.).

Otherwise, any combination between the primary selections (Holiday, Virtual Reality, Music, Book) and the secondary selections (Seaside, Beach, Ancient Castles, First Computer, Forgiveness, Love, etc.) is supported by the proposed LabVIEW BCI & Chat GPT application. Moreover, multiple parts can be added in the resulted question composed of a single mandatory primary part, one mandatory secondary part and countless optional secondary selections. Nevertheless, the LabVIEW state-machine paradigm was designed with respect to the condition aimed to avoid the repetition of similar optional secondary selections that can be indicated for multiple times in a single question addressed to ChatGPT. Moreover, the LabVIEW Front Panel displays a picture corresponding to each secondary selection determined by the state-machine paradigm. According to Fig. 9, after the execution of each secondary state (examples: First Computer, Ancient Castles, Mountain, Foreign Languages, Adventure Games, and the others), the LabVIEW state-machine continues to run in the subsequent states called as in the following examples: Next First Computer, Next Ancient Castles, Next Mountain, Next Foreign Languages, Next Adventure Games, so that all the states contain the Next term. The role of these subsequent states is to stop the continuous sending of the secondary part of the ChatGPT request, for example the previously mentioned phrases: enjoy seaside, play adventure games, express hope feelings, and all the others. Also,

236

O. A. Rusanu

Fig. 9. The third sequence of the State-Machine sequence flow until reaching the result – displaying the ChatGPT response or searching it into Internet.

the states including the Next term are aimed at providing the possibility of addressing additional content (third or fourth part) to the ChatGPT request, by checking if another UDP based command was received. Moreover, the Next based states contain the option of selecting the ChatGPT so that the formulation of the resulted question is finished, and the user waits for a response. Figure 3 presented the content of the ChatGPT state related to using the Python LabVIEW toolkit to call the OpenAI API. Further, it is necessary to call the Python function only once a time so that the ChatGPT state is automatically followed by the Next ChatGPT state. According to Fig. 9, the Next ChatGPT state is aimed at comparing the extracted UDP data with three values corresponding to the three options aimed at searching the ChatGPT response into Internet by accessing YouTube (UDP = Y), Facebook (UDP = K), or Google (UDP = g).

5 Experimental Results The live experimentation session of the presented brain-computer interface was performed by a single subject (girl, 31 years) and recorded by the video demonstration uploaded to the YouTube link [13]. The results provided by the 16 experiments are described in the Table 1. The maximum accuracy of 100% percent was reported for 13 out of the 16 experiments. The lower accuracy values of 75% were caused by the subject’s tiredness. The time interval between P300 based selection and LabVIEW command extraction was equal to tens of milliseconds. There were counted 10–30 s between sending the subject’s request and receiving the Chat GPT response. Each experiment comprised a main, a secondary, and two search selections. The calibration session included 6 items and 30 flashes per each item. The settings of the testing session were the following: 16 flashes per item, flash time = 100 ms, and dark time = 75 ms.

A Brain-Computer Interface Application Based on P300 Evoked EEG

237

Table 1. The results obtained by the subject after performing the 16 BCI experiments. Exp.

Main Selection

Secondary Selection Search Selection

Search Selection

Acc.

1

Holiday

Pass

Seaside (S)

Pass

Chat GPT

Pass

Facebook

Pass

100%

2

Holiday

Pass

Beach (b)

Pass

Chat GPT

Pass

Facebook

Pass

100%

3

Holiday

Pass

Mountain (m)

Pass

Chat GPT

Pass

Facebook

Pass

100%

4

Holiday

Pass

Fresh Air (f) Pass

Chat GPT

Pass

Facebook

Pass

100%

5

VR

Pass

Computer (C)

Pass

Chat GPT

Pass

Google

Pass

100%

6

VR

Pass

Castles (A)

Pass

Chat GPT

Pass

Google

Pass

100%

7

VR

Pass

Languages (L)

Pass

Chat GPT

Pass

Google

Pass

100%

8

VR

Pass

Games (E)

Pass

Chat GPT

Pass

Google

Pass

100%

9

Music

Pass

Love (O)

Pass

Chat GPT

Fail

YouTube

Pass

75%

10

Music

Pass

Loneliness (I)

Pass

Chat GPT

Pass

YouTube

Pass

100%

11

Music

Pass

Forgiveness (F)

Pass

Chat GPT

Pass

YouTube

Pass

100%

12

Music

Pass

Hope (P)

Pass

Chat GPT

Fail

YouTube

Pass

75%

13

Book

Pass

Mountains (m)

Pass

Chat GPT

Pass

Google

Pass

100%

14

Book

Pass

Computer (C)

Pass

Chat GPT

Pass

Google

Pass

100%

15

Book

Pass

Castles (A)

Pass

Chat GPT

Pass

YouTube

Fail

75%

16

Book

Pass

Love (O)

Pass

Chat GPT

Pass

Facebook

Pass

100%

6 Conclusions This paper provided a new way of creating a brain-computer interface aimed at enabling the communication between the disabled users and the Chat GPT by accomplishing the integration between LabVIEW and Python. Therefore, by using the Unicorn EEG headset embedding eight sensors, it was achieved the simple, quick, and reliable detection of the P300 evoked EEG potentials from the official Unicorn Speller board. By focusing the attention and the eyesight to graphical symbols, a specific question composed of a main and multiple selections is generated to be further addressed to Chat GPT. The proposed brain-computer interface application could be leveraged not only as a communication system integrating the artificial intelligence, but also as a training virtual environment for people with neuromotor disabilities.

238

O. A. Rusanu

References 1. Kurczak, J., Białas, K., Chalupnik, R., Kedziora, M.: Using Brain-Computer Interface (BCI) and artificial intelligence for EEG signal analysis. In: Szczerbicki, E., Wojtkiewicz, K., Van Nguyen, S., Pietranik, M., Krótkiewicz, M. (eds.) Recent Challenges in Intelligent Information and Database Systems: 14th Asian Conference, ACIIDS 2022, Ho Chi Minh City, Vietnam, November 28-30, 2022, Proceedings, pp. 214–226. Springer Nature Singapore, Singapore (2022). https://doi.org/10.1007/978-981-19-8234-7_17 2. Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70, 510 (1988) 3. Cao, Z.: A review of artificial intelligence for EEG-based brain−computer interfaces and applications. Brain Sci. Adv. 6(3), 162–170 (2020). https://doi.org/10.26599/BSA.2020.905 0017 4. Tang, J., LeBel, A., Jain, S., et al.: Semantic reconstruction of continuous language from non-invasive brain recordings. Nat. Neurosci. 26, 858–866 (2023). https://doi.org/10.1038/ s41593-023-01304-9 5. O’Leary, K.: MRI decoders translate thoughts into words. Nature medicine, Advance online publication (2023). https://doi.org/10.1038/d41591-023-00044-4 6. G.tec BCI & Neurotechnology Spring School 2023. https://www.gtec.at/spring-school-2023/. Accessed 17 July 2023 7. Fernández-Rodríguez, Á., Medina-Juliá, M.T., Velasco-Álvarez, F., et al.: Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm. Med. Biol. Eng. Comput. 59, 869–881 (2021). https:// doi.org/10.1007/s11517-021-02340-y 8. Renton, A.I., Mattingley, J.B., Painter, D.R.: Optimising non-invasive brain-computer interface systems for free communication between naïve human participants. Sci. Rep. 9, 18705 (2019). https://doi.org/10.1038/s41598-019-55166-y 9. OpenAI Pricing –GPT-3.5 Turbo. https://openai.com/pricing. Accessed 17 July 2023 10. ChatGPT API in Python. https://youtu.be/c-g6epk3fFE?t=1. Accessed 17 July 2023 11. Rus, anu, O.A.: A brain-computer interface-based simulation of vending machine by the integration between Gtec Unicorn EEG headset and LabVIEW programming environment using P300 speller and UDP communication. In: The 16th International Conference Interdisciplinarity in Engineering. Inter-Eng 2022 (2023). https://doi.org/10.1007/978-3-031-223754_68 12. Integrating Python Code in LabVIEW. https://www.ni.com/en/support/documentation/supple mental/18/installing-python-for-calling-python-code.html. Accessed 29 July 2023 13. Original experiment. https://youtu.be/ArLytIKK0D0. Accessed 29 July 2023

Self-management of Type-2 Diabetes Using a Mobile Application: A Pilot Study Soulakshmee D. Nagowah1(B) , Abha Jodheea-Jutton2 , Kavi Kumar Khedo1 , Shakuntala Baichoo1 , Sudha Cheerkoot-Jalim1 , Leckraj Nagowah1 , and Zahra Mungloo-Dilmohamud1 1 FoICDT, University of Mauritius, Réduit, Mauritius

[email protected] 2 Faculty of Medicine and Health Sciences, University of Mauritius, Réduit, Mauritius

Abstract. There is a steady rise in the number of patients living with diabetes worldwide and the threat of Type 2 Diabetes Mellitus (T2DM) on public health is urging policy makers, researchers and clinicians to investigate novel methods that can help to reduce the burden of diabetes and related complications. The use of mobile applications is emerging as the latest technology in the follow-up and management of patients with diabetes through registries, recall systems and monitoring. The primary objective of this research was to develop a mobile application entitled DiaMon, and to assess its acceptability to support patients of T2DM in self-managing their health conditions, specifically adapted to the Mauritian eating habits. Patients with T2DM or pre-diabetes were invited to participate in a study, over a period of 12 weeks, to evaluate the application. The glycaemic control (HbA1c, Fasting Blood Glucose (FBG)) was conducted at baseline and at the end of the study for each participant. A positive behavioural change was noted among the participants. There were statistically significant improvements in glycaemic control: the HbA1c showed an improvement of 1.2% over the 3-month period, with a mean of 6.53 (SD 1.66) %. Upon concluding the study, HbA1c was found to be within the recommended range for mitigating the risk of complications associated with T2DM. There was also a significant reduction in the mean FBG when the pre-intervention blood glucose values were compared to post-intervention. This research has demonstrated the positive impact of the use of a digital tool for the self-management of T2DM. Keywords: T2DM · Self-Management · Mobile Application · Behavioural Change

1 Introduction According to the World Health Organisation, the prevalence of type 2 diabetes mellitus (T2DM) has risen considerably in the last three decades in countries of all income levels [1]. The International Diabetes Federation [2] reports that the number of people living with T2DM worldwide was 463 million in 2019 and this figure is expected to increase to 700 million by 2045. The global health expenditure due to diabetes which © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 239–250, 2024. https://doi.org/10.1007/978-3-031-56075-0_23

240

S. D. Nagowah et al.

was 760 billion USD in 2019 is predicted to escalate to 845 billion USD by 2045. The hyperglycaemic state of T2DM predisposes people living with diabetes to increased risks of microvascular and macrovascular complications [3]. Several landmark studies such as the Diabetes Control and Complications, the ADVANCE and the ACCORD trials, have shown the link between tight glycaemic control and prevention of complications [4–6], which has been reinforced by established guidelines such as American Diabetes Association [7]. Over the last few decades, newer methods and technologies have been put to test in the quest to improve glycaemic control [8]. There has been an endeavour to integrate technology to promote self-care, empowerment and diabetes control [9]. The emergence of a number of mobile frameworks to facilitate diabetes control [10] have consequently been observed. Mobile health technology has emerged as an innovative approach that can address the motivational need of patients [11], leveraging the potential of mobile devices such as smartphones and tablet computers. Continuous advancements in mobile technology have led to increased adoption and enhanced capabilities [12]. Applications of these devices can assist both the patient and healthcare provider to meet self-management practices. They can induce behavioural changes in patients, resulting in preventing or at least delaying the complications associated with the disease. Mauritius has one of the highest Diabetes Mellitus (DM) prevalence globally (25.3% prevalence for ages 20 – 79) [13]. Approximately 2,648 adults died due to diabetes in 2019, and the average cost of DM related problems amounted to USD 28.7 million. The Mauritius Non-Communicable Diseases (NCD) Survey [14] reports an estimated 257,442 people between the ages of 25 and 74 with DM in Mauritius. A high incidence of pre-diabetes is also observed, which if not effectively managed, may lead to the development of diabetes and heart disease. The growing T2DM population in Mauritius is imposing on the healthcare budget and the health system [15]. The DM epidemic has a significant impact locally, calling for urgent remedial strategies to curb the spread. The aim of this study is to illustrate the implications of a digital tool in the self-management of DM, within a population characterized by a substantial prevalence of DM. The objectives of this research work are to (1) develop a mobile application entitled DiaMon (Diabetes Monitoring) in order to support patients of T2DM in self-managing their health conditions, (2) evaluate the acceptability of such a system among patients living with diabetes and pre-diabetes and determine the clinical implications of using the developed application and (3) assess behaviour change (for example diet, physical activity) through the use of the mobile application. The rest of the paper is structured as follows: Sect. 2 describes related works. Section 3 details the proposed mobile application and its features. Section 4 describes a two-step evaluation conducted to determine the usability of DiaMon. Section 5 discusses the strengths and limitations of the study and finally, Sect. 6 concludes the paper.

2 Related Works This section presents the related works on the usage of mobile applications for diabetes self-management. The review conducted by [16] showed that mobile applications have the potential for improved diabetes self-management. The use of applications showed

Self-management of Type-2 Diabetes Using a Mobile Application

241

an improvement in health habits, like regular blood sugar testing, increase in physical activities and adherence to a healthy diet. A meta-analysis [17] indicated a significant improvement in glycated haemoglobin (HbA1C) readings throughout a 12-month period among the intervention group. By providing better information and health education, the applications strengthen the self-care confidence of patients. Applications which contribute to a better glycaemic control are those which provide storage and feedback on blood glucose data, advice on diet plans and physical activity, assistance in drug therapy and access to healthcare professionals. McMillan et al. [18] have conducted a systematic and integrated review of mobile-based technology to promote active lifestyles in people with T2DM. Results of the review show that the papers examined have not investigated the feasibility and acceptability of using mobile technology to provide sustained lifestyle change. Diabetes self-management encompasses the self-care of pre-diabetes and diabetes patients, including the activities and behaviours needed to manage their conditions continually [18, 19]. This concept has been replicated worldwide [20] with good outcomes. Hunt et al. [21] reported that technology can support daily diabetes self-management activities such as blood glucose monitoring, physical activity, healthy eating, taking medication, monitoring for complications, and problem-solving. The technologies that were mostly used for the self-management of diabetes were mobile phones, gaming systems, and internet-based systems. Sahu et al. [22] conducted a systematic review to investigate the contribution of mobile phone technologies in improving health outcomes of patients suffering from chronic diseases in Asian and African countries. Their investigations revealed a substantial presence of applications dedicated to health education particularly focusing on conditions such as diabetes and heart disease. Mobile phone, text messaging (SMS) and video telephony (MMS) were widely used for disease self-management, health education, providing recommendations to patients as well as sending reminders and alerts about diet, physical activity, drug schedules, upcoming tests, and appointments. Haddad et al. [23] performed a feasibility study to assess the effect of SMS on the education and self-management of Iraqi adults suffering from T2DM. Because of the increasingly widespread use of mobile phones in Iraq, the method was considered as widely acceptable, feasible and cost-effective in providing ongoing health care support to individuals. Bant II, a Canadian application for patients with DM is set to target behavioural change and self-care, as determined by the pilot testing of the Bant I Application, which was centred on blood glucose monitoring [24]. However, the currently available interventions are not suited for daily sensor-based monitoring of eating habits and physical activity, which are the key aspects of behaviour change in T2DM.

3 DiaMon Mobile Application DiaMon is a mobile application implemented using Android Studio 3.0.1 with an embedded SQLite database. The information through the mobile application, such as the blood glucose level, food intake, and physical activities are initially stored in an embedded internal SQLite database. When an Internet connection is available, this information is securely transferred to the Firebase online real-time database. The DiaMon mobile

242

S. D. Nagowah et al.

application requires that the users register themselves prior to using the application. Registration process entails the use of an email address and a password. Participants were also advised to fill in other demographic details such as smoking status, occupation and alcohol consumption.

Fig. 1. Weekly Chart for Breakfast and Dinner

Fig. 2. Morning Food Intake

3.1 Blood Glucose Monitoring DiaMon enables users to log their blood glucose levels before and after meals, providing a visual tracking of these readings over time (Fig. 1). DiaMon utilizes graphic charts derived from stored readings for breakfast, dinner, or for both enhancing the interpretation of data provided by participants. We anticipate that the visually presented information will motivate participants to alter their habits contributing to spikes in blood glucose levels. 3.2 Food Intake Tracker One of the primary aims of DiaMon is to facilitate food intake monitoring. The mobile application allows users to monitor their dietary intake by recording the foods consumed at four different times of the day (morning, noon, afternoon, night) and assess the postmeal blood glucose response. The selection of food items included (Fig. 2) in DiaMon have been customized to align with the Mauritian cultural lifestyle. The list of food items within DiaMon have been sourced from the United States Department of Agriculture Food Composition Databases [25]. The application provides valuable feedback before and after a meal to its users, guided by the mean glucose levels (Fig. 3).

Self-management of Type-2 Diabetes Using a Mobile Application

Fig. 3. Pre-Meal Feedback

243

Fig. 4. BMI Calculator and Feedback

3.3 Health and Fitness Monitoring In DiaMon’s Health and Fitness section, users can monitor their physical activity and record the type and duration of their activities, alongside tracking their Body Mass Index (BMI). The user is also presented with textual and visual feedback based on the calculated BMI (Fig. 4). This value is displayed along a SeekBar featuring varying colours and an indicative table appears at the bottom of the screen based on the feedback.

4 Methods DiaMon was used by a group of selected participants, for a minimum duration of 12 weeks to assess the feasibility and acceptability of the intervention. A two-step evaluation was conducted to determine the usability of DiaMon, including a questionnaire based survey and an assessment of the change in clinical biomarkers following the use of DiaMon. Ethical clearance was granted by the University of Mauritius Ethics Committee. All procedures were in line with the World Medical Association Helsinki Declaration. The study was registered on ClinicalTrials.gov (number: NCT05027334). Patients with T2DM or pre-diabetes were invited to participate in the study. They were recruited through adverts in the local area and neighbouring institutions through social media. They attended an initial screening workshop during which the baseline information such as medical history, fasting blood sugar, smoking and drinking history were recorded. Only participants meeting the inclusion and exclusion criteria were enrolled in the study. The inclusion criteria included patients aged 18–65 years old, diagnosed with T2DM and having an HbA1c of less than 8% and not taking insulin. Patients

244

S. D. Nagowah et al.

who are known with a diagnosis of pre-diabetes based either on an oral glucose tolerance test or two fasting blood glucose readings showing a level of 5.6 mmol/L of glucose or more were also considered. Individuals who did not have access to a smartphone or were not conversant with using technology, as well as individuals who were worried about regular blood glucose monitoring were excluded. There were 25 participants included in this pilot study: 20 patients with T2DM and 5 with pre-diabetes. All the participants had DiaMon uploaded on their smartphone and were provided with an educational session and written information to help them navigate the mobile application. Participants were further provided with fully equipped glucometers to monitor their glucose levels four times daily. They were requested to input their pre-meal and post-prandial glucose for 2 meals daily over 3 months. They were empowered to correlate post-prandial spikes to their dietary content, and hence amend their diet accordingly, based on the personalized feedback provided through DiaMon. All other variables including physical activity and food consumed were daily entered in the mobile application. Baseline biomarkers including, fasting blood glucose and HbA1c were recorded and compared to post intervention biomarkers along with other secondary variables such as weight, fasting blood glucose and pre and post meal glucose as part of the evaluation process. All blood tests were analysed in certified laboratories through standardised clinical chemistry and haematology protocol. The sample size was estimated using 5% marginal error and 80% power providing a 1% improvement in glycaemic control. The minimum sample size for the trial would be 250 patients. A 20% sample size was determined by the group to assess feasibility of the study. Data was entered and analysed using Microsoft Excel and SPSS version 23. The glycaemic control (HbA1c) and clinical markers following the use of DiaMon was tested using paired t-test, with a p-value of less than 0.05 considered as significant. Daily blood glucose levels were charted to see the difference in the pre and post meals BG with time. Linear regression was used to demonstrate the relationship between self-monitored blood glucose and HbA1c, with r-value, defined between −1 and 1. 4.1 Participant Feedback A semi-structured questionnaire was used to collect feedback on the use, receptiveness and usability of the mobile application by the participants. The questionnaire consisted of a number of short close ended questions, geared at assessing how a participant rated the application, and whether the use of DiaMon motivated a change in the lifestyle. Generally, the mobile application has been reported as user friendly with 50% of participants finding the DiaMon application very easy to use and the other 50% found it quite easy to use. None of the participants reported the application as difficult to use. The average user-friendliness score of the application was 4.3 over 5, where participants were asked to rate the application between 1 and 5. The charts were the most appreciated feature of the application (50%). Other commendable features include the speed of recording entries and navigation throughout the application. There were 60% of the participants who recommended improvement in the food entry section, as the variety of food in the

Self-management of Type-2 Diabetes Using a Mobile Application

245

database did not tally with their consumption. Most of the participants (60%) were willing to use the application again while 20% felt they had already mastered the concept of diabetes self-management and another 20% reported that they might possibly use the application again. All (100%) the participants, who responded to the questionnaire, agreed they would recommend the application to another friend or patient with T2DM and pre-diabetes. At the conclusion of the study, participants reported confidence and a sense of empowerment in effectively self-managing their diabetes condition through the adoption of lifestyle changes such as self-monitored blood glucose, balancing physical activity and eating habits. The confidence rating in managing diabetes was 4.1/5, where ‘5’ is described as fully confident in diabetes self-management. The study has demonstrated that a digital tool designed to empower patients with T2DM can be highly valuable in imparting self-efficacy and diabetes self-care. DiaMon has integrated important features of diabetes self-management that are deemed fundamental for diabetes self-care. Acceptability depends on several factors including age and features of the application but as reported by Torbjørnsen et al. [26], digital applications can facilitate the control of DM. 4.2 Clinical Outcomes The primary outcome measures included change in glycaemic control (HbA1c), fasting blood glucose, weight and BMI and secondary outcome measures included behaviour modification practices such as change in eating habits, physical activity and blood glucose monitoring. 4.2.1 Primary Outcomes Participants were divided according to diabetes status. Table 1 shows the change in the outcome of the 20 participants with T2DM. There was a significant improvement in the glycaemic control of patients with T2DM. The HbA1c has been reduced by 1.2% (P = 0.007) over the 3 months period of blood glucose monitoring (Table 1). A statistically significant reduction in the mean fasting blood glucose (P = 0.019) and weight (P = 0.002) were also observed post-intervention. There was a noted decrease in BMI, which was statistically non-significant. Additionally, 50% of the participants reported losing weight with the intervention, while 43.8% were not sure if they have lost weight. These findings reflect the weight differences observed following the intervention (Table 1: P-value = 0.002, CI 1.35–4.38). Table 2 displays the outcomes related to the 5 patients with pre-diabetes (n = 5). The mean age was 51.6 (SD 10.9; CI 95%) years. The mean fasting blood glucose level (FBG) was 5.74 mm/L (SD 0.34; CI 95%). A 0.2 unit decrease in the FBG was noted but was not significant. There was a statistically non-significant reduction in weight and BMI pre-intervention and post-intervention (Table 2).

246

S. D. Nagowah et al. Table 1. Pre and post-intervention changes in patients with T2DM Pre-intervention

Post-intervention

CI [95%]

P-value

FBG/(mmol/L)

6.9 (SD 1.58)

6.18 (SD 0.67)

0.15–1.47

0.019

HbA1c (%)

7.6 (SD 1.28)

6.53 (SD 1.66)

5.48 – 7.58

0.007

Weight (Kg)

77.9 (SD 18.5)

75.25 (SD 17.2)

1.35–4.38

0.002

BMI (Kg/m2 )

26.2 (SD 4.75)

25.94 (SD 5.14)

−5.99–7.61

0.794

FBG: Fasting Blood Glucose; BMI: Body mass index; SD: Standard Deviation

Table 2. Pre and post-intervention changes in patients with pre-diabetes Pre-intervention

Post-intervention

P-value

FBG (mmol/L)

5.74 ± 0.34

5.54 ± 1.17

0.35

Weight (Kg)

64.9 ± 11.2

58.5 ± 12.6

0.48

BMI (Kg/m2)

23.4 ± 2.6

21.8 ± 3.0

0.40

FBG: Fasting Blood Glucose; BMI: Body mass index; SD: Standard Deviation

4.2.2 Secondary Outcomes: Behaviour Change It is stipulated that regularly monitoring blood glucose, especially post-prandial glucose levels will induce participants to alter their behaviour to improve the post-prandial glucose levels. The change in behaviour includes self-monitored blood glucose monitoring, change in eating habits and an improvement in physical activity. • Self-monitored blood glucose (SMBG) frequency 94% of the participants reported that they will self-monitor their blood glucose more frequently in the future. There was further a slight inverse relationship between the frequencies of blood glucose monitoring (r = -0.55) and HbA1c, with a borderline significance (P = 0.06) on regression analysis, implying that self-monitoring of blood glucose improves glycaemic control. Throughout the study’s duration, there was progressive decline in the post-prandial blood glucose levels, with a mean of 8.35. 63% of the participants were familiar with SMBG prior to entering the study while 25% were not familiar with monitoring their blood glucose levels. They found SMBG fairly easy and rated it as 4.3 over 5. All of them were keen to continue self-monitored blood glucose while 90% would favourably recommend the intervention to friends and other patients. 80% reported an improvement in their overall blood glucose. • Physical activity Participants self-reported an increase in the rate of physical activity with 56.3% engaging in physical activity prior to entering the study and 62.5% reported an increase in the amount and level of physical activity towards the end of the study. Up to 80% of participants also reported a decrease in their glucose levels with physical activity.

Self-management of Type-2 Diabetes Using a Mobile Application

247

There was an overall increase in daily duration of physical activity as assessed by both the application and feedback form, which relied on self-reported data from participants. Nevertheless, the observed increase did not reach statistical significance. • Eating habits One of the most important features of this application is food charting with a combination of pre-meal and post-prandial glucose. While a majority of the participants (81.3%, n = 13) successfully identified the food contributing to spikes in blood glucose (based on comments from the feedback questionnaire), the application struggled in accurately tracking total carbohydrate and calorie intake. It was observed that 81.3% of participants noticed an improvement in their eating habits and consumed healthier food, while the rest had no change and 6% of participants reported worsening of his/her eating habits. Rice and refined carbohydrate were important aspects of the daily diet of Mauritians. Although some people considered complex carbohydrates, it was not very popular among the participants. Some information on fruit and vegetable consumption was obtained through the application but the number of fruits and vegetables consumed daily was not captured. We report a positive change in behaviour by combining the use of technology and SMBG through DiaMon. Patients with T2DM who do not require insulin therapy require support to prioritize self-care and embrace behaviour change for optimal glycaemic control. While behaviour change can be challenging, numerous studies have attempted to test the efficacy of the tool in behaviour change [24, 29]. Quinn et al. [12] report the clinical trial results of using mhealth in terms of a web portal system where patients can freely discuss their management such as healthy eating, blood monitoring and medications with diabetes educators. The intervention led to a decrease in HbA1c levels by 0.54%, which is similar to our study. Although our participant size is much smaller, the response to a reduction in HbA1c is promising. Extension of the use of intervention among regularly attending patients with T2DM in the primary health care setting combined with adequate training and support can be beneficial. A recent systematic analysis of all systematic reviews from the year 1996 to 2015 showed that mobile interventions can boost self-care and reduce HbA1c in patients with T2DM by 0.8% [27]. Cui et al. [17] reported a moderate improvement in HbA1c following the use of mobile interventions through their meta-analysis of 13 studies involving patients with T2DM [17]. Ryan et al. [28] also showed a similar pattern among patients with type 1 DM, where there was a statistically significant improvement in HbA1c. This study reports a promising improvement in the clinical biomarkers related to DM with the use of DiaMon, consistent with evidence.

5 Strengths and Limitations The study was undertaken in Mauritius, a country well known for its ethnic diversity and high prevalence of T2DM. Recent modernization and westernisation have accentuated the risk of developing NCDs. At present, almost 1 in 2 adults either have T2DM or have the risk of developing T2DM [14]. Hence, this provides us the perfect environment to observe the state of diabetes and the behaviour of the population and to test the feasibility

248

S. D. Nagowah et al.

and acceptability of modern technology in DM self-care. Our sample consisted of patients with reasonably controlled DM at baseline, with an average HbA1c of 7.7% to reducing the risk of medication change, as a potential confounding effect. We could thus ascertain the changes in glycaemic control and behaviour might have been triggered essentially through the use of the mobile application. No changes in medications have been reported by the participants. The DiaMon application has integrated charts for daily charting of the food intake and physical activity, which enables the participants to review their metabolic input and output in relation to their glucose levels. Participants were then motivated to adapt their lifestyle to achieve their targeted glycaemic level. Most participants in this study found this feature useful and interesting and confirmed that their eating habits have improved. Weight reduction has also been observed to quantify the behavioural change. However, the study was found to have some limitations. DiaMon was developed for Android mobile phones. During the recruitment process, some participants were not able to enrol in the study as they had iPhones on which the Android application could not be installed. The development of an iOS-compatible application is, however, under consideration. Some participants reported that they were not able to correctly report their food intake while the application worked fine for the other sections. Improvement in physical activity, however, proved to be more challenging to achieve. Eating, being part of the daily routine, facilitates review and reflection compared to physical activity which is still not part of many people’s routine. Thomas et al. [30] estimated that 34% of patients with DM performed regular physical activity while only 9% reached the recommended heart rate. Although the proposed sample size was much larger than currently enrolled, this pilot study was successfully completed in a country where clinical research among the public and in healthcare settings is still alienated.

6 Conclusion Technology-assisted self-management of T2DM is getting more attention globally and there is an urge to develop effective tools that will enable easy monitoring of patients with T2DM. This research has led to the development of a mobile application (DiaMon) that can be used by individuals with T2DM and those with pre-diabetes conditions. The application enables users to daily log their blood sugar levels before and after breakfast and dinner, in addition to documenting the quantity and type of food consumed and the duration of physical activity. It subsequently tracks the glucose levels and displays them on a graph over several days. Users can thus establish correlations between their blood sugar levels and the foods they have consumed and their level of physical activity. Following a survey that has been conducted, it was noted that DiaMon has been well accepted and the tool can have major future ramifications. This pilot study has demonstrated the link between the self-management of T2DM and glycaemic control. There was a significant improvement in the endpoints such as glycated haemoglobin and fasting blood glucose post-intervention. Moreover, important changes were seen in secondary outcomes such as weight, BMI and physical activity. The widespread use of such a tool or similar principles can be used to encourage positive changes among patients living with T2DM. Nonetheless, it is advisable to conduct further rigorous studies to validate the findings of this research.

Self-management of Type-2 Diabetes Using a Mobile Application

249

References 1. Global report on diabetes. World Health Organisation. https://www.who.int/health-topics/dia betes#tab=tab_1. Accessed 5 Sept 2023 2. IDF Diabetes Atlas. International Diabetes Federation. 9th edition 2019. https://www.diabet esatlas.org/. Accessed 5 Sept 2023 3. UK Prospective Diabetes Study (UKPDS) Group: Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). Lancet 352, 854–865 (1998). PMID: 9742977 4. Diabetes Control and Complications Trial Research Group, Nathan, D.M. Genuth, S., Lachin, J., Cleary, P., Crofford, O., et al.: The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N. Engl. J. Med. 329, 977–986 (1993). https://doi.org/10.1056/NEJM199309303291401 5. ADVANCE Collaborative Group, Patel, A., MacMahon, S., Chalmers, J., Neal, B., Billot, L., et al.: Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes. N. Engl. J. Med. 358, 2560–2572 (2008). https://doi.org/10.1056/NEJMoa0802987 6. ACCORD Study Group, Gerstein, H.C., Miller, M.E., Byington, R.P., Goff, D.C., Bigger, J.T., et al.: Effects of intensive glucose lowering in type 2 diabetes. N. Engl. J. Med. 358, 2545–2559 (2008). https://doi.org/10.1056/NEJMoa0802743 7. Introduction: Standards of Medical Care in Diabetes-2019. Diabetes Care 42, S1–S2 (2019). https://doi.org/10.2337/dc19-Sint01 8. Bailey, T.S., Walsh, J., Stone, J.Y.: Emerging technologies for diabetes care. Diabetes Technol. Ther. 20, S278–S284 (2018). https://doi.org/10.1089/dia.2018.0115 9. From innovation to implementation: eHealth in the WHO European region. World Health Organization 2016. Regional Office for Europe. https://www.euro.who.int/en/publications/ abstracts/from-innovation-to-implementation-ehealth-in-the-who-european-region-2016. Accessed 5 Sept 2023 10. Moonian, O., Jodheea-Jutton, A., Khedo, K.K., Baichoo, S., Nagowah, S.D., Nagowah, L., et al.: Recent advances in computational tools and resources for the self-management of type 2 diabetes. Inform. Health Soc. Care 45(1), 77–95 (2020). https://doi.org/10.1080/17538157. 2018.1559168 11. Kirk, J.K., Stegner, J.: Self-monitoring of blood glucose: practical aspects. J. Diabetes Sci. Technol. 4, 435–439 (2010). https://doi.org/10.1177/193229681000400225 12. Quinn, C.C., Butler, E.C., Swasey, K.K., Shardell, M.D., Terrin, M.D., Barr, E.A., et al.: Mobile diabetes intervention study of patient engagement and impact on blood glucose: mixed methods analysis. JMIR Mhealth Uhealth 6, e31 (2018). https://doi.org/10.2196/mhe alth.9265 13. Countries ranked by Diabetes prevalence (% of population ages 20 to 79). Index Mundi. (2019). https://www.indexmundi.com/facts/indicators/SH.STA.DIAB.ZS/rankings. Accessed 5 Sept 2023 14. The Trends in Diabetes and Cardiovascular Disease Risk in Mauritius- Mauritius NCD Survey 2015. Ministry of Health and Quality of Life. https://untobaccocontrol.org/impldb/wp-con tent/uploads/mauritius_2018_annex-1_NCD_survey_report_2015.pdf. Accessed 5 Sept 2023 15. Musango, L., Timol, M., Burhoo, P., Shaikh, F., Donnen, P., Kirigia, J.M.: Assessing health system challenges and opportunities for better noncommunicable disease outcomes: the case of Mauritius. BMC Health Serv. Res. 20(1), 184 (2020). https://doi.org/10.1186/s12913-0205039-4.PMID:32143648;PMCID:PMC7059264 16. Hou, C., Carter, B., Hewitt, J., Francisa, T., Mayor, S.: Do Mobile Phone Applications Improve Glycemic Control (HbA1c) in the self-management of diabetes? a systematic review, metaanalysis, and GRADE of 14 randomized trials. Diabetes Care 39, 2089–2095 (2016). https:// doi.org/10.2337/dc16-0346

250

S. D. Nagowah et al.

17. Cui, M., Wu, X., Mao, J., Wang, X., Nie, M.: T2DM self-management via smartphone applications: a systematic review and meta-analysis. PLoS ONE 11, e0166718 (2016). https://doi. org/10.1371/journal.pone.0166718 18. McMillan, K.A., Kirk, A., Hewitt, A., MacRury, S.: A systematic and integrated review of mobile-based technology to promote active lifestyles in people with type 2 diabetes. J. Diabetes Sci. Technol. 11(2), 299–307 (2017). https://doi.org/10.1177/1932296816656018 19. Beck, J., Greenwood, D.A., Blanton, L., Bollinger, S.T., Butcher, M.K., Condon, J.E., et al.: National standards for diabetes self-management education and support. Diabetes Care 40, 1409–1419 (2017). https://doi.org/10.2337/dci17-002517 20. Gallé, F., Di Onofrio, V., Cirella, A., Di Dio, M., Miele, A., Spinosa, T., et al.: Improving selfmanagement of type 2 diabetes in overweight and inactive patients through an educational and motivational intervention addressing diet and physical activity: a prospective study in naples. South Italy. Diabetes Ther 8, 875–886 (2017). https://doi.org/10.1007/s13300-0170283-2 21. Hunt, C.W.: Technology and diabetes self-management: An integrative review. World J. Diabetes 6, 225–233 (2015). https://doi.org/10.4239/wjd.v6.i2.225 22. Sahu, M., Grover, A., Joshi, A.: Role of mobile phone technology in health education in Asian and African countries: a systematic review. Int. J. Electron. Healthc. 7, 269–286 (2014). https:// doi.org/10.1504/IJEH.2014.064327 23. Haddad, N.S., Istepanian, R., Philip, N., Khazaal, F.A.K., Hamdan, T.A., Pickles, T., et al.: A feasibility study of mobile phone text messaging to support education and management of type 2 diabetes in Iraq. Diabetes Technol. Ther. 16, 454–459 (2014). https://doi.org/10.1089/ dia.2013.0272 24. Goyal, S., Morita, P., Lewis, G.F., Yu, C., Seto, E., Cafazzo, J.A.: The systematic design of a behavioural mobile health application for the self-management of type 2 diabetes. Can. J. Diabetes 40, 95–104 (2016). https://doi.org/10.1016/j.jcjd.2015.06.007 25. Haytowitz, D.B., Pehrsson, P.R.: USDA’s National Food and Nutrient Analysis Program (NFNAP) produces high-quality data for USDA food composition databases: Two decades of collaboration. Food Chem. 238, 134–138 (2018). https://doi.org/10.1016/j.foodchem.2016. 11.08225 26. Torbjørnsen, A., Ribu, L., Rønnevig, M., Grøttland, A., Helseth, S.: Users’ acceptability of a mobile application for persons with type 2 diabetes: A qualitative study. BMC Health Serv. Res. 19(1), 641 (2019). https://doi.org/10.1186/s12913-019-4486-2 27. Kitsiou, S., Paré, G., Jaana, M., Gerber, B.: Effectiveness of mHealth interventions for patients with diabetes: An overview of systematic reviews. PLoS ONE 12, e0173160 (2017). https:// doi.org/10.1371/journal.pone.0173160 28. Ryan, E.A., Holland, J., Stroulia, E., Bazelli, B., Babwik, S.A., Li, H., et al.: Improved A1C levels in type 1 diabetes with smartphone app use. Can. J. Diabetes 41, 33–40 (2017). https:// doi.org/10.1016/j.jcjd.2016.06.001 29. Yu, K., Wu, S., Liu, R., Chi, I.: Harnessing mobile technology to support type 2 diabetes self-management among Chinese and Hispanic immigrants: a mixed-methods acceptability study. J. Ethn. Cult. Divers. Soc. Work 32(4), 171–184 (2023) 30. Thomas, N., Alder, E., Leese, G.P.: Barriers to physical activity in patients with diabetes. Postgrad. Med. J. 80, 287–291 (2004). https://doi.org/10.1136/pgmj.2003.010553

Plant Disease Prediction Using Deep Learning Techniques Widaad Fayid Hulkury and Leckraj Nagowah(B) University of Mauritius, Réduit, Mauritius [email protected], [email protected]

Abstract. With the rising importance of agriculture in ensuring food security, early disease detection is crucial to mitigate yield losses and economic impacts caused by crop diseases. In this paper, a lightweight mobile application, BotaniCare, is presented which aims at revolutionizing agricultural practices by using deep learning techniques to accurately detect and diagnose diseases in plants. The system has been trained on a dataset of images of both healthy plants and those affected by various diseases. Through the implementation of the model, the system could recognize patterns and anomalies in plant health, enabling precise identification of plant diseases with minimal human intervention. The mobile application also included remedial actions to health the plants from the identified disease. The development of the plant disease prediction system involved data collection, pre-processing, model selection, testing and evaluation. The model was trained on labeled data and evaluated using appropriate metrics to ensure the reliability of the model. Different CNN architectures were compared and evaluated to be able to choose the most suitable one. By using transfer learning, MobileNetV2 was used in BotaniCare and a training accuracy of 98.7% and a validation accuracy of 96.4% was achieved during the model development and evaluation process. BotaniCare was thoroughly assessed using real-life images and validated against expert diagnosis, demonstrating its high accuracy and reliability in disease prediction. It is anticipated that the mobile application will be widely used by farmers in Mauritius to identify the frequent diseases of the common plants and apply appropriate remedial actions. Keywords: plant disease prediction · deep learning · CNN · transfer learning · mobile application

1 Introduction In Mauritius, there are many farmers who rely on plantations as a means of livelihood. However, when their plants are affected by diseases, it causes a significant threat to agricultural productivity and food security [1]. The affected plants exhibit symptoms that are mostly visible on their leaves. Unfortunately, many planters lack specific knowledge about the different types of diseases and the appropriate remedies to be taken. As a result, they often resort to human visual examination and experience to identify the crop diseases and treat them. While this method is commonly used, it is not always accurate. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 251–263, 2024. https://doi.org/10.1007/978-3-031-56075-0_24

252

W. F. Hulkury and L. Nagowah

In other cases, farmers often have to seek the assistance of experts which inevitably incurs additional costs while also leading to a delay in the treatment of the infected plants [2]. With the widespread use of smartphones, there is an opportunity to leverage latest technologies for more efficient and accurate disease prediction [3]. The problem at hand revolves around the agricultural practices in Mauritius and also in other parts of the world, where planters face a critical issue which is the lack of accessible and accurate plant disease identification methods. Currently, planters heavily rely on visual examination to identify crop diseases, which may also be time-consuming [4]. The existing reliance on human visual inspection for disease identification may lead to delayed responses to take remedial actions. Furthermore, the availability of skilled agricultural experts may be limited, hindering effective disease diagnosis and management [5]. The primary aim of this work is to develop a mobile application for a plant disease recognition model based on leaf image classification by using deep learning techniques. The model aims at identifying the main types of diseases of the common plants in Mauritius and thereafter recommend treatment practices. The remedial measures are recommended after the accurate recognition of the disease which can therefore be beneficial for farmers and other agricultural organizations. The remainder of this paper is organized as follows. Section 2 presents a literature review on existing systems for plant disease prediction along with a comparative analysis. Section 3 highlights the architecture of the system BotaniCare. The system prototype and testing of the application are discussed in Sect. 4. An evaluation of the system is carried out in Sect. 5 and finally Sect. 6 concludes the paper.

2 Literature Review Research about plant disease prediction and detection has been conducted by several researchers as it is a major challenge that leads to production and economic losses. In this section, some recent related works are highlighted. Pandhare et al. [6] conducted research centered on the application of Convolutional Neural Networks (CNN) for the prediction of plant diseases, with a specific focus on recognizing cotton leaf diseases through a web-based system. This system comprised of two primary components: a training model and image processing. Using a dataset featuring various cotton diseases, the model achieved an accuracy rate of 89%. Rautaray et al. [7] employed a transfer learning approach for the efficient detection of diseases in paddy plants, using Python for its deep learning capabilities. The model comprised of two main processes: classification and detection, utilizing CNN with the VGG-16 architecture. By extracting features from paddy leaves, the model achieved 92% training and 90% testing accuracy in identifying plant diseases. Karnik and Suthar [8] adopted a two-step classification approach to categorize images of diseased plants. The researchers used a dataset of 1000 plant leaf images and conducted image pre-processing to enhance model performance, including noise reduction using a median filter. The first classifier stage involved the implementation of the YOLOv3 algorithm for pre-processing, while the second classifier stage utilized a ResNet50based CNN for disease prediction. This approach yielded 97% accuracy in detecting leaf diseases.

Plant Disease Prediction Using Deep Learning Techniques

253

Militante et al. [9] have used CNN to be able to detect various plants and their diseases. The CNN model applied a neural network application program interface which was written in Python. The PlantVillage dataset was used to train the model and the proposed model achieved an accuracy of 96.5% for detecting the plant diseases. Gayathri et al. [10] used a deep CNN named LeNet to detect diseases of tea leaves. They have used the PlantVillage dataset whereby 80 images have been selected and categorized into four classes namely, tea leaf blight, blight disease, red spot, red scab. The image was passed through a single flattening convolutional layer which was followed by two fully connected layers. The accuracy of the model was 94% for tea red scab leaves and for the other remaining leaves, it was between 84% and 93%. Chohan et al. [11] used a deep learning model to detect several diseases of plants from their leaves. This system used a classification algorithm which focused on the feature extraction of CNN. The PlantVillage dataset was used for training the model. Afterwards, a CNN with multiple convolutional layers and pooling layers was created to classify and predict the name of disease of the different plants. The model achieved 95% testing accuracy. Zaw et al. [12] used SVM classifier algorithm in MATLAB R2017a to classify the diseases of plant leaves. The procedures of the proposed model were categorized into four main phases namely: image pre-processing, image segmentation, feature extraction, disease detection and classification. The model took about 0.24 min for execution and achieved an accuracy of 83%. Jasim and AL-Tuwaijari [1] used CNN for the classification and detection of plant leaf diseases. The main aim of the study was to detect diseases for the following types of plants: tomatoes, pepper and potatoes. Many processes like image acquisition, image processing, designing CNN model, training and testing were carried out to be able to accurately detect the diseases of plants. The PlantVillage dataset was chosen which consists of around 20600 images with 15 different classes on which the proposed model was trained. The model achieved a training and testing accuracy of 98%. Krishnamoorthy et al. [13] have utilized InceptionResNetV2 with transfer learning approach for the detection of diseases in rice leaves. By using Keras framework and TensorFlow, the researchers have trained the deep neural network using a dataset downloaded from Kaggle website. This proposed model achieved an accuracy of 95.67%. Ibrahim and Atya [14] used several machine learning algorithms to classify rice leaf diseases. They initially acquired the images in real-time environment by using a highresolution camera. Image processing techniques were then applied which included noise removal and feature extraction. Disease classification was then carried out using J48, Random Forest and Naïve Bayes algorithms. The authors reported an accuracy of 95% using the n-fold cross validation technique. Chen et al. [15] conducted research where a transfer learning of deep CNN VGGNet pre-trained model on ImageNet and the Inception module was selected for their proposed system. Fujian Institute of Subtropical Botany provided the images dataset, and then, image processing techniques were applied to the images. Data augmentation was also used to obtain more sample data to attain a better accuracy. The system achieved an accuracy of 92% in detecting rice plants diseases.

254

W. F. Hulkury and L. Nagowah

Advanced computer vision and machine learning techniques were employed to detect 20 distinct diseases affecting five common plant species by Kulkarni et al. [16]. Using the PlantVillage dataset, the researchers carried pre-processing, feature extraction and used the Random Forest classification algorithm for disease recognition, which led to a 93% accuracy. Table 1 gives comparative analysis of the related works that focused on the classification of plant diseases including the datasets, algorithm used, metrics used and the main features of the applications. It was observed from the related works that various datasets have been used. Pandhare et al. [6] created a real-time dataset consisting of 520 images, while Krishnamoorthy et al. [13] obtained images from the Kaggle website. Chen et al. [15] obtained their dataset from the Fujian Institute of Subtropical Botany. However, to a large extent, the PlantVillage is the most commonly used dataset among the researchers for the development of their systems for plant disease detection [1, 9–11, 16]. Table 1. Comparison table for related works Paper Reference

Dataset

Algorithm

Metrics

Features

Pandhare et al. [6]

Real-time dataset made up of 520 images

CNN

Accuracy: 89%

Recognize cotton leaf diseases Analyze dead leaf images

Rautaray et al. [7]

N/A

VGG-16 Transfer Learning

Accuracy: 90%

Predict disease type Classify into 3 categories with probability

Karnik and Suthar [8]

N/A

Pre-processing YOLOv3 Prediction- Resnet50

Intersection over union: 97%

Classify images of plants Identify name of disease

Militante et al. [9]

PlantVillage

CNN

Accuracy: 96.5%

Classify plant leaves Recognize 32 different plants Identify plant disease

Gayathri et al. [10] PlantVillage

LeNet5

Accuracy: 90.23% Discover tea plant Sensitivity: 86 disease – 94%

Chohan et al. [11]

CNN

Accuracy: 98.3%

PlantVillage

Tells the plant type Detect whether there is a disease and the type of disease

(continued)

Plant Disease Prediction Using Deep Learning Techniques

255

Table 1. (continued) Paper Reference

Dataset

Algorithm

Metrics

Features

Zaw et al. [12]

N/A

Segmentation K-means clustering Classification Support Vector Machine

Accuracy: 83%

Classify disease Identify diseased leaf correctly

Jasim and AL-Tuwaijari [1]

PlantVillage

CNN

Accuracy: 98.029%

Classify leaves Types of plants: tomatoes, pepper and potatoes Detect plant diseases

Krishnamoorthy et al. [13]

Download rice leaf InceptionResNetV2 Accuracy: 95.67% images from Kaggle with transfer learning Precision: 0.96 website Recall: 0.98 F1 Score:0.96

Classify leaf of plants Recognize diseases in rice leaf

Ibrahim and Atya [14]

1000 images of healthy and diseased rice plant leaf

Random Forest

Accuracy: 95.5%

Classify and detect rice disease into 3 categories: Brown Spot, Bacterial Blight, Leaf Smut

Chen et al. [15]

Fujian Institute of Subtropical Botany

VGGNet pre-trained model and Inception module

Accuracy: 92% Sensitivity: 80%

Different diseases of rice, maize Detect disease category

Random Forest

Accuracy: 93% F1 Score: 0.93

Web app for detecting plant disease Deployed on free cloud hosting server

Kulkarni et al. [16] Plant Village

With respect to the preferred models of the researchers, Pandhare et al. [6], Militante et al. [9], Gayathri et al. [10], Jasim and AL-Tuwaijari [1] and Chohan et al. [11] used CNNs for their studies. Rautaray et al. [7], Krishnamoorthy et al. [13] used CNN with transfer learning namely VGG-16, InceptionResNetV2 and EfficientNet respectively. Chen et al. [15] used a pre-trained VGGNet model with Inception module while Karnik and Suthar [8] utilized YOLOv5 for processing and ResNet50 for prediction. Ibrahim and Atya [14] and used Random Forest, Naïve Bayes, SVM for their classification models. Zaw et al. [12] used segmentation with K-means, GLCM for feature extraction, and SVM for classification of plant leaves. Two of the lowest accuracies obtained were 89% [6] and 83% [12]. Most of the other studies had an accuracy of above 90% with a whopping 98.3% obtained in [11].

256

W. F. Hulkury and L. Nagowah

3 System Architecture To cater for the main diseases of the common plants of Mauritius, BotaniCare has been proposed and implemented. This section provides the architectural design of BotaniCare along with the description of the main processes and the dataset. Figure 1 depicts the detailed design of the proposed system.

Fig. 1. Architecture of BotaniCare

The dataset contained 5 categories of images namely Sugarcane, Maize, Potato, Tomato and Strawberry which are the common crops planted by farmers in Mauritius. A customized dataset was created from the publicly available datasets, namely PlantVillage, Potato Leaf, Sugarcane Disease, Red Rot Sugarcane Disease Leaf, and Sugarcane Leaf Disease dataset. Then, the custom dataset has been split into 3 main folders namely train, test and valid. The trained folder of the custom dataset has been augmented by using different techniques to increase the number of images to train the model. Then, preprocessing techniques like resizing and normalization were applied to the input images before feeding them into different models. The models were then trained using transfer learning approach. The model with the highest accuracy was optimized by applying regularization. The model was then saved in the.h5 file format, which was converted to the TensorFlow Lite format, optimizing its size, and enabling efficient deployment. The trained model was deployed and integrated into Android studio, allowing for real-time predictions on Android devices.

4 Prototype Implementation and Testing Different tools and technologies were used in the implementation of the proposed system for plant disease prediction using deep learning. Python was used as programming language given its simple syntax and small learning curve. Google Colab was chosen

Plant Disease Prediction Using Deep Learning Techniques

257

for the IDE for its pre-installed packages and libraries. Deep learning libraries such as TensorFlow and Keras were used as both are highly flexible and scalable. Image processing libraries like Matplotlib and NumPy were used for creating plots and charts for better visualization of data. The dataset exhibited varying class sizes which lead to class imbalance. To mitigate this issue, data augmentation was applied exclusively to the training data to address class imbalances and create a more balanced representation of each class as shown in Fig. 2. Techniques like rotation, zooming and flipping were used. The dataset contained 5 distinct plants along with the 10 most common diseases resulting in a total of 15 classes. Prior to data augmentation, the dataset was imbalanced with a total of 6600 images. After data augmentation, each class had 800 images, summing up to 12000 images for the whole dataset.

Fig. 2. Visualization of augmented train set

In the pursuit of optimizing model performance, several image pre-processing techniques were examined including edge detection, denoising, grayscale conversion, resizing, and scaling and image normalization. After careful evaluation, it was determined that scaling and normalization were particularly well-suited for this study’s objectives. As a result, these two techniques were applied to the input images which led to notable improvements in the model ensuring that the model could effectively classify patterns and features within the images. In the deep learning model implementation, six individual models were trained: MobileNetV2, AlexNet, Xception, Inception, DenseNet201 and ResNet50. Table 2 shows the accuracy of each model that were trained on 10 epochs each. After evaluating the models, it was observed that AlexNet and ResNet had lower accuracies compared to the other models. Further training on the remaining models with a higher number of epochs was done to identify the best model for the final classification. Table 3 shows the accuracy of each model that were trained on 25 epochs. Among the evaluated models, Inception, XceptionV3, DenseNet201, and MobileNetV2, all achieved high accuracies, indicating their strong performance in learning the data patterns. However, MobileNetV2 stood out as the preferred model for the mobile app due to its validation accuracy of 95.1% and the other metrics.

258

W. F. Hulkury and L. Nagowah Table 2. Accuracy of models trained (10 epochs)

Models

Training Accuracy

Validation Accuracy

Inception

93.3%

86.6%

Xception

95.8%

88.0%

DenseNet201

94.1%

94.4%

MobileNetV2

96.7%

94.2%

AlexNet

89.7%

72.3%

ResNet50

60.1%

53.1%

Table 3. Accuracy of models trained (25 epochs) Models

Training Accuracy

Validation Accuracy

Precision

Recall

Inception

95.9%

88.7%

95.9%

95.9%

XceptionV3

95.6%

93.7.%

96.2%

95.4%

DenseNet201

95.3%

94.2%

95.2%

92.1%

MobileNetV2

97.8%

95.1%

97.4%

95.8%

The next step in the process involved hyperparameter tuning, which aimed to optimize the selected model further by adjusting parameters such as learning rate and optimizer. While conducting this tuning, RMSprop optimizer was also used. However, it returned lower accuracies compared to the Adam optimizer. Table 4 highlights the model accuracy of with the Adam optimizer along with various learning rates. Table 4. Hyperparameter Tuning Results Optimizer

Learning Rate

Training Accuracy

Validation Accuracy

Precision

Recall

Adam

0.01

93.2%

91.4%

95.2%

92.4%

0.001

97.5%

95.6%

97.8%

97.3%

0.004

92.9%

91.5%

93.5%

92.1%

0.0001

95.9%

94.6%

96.3%

94.8%

0.0003

96.8%

94.9%

97.8%

96.5%

Plant Disease Prediction Using Deep Learning Techniques

259

As depicted in Table 4, Adam with learning rate 0.001 achieved the best results. Therefore, this model was used with a regularization technique to prevent overfitting and improve model generalization. Then the model attained a training accuracy of 98.7% and a validation accuracy of 96.4%, indicating good performance as shown in Fig. 3.

Fig. 3. Accuracy and loss graph for MobileNetV2

The application was then tested to ensure that it met its objectives. BotaniCare was tested on healthy plants, plants with diseases and also on images with noises, i.e., blurred images. Figure 4 shows the correct classification of a healthy potato leaf and Fig. 5 shows a diseased plant being correctly identifies by the application along with the accuracy as a percentage. The causes, symptoms, preventions, and treatment measures can be explored as well along with an ontology diagram.

Fig. 4. Healthy plant from gallery

Fig. 5. Diseased plant and details explored.

Figure 6 shows that BotaniCare was also able to correctly classify diseases even if the captured image was blurred.

260

W. F. Hulkury and L. Nagowah

Fig. 6. Blurry leaf from Camera

5 Evaluation An evaluation of BotaniCare was carried out firstly with respect to an existing application and secondly by human experts who are knowledgeable in the frequent diseases of the common plants of Mauritius. 5.1 Evaluation with Respect to Existing System To assess the accuracy of the BotaniCare application, its classification was compared to an existing system, namely PictureThis, using an identical set of leaf images. Table 5 presents the results obtained during this evaluation process. After the evaluation, it can be concluded that all the disease classifications in the BotaniCare application were done correctly. Moreover, our proposed application offered a better approach to detect plant diseases unlike the existing app where it indicates only whether the plant is sick or not. BotaniCare also provided the exact name of the disease which affected the plant and also some remedies to treat the plant.

Plant Disease Prediction Using Deep Learning Techniques

261

Table 5. Evaluation results with respect to similar system PictureThis (Existing System)

BotaniCare (Proposed System)

Classified correctly

5.2 Evaluation by Human Experts In order to assess the reliability of the application BotaniCare, an independent research officer at the MSIRI was assigned to labeling the leaves of sugarcane. Two local farmers were requested to label the remaining leaves. The same leaves were then processed by the BotaniCare application. A sample of the classification done by BotaniCare and those done by the expert is presented Table 6. It was deduced that BotaniCare correctly identified the diseases while even the expert and farmers had difficulties in getting the exact names of the plant diseases.

262

W. F. Hulkury and L. Nagowah Table 6. Evaluation results

Photo

Expert Labelling

BotaniCare Classification

Classified correctly

Potato Disease

Sugarcane Red Rot

Legend

Expert has been able: • To identify plant name and correctly label plant status • To identify the exact name of disease if affected Expert has been able: • To identify plant name and correctly label plant status Expert not been able to identify the exact name of disease

6 Conclusion In this paper, a lightweight mobile application, BotaniCare was developed that could identify the diseases of the main crops in Mauritius, namely Sugarcane, Maize, Potato, Tomato and Strawberry. By using transfer learning, MobileNetV2 was used to implement the deep learning model with a training accuracy of 98.7% and a validation accuracy of 96.4%. It can be concluded BotaniCare met all the requirements with respect to plant disease prediction. It also provided plant care tips and also facilitated easy access to reputable plant institutes like MSIRI and FAREI, allowing users to seek further guidance and assistance. Users could capture leaf images and obtain disease predictions with confidence scores. Treatment and preventive measures were provided through ontology design. Overall, the system provided a seamless user experience, empowering users to make informed decisions about plant care and disease prevention. As future works, the plan is to enhance the plant recognition system by expanding its dataset and refining its capabilities. By acquiring more local images of plants found in Mauritius, the dataset will grow, and the application would cover a wider range of

Plant Disease Prediction Using Deep Learning Techniques

263

plant categories and their diseases. Employing ensemble techniques will be investigated which may lead to more accurate predictions. User feedback will also be taken into consideration to ensure the system meets real-world needs, making it a valuable tool for plant disease recognition in Mauritius.

References 1. Jasim, M.A., AL-Tuwaijari, J.M.: Plant leaf diseases detection and classification using image processing and deep learning techniques. In: 2020 International Conference on Computer Science and Software Engineering (CSASE) (2020) 2. Simhadri, C.G., Kondaveeti, H.K.: Automatic recognition of rice leaf diseases using transfer learning. Agronomy 13(4), 961 (2023) 3. Mohanty, S.P., Hughes, D.P., Salathé, M.: Using deep learning for image-based plant disease detection. Front. Plant Sci. 7, 1419 (2016) 4. Pani, S., Rout, J., Afroz, Z., Dey, M., Sahoo, M.K., Das, A.K.: Diagnosis of plant diseases by image processing model for sustainable solutions. In: Mohanty, S.N., Diaz, V.G., Satish Kumar, G.A.E. (eds.) Intelligent Systems and Machine Learning: First EAI International Conference, ICISML 2022, Hyderabad, India, December 16-17, 2022, Proceedings, Part II, pp. 181–192. Springer Nature Switzerland, Cham (2023). https://doi.org/10.1007/978-3-03135081-8_15 5. Bhujel, A., et al.: Detection of gray mold disease and its severity on strawberry using deep learning networks. J. Plant Dis. Prot. 129(3), 579–592 (2022) 6. Pandhare, N., Panchal, V., Mishra, S.S., Tambe, M.D.: Cotton plant disease detection using deep learning. Int. Res. J. Modern. Eng. Technol. Sci 4(04) (2022) 7. Rautaray, S.S., Pandey, M., Gourisaria, M.K., Sharma, R., Das, S.: Paddy crop disease prediction - a transfer learning technique. Int. J. Recent Technol. Eng. 8(6), 1490–1495 (2020) 8. Karnik, J., Suthar, A.: Agricultural plant leaf disease detection using Deep Learning Techniques. SSRN Electron. J. (2021) 9. Militante, S.V., Gerardo, B.D., Dionisio, N.V.: Plant Leaf Detection and disease recognition using Deep Learning. In: 2019 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE) (2019) 10. Gayathri, S., Wise, D.J.W., Shamini, P.B., Muthukumaran, N.: Image analysis and detection of tea leaf disease using deep learning. In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), pp. 398–403. IEEE (2020) 11. Chohan, M., Khan, A., Chohan, R., Katpar, S.H., Mahar, M.S.: Plant disease detection using deep learning. Int. J. Recent Technol. Eng. 9(1), 909–914 (2020) 12. Zaw, K.K., Myo, Z.M.M., Thoung, D.T.H.: Support vector machine based classification of leaf diseases. Int. J. Sci. Eng. Appl. 7, 143–147 (2018) 13. Krishnamoorthy, N., Prasad, L.N., Kumar, C.P., Subedi, B., Abraha, H.B., Sathishkumar, V.E.: Rice leaf diseases prediction using deep neural networks with transfer learning. Environ. Res. 198, 111275 (2021) 14. Ibrahim, D.A.W., Atya, D.B.: Detection of diseases in rice leaf using Deep Learning and Machine Learning Techniques. Webology 19(1), 1493–1503 (2022) 15. Chen, J., Chen, J., Zhang, D., Sun, Y., Nanehkaran, Y.: A. Using deep transfer learning for image-based plant disease identification. Comp. Electron. Agric. (2020) 16. Kulkarni, P., Karwande, A., Kolhe, T., Kamble, S., Joshi, A., Wyawahare, M.: Plant disease detection using image processing and machine learning. arXiv preprint arXiv:2106.10698 (2021)

From Headsets to Mindsets – Human-Centred Extended Reality

3D Transformation of 2D Captured Museum Objects at Risk Maxim Goynov1 , Dušan Tati´c2 , Desislava Paneva-Marinova1(B) , Radomir S. Stankovi´c2 , Detelin Luchev1 , Emanuela Mitreva1 , and Lilia Pavlova3 1 Institute of Mathematics and Informatics at the Bulgarian Academy of Sciences, Sofia,

Bulgaria [email protected], [email protected] 2 Mathematical Institute of the Serbian Academy of Sciences and Arts, Belgrade, Serbia 3 Laboratory of Telematics at the Bulgarian Academy of Sciences, Sofia, Bulgaria

Abstract. This paper presents a methodology for 2D to 3D object transformation, based on 2D captured museum assets (such as old photos, scans, or other 2D digitized content) for items with limited access due to their age, value or fragile nature. The technology simplifies 3D object generation in order to reduce cost of hardware or personnel for 3D production. It is applicable for improved display of valuable artefacts in 3D web-based virtual museums, galleries, educational games, etc. Keywords: 2D-3D Transformation · 3D Design · 3D Virtualization · Museum Objects in Risk

1 Introduction Digital technologies are introducing new solutions for museums, galleries and libraries and their new presentation in virtual space. Alluring solutions are offered by 3D technologies, but they come with challenges related to 3D design for content tinkering and making. Technologies like 3D scanning and photogrammetry are usually used to create a 3D visualization. But those technologies require a large number of resources (specialized equipment, competencies, skills, time, etc.) and sometimes we cannot implement them due to various constraints or circumstances related to physical access to the target objects. This is usually the case for the most valuable museum items with limited access due to their bad condition (due to their age or fragile state), and risk of damage or destruction of the item, which makes them not suitable for displaying or makes the actual display difficult. Besides the artefacts, the museums store photos of valuable artefacts that are irretrievably lost for which solutions for 3D revival would be of most importance. This paper presents a methodology for 2D to 3D object transformation, based on 2D captured museum assets (such as old photos, scans, or other 2D digitized content) for artefacts with limited access. The methodology offers a solution for simplifying 3D object generation including the usage of AI tools, reducing the need for expensive © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 267–276, 2024. https://doi.org/10.1007/978-3-031-56075-0_25

268

M. Goynov et al.

hardware and highly qualified personnel for 3D production, but applicable for improved display of valuable artefacts in 3D web-based virtual museums, galleries, educational games, etc. The paper also offers an applicable framework for enhanced existing digital competencies and skills for 3D modelling (tinkering and making). Experiments have been realized with medals from Serbian history and in particular the war between Serbia and Turkey in 1912 and the suppression of the Timoˇcka Rebellion in 1883. They were awarded to members of the Royal Serbian Army for proven service to the king and the fatherland or for exceptionally excellent and zealous service during peace, emergency or war.

2 Approach 2.1 Methodology for 2D to 3D Object Transformation The presented methodology aims to show the basic steps for the creation of a simplified and web-optimized 3D object based on a limited number of 2D images of the corresponding object. It is suitable for objects having relatively standard geometry like coins, medals, vases, etc. Unlike other techniques like 3D scanning and photogrammetry the proposed one targets to create a low polygon object at the very beginning of the process, making it render-friendly for older or cheaper devices (PCs, mobile phones, tablets, SMART TVs, etc. devices with average graphic cards). The process starts with just an image (Fig. 1) and the first step is to trace and vectorize the contour of the object for which we use Inkscape open-source vector graphics software [1]. The next step is to import the vectorized contour in a 3D modelling software (we use Blender [2]) and make a 3D mesh from the 2D vector graphic (Fig. 2) - for medals and coins, a solidify modifier is used (surface extrusion), for cylindrical objects (like vases) a lattice modifier can be used. After the geometry of the objects is created, we need to define its materials and colors. Colors are usually defined using diffuse maps. A diffuse map is a standard bitmap image. The original 2D image of the object can be used as a diffuse map. There should be a relation between the original image and the created 3D geometry. This relation is called UV map. UV maps define how every polygon of the 3D geometry is positioned at the diffuse bitmap (Fig. 3). One of the most important parts of 3D modelling (especially when we talk about presenting an object’s small details) is normal mapping. The normal mapping defines the orientation of a surface toward a light source. Usually, a normal map is a matrix of RGB pixels mapped to the object according to the UV map. Every RGB pixel defines an XYZ vector (RGB colors are mapped as XYZ vectors). The normal vector is perpendicular to every part of the object’s surface, so the light reflection changes accordingly and the 3D effect is achieved on a flat surface. This technique is very common in 3D graphics because it doesn’t require more polygons for the objects and this is very important for the rendering process - the fewer polygons the faster is rendering and viewing the final 3D result [3].

3D Transformation of 2D Captured Museum Objects at Risk

269

Fig. 1. Source 2D images (first raw: Charter 1912, established by the Decree of October 31, 1913 after the victorious end of the war between Serbia and Turkey in 1912, Odin; second raw: Medal for Military Virtues, established after the suppression of the Timoˇcka Rebellion in 1883, by the decree of King Milan Obrenovi´c of December 21, 1883).

Generating a normal map from the object’s picture is a task which may have many solutions. The most common solution is using the Sobel algorithm (or Scharr filter) which aims to detect the edges of the object and based on that to create a normal map after that. A relatively new approach is the usage of neural network models which takes the picture of the object as input and generates a normal map as an output (Fig. 4). We have used a pre-trained model to show the result of such normal map generation [4].

270

M. Goynov et al.

Fig. 2. A simple 3D model, created from the given 2D picture.

Fig. 3. UV map - polygons mapped to image

Figure 5 shows the final rendering with applied normal maps and using advanced lighting (a spotlight is added in front of the objects).

3D Transformation of 2D Captured Museum Objects at Risk

271

Fig. 4. A normal map (left side is Sobel algorithm generated, right sight is AI generated).

2.2 Used Technologies The tools we used for the creation of 3D objects from 2D images include open source software - Inkscape - in order to trace the image contours, Blender - for all of the 3D modelling. The following GitHub projects were used in order to create the normal maps: • Normal generator using Sobel or Scharr algorithms [5]. • A normal map generator based on pre-trained AI models [4]. For the web presentation of the final results, the following technologies were used: • THREE.JS - a powerful javascript library for rendering 3D environments on the web [6]. • glTF - a lightful format for storing three-dimensional data using JSON and binary data [7].

272

M. Goynov et al.

Fig. 5. 3D objects final rendering.

3 Virtual Demonstration of Created 3D Museum Objects 3.1 In Holographic Pyramid Presenting 3D objects using holographic pyramids (also known as fake holograms) is an impressive way of visualization of artificially created objects in real space. The objects are projected, in fact, on a display (phone, tablet or another horizontally positioned screen) and the pyramid, which is built from tinted glass materials, reflects the projection [8]. Figure 6, Fig. 7 and [9] present screenshots and demo of the final results. The object can be observed from all 4 sides of the pyramid thus making this approach suitable for spaces where viewers are all around. 3.2 In Inclined Plane Projection System The created 3D museum objects could be presented also in the inclined plane projection system, which is specifically designed technical solution for the virtual storytelling for museum objects during exhibits [10]. This system consists of two synchronized video presentations that are displayed one above the other on separate projection surfaces

3D Transformation of 2D Captured Museum Objects at Risk

273

Fig. 6. A tablet presenting the four sides of an object.

Fig. 7. A holographic pyramid placed on the tablet.

[11]. The upper projection surface is an inclined plane realized as a see-through display designed for the presentation of 3D museum objects. The projection is made on the semitransparent glass to achieve the simulation of a holographic effect similar to a holographic pyramid. The bottom surface is the horizontal plane with special projection foil used for video presentation projected from a projector hidden in the box of the system. This video presentation is used to provide additional and more detailed information about 3D museum objects projected on an inclined plane. Currently, the solution is used in the Digital Museum located in Fortress of Niš to display stories and digital reconstructions of objects from different historical periods of the city of Niš. This relates to the presentations

274

M. Goynov et al.

such as the archaeological site Mediana, Serbian houses that existed in the Ottoman period and details about Niš as a war capital. The presented medals from Serbian military history will enrich the current digital collections. 3.3 In Universal Electronic Museum Guide Another platform which is suitable for the presentation of the created 3D objects is the universal electronic museum guide, a mobile application that could be adapted for various museum exhibitions [12]. This electronic museum guide is realized as a more advanced solution compared to classical museum audio guides. The developed modules can display various multimedia formats such as text, image, audio, video or 3D objects. This gives visitors the choice of familiarize themselves with the information about the display in the museum in their preferred format. The engaging feature of this guide is achieved with the integration of VR and AR technologies where 3D museum objects can be visualized in an appealing way. 3.4 Possible Usage of the 3D Created Museum Objects for Educational Purposes The created 3D museum objects could be used for educational purposes, especially in supporting VR/AR presentations of the reconstructed objects in serious educational games or other interactive learning materials. More precisely, students can examine the symbols in the above medals given for bravery, dedication and proven service to the king and the fatherland in interactive educational resource for historic events from the Serbian history. Similarly, as the national symbols (coat-of-arms and flag) the medals hold their symbolism, which is rarely recognized and understood by the current generation. In that way this precious knowledge about the national memory can be passed on in understandable and easy way to the students. Some of the presented approaches and some test objects, mainly coins, weapons of war (swords, sabers, etc.) were experimentally embedded in our current development, the Aquae Calidae serious game [13, 14]. Students can explore the multilayered archeological excavations in the town of Aquae Calidae, situated in the Burgas mineral baths region in Bulgaria. Through immersing in the 3D reality of the ancient complex, and playing intuitive educational mini-games, students improve their knowledge and understanding of the ancient civilizations on the Balkan peninsula. The player, for example, visits a hall with artefacts of Thracian treasures, and, in gaming mode, explores 3D transformed objects in detail and perceives crucial aspects of the Thracian culture and civilization [15].

4 Conclusions and Future Work In this paper a methodology for 2D to 3D object transformation, based on 2D captured museum assets is proposed. It represents a symbiosis between traditional methods and algorithms for 3D modelling and advanced technologies based on artificial intelligence. Possible applications of the reconstructed 3D objects could be the representation of ancient artifacts, treasures and valuables in a more realistic way, enlarged, voluminous,

3D Transformation of 2D Captured Museum Objects at Risk

275

and in all around view. Moreover, in order to present more complete content, complex scenes can be created using combinations of real and artificial 3D objects, specific animations and transitions. As a next step, we plan to optimize and improve the offered framework towards maximum use of generative AI for creative co-creation of immersive and authentic 3D applications. Acknowledgements. This research work was carried out and is supported partly by the joint research project “Development of Software Tools and Multimedia Technologies for Digital Presentation, Preservation and Management of Cultural Heritage” between the Institute of Mathematics and Informatics, Bulgarian Academy of Sciences and the Mathematical Institute of the Serbian Academy of Sciences and Arts (2023–2025) and CLaDA-BG, the Bulgarian National Interdisciplinary Research e-Infrastructure for Resources and Technologies in favor of the Bulgarian Language and Cultural Heritage, part of the EU infrastructures CLARIN and DARIAH, Grant number DO01–167/28.07.2022, https://clada-bg.eu/bg/. It is also supported by the program “Increasing Research Capacity in the Field of Mathematical Sciences” (PICOM, Agreement DO1–67/05.05.2022), financed by the Bulgarian Ministry of Education and Science, according to Decree of the Council of Ministers No. 54/15.04.2022.

References 1. Teng, J., Wang, F., Liu, Y.: An efficient algorithm for raster-to-vector data conversion. Geogr. Inf. Sci. 14(1), 54–62 (2008). https://doi.org/10.1080/10824000809480639 2. Baechler, O., Greer, X.: Blender 3D By Example: A project-based guide to learning the latest Blender 3D, EEVEE rendering engine, and Grease Pencil, 2nd Edition, Packt Publishing (2020) 3. Preppernau, C. Normalizing the Normal Map. Cartographic Perspectives 96, 61–74. https:// doi.org/10.14714/CP96.1669 4. AI Material Map Generator. https://github.com/joeyballentine/Material-Map-Generator. Accessed 14 July 2023 5. Sobel/Scharr Normal Map Generator. https://cpetry.github.io/NormalMap-Online/. Accessed 12 July 2023 6. Danchilla, B.: Three.js framework. In: Beginning WebGL for HTML5. Apress, Berkeley, CA, pp 173–203 (2012). https://doi.org/10.1007/978-1-4302-3997-0_7 7. Thomas, E., Potetsianakis, E., Stockhammer, T., Bouazizi, I., Champel, M.L.: MPEG media enablers for richer XR experiences. n. pag. arXiv abs/2010.04645 (2020) 8. Yamanouchi, T., Maki, N., Yanaka, K.: Holographic pyramid using integral photography. In: Proceedings of the 2nd World Congress on Electrical Engineering and Computer Systems and Science (EECSS’16), Budapest, Hungary – August 16–17, 2016, Paper No. MHCI 109, pp. 1–4 (2016). https://doi.org/10.11159/mhci16.109 9. Demonstration of the results. https://demo.bg73.net/3d/museum.html?room=a7/. Accessed 15 July 2023 10. Radmanovi´c, M., Tati´c, D., Gaji´c, D. One solution for building reconfigurable multiprojection systems using the Adobe AIR platform. In: Milovanovi´c, B. D. (Ed.) Proceedings of Papers of the 49th International Scientific Conference on Information, Communication and Energy Systems and Technologies 2014 - ICEST 2014, pp. 121–124 (2014) 11. Tati´c, D., Stankovi´c, R. S., Stojanovi´c, J., Jovanovi´c, M. Application of Information Technologies in Presentation of Historical Heritage of Niš. In: Proceedings of Conference Digitalization of Cultural Heritage in Niš Region, pp. 47–52 (2019)

276

M. Goynov et al.

12. Tati´c, D., Stankovi´c, R. S., Stojanovi´c, J., Jovanovi´c, M. Universal electronic guide for museum exhibitions. In: Proceedings of Conference Digitalization of Cultural Heritage in Niš Region, pp. 54–62 (2019) 13. Aquae Calidae serious game. https://demo.bg73.net/3d/aquae-calidae.html/. Accessed 15 July 2023 14. Aquae Calidae serious game (video presentation). https://demo.bg73.net/3d/aquae-calidaeshort.mp4/. Accessed 15 July 2023 15. Paneva-Marinova, D., Goynov, M., Pavlova, L., Zlatkov, L., Luchev, D.: Studying the Ancient Civilizations on the Balkan Peninsula through Serious Game and Storytelling. In: Auer, M.E., Tsiatsos, T. (eds.) New Realities, Mobile Systems and Applications: Proceedings of the 14th IMCL Conference, pp. 537–546. Springer International Publishing, Cham (2022). https://doi. org/10.1007/978-3-030-96296-8_48

From Data Abundance to Informed Citizenship: The Empowering Potential of the Dali Life Game for Data Literacy Petros Lameras(B) , Sylvester Arnab, and Mark Lewis Coventry University, Coventry, UK {ab3430,aa8110,ac6055}@coventry.ac.uk

Abstract. In a modern society characterised by an exponential proliferation of data, the concept of data literacy has risen as an essential construct for becoming an active citizen. This development has been catalysed by the concerted efforts of scholars, policymakers, and technological visionaries aimed at equipping individuals with the essential skills to navigate the complexities and opportunities embedded within the data overflow. This paper presents and evaluates the Dali Life game as a multimodal digital tool designed to empower citizens in comprehending, mitigating, and evaluating the spectrum of risks and challenges accompanying their daily data interactions. Employing a mixed methods approach involving a cohort of adult citizens (n = 37), this study aims to understand how the game is used and what data literacy aspects are important to be nurtured for becoming a data literate citizen. Two overarching categories were prevalent: (1) understanding data literacy and (2) engaging through data. Both dimensions reinvigorate the procedural rhetoric of data literacy via encouraging players to create their own cognitive structures and skills as means to foster critical practice for tackling real world problems. The research findings offer insights to technology experts and data literacy advocates guiding the formulation of data literacy interventions with targeted focus on connecting data literacy to concepts such as games and play, citizenship, empowerment, and social innovation thus accentuating its role as a pragmatic tool for effectively resolving everyday challenges. Keywords: Data literacy · games · skills · data · learning · citizens

1 Introduction Increased availability of data in societies fuels the need to develop data literacy skills for making decisions that are data-driven, accurate and inimitable. The ability to find, understand and make meaningful inferences of data is a useful skillset that citizens need to be equipped for actively participating in modern societies. The overarching question that this paper raises is “How do citizens become data literate through the use of a game?” The general view is that different resources, applications, and tools may be deployed for understanding data literacy such as digital information found in podcasts, blogs, mini games and through informal events such as conversations unfolded © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 277–286, 2024. https://doi.org/10.1007/978-3-031-56075-0_26

278

P. Lameras et al.

in online learning communities or via workshops where participants have the opportunity to experience the theoretical tenets of data literacy but also to apply what it has been learned. There is a predominant belief that a data literate society is constructed from the learning experiences and foundational knowledge attained during formal education that are contextualised and ultimately lead to transferable skills (e.g., [1]). Such transferable skills may enable citizens to identify disinformation content and other uses and abuses to which data can be put. The paper aims to investigate the use of the Dali Life board game as a scaffold for helping citizens to create an awareness of data literacy and how it may be employed in real world situations. The paper starts by situating the study in the context of data literacy in terms of its meanings and discourses and how it can be developed through analogue and digital games. It then presents the Dali Life game and its features before elucidating on the method and findings. A discussion then follows around data literacy, skills, limitations, and future research.

2 Background In this section, we set the stage around research on data literacy with focus on meanings, practices, and discourses. Specifically, studies that demonstrate how data literacy is enacted, cultivated, and nurtured from an informal and formal learning perspective. Investigations on research studies that provide meaningful insights on how games, both analogue and digital, are utilised for creating an awareness on data literacy are highlighted. 2.1 Data Literacy as the Foundation for Data Literate Citizens Data literacy as a term has been used to denote a set of skills and capabilities around the use of data for solving problems and challenges that citizens are facing as part of their everyday life [1]. Therefore, data literacy is a skill for life, as interactions with data are occurring regularly from simple tasks such as creating social media accounts to more complex and ill-defined deliberations spanning from data collection to analysis and interpretation. A key assumption that defines data literacy as a skill for life is because with such skills citizens may tackle risks and harms, social, personal, and financial that are likely to increase their ability to act as proactive citizens in a datafied society. [2] explored how the field of data literacy should address harms and risks in terms of what skills, cognitive processes and actions need to be in place for preventing malicious acts like dis-, mis- and mal- information. Investigations are carried out from the lenses of data citizenship as a substrate of data literacy that moves away from individual literacies to networked literacies encompassing learning traits for critical thinking about the online ecosystem and making informed judgments of how data-driven decisions may influence not only individuals but also communities for promoting social justice and reducing social, political, and economic inequalities. As such, a data-literate citizen should be able to critically assess data and their sources and determining data misuse for alleviating dis, mis and mal- information [3].

From Data Abundance to Informed Citizenship: The Empowering Potential

279

Wolff (2016) [1] identified a set of data literacy competencies that are categorised in five themes forming an iterative inquiry cycle: Problem, Plan, Data, Analysis and Conclusions constituting the PPDAC as an inquiry process, like other inquiry models (e.g.,[4]) that represent a developmental progression of a process, skill, or capability. The inquiry process involved the planning, implementation and monitoring courses of action and the undertaking of data inquiry processes. The foundational data literacy competencies involved the: ethical use of data, use of data to solve real problems, role, and impact of data in society, identification of questions and problems that can be solved with data, hypothesis and data identification, data collection, data analysis, data interpretation and evaluation of data-driven decisions. [3] have taken a storytelling approach to offering a model for developing data literacy skills. The OCEI.AI paradigm integrates storytelling for helping students to learn data science and develop distinct data literacy skills through five story-based steps: (1) A life story for guiding students to identify a problem or a challenge; (2) A data story for linking data with real world problems; (3) a model story for selecting, justifying and conducting experimentations; (4) A user story for visualising data for helping users to consolidate and assimilate data in practical ways; and (5) a societal story for evaluating the ethical, social, and cultural implications. The model incorporates a set of open-ended questions that help students to probe more on the ‘who’, ‘what’, ‘when’ ‘where’ ‘why’ and ‘how’ to use data associated to specific target users and contexts. 2.2 Games as a Medium for Data Literacy Data literacy encompasses the ability to collect, find, analyse, and make informed decisions about how data will be used for solving problems. It is an inquiry process that can be integrated into resources, applications, and tools for helping citizens be data literate beyond focusing on the purely technical and linear-centric propagation to connecting broader concepts such as ownership, empowerment, and citizenship. [5] coined the term ‘creative data literacy’ to draw attention to data literacy skills that are not predominantly focused on technical knowledge. The theory and practice of literacy, as a process, may be propagated by introducing rich multimodal tools such as games for empowering citizens to develop a foundation about what data literacy means to them whilst contributing to having a plethora of multimedia applications and tools deployed by citizens for learning data literacy. Play arguably manifests itself in games and speculatively games may provide a form or structure for citizens to demarcate the foundations of data literacy and lead to an escalating logic of helping citizens to use associated knowledge and skills for undertaking challenges encountered in their everyday endeavours. To demonstrate how analogue and digital games have been designed and used for creating awareness, knowledge and skills on data literacy, a series of studies are presented. [6] developed an analogue card game prototype, as an epistemic object, on the political economy of app publishing which generated discussions not about data per se but in terms of how a data-oriented process help citizens to accomplish certain tasks (e.g., how app developers plan an app launch as means to exceed competition?), thereby placing emphasis on the ‘freedom to act’ that is achieved through the literacy process. [7] created a web-based game for teaching data literacy, particularly for teachers to aid students on how to use data, charts, and insights for triggering decisions. The main game mechanic

280

P. Lameras et al.

is a slide deck containing data about a given scenario and the goal is for students to complete the slide deck by making decisions that prioritise data over subjective inferences. [8] designed and assessed a gamified application to teach data literacy to university students. Findings suggested that gamified applications improved learning outcomes related to data-literacy especially when incorporating interactive charts and tools, with real datasets that address societal challenges. [9] showed how ‘StoryLet’ a game where biology students have the role of junior editors tasked to choose appropriate graphics and titles for different stories. Results showed positive correlations between choices and data-literacy related learning outcomes and that the in-game data visualisations led students to make data-informed choices.

3 The Dali Life Game Dali Life an analogue game that aims to inform and generate discussion around data literacy and the dangers associated with data usage in daily life. The game utilises a board (see Fig. 1) which the players move around and two sets of cards (see Fig. 1) which provide information relating to data literacy concepts, allowing a facilitator (e.g., educator, librarian, trainer, parent, etc.) to begin conversations around them. For example, one card discusses the use of public Wi-Fi, which leaves devices vulnerable to hackers. Variations of each card can provide positive and negative outcomes. In a positive outcome, the player is rewarded with advancement on the board, in a negative outcome, the player is inhibited or sent backwards. This reflects how personal use of data can affect our daily life in both positive and negative ways. Dali Life is a multiplayer game for 2–5 players. The first player that reaches the circle that marked ‘finish’ wins the game. Play time can range between 30 min to an hour depending on how the game is played. The first player rolls the dice and moves their counter the number of places shown on the dice and then the player may decide which route to take. If a player lands on a space marked with an ‘Event’ icon, the player must draw an ‘Event’ card (blue) and read its contents aloud. Event cards can have either a negative or positive effect on the player or their opponents and can only be blocked by specific action cards. Once an ‘Event’ card has been used, it is placed on the discard pile. When any event cards have played out, the player may play up to three ‘Action’ cards (black). Players can be targeted by ‘Action’ cards and can counter their effects by playing their own action cards. Action cards which the player draws, holds, and can utilise at specific points within the game. Once an ‘action’ card is used, it is placed in the discard pile. Once the action phase is complete, the player ends their turn, the sequence begins again, with the next player. The learning objective is for the players to demonstrate understanding and knowledge of the in-game data-literacy aspects within the game’s card’s contents. Players should discern awareness on both positive and negative aspects of data literacy and how to better protect themselves against the challenges and implications of data misuse such as accessing personal data, or through recognising fraudulent situations where mis- dis- or mal- information is provided to them.

From Data Abundance to Informed Citizenship: The Empowering Potential

281

Fig. 1. Dali Life board, action, and event cards

4 Method We conducted a usability study for evaluating the efficacy of Dali Life as a multimodal and semiotic analogue board game for helping citizens to enhance data literacy skills. We have organised and executed twelve (12) inter-generational field trials at Coventry in the UK, with the assistance of the Coventry City Council and City library, for delineating how the game was used for learning data literacy. By adopting an inter-generational approach, we aimed at understanding data-literacy through Dali Life between family members and between non-family members as to gauge a more open and varied perspective of the phenomenon investigated. We have trained six (6) facilitators in total from Coventry City Council and Coventry City library as to how to play the game for supporting gameplay with participants and facilitating discussions during the trials. Thirty-seven (n = 37) adult citizens (30–64 years old) from the city of Coventry participated in the field trials and were supported by the facilitators in playing Dali Life, and at the end of the trials they completed an online survey as means to delimit game usability aspects, beliefs, and actions on how Dali Life was used for enhancing data literacy. The survey was designed and delivered via the JISC online surveys tool and was an assortment of closed (e.g., Likert scale) and open questions (e.g., text-response). Examples of closed questions included previous experiences of playing games (e.g., when was your last time you played games?), ability to act (e.g. the game gives me confidence to act), easiness, visual, aesthetics (e.g. the game is easy to play, visuals and text are clear) and open questions such as experiences of learning through the game (e.g. what do you feel you have learned in the game?). We have employed a mixed-study approach for analysing quantitative and qualitative data. MS Excel was used for inserting, organising, and filtering data, removing duplicates, calculating question averages, and creating charts. We have used content analysis for creating codings using MS Excel, as means to discern meaning of the qualitative data. The research study has received ethical approval from Coventry University through carrying out an ethical application where challenges have been raised,

282

P. Lameras et al.

participant information sheets and consent forms were validated, and methods of data collection, storage, access, and analysis were detailed. 4.1 Results We attempted to encompass participants with varied previous experiences, game routines and future intentions of playing games. Participants had experience mostly in analogue games (73%), which correlate with the predominant analogue nature of the Dali Game (see Fig. 2: top-left chart). Since citizens were not presumed as avid gamers, they had only occasionally played games (48.6%) with some often-playing games (24.3%). Some participants used to play games (10.8%) and (16.2%) never played games (see Fig. 2: top-right chart). The last time participants played games were last week (37.8%) which meant that there was a general understanding of the mechanics and dynamics and logic inherent to the value of game-play. More than a year was also reported with (35.1%) and (16.2%) never played games, meaning that some individuals needed some time during the trials to familiarise with Dali Life as a medium for play and learning (see Fig. 2: bottom left chart). There was motive and intention to play games in the future, with (67.6%) being positive to play games in the long-term (see Fig. 2: bottom-right chart).

Fig. 2. Top-left chart: Experience in games. Top-right chart: Game routines. Bottom-left chart: last time played games. Bottom-right chart: Intention to play games in the future.

It was evident from the data that the game enabled participants to make decisions (81%) - (see Fig. 3, In.1) especially with regards to how the knowledge gained could be used for preventing personal data misuse especially when interacting with social media websites. This, in turn, seemed to increase emotional agency (>70%) related to wonder, delight, excitement and / or surprise (see Fig. 3, In.2). Confidence to act (78%) was seen as twofold: Both in terms of acting during game-play to challenge and respond to attacks and counter-attacks from other players and in terms of becoming more confident to act on refining understanding and skills on data literacy (see Fig. 3, In.3). Dali Life is premised on encouraging dialogue on what data literacy means and engaging into conversations and as such there was strong agreement that the game encouraged peer-support in terms of asking questions, co-learn and collaborate with the trainers and the players (see Fig. 3, In.4). Participants felt that the visuals and textual elements of Dali-Life are clear (63%), (see Fig. 4, In. 5) especially the visuals of the board and the text to describe the dataliteracy aspects represented in the cards (73%) (see Fig. 4, In.6). There were some minor visual aspects that participants noticed in terms of some visuals (e.g., arrows on the

From Data Abundance to Informed Citizenship: The Empowering Potential

283

Fig. 3. Agency enactment for decisions (In.1), emotional reactions (In.2), confidence (In.3) and peer-support (In.4)

board) that needed refinements as for having a clearer path for counters to be moved to the next box, Overall, Dali Life was easy to understand in terms of its aims, rules, and winning conditions (79%) (see Fig. 4, In.7).

Fig. 4. Visual elements (In.5), text elements (In.6) and game usage (In.7)

Dali Life’s content consisted predominantly of the prompts included in the ‘event’ and ‘action’ cards that triggered player understandings on what data literacy means to them. For example, an ‘event’ card included a prompt “To ensure sellers are genuine, check online reviews and only use sites with the secure https: prefix”. A simple suggestion such as this was perceived as relevant to a real problem or situation that participants found themselves in at some point (75%) (see Fig. 5, In.8). The time necessary to finish the game was proportional to how game-play was facilitated by the trainers. If for example, the trainers along with the participants decided to roll a single pair of dice, then game-play would be slow, whereas if two pair of dice were tossed, then pace would be distinctively accelerated. To this end, (71%) perceived that time dynamics were balanced with an appropriate flow to accumulate and consolidate learning but also to stay engaged and motivated until the end of the game (see Fig. 5, In.9). It is likely to be in congruence to engender content with the fun aspect of the game. It seems there was agreement that the game was fun to play (80%) (See Fig. 6, Int.10) and that also the game was a scaffolding medium to learn data literacy while playing (see Fig. 5, In.11). This combination between fun and learning was prevalent amongst

284

P. Lameras et al.

Fig. 5. Game content relevancy (In.8) and in-game time dynamics (In.9)

participants both in terms of developing cognitive schemata (knowledge) but also in terms of testing and improve cognitive understandings through learning relational perspectives (e.g., data shared between applications may lead to software vulnerabilities exploited by hackers) with (78%) perceived that the game helped them to assess their learning (see Fig. 5, In.12).

Fig. 6. Fun (In.10), learning (In.11) and learning assessment (In.12)

To shed light on how learning has been experienced through Dali Life, we have qualitatively investigated how in-game data literacy was perceived by the participating citizens. Two overarching categories have been revealed: (1) understanding data and (2) engaging through data (see Table 1). We observed that participants did not provide linear understandings of what data literacy is, but rather they provided statements linked to actual applications of data literacy in their everyday interactions with data. For example, data as a term was linked with privacy and techniques or measures for achieving privacy. Data awareness was perceived as an aspect of data literacy comprising a spectrum of skills associated with it. The value of data literacy was connected to being able to assess and evaluate knowledge. Sharing data was associated with applying data protection policies and privacy strategies that may inhibit malicious and harmful actions.

From Data Abundance to Informed Citizenship: The Empowering Potential

285

Table 1. Citizens’ experiences of data literacy through Dali Life Category

Sub-category

Quotation(s)

Frequency

Understanding data literacy

Meaning of data

“It is about privacy, like having a VPN”

25

Data awareness

“It is being data literate”

21

Data literacy value

“Assessing and validating previous knowledge on data literacy”

7

Sharing data

“Knowing GDPR and privacy features when I am online”

14

Making decisions

“I have used a VPN and a password manager for blocking malicious content”

6

Knowledge application

“Applied my knowledge gained from the game to actual challenges I experienced when entering passwords online”

4

Critical thinking

“I am more careful when sharing data like having more complex passwords and not entering my pet’s name”

2

Engaging through data

5 Discussion and Conclusions We understand that the two categories may not signify a holistic representation of data literacy discourse, but they do provide a proffered contingency that data literacy adheres to forging awareness, values, and data sharing. Engaging through data literacy is characterised as a ‘transition from theory to practice’ with focus on the ‘praxis’ of establishing a coalition between the abstract nature of theoretical underpinnings and the realistic and tangible reality for tackling challenges encountered in real-life settings. The design of Dali Life was based on the principle that data literacy interventions do not need to be limited to technical content. Dali Life is built around the concept of creative data literacy aligned with habituating strategies and playing styles that activate different forms of cognitive engagement. We appreciate the limitations of the study, especially in relation to implementing more profound and thorough methodology aspects that would demarcate deeper understandings of how data literacy was perceived by the citizens. The study will be extended for providing a comprehensive analysis on citizens’ experiences of data literacy using games.

286

P. Lameras et al.

Acknowledgements. The DaLi project received funding from the European Commission, Erasmus + with project number 2020-1-N001-KA204-076492.

References 1. Wolff, A., Gooch, D., Cavero, J.J., Montaner, U.R., Kortuem, G.: Creating an understanding of data literacy for a data-driven society. J. Commun. Inform. 12(3) (2016). https://doi.org/10. 15353/joci.v12i3.3275 2. Carmi, E., Yates, S.J., Lockley, E., Pawluczuk, A.: Data citizenship: Rethinking data literacy in the age of disinformation, misinformation, and malinformation. Internet Policy Rev. 9(2), 1–22 (2020). https://doi.org/10.14763/2020.2.1481 3. Li, Y., Wang, Y., Lee, Y., Chen, H., Petri, A., Cha, T.: Teaching data science through storytelling: improving undergraduate data literacy. Think. Skills Creat. 48, 101311 (2023). https://doi.org/ 10.1016/j.tsc.2023.101311 4. Lameras, P., Arnab, S., de Freitas, S., Petridis, P., Dunwell, I.: Science teachers’ experiences of inquiry-based learning through a serious game: a phenomenographic perspective. Smart Learn. Environ. 8(1) (2021). https://doi.org/10.1186/s40561-021-00152-z 5. D’Ignazio, C.: Creative data literacy: Bridging the gap between the data-haves and data-have nots. Inform. Design J. 23(1), 6–18 (2017). https://doi.org/10.1075/idj.23.1.03dig 6. Werning, S.: Making data playable: a game co-creation method to promote creative data literacy. J. Media Literacy Educ. 12(3), 88–101 (2020) 7. Pandeliev, V., Namanloo, A., Lyons, K., Bliemel, M., Ali-Hassan, H.: A serious game for teaching data literacy. In: 2022 IEEE Games, Entertainment, Media Conference (GEM), pp. 1–6 (2022). https://doi.org/10.1109/GEM56474.2022.10017613 8. Legaki, Z., Fernandez Galeote, D., Hamari, J.: The Impact of different gamification types in the context of data literacy: An online experiment. In: Bujic, M., Koivisto, J., Hamari, J. (eds.) Proceedings of the 6th International GamiFIN Conference 2022 (GamiFIN 2022). CEUR Workship Proceedings, vol. 3147, pp. 22–32 (2022). https://urn.fi/URN:NBN:fi:tuni-202207 045964 9. Chin, D.B., Blair, K.P., Schwartz, D.L.: Got Game? a choice-based learning assessment of data literacy and visualization skills. Tech. Know Learn. 21(2), 195–210 (2016). https://doi.org/10. 1007/s10758-016-9279-7

Rollick Games: A Formal Language Based Platform for Location-Based Pervasive Games George Anestis1 , Nektarios Gioldasis1 , Stefanos Karasavvidis2 , Tzanis Palioudakis1 , Nektarios Moumoutzis1(B) , and Stavros Christodoulakis1 1 Laboratory of Distributed Multimedia Information Systems and Applications, School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece {ganestis,ngioldasis,nmoumoutzis,schristodoulakis}@tuc.gr, [email protected] 2 Software Technology and Network Applications Laboratory (SoftNet), School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece [email protected]

Abstract. In this paper, we present the Rollick Games platform and its Pervasive Game Modeling Language that enables the description of location-based pervasive games using a GUI provided by the Game Studio web app. The runtime environment (Pervasive Game Execution Environment) is responsible for generating the Pervasive Game Execution Language from a game model and loading it to an instance of the game’s Execution Engine. The Execution Environment includes a set of agents responsible for monitoring the players’ context and interaction with other game elements and among each other. The Engine follows an event-driven paradigm and uses an event loop waiting for events to be dispatched from the agents to carry out its computations based on its current state, context, and event data. The proposed approach allows for both sharing and reusing game models, significantly reducing time to market and development costs. Keywords: pervasive mobile games · location-based games · game development

1 Introduction Pervasive Games (PGs) can use real-world locations for gameplay and support the projection of a virtual world onto the real one, creating the feeling that real and virtual objects coexist in space and time, a mixed-reality environment, allowing the player to interact with the objects and the real-world locations. This way, contextualized experiences can be offered based on direct or indirect recognition of spatial context [1–4]. The involved technologies include the player’s mobile devices, wireless communications, and sensors that capture the player’s current context (GPS, beacons, camera, compass, accelerometer, etc.) [1]. A PG can be defined as a context-aware game system with the above-mentioned characteristics [5]. Creating a PG involves professionals such as game designers, artists, graphic designers, and developers. It is a costly and time-consuming process with two phases: preproduction and production. During pre-production, most of the “creative” work of the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 287–296, 2024. https://doi.org/10.1007/978-3-031-56075-0_27

288

G. Anestis et al.

game takes place and the main actors are the game designers and the artists. The game designers work mainly on defining the script and plot (storyline), virtual characters and their behavior, and map elements and interactions with locations and objects. The artists create graphic elements, animations, sound effects and take care of the soundtrack. During production, the developers deliver the game as software. The collaboration in such a multidisciplinary process is quite complicated [6–10]. The intrinsic characteristics of pervasive mobile games motivate the development of appropriate platforms [5, 11–19]. These platforms aim to support the pre-production phase while making the second phase entirely transparent for the game’s creators through automatic processes, thus minimizing implementation time and cost. This paper proposes a formal, platform-independent, Pervasive Game Modeling Language (PGML). This domain-specific language provides the modeling primitives for the design of locationbased pervasive games. The language allows for the definition of a rich set of game mechanics adaptable to the game plot in different domains, i.e., education, cultural heritage, tourism, entertainment, etc. The Game Studio, a Graphical User Interface (GUI) tool based on PGML, facilitates the game design. It standardizes primitive design elements and the core functionality of interoperable authoring and design tools. PGML allows for the sharing and reusing of successful and broadly accepted game models to the broader community. Those game models could be created by domain experts and game designers and freely offered to the broader creative community practitioners for further adaptation, customization, and reuse [20]. The concept of a Game Definition Language has been used in General Game Playing [21–24]. To our knowledge, no previous work is related to defining a formal language for pervasive mobile games. The support of the intrinsic features of pervasive games using such a modeling language can benefit the creation of pervasive game applications. The PGML language is at the core of the Rollick Games Platform [25]. The platform supports multiplayer gameplay via the Pervasive Game eXecution Environment (PGXE), a complete gameplay infrastructure, which compiles a platform-independent game specification, expressed in PGML, to an intermediate representation language that is fed to the Pervasive Game Engine for execution. A set of environment-sensing agents senses the player’s context and dispatches events to the engine during the execution of the game. A player app for Android and iOS is also provided. The following sections describe the Pervasive Game Modeling Language, a platformindependent, domain-specific modeling language (Sect. 2). Section 3 presents the Pervasive Game Execution Environment, a complete gameplay infrastructure for playing the games described with PGML. Section 4 describes platform implementation details. Section 5 offers related work and compares it with our work. Section 6 concludes and reports on the preliminary empirical evaluation done.

2 PGML: Pervasive Game Modeling Language The Pervasive Game Modeling Language is a platform-independent, domain-specific modeling language tailored for game designers. It allows for the representation of various pervasive mobile games and can be compiled to generate executable instances of these games. The language consists of several key constructs and relationships that define the game world, entities, interactions, mechanics, and plot. In particular:

Rollick Games: A Formal Language Based Platform for Location

289

The Game’s World: The game’s world combines elements from the real world and a fictional/virtual world. It comprises two components: Design Time and Play Time. Design Time describes the world’s construction, including space, time, living beings, culture, laws, objects, history, and myths. Play Time defines gameplay interactions and their semantics. It is described in the next section. World’s Territory: It is a spatial representation on a real-world map. It can be defined by real-world area names, geo-coordinates, or both. Scarce resources exist in finite amounts within the world. Laws govern the rules and activities related to resources and living beings. Scenes: The world territory is composed of scenes (defined areas on a map) representing real-world areas where most of the game action occurs. Scenes also allow for a better structuring of game plots, providing a scope for event handling and can form a hierarchy with non-overlapping sub-scenes. Game Objects: Each game can have various objects with data and behavior. Data represents object characteristics, ID, name, creator ID, icon, etc. Designers can also define additional custom properties for objects. Object Behavior: Object behavior defines how players interact with it and the effects on the player, the world, and itself. Behavioral Interfaces include collectable, redeemable, droppable, destroyable, exchangeable, solvable, etc. Designers can also define custom object behavior. Real-World Objects (RWOs): Existing real-world objects can be included in the game’s world. RWOs receive digital surrogates for representation in the game’s augmented world. Virtual Objects (VOs): VOs are fictional objects defined by designers. They can be hidden or activated based on proximity criteria. Game Object Instances: Instances can be created automatically or by players according to defined object creation policies, and once put in the game scenes, players can interact with these instances. Missions and Progression: Player progress can be structured in missions that may have sub-missions. Mission accomplishment conditions are based on player and/or world state. Accomplished missions yield rewards (game currency, skills, items, etc.). Players and Avatars: Players interact with the game’s world to accomplish missions and goals. Avatars represent players in the virtual world. Marketplace: The game may have a marketplace where players can buy scarce resources (skills and object instances). In-game trading enhances player engagement through collaboration and/or competition. Non-Player Characters (NPCs): Designers can create NPCs with names and visual representations. NPCs can be engaged in conversations with players by expressing statements to which players respond through options, which may affect the game state.

290

G. Anestis et al.

Events and Conditionals: Events are captured and generated during gameplay, and their handling drives the gameplay. Conditionals are logical expressions about player and world state evaluating to true or false and can be used to drive or finetune gameplay. A UML class diagram of the above modeling constructs is shown in Fig. 1. The formal description of the language grammar and a parser for it using Jison (http://zaa. ch/jison) can be found at: https://bit.ly/3lBOFr9.

Fig. 1. The visual representation of PGML elements using UML class diagram.

3 PGXE: Pervasive Game Execution Environment The Pervasive Game eXecution Environment (PGXE) enables gameplay, employing an event-driven architecture. Handling of events triggered by players or the game itself leads to changes in the player’s and game world’s states. PGXE employs several agents to sense, emit, and process events. The Location Tracking Agent (LTA) provides locationbased event intelligence, playing a crucial role in unfolding the game’s plot. The Scan Tracking Agent (STA) is Responsible for recognizing QR code scans. The Object Behavioral Agent (OBA) identifies player interactions with game objects. The World Watcher Agent (WWA) monitors the state of the game’s world and the player. These agents capture events, process raw data if needed, create higher-level events, and dispatch them to the game engine. This modular approach allows for easy integration of new agents into the platform through a well-defined and compact interface. For each event type, a corresponding event handler implements the game logic and manages side effects. The platform provides skeleton implementations for these handlers, and designers can configure them through a GUI to align with the game’s requirements. Events may have explicit connections with scenes when players enter or leave them. Events may be implicitly linked to scenes through game objects positioned within those scenes. When an object is placed in or removed from a scene, the corresponding event

Rollick Games: A Formal Language Based Platform for Location

291

handler is attached or detached, allowing for the management of events triggered by player interactions with objects. Events can also be implicitly associated with the player’s state via game objects collected and stored in their inventory. For instance, players can redeem collectibles to increase their in-game currency balance or exchange them for other collectibles or skills. The Pervasive Game Execution Language (PGEL) serves as an intermediate representation language for execution by the Pervasive Game Engine (Engine). It allows game models expressed in PGML to be compiled into an executable format suitable for the Engine. PGEL is currently implemented using JavaScript but can be adapted to other languages. The transpiler parses game models described in PGML and generates an executable game configuration in PGEL, including the attachment of event handlers to scenes and game objects. The Engine operates as an abstract construct comprising a finite set of scenes. At any given time, a player is within a given scene, and the engine responds only to events of interest to this scene. It continuously waits for events to occur and interprets them based on the current scene, the event itself, and any accompanying parameters or conditionals. Events that cannot be handled by the current scene are propagated recursively to parent scenes until they are handled or ignored. The Engine can operate in different modes: (a) Sleep Mode: The gameplay is paused, the current state is persistently stored, and agents are deactivated. From this mode, gameplay can resume or terminate; (b) Terminate Mode: The gameplay is terminated, agents are deactivated, resources are released, and the Engine is destroyed. Multiplayer support is also a feature of the platform. Players can join and interact in the game’s world, sharing game objects and communicating with each other in real time. Multiplayer games are synchronized between the back end and front end, ensuring consistency and enabling real-time interactions. The platform offers data collection and game analytics functionality, including monitoring and recording player behavior respecting player privacy and security guidelines. Push notifications inform players of important events and milestones, enhancing their engagement. In the next section, the implementation details of the Rollick Games Platform will be described, focusing on the Game Studio, Player App, and PG Server. This platform architecture facilitates the design and execution of highly interactive and engaging games, fostering player involvement and enjoyment.

4 Platform Implementation The Game Studio is a web application for designing location-based pervasive mobile games. More specifically, it gives the designer a rich GUI for game design. Google Maps are used to define the area in which the game will take place. Initially, the designer describes the game’s scenes that compose the game’s world (Fig. 2). Then, the designer determines the game Objects and positions their instances in scenes. Its implementation is based on the React library. The Game Designer describes the plot of the game via interactions of the Player with game Objects, selecting what Behavioural Interfaces will be exposed by the Objects and

292

G. Anestis et al.

under what Conditions that behaviour will be available. That interaction will be carried out via GUI elements implemented by the Player App. The laws governing the game’s World are represented with rules described as conditional statements, implemented using the JsonLogic (https://jsonlogic.com) library. The Studio also supports importing all the necessary multimedia resources, such as icons, images, photos, video, and audio files, for configuring the game’s aesthetics. Furthermore, a Game Studio extension module supports the Game Master role, allowing for interaction at Play Time. The Game Master can interfere in the game world by communicating with the players or modifying the game world (placing new or removing object instances, resetting asset quantities, etc.) during playtime, enriching the gameplay process and enhancing the play experience. The design of a game is an iterative, multi-phase process. To support this process, the Game Studio provides an extensive set of editing capabilities for the aforementioned game authoring process till the final result. When ready, the designer exports the game description in PGML, which is permanently stored in JSON format in the back-end infrastructure.

Fig. 2. Game designers use the Game Studio to define the scenes of the world of the game.

The Player App, called Rollick, has been implemented using the React Native framework and is available for Android and iPhone devices. The app is part of the Pervasive Game Execution Environment that consists of the Engine and a set of agents that monitor various events and, when necessary, dispatch events to the Engine, which may change the game’s state. In addition, the app implements the required interactive GUI elements as React Native components. The platform’s high-level architecture is depicted in Fig. 3 (Right). The PG Server provides core services such as real-time communication and persistent storage. Its implementation is based on Node.js, Socket.io, and MongoDB.

Rollick Games: A Formal Language Based Platform for Location

293

Fig. 3. Left: The player can see the assigned missions and their status (accomplished or pending). Middle: In the course of a mission the player location (blue dot) in a scene (pink circle) is shown; icons represent virtual game objects. Right: Pervasive Game platform high-level architecture.

5 Related Work Pervasive Games bring computer entertainment to the real world [8] and have evolved in successive generations following advances in mobile/pervasive computing technologies. In [9], three generations are determined using criteria such as localization-user tracking, communication, context, orchestration, and player equipment. A template-based language that extends traditional use case templates to design pervasive game activities in the conceptual design phase and to help the transition from pre-production to production stages is described in [10]. The proposed text-based tool facilitates the involved stakeholders in the conceptual design phase but remains at the requirements specification level. Our work focuses on detailed design and platform implementation for generating pervasive games. ARIS [11, 12] is a rich, open-source platform for creating and playing augmented reality, interactive storytelling experiences on mobile devices based on player location (GPS), QR codes, beacons, maps, and media collection. The platform provides a game editor, a client app to play games, and a cloud database server. It mainly focuses on interactive stories, scavenger hunts, tours, and data collection activities. Although ARIS is one of the most full-featured platforms, its player app is available only for iOS. It does not support player communication nor single sign-on for the authentication process. Finally, it does not provide connectionless gameplay or game analytics functions. TaleBlazer [13] is a location-based augmented reality game platform with an online game editor that allows designers to edit games and a game repository server that stores games. A player app exists for both iOS and Android. Its conceptual model contains agents, regions, scenarios, and roles. Authors can define custom traits that can be modified through a visual, block-based scripting language. TaleBlazer does not support multiplayer games. Game analytics features are available only for officially recognized partners. QR codes are not supported. LAGARTO [14] is an authoring tool for designing location-based games with AR features, mission ordering, interaction with digital content, and mission sharing between groups of players. It is composed of a Game Editor, a Game Server, and a Mobile

294

G. Anestis et al.

application for gameplay. Using a GUI, the tool targets users without programming skills to create, build, and run location-based games. It does not provide game analytics, nor does it reference indoor gameplay with QR codes or Bluetooth Low Energy technology. CREANDO [15] is a platform which supports the creation of pervasive gaming experiences only in closed spaces. A story editing tool allows for the creation and editing of game experiences, adding audio, video, images, and HTML code, while the supported interaction elements are Beacons, QR codes, and AR markers. When the game configuration is done, it is possible to play the game via the provided mobile application. Geolympus [16] is a cloud platform for managing location data for applications and pervasive game experiences. The platform focuses on geolocated entities, areas, groups, and routes. Geopositioning is achieved via GPS. Beacons and QR codes are used for indoor positioning. It focuses on geolocation entities and lacks alternative game mechanics such as virtual objects, missions, player inventory, etc. Tidy City Scout [17] supports authoring tools for in-situ and web-based usergenerated content for location-based games. The games are organized as missions composed of riddles described by name, difficulty level, category, clue text, and image. Only pictures and texts are supported in its mechanics. No support for indoor gameplay. WeQuest [18] is an authoring tool that allows end-users to generate Alternative Reality Game (ARG) stories that can be executed automatically on geo-location-aware mobile devices. ARG stories are represented by a directed, acyclic graph where nodes represent events and arcs impose constraints on story event visitation order. It does not provide game analytics and does not reference indoor gameplay. fAR-PLAY [19] is a framework for developing augmented/alternative reality games in the treasure-hunt style. It is composed of the Layar augmented reality browser and the BeeTagg QR code reader, a game engine, tools to support virtual worlds, and an authoring environment. It lacks player communication and game analytics mechanisms, while the authoring environment combines three different tools.

6 Conclusions and Future Work We have presented in this paper the Pervasive Game Modeling Language (PGML), a platform-independent, domain-specific modeling language that allows the modeling of a rich set of pervasive games. The language has been used in the Rollick Games Platform that allows the design, deployment, execution, and management of location-based pervasive games. The platform provides a transpiler that compiles PGML into PGEL (Pervasive Game Execution Language) programs that can be deployed and executed on its Pervasive Game eXecution Environment. PGXE supports multiplayer gameplay following an event-driven architecture. It senses the player’s context and game state changes to generate events. These events are dispatched to the game engine, and if a handler is found, the corresponding computation takes place based on the event data and the game’s internal state. Such computations can affect the player or world state and trigger other events. The platform implementation includes the Game Studio, a web application that allows for the intuitive design of location-based mobile games, a Player mobile App available for Android and iPhone devices, and the Pervasive Game Server, which provides a set of core services for supporting multiplayer games such as real-time communication and persistent storage.

Rollick Games: A Formal Language Based Platform for Location

295

We developed the PGML and the accompanying platform to facilitate the design, implementation, and deployment of location-based pervasive game models by people without programming knowledge, reducing cost and shortening the time to market. Essential success factors of our work include (1) the expressive power of PGML and (2) the platform’s usability. To have some preliminary feedback on these factors, a group of volunteer undergraduate students in the School of Architecture of the Technical University of Crete were engaged in the design of location-based pervasive games. They were initially informed about the modeling primitives. They were also given paper game cards representing scenes, game objects, NPCs, missions, skills, inventory, game currencies, and other modeling primitives and a city map. Two groups were created, and each group designed its own game. Subsequently, each group presented its game, and later, participants decided to work together, combining the two games. The result was a game named “Sentinels Awakening”. Finally, the students tried to use the Game Studio to describe and create their game. After finishing the implementation, the students were asked to describe their experiences through a structured interview focusing on the two success factors mentioned above: (1) whether the offered modeling elements were adequate to design a challenging game and (2) the usability of the Game Studio. Most students considered that the game modeling elements offered were sufficient. However, at the beginning of their acquaintance with modeling elements and tools, they needed help understanding and complying with some restrictions and rules. They also found using the Game Studio to be difficult to some extent. These results, though preliminary, were encouraging regarding the PGML power and value. These usability problems related to the Game Studio can be explained by the fact that it was the first version of the software. We plan to proceed with developing different games and a thorough usability evaluation of the tool. Acknowledgments. This work has been carried out in the context of the “G4M: Games for Marketing” pilot project (MIS code 5028310) funded through the Smart Specialisation Strategy of Crete (RIS3Crete) with the financial support of the European Regional Development Fund.

References 1. Benford, S., Magerkurth, C., Ljungstrand, P.: Bridging the physical and digital in pervasive game. CACM 48(3), 54–57 (2005) 2. Montola, M.: Exploring the edge of the magic circle: defining pervasive games. In: Proceedings of Digital Arts and Culture (DAC’ 2005), pp. 1–3 (2005) 3. de Souza, A.: Location based mobile games: blurring the borders between physical and virtual spaces. In: Proceedings of the Inter-Society for the Electronic Arts (ISEA) Symposium, pp. 99–102 (2004) 4. Björk, S., Holopainen, J., Ljungstrand, P., Mandryk, R.: Special issue on ubiquitous games. Person. Ubiquit. Comput. 6, 358-361 (2002). (Springer-Verlag) 5. Valente, L., Feijó, B., do Prado Leite, J.C.S.: Mapping quality requirements for pervasive mobile games. Require. Eng. (22), 137–165 (2015). (Springer-Verlag London) 6. Callele, D., Neufelf, E.: Requirements engineering and the creative process in the video game industry. In: Proceedings of the 13th IEEE International Conference on Requirements Engineering, pp. 240–252 (2005)

296

G. Anestis et al.

7. Alves, C., Ramalho, G., Damasceno, A.: Challenges in requirements engineering for mobile games development: the meantime case study. In: Proceedings of the 15th IEEE International Requirements Engineering Conference, pp. 275–280 (2007) 8. Magerkurth, C., Cheok, A.D., Mandryk, R.L., Nilsen, T.: Pervasive games: bringing computer entertainment back to the real world. Comput. Entertain. 3(3), 4–4 (2005). https://doi.org/10. 1145/1077246.1077257 9. Kasapakis, V., Gavalas, D.: Pervasive gaming: Status, trends and design principles. J. Network Comput. Appl. 55, 213–236 (2015). https://doi.org/10.1016/j.jnca.2015.05.009 10. Valente, L., Feijó, B.: Extending use cases to support activity design in pervasive mobile games. In: SBC - Proceedings of the SBGames, pp. 876–885 (2014) 11. Holden, C.: ARIS: Augmented Reality for Interactive Storytelling. Mobile Media Learning: Innovation and Inspiration, pp. 68–83, ACM (2015) 12. ARIS manual. https://manual.arisgames.org 13. TaleBlazer. http://taleblazer.org 14. Maia, L.F., et al.: LAGARTO, A LocAtion based Games AuthoRing TOol enhanced with augmented reality features. Entertain. Comput. 22, 3–13. Elsevier (2017) 15. Alonzo-López, J., Valdivieso, C.C.C., Collazos, C.A., Vela, F.L.G., Moreira, F.: CREANDO: Tool for creating pervasive games to increase the learning motivation in higher education students. Telemat. Inform. 38, 62–73. Elsevier (2019) 16. Forte, J.L.B., Gázquez, D.P., Arango-López, J., Vela, F.L.G., Moreira, F.: Geolympus - cloud platform for supporting location-based applications: a pervasive game experience. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S. (eds.) New Knowledge in Information Systems and Technologies: Volume 3, pp. 256–266. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-16187-3_25 17. Wetzel, R., Blum, L., Oppermann, L.: Tidy City - A location-based game supported by insitu and web-based authoring tool to enable user-created content. In: Proceedings of the International Conference on the Foundations of Digital Games, pp. 238–241, ACM (2012) 18. Macvean, A., et al.: WeQuest: scalable alternate reality games through end-user content authoring. In: Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology, pp. 1–8, ACM (2011) 19. Gutierrez, L., et al.: faR-PLAY: a framework to develop Augmented/Alternative Reality Games. In: IEEE International Conference of Pervasive Computing and Communications Workshops, pp. 531–536 (2011) 20. Fidas, C., Sintoris, C., Yiannoutsou, N., Avouris, N.: A survey on tools for end user authoring of mobile applications for cultural heritage. In: 6th International Conference on Information, Intelligence, Systems and Applications (IISA), IEEE (2015) 21. Geneserth, M., Love, N., Pell, B.: General game playing: overview of the AAA competition. AI Mag. 26(2), 62–72 (2005) 22. Love, N., Hinrichs, T., Genesereth, M.: General Game Playing: Game Description Language Specification Technical Report LG-2006.-01, Stanford University (2006) 23. Marques, E., Balegas, V., Barroca, B.F., Barisic, A., Amaral, V.: The RPG DSL: a case study of language engineering using MDD for generating RPG games for mobile phones. In: Proceedings of the 2012 Workshop on Domain-Specific Modeling, pp. 13–18, ACM (2012) 24. Herzig, P., Jugel, K., Momm, C., Ameling, M., Schill, A.: GaML- a modeling language for gamification. In: Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing. pp. 494–499 (2013) 25. The Rollick Games Platform. https://rollick.tuc.gr

Perceptions and Challenges of Implementing XR Technologies in Education: A Survey-Based Study 1 ˇ Filip Skola , Alexandra Karanasiou2 , Mike Triantafillou2 , Haris Zacharatos3 , and Fotis Liarokapis1(B) 1

CYENS – Centre of Excellence, Nicosia, Cyprus [email protected] 2 KAINOTOMIA, Larissa, Greece 3 CELLOCK, Nicosia, Cyprus

Abstract. This paper presents a user-centered design approach employed in the development of initial user requirements for the XR4ED EU project. The process involves the utilization of questionnaires to gather valuable insights regarding the user requirements for experts XR in education. Understanding the perspectives of experts and identifying barriers to implementation are crucial for fostering its effective utilization. Results highlight the positive outlook among experts regarding the potential of XR as a transformative tool in education as well as the importance of addressing financial, technical, and infrastructural challenges to facilitate successful implementation.

Keywords: augmented reality reality · virtual reality

1

· education · extended reality · mixed

Introduction

The rapid advancement of technology has revolutionized various aspects of society, and education is no exception. In recent years, extended reality (XR) technologies have emerged as powerful tools for facilitating knowledge transmission and enhancing engagement within educational environments [8]. XR encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR), offering immersive and interactive experiences that transcend traditional teaching methods. As XR technologies continue to evolve, it is essential to examine the perceptions of experts and educators regarding their integration into academic settings. The European education technology (EdTech) sector is playing an increasingly important role in driving forward digital transformation in Europe covering all education and training sectors. The European XR industry has evolved and maintained a leading role globally in software and content production. According to prior studies, XR technologies can be an excellent learning tool when used c The Author(s), under exclusive license to Springer Nature Switzerland AG 2024  M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 297–306, 2024. https://doi.org/10.1007/978-3-031-56075-0_28

298

ˇ F. Skola et al.

in conjunction with more conventional techniques [2,3,5,6]. The use of XR can expand the range of activities through which students can gain hands-on experience, enabling them to go beyond abstract knowledge and supporting skills-based teaching and learning. Currently, developing XR applications for education requires specialized knowledge and strong XR development skills. The process takes a lot of effort, time, and money. This study is part of the XR4ED EU project [10] that focuses on the sectors that must be further developed to make Europe a leader in cuttingedge technologies for education. The XR4ED concept is to (a) bring together EdTech and XR community and resources, overcoming fragmentation issues and encouraging the acceleration of innovation for personalized, innovative, efficient, and inclusive learning using XR; and (b) build a one-stop shop and open marketplace for XR applications for learning, training, and education that will act as a European reference platform on learning and teaching. This paper aims to investigate the perceptions and challenges associated with implementing XR technologies in the European EdTech sector. Through a survey-based study, the study analyzes expert opinions and explores their views on the didactic usefulness of XR, its potential as a teaching tool, and the barriers to its adoption. To gather insights, a sample of 48 experts was surveyed, consisting of individuals with a median of 5 years of experience with XR technology. Participants represented diverse backgrounds and expertise, providing a comprehensive overview of the field. The survey encompassed questions about the perceived importance of XR in education, its impact on students’ perception of teaching environments, barriers to implementation, and participants’ experience with XR technology. The results of the survey revealed a strong consensus among experts regarding the potential of XR technologies in educational settings. Participants widely believed that XR would become a standard tool for knowledge transmission within universities, with a median likelihood rating of 4 out of 5. They also agreed that offering XR content would influence students’ perception of a university’s ability to provide a progressive and modern teaching environment. However, challenges to the implementation of XR technologies were also identified. Cost/funding, technical concerns/technical know-how, and limited technology infrastructure were reported as the primary barriers. Notably, participants with previous experience in designing educational XR applications rated the importance of didactic usefulness and immersiveness of XR higher than their counterparts without such experience.

2

XR4ED Overview

The main objective of the XR4ED EU project is to conceive, establish, and uphold a sustainable on-demand educational platform with an emphasis on promoting the collaborative efforts of the European Union’s EdTech and XR communities. It aims in promoting Europe’s leadership in ground-breaking innovations while consolidating and streamlining access to existing solutions.

XR in Education

299

XR4ED will offer a platform where educators without prior programming experience or computer science knowledge will be able to build their own XR classes. A framework will be developed that will enable any instructor to design a virtual classroom, incorporate 3D content obtained from our marketplace, and then customize the learning environment for each student using a simple-to-use GUI. XR4ED aims to facilitate educational, learning, and training endeavors serving as a dynamic platform. It seeks to invigorate and streamline the adoption of XR systems across educational domains. This effort addresses accessibility concerns by providing a centralized platform for all parties involved in the XR technology and education sectors. To accomplish that, XR4ED encompasses an exhaustive catalog of XR solutions coupled with a marketplace, which empowers both the EdTech and XR communities to proffer and endorse their market-ready solutions. Educators, students, guardians, and educational administrators are poised to efficiently navigate and locate fitting educational resolutions tailored to their distinctive needs, concurrently acquainting themselves with the array of products available within the market landscape. The marketplace will be a web-based application offering services that will include a brief description and a preview that users can see. Any user will have the ability to develop or prepare third party applications that can be connected to XR4ED services via the marketplace.

3 3.1

Methodology User Requirements

The user requirements and analysis methodology consist of essential parts of XR in education as they serve as the foundation for any successful collaboration or project. A clear understanding of the needs, expectations, and preferences of the users or stakeholders involved will lead to higher customer satisfaction and a greater likelihood of achieving the desired outcomes. Users’ requirements capture has two objectives: (a) Find the “true” requirements and (b) Represent the “true” requirements in a suitable way for the users, customers, and developers. The term ‘true requirements’ refers to the implemented requirements that will add value to the users. The resulting description of the requirements must be understandable by users and customers. To build a system that truly meets the customers’ needs, it is essential to clearly define the problem. 3.2

Overview of Users

The end-user profile for XR applications for education is diverse and includes various groups. Each group accesses the tool in different ways based on their specific needs and interests as shown below.

300

ˇ F. Skola et al.

– XR Developers—This target group consists of developers who are interested in creating educational XR applications. They access the tool to leverage its features and resources for designing and implementing XR experiences in the field of education. They utilize the development tools, frameworks, and documentation provided by the system to create engaging and interactive XR applications tailored for educational purposes. – Business Community—The business community, including SMEs (small and medium enterprises) and companies operating at the national or European level in XR platforms, repositories, libraries, instructional design, system integration, investments, and hardware providers, utilizes the tool to explore opportunities in the educational XR market. They access the system to access XR content repositories, integrate XR technology into their existing platforms or services, invest in XR-based educational solutions, and provide hardware support for XR implementation in educational settings. – Scientific Community—The scientific community, comprising universities, research centers, and schools, particularly those specializing in STEM (Science, Technology, Engineering, and Mathematics), history, healthcare, and related fields, benefits from the tool to enhance their educational practices. They utilize the system’s XR resources, educational materials, and learning modules to create immersive and interactive learning experiences for their students. Additionally, researchers and educators within this group may contribute to the development of XR applications and collaborate with other stakeholders. – Young Professionals—Young professionals seeking training and up-skilling in industries such as healthcare, medical, manufacturing, construction, and engineering utilize the tool to access XR-based training programs. They can undergo virtual simulations, practical exercises, and hands-on experiences through XR applications provided by the system. This group benefits from immersive learning opportunities that help them acquire practical skills and knowledge in their respective fields. – Social Innovators—Social innovators involved in ethics and policy regulation for XR applications in education at the national or state level utilize the tool to stay informed and contribute to the development of ethical guidelines and policy frameworks. They access the system to gather insights, review XR applications’ educational impact, and provide recommendations for responsible and effective use of XR technology in educational settings. Therefore, the XR tool serves a diverse range of end-users including XR developers, the business community, the scientific community, young professionals, and social innovators. Each group accesses the tool in distinct ways, leveraging its features and resources to create, integrate, enhance, train, and regulate XR applications for educational purposes. Therefore, the XR tool serves a diverse range of end-users including XR developers, the business community, the scientific community, young professionals, and social innovators. Each group accesses the tool in distinct ways, leveraging

XR in Education

301

its features and resources to create, integrate, enhance, train, and regulate XR applications for educational purposes. 3.3

Questionnaires

The objective of the questionnaire to be used for the requirements-capturing process is to define the day-to-day requirements that users will have from the XR4ED system. Prior to building a questionnaire that fully captures the customers’ needs, it is deemed essential to answer some basic questions regarding the eventual usage of the system, namely, who will be the eventual users of the system, hence who should be responding to this questionnaire, and what are the processes and general features we want to examine. The integrated questionnaires were constructed from 5 validated questionnaires [1,4,7,9,11]. One questionnaire was designed for students (over 18 years old), and one targeted experts coming from the scientific and business community (universities, colleges, schools, research centers, XR developers, and experts in the partnership).

4 4.1

Results Demographics

The sample consisted of 48 expert participants, of which 10 were female. The participants reported a median duration of 5 years of experience with XR technology. When asked about their frequency of XR technology usage, the most common response was “sometimes” (median answer). In terms of awareness in the field of XR technologies and their use for educational purposes, participants rated themselves as “aware” with a median score of 4 (1 = Unaware, 5 = Very aware). Similarly, participants expressed confidence in their ability to develop or design new educational tools based on information and communication technologies, with a median rating of 4 (1 = Very imperceptive, 5 = Very perceptive). The general level of XR knowledge among participants was slightly above average, with a median rating of 3.5 (1 = Very poor knowledge, 5 = Very good knowledge). The majority of the sample (30 participants) did not have experience using VR/AR/MR technology to assist in education, while 16 participants reported having such experience, and 2 participants indicated that the question was not applicable to them. 4.2

Comparing XR Utilization in Education

Participants in the study attributed a median rating of 4 (1 = Unimportant, 5 = Very important) to the level of importance they assigned to the didactic usefulness of XR in the design of educational experiences. This indicates a significant degree of importance placed on XR as a valuable tool in enhancing educational methods and practices.

302

ˇ F. Skola et al.

Participants who had prior experience in utilizing XR technology for educational purposes provided a notably higher rating (median 5) regarding the importance of the didactic usefulness of XR when designing educational experiences, in comparison to those without such experience (median 3.5). This difference was determined to be statistically significant with p = 0.008, based on the Wilcoxon rank sum test (W = 350). These results indicate that previous experience in designing educational XR applications reinforces the perception of their didactic usefulness. Although the two groups did not differ significantly in their ratings of the importance of realism and 3D aspects in XR educational applications, participants with prior experience in designing applications rated the importance of immersiveness higher (median 5) compared to the inexperienced group (median 4). This difference was also statistically significant, with p = 0.017 and W = 338. These findings suggest that previous experience in designing XR applications enhances the recognition of the importance of immersiveness in educational contexts. A portion of the surveyed participants (15 individuals) tend to perceive XR as a passing trend to some extent, while the majority (18 participants) do not consider it a passing trend at all. A noteworthy difference exists between experts who already incorporate XR in education (median answer 1 = not at all a passing trend) and those who do not (median answer 3 = moderately passing trend). This disparity was found to be statistically significant (p = 0.022, W = 144.5). Similarly, a between-group difference was observed in their responses to the statement “XR is a powerful tool and should be used in teaching within educational environments”, although there is no difference in median answers (median 4 = Considerably). Both groups agree that XR may present a slight risk of distracting students from the teaching content. Additionally, both groups concur that teaching must continually evolve and adapt. However, experts who have prior experience using XR in education exhibit slightly higher confidence in this aspect (W = 313, pvalue < 0.05), although the median rating remains the same (5) for both groups. 4.3

Importance of XR in Education

Regardless of group, experts widely anticipate that XR will become a standard tool for facilitating knowledge transmission in university environments, with a median likelihood rating of 4 (Likely). They also agree that the availability of XR content will significantly influence students’ perception of a university’s capacity to deliver a progressive and modern teaching environment, again with a median likelihood rating of 4. Furthermore, both groups express optimism regarding the practical integration of XR technologies into university and school curricula by providing a sustainable and appropriate framework within the next 5 years, also with a median likelihood rating of 4. It is widely acknowledged that ongoing support for the technology is essential for its viability and usefulness, as indicated by a median rating of 4 (Important) in terms of importance.

XR in Education

303

Fig. 1. Responses to In your opinion, how likely is it that offering XR (AR, VR, MR) content will influence how current/future students think about university’s ability to offer a progressive modern teaching environment?

Experts widely hold the belief that XR will become a commonplace tool for facilitating knowledge transmission in university settings, as indicated by a median likelihood rating of 4. Furthermore, their consensus is mostly reflected in Fig. 1, where it is evident that experts largely agree on the impact of offering XR content on students’ perception of a university’s capacity to provide a modern and progressive teaching environment, also indicated by a median likelihood rating of 4. Experts also concur with the notion that XR is a powerful tool that should be utilized in teaching within educational settings. The survey results reveal a considerable level of agreement among the experts, with a median rating of 4 (considerably agree) and a substantial number of respondents selecting the option “Extremely considerably.” Refer to Fig. 2 for a comprehensive overview of the responses. Figure 3 provides a comprehensive breakdown of the participants’ opinions on the biggest barriers to implementing XR technologies within universities and schools in the next 5 years. The majority, consisting of 38 participants, identified cost and funding as the primary barriers. Technical concerns and a lack of technical know-how were cited by 30 participants, while limited technology infrastructure was recognized as a significant obstacle by 25 participants. Conversely, a minority of 11 participants regarded the disruption to traditional learning as a substantial impediment.

304

ˇ F. Skola et al.

Fig. 2. Responses to XR is a powerful tool and should be used in teaching within educational environments.

Fig. 3. Responses to Where do you see the biggest barriers for implementation of XR (AR, VR, MR) technologies within the next 5 years at the universities/schools?

Overall, the findings emphasize the potential of XR as a transformative tool in educational environments while highlighting the financial, technical, and infrastructural challenges that must be overcome for successful implementation.

5

Conclusions

This research paper provides insights into the perceptions and challenges surrounding the integration of XR technologies in educational environments. The study highlights the positive outlook among experts regarding the potential of XR as a transformative tool in education. Moreover, it underscores the importance of addressing financial, technical, and infrastructural challenges to facilitate successful implementation. By understanding the perceptions of experts and identifying barriers, educational institutions can make informed decisions about incorporating XR technologies into their teaching practices. As further research and advancements in

XR in Education

305

XR continue to unfold, it is imperative to explore innovative ways to leverage its potential for enhancing teaching and learning experiences. Acknowledgments. This research was part of the XR4ED (eXtended Reality for Education) project, which has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101093159. This research was partially supported by the project that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. The authors would like to thank all users who participated in the study.

References 1. Dalager, S., Majgaard, G.: Development of an educational AR tool for visualization of spatial figures and volume calculation for vocational education. In: Chen, J.Y.C., Fragomeni, G. (eds.) Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and Industry, pp. 14–30. Springer, Cham (2022). ISBN: 978-3-031-06015-1. https://doi.org/10.1007/978-3-031-06015-1 2 2. Dengel, A.R., et al.: A review on augmented reality authoring toolkits for education. Front. Virtual Reality (2022). https://doi.org/10.3389/frvir.2022.798032, https://api.semanticscholar.org/CorpusID:248399727 3. Jaligama, V., Liarokapis, F.: An online virtual learning environment for higher education. In: 2011 Third International Conference on Games and Virtual Worlds for Serious Applications, pp. 207–214 (2011). https://doi.org/10.1109/VS-GAMES. 2011.44 4. Kluge, M.G., et al.: Current state and general perceptions of the use of extended reality (XR) technology at the University of Newcastle: interviews and surveys from staff and students. SAGE Open 12(2), 21582440221093348 (2022). https:// doi.org/10.1177/21582440221093348 5. Liarokapis, F.: Using activity led learning for teaching computer graphics principles through augmented reality. In: Bourdin, J.-J., Shesh, A. (eds.) EG 2017 Education Papers. The Eurographics Association (2017). https://doi.org/10.2312/ eged.20171025 6. Liarokapis, F., Anderson, E.F.: Using augmented reality as a medium to assist teaching in higher education. In: Kjelldahl, L., Baranoski, G.V.G. (eds.) 31st Annual Conference of the European Association for Computer Graphics, Eurographics 2010 - Education Papers, Norrk¨ oping, Sweden, 3–7 May 2010. Eurographics Association, pp. 9–16 (2010). https://doi.org/10.2312/eged.20101010 7. Meccawy, M.: Creating an immersive XR learning experience: a road-map for educators. Electronics 11, 21 (2022). ISSN: 2079–9292. https://www.mdpi.com/20799292/11/21/3547. https://doi.org/10.3390/electronics11213547 8. Slater, M., Sanchez-Vives, M.V.: Enhancing our lives with immersive virtual reality. Front. Robot. AI 3 (2016). ISSN: 2296–9144. https://doi.org/10.3389/frobt.2016. 00074/ (visited on 03/26/2018) 9. Vergara, D., et al.: Virtual reality as a didactic resource from the perspective of engineering teachers. Comp. Appl. Eng. Educ. 30(4), 108–1101 (2022). https:// doi.org/10.1002/cae.22504. https://onlinelibrary.wiley.com/doi/pdf/10.1002/cae. 22504

306

ˇ F. Skola et al.

10. XR4ED: Accessed 10 Sept 2023 (2023). https://xr4ed.eu/ 11. Yang, K., Zhou, X., Radu, I.: XR-Ed framework: designing instruction-driven and learner-centered extended reality systems for education (2020). arXiv: 2010.13779 [cs.HC]

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids Aged 7–12 Using Gaming Panagiotis Petridis1(B) , Vladlena Benson1 , Mariam Garibyan1 , Gonçalo Meireles2 , Alex Carpov3 , Dimitra Dimitrakopoulou4 , and Marisa Teles5 1 OIM, Aston University, Birmingham, UK

[email protected]

2 Advancis Business Services, Lda, Matosinhos, Portugal 3 CEIPSO Maestro Rodrigo, Madrid, Spain 4 Ellinogermaniki Agogi, Pallini, Greece 5 Boon, Idanha-a-Nova, Portugal

Abstract. Artificial Intelligence is becoming popular across various sections of the industry varying from education to military and health, however the public understanding of those technologies is often limited. In the education domain there is a need to increase AI literacy of the students and the educators. However, the education systems are unprepared for addressing those complex issues and therefore there is a need to raise the awareness of about the potential, benefits, limitations, breadth of technologies used in AI (machine learning, deep learning), to make available resources that educator can use, and equipping them with all the necessary tools needed to engage with the students. This paper is focused on addressing those issues, and it proposes the use of a toolkit that includes games, educational materials and learning resources. The paper introduces the evaluation of a game by 68 participants, aged between 7–12 and several primary school educators. From the preliminary analysis we have identified that the game was perceived positively from participants and educators as an educational tool. Keywords: Games · Artificial Intelligence · Education · Serious Games · Game Base Learning

1 Introduction Artificial intelligence (AI) is becoming an increasingly important part of our lives, whether we knowingly embrace it or not. From social media to online movie platforms, AI systems penetrate human existence to offer benefits such as personalized experiences and simplify daily tasks, deeply changing the way we live. Rapid technological advances in the fields of artificial intelligence (AI) and machine learning (ML) promise significant economic benefits (e.g. UK GDP to be 10% higher in 2030 as a direct result of AI through increased productivity and increased quality of products) [1]. As AI systems become embedded into our daily lives and behaviours, we can also expect an increasingly higher impact of these systems on our future, namely on jobs and competences required to successful careers. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 307–315, 2024. https://doi.org/10.1007/978-3-031-56075-0_29

308

P. Petridis et al.

With the AI market estimated to reach around 15 trillion USD by the year 2030, the push for integrating AI in the school curriculum is only growing [1]. The delivery of this innovative gamification training package for schools, whilst exposing schoolteachers to cutting-edge research environment, has enabled the Adventure in AI (2AI) project to engage and communicate effectively with school pupils, educators, policy makers, software engineers and data scientists involved in the application of AI, whilst inspiring them to become the next generation of leading innovators in that field [1, 2]. Increasing the AI literacy for all the parties involved is important. AI literacy begins with a basic understanding of what AI is, the language surrounding the technological and social aspects of AI, how AI works and how it is currently playing a role in our daily lives, in addition to potential implications in jobs and careers. In a way, the objective of AI literacy is to eradicate the misconceptions around AI and to create an all-inclusive ecosystem where all members of the community are equipped with the basic skills needed to pursue further learning to better adapt to a changing world where AI will be prevalent. However, currently the education systems are unprepared for addressing those complex topics and it is, therefores necessary to: • Raise awareness about the importance of AI literacy • Make available learning resources on these topics • Equip educators with the necessary tools and resources to engage with the students through a variety of learning activities. The 2AI project aims to address those issues and the consortium has developed a gamified approach to increase children’s (7–12 year olds) AI literacy, and to raise awareness and equip educators to address AI topics in the classrooms and in other learning contexts (including non-formal and informal learning). Section 2 will contain the background information on the topic and it will focus on the current approach the schools are taking to address those issues. Section 3 will introduce the tools that the consortium has developed, Sect. 4 will include the evaluation of those tools with students and educators, and Sect. 5 will include the conclusions and further work.

2 Background Artificial Intelligence can be defined in a number of ways. According to Baker and Smith (2019) [3], a broad definition of AI is: “Computers which perform cognitive tasks, usually associated with human minds, particularly learning and problem-solving”. According to Baker and Smith [3], AI is a combination of various technologies across different disciplines (i.e. machine learning, natural language, data mining, neural networks, etc.). Another broader definition of AI is the “intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. Recent advances in artificial intelligence have perched AI at the apex of the hype curve, promising to transform fields such as healthcare, finance, education, security and law enforcement among other is enormous [3].

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids

309

Artificial Intelligence (AI) has fed the fears and hopes of various social and professional sectors, hand in hand with both scientific dissemination and research [2]. The educational world has not been an exception to the norm [4] and, although its implementation has occurred more slowly than in sectors such as medicine or finance, AI applied to teaching and learning is emerging as a more widespread reality whose implications on meaningful student learning is viewed with both suspicion and expectation [2, 5, 6]. The qualities previously mentioned set off alarms in some sectors of the educational world due to their possible consequences on the preponderance of human intelligence as superior to all others [3]. The emergence of AI was concentrated mostly (although not only) in three differentiated computer applications: the explosion of Big Data, the appearance of the Internet as a tool for daily use, and the proliferation of ICT in many countries of the world [7]. Since then, AI has spread into the unthinkable. The paths that AI has taken are as follows: • Microworlds: defined by some theorists as a consistent programming environment and therefore capable of offering the exploration of a series of knowledge, whether mathematical or related to literacy, and can be seen in programs such as the famous Logo programming language [8]. • Intelligent Tutoring: instructional system that allows a training adapted, and therefore more individualized, to the needs and profiles of the students based on the content module, which has the knowledge to transmit information to the student. This learning is facilitated by the teacher, who indicates when a certain material should be presented to the students, allowing for essential communication between student and program. A good example of this aspect of AI in pedagogy is the Brazilian Geekie [9]. • Robotics: AI technique in charge of building devices to perform physical activities similar to those performed by living beings. Through initiatives [7] such as Robotics from Lego Education1 , this methodology can be used both to teach principles related to robotics in itself and to train students in other curricular content. The School Education Gateway [10] has produced a tutorial for successfully embedding AI in education, suggesting the following 3 ways: 1. Learning with AI, i.e. integrating AI technologies into the classroom to enhance student learning and improve instruction. 2. Learning for AI, that is, acquiring new skills required for life and work in an AIshaped world. To unlock the potential of AI and to deal with challenges in an AIshaped world, students need to be equipped with computational thinking and problemsolving skills, as well as coding and data literacy skills. 3. Learning AI, or applying AI-related skills to effectively use AI and build new AI tools and technologies. Giannakos, M. et al. (2020) [11] give a good overview and further useful perspectives on using games in AI education [12]. There is a need for additional research in the area of AI to investigate what new competencies will be necessary in a future in which AI transforms the way that we communicate, work, and live with each other and with machines- this term is referred to as AI Literacy [13]. AI literacy as a set of competencies

310

P. Petridis et al.

that enables individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace [13]. The Adventure in AI - Developing Children’s AI Literacy (2AI) project aimed to develop a gamified approach to increase children’s (7–12 year olds) AI literacy, and to raise awareness and equip educators to address AI topics in the classrooms and in other learning contexts (including non-formal and informal learning) [14].

3 Our Approach 3.1 Game Description The 2AI Adventure game main objective (see Fig. 1) is to increase children’s AI Literacy and consequently eradicate the misconceptions around AI and create an all-inclusive ecosystem where all members of the community are equipped with the basic skills needed to pursue further learning to better adapt to a changing world where AI will be prevalent.

Fig. 1. 2AI Game Screenshots from Various Levels

Thus, as already mentioned above, the Game promotes a basic understanding of what AI is, the language surrounding the technological and social aspects of AI, how AI works and how it is currently playing a role in our daily lives (and children’s lives in particular), the risks and opportunities AI bares and AI’s potential impact in jobs and other areas of society. The game can be used as a standalone tool where the kids can play and explore the different AI concepts that are introduced in the game. Alternatively, it can be easily integrated in the curriculum and can be used as a triggering discussion on AI, or as a way to test the knowledge of the students in that particular domain. The game uses a pool of 30 questions focused on AI, that were carefully created by school educators (theeducators from the UK, Greece, Spain and Portugal) and are specifically designed for key stage 3 students. The questions are randomly displayed at the end of each level. This pool of questions is given to the educators and can be used as a standalone tool. These questions are set to test whether the students are grasping the concepts of the AI, and at the end of each level of the game the student has to answer these questions.

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids

311

However, answering these questions wrong will not impede the player/student progress. In order to ensure learning we the consortium has created a glossary within the gamerelated to AI- that allows the player to familiarize themselves with specific AI-terms. The game is divided into 3 levels focusing on various AI topics. The table below shows the lists of topics that are covered in the 3 levels of the game (Table 1). Table 1. AI Concepts Linked with Game levels Level 1

Level 2

Level 3

Artificial Intelligence (AI)

Algorithmic bias

Trolley Problem

Data

Labelled Data

Turing Test

Sensors

Speech Recognition

Data bias

Agents

Speech Generation

Algorithm

Weak/Narrow AI

Natural Language Processing

Strong AI

Neural Networks

Lidar Sensor

Super AI

Unlabeled Data

4 Usability Evaluation In this section we are going to present our preliminary findings from the evaluation of the usability of the developed game. 4.1 Apparatus and Visual Content The participants were provided with laptops, where the game had been pre-installed, and the questionnaires were either given on electronic forms (e.g. google forms) or they had been printed. The main reason for that is that different schools have different levels of expertise so, the educators/trainers had to decide what was best for their students. 4.2 Participants Sixty-eight participants in total were used from our selected primary schools from the UK, Spain, Greece and Portugal. The participants were between 6–12 year old students, studying at selected schools (Fig. 2). 30 males and 36 females participants and 4 preferred not reveal their genders. A within subject design (i.e. every participant played all the different levels of the game) was followed. The educator was supervising the process and was making sure that the experiment ran smoothly. From the 68 participants, 6 were removed from the study, because they failed to answer 2 or more questions from the required questionnaire section.

312

P. Petridis et al.

Fig. 2. Students During the Evaluation of the Game

4.3 Procedure It was ensured that each student was comfortable and at ease prior to the start of the experiment. Our target group were children between the ages of 6–10 across the four member states (the UK, Portugal, Spain and Greece). The students were told that the participant’s data would be used anonymously; along with the data of several others and that the experiment would be divided into two main stages. During the first stage of the experiment the participants had to fill a mini questionnaire with open-ended questions designed to identify their prior knowledge in relation to AI. The questions were designed/formulated with a focus on our target audience (kids between 6–10). After completing the questionnaire, they were given a presentation of the concepts of AI: machine learning, reflection on the advantage and disadvantages of AI in our society and presentations of example of AI in our society. During the second stage the participants were asked to play the first two levels of the Adventure in AI game. The educators were asked to supervise the process and record (by filling a different questionnaire at the end of the session) the progress of the students and any potential issues that they had identified. Upon completion of the activity, the participants filled a usability questionnaire which was modified with our target audience in mind. The questionnaire was based on a 10- Likert Scale. The questionnaire was divided into 7 sections focussing on evaluating the Game Concept Idea, The Game technical elements, and the participant’s satisfaction. The instructions and the statements that were used during the preliminary testing were standardized for all four countries. Both questionnaires were presented in a digital form using Google Forms and then transcribed into EXCEL/ SPPS v13.0 for analysis. 4.4 Analysis. – Evaluation of the Game The Usability questionnaire was divided into 3 sections. The first section was focussing on the game element evaluation such as the game idea or Concept, the complexity/difficulty of the game, the luck vs skill, the playing time and the text size. The graph below shows the responses of the 62 students (Fig. 3).

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids

313

Fig. 3. Summary of Participant responses focusing on the game elements evaluation

From the graph above we can see that 72% of the participants liked the game idea/concept of the developed game (i.e. gave a score greater or equal to 6 in the Likert scale. An interesting finding within the participants is that the game difficulty is reduced as the ages of the participant increases. This may be attributed to the familiarity of those types of games to the participants. The table below shows the participant responses in correlation with the player age. Player’s Age

Complexity/Difficulty:

std

7

7.5

0.5

8

7.16

1.34

9

7

0.82

10

7

1.53

11

6.45

2.13

11.5

5.66

0.47

12

5.32

2.10

The same pattern is observed in the Luck vs Skill question, the higher the age of the player the more the players are lining towards skill rather than Luck (Table 2). Table 2. . Age

Average of Luck vs. Skill:

Std

7

10

0

8

8.83

1.46

9

9.67

0.47

10

6.33

2.87

11

5.59

2.51

11.5

8

1.41

12

7.16

2.64

314

P. Petridis et al.

The second part of the questionnaire was focussing on the Graphic Design, Control, sound/music, creativeness, and fun aspects of the game. The result from the evaluation is shown in the table below. 18% of the participants had issues with the controls, but based on their comments and the peer observation of the educators this can be attributed to the type of game. Based on the feedback received from the students, the sound/music of the game should be improved, as several students mentioned this can be improved with the introduction of sound effects withing the various levels (Fig. 4).

Fig. 4. Graphics Design, Control/music and Fun Factor of the Game

However this didn’t have an impact on how the students perceived the game (67% of the participants had a positive view of the game). Checking the final version of the questionnaire, we are able to see that the same pattern is followed, and the participants had a very positive view of the three levels of the game (Fig. 5).

Fig. 5. Student’s Perception of the Game

To conclude, the game was perceived positively from the participants, and according to our preliminary evaluation the game is a successful tool that can be used in classroom and be used as a trigger point for discussion for the potential benefits or limitations of AI, and increasing the AI literacy of our participants.

5 Conclusions Artificial intelligence is a topic that has received a significant boost in terms of its effectiveness in recent years. Nowadays it is important to increase the AI literacy amongst the young generation, educators, policymakers etc. Educators have an opportunity to engage and empower the imagination of the younger generations to shape and improve the world they are going to inherit, especially as children create unique perspectives as the first generation surrounded by AI systems. For all these reasons, it is critical to introduce or reinforce the approach to this

Adventure in AI Project (2AI): Promoting AI Knowledge for Kids

315

topic with children and improve their AI literacy. Our approach is focussed towards creating EU/UK platform for outstanding teacher and pupil training in AI technologies and its societal implications. Our platforms aims to create new learning software solution that will aid the Road to AI and the EU Single Digital market initiative using AI in the future; it will produce novel interactive learning AI techniques to help users understand their benefits and societal implications.

References 1. Forum, W.E.: By 2030, AI will contribute $15 trillion to the global economy (2023). https:// www.weforum.org/agenda/2019/08/by-2030-ai-will-contribute-15-trillion-to-the-global-eco nomy/#:~:text=AI%2C%20in%20all%20its%20applications%2C%20is%20predicted%20t o,combined%20GDP%20of%20China%20and%20India%20in%202018. Accessed 9 July 2023 2. Crompton, H., Burke, D.: Artificial intelligence in higher education: the state of the field. Int. J. Educ. Technol. High. Educ. 20(1), 22 (2023) 3. Baker, T., Smith, L.: Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges (2019). Retrieved from Nesta Foundation website. https://media.nesta. org.uk/documents/Future_of_AI_and_education_v5_WEB.pdf 4. Ocaña-Fernandez, Y., Valenzuela-Fernandez, L., GarroAburto, L.: Artificial intelligence and its implications in higher education. Propósitos y Representacione 7(2), 536–568 (2019) 5. Lameras, P., Arnab, S.: Power to the teachers: an exploratory review on artificial intelligence in education. Information 13 (2022) 6. Winstone, N.E., et al.: Measuring what matters: the positioning of students in feedback processes within national student satisfaction surveys. Stud. High. Educ. 47(7), 1524–1536 (2022) 7. Zawacki-Richter, O., et al.: Systematic review of research on artificial intelligence applications in higher education – where are the educators? Int. J. Educ. Technol. High. Educ. 16(1), 39 (2019) 8. Lego Foundation (2019). https://el.media.mit.edu/logo-foundation/index.html. Accessed 9 July 2023 9. Gil, N.: How software that learns as it teaches is upgrading Brazilian education.(2016). https://www.theguardian.com/technology/2016/jan/10/geekie-educational-sof tware-brazil-machine-learning. Accessed 9 July 2023 10. S.E.G. How can artificial intelligence be embedded in education? A tutorial (2021). https:// www.schooleducationgateway.eu/en/pub/resources/tutorials/ai-in-education-tutorial.htm 11. Giannakos, M. (ed.): Non-Formal and Informal Science Learning in the ICT Era. LNET, Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-6747-6 12. Luckin. AI in UK schools? I’d give us 5 out of 10. (2019). https://www.tes.com/news/ai-ukschools-id-give-us-5-out-10 13. Long, D., Magerko, B.: What is AI Literacy? competencies and design considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16. Association for Computing Machinery, Honolulu, HI, USA (2020) 14. Care, E., Griffin, P.: Assessment of Collaborative Problem-Solving Processes. OECD Publishing, Paris (2017)

Remote and Online Laboratories

Work-in-Progress: “Smart Print Automation” Remote Lab and Cloud Connector C. Madritsch(B) , P. Hohenberger, B. Heindl, and V. Smoly Carinthia University of Applied Sciences, Villach, Austria [email protected]

Abstract. The paper discusses a project to develop a “Smart Print Automation” system that allows remote access to the lab and is an integral part of Automation 4.0 education. This system consists of a print head, a gantry, a controller unit and an edge device designed and built by a group of students. A programmable logic controller (PLC), specifically the SIMATIC S7-1200, coordinates the system, primarily the three stepper motor drivers and motors. It uses OPC UA to communicate real-time production data with the Raspberry Pi edge device, providing remote monitoring and control via a web interface built with Node-RED. The print head and mechanical design of the T-Bot Gantry ensure fast and accurate operation. The PLC program was created using TIA Portal V17 and Function Block Diagram (FBD) for precise motor control. Docker facilitated the installation of programs on the edge device, including Node-RED, influxDB, Portainer and Grafana. While the system is currently standalone, the next phase is to enable it as a remote lab and develop a cloud connector to make the edge device data easily accessible online for machine learning applications. Keywords: Automation 4.0 · Remote Lab · Edge Device · PLC

1 Introduction To cover the topics of the Smart Automation [1] lecture from a practical point of view, and to enable students from abroad to use the laboratory hardware, a Smart Print Automation Plant was developed and will be made ready for use as a Remote Laboratory Experiment (Remote Lab) [2], mobile technology – at any place, at any time. The project was carried out by a group of students, supervised by the Professor, and will be continued during the next six month until its completion. In Sect. 2, the discussion opens with a comprehensive “System Overview” where we illuminate the four main components of the automated printing system: the printer head, the gantry, the controller unit, and the edge device. This chapter includes an extensive description of how each component was designed and manufactured, emphasizing the critical role of PLC and how it communicates with Raspberry Pi via OPC UA for remote monitoring and control. Section 3, titled “Hardware” provides an exhaustive analysis of the system’s hardware elements. Here, we expound on the role and functionality of the PLC (SIMATIC S7-1200 © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 319–325, 2024. https://doi.org/10.1007/978-3-031-56075-0_30

320

C. Madritsch et al.

CPU 1215C DC/DC/DC), the Stepper Motor Driver (TB6600), and the Stepper Motors (NEMA11 & NEMA17), explaining how they collaboratively contribute to the system’s operation. The chapter concludes with a discussion on the Edge Device: Raspberry Pi, emphasizing its function as a communication interface between the PLC and the Node-RED interface. In Sect. 4, we delve into the “Mechanical Design” of the system. This chapter provides an in-depth explanation of the design, development, and functionality of the printer head and the T-Bot Gantry. It also sheds light on how these mechanical components facilitate the precise movement of the printer head, thus ensuring fast and accurate operation. Section 5, “Software” explicates the software components of the system, focusing on the PLC and Edge Device programming. We look at how the PLC program was created in TIA Portal V17 and implemented using Function Block Diagram (FBD) for precise control over the motors. Furthermore, we delve into how Docker was employed to install programs on the edge device, facilitating the functionality of an edge device with programs like Node-RED, influxDB, Portainer, and Grafana. Section 6, “Conclusions and Future Work” gives a summary about the work done so far and shows which elements are still to be developed by the next cohort of students.

2 System Overview As a project, an automated printing system was built, which prints the universities logo. The printing device consists of four main parts as seen in Fig. 1, the printer head on the top left, the gantry on the top right which carries the printer head, the controller unit in the centre, and an edge device on the bottom centre. All custom parts of the printer head and the traversing axes were designed by the project group and manufactured with a 3D printer. The printer head and the gantry are both controlled by a PLC (programmable logic controller) which communicates relevant production data to a Raspberry Pi via

Fig. 1. Schematic System Overview

Work-in-Progress: “Smart Print Automation” Remote Lab and Cloud

321

OPC UA [3]. This allows the user to remotely monitor and control the printer on a web-interface built with Node-RED [4].

3 Hardware 3.1 PLC: SIMATIC S7-1200 CPU 1215C DC/DC/DC In the given setup, a programmable logic controller (PLC), the SIMATIC S7-1200 Compact CPU 1215C DC/DC/DC [5] was used. It acts as the central control unit for the system. It is responsible for coordinating and executing commands to operate the three stepper motor drivers. These drivers, in turn, interface with the corresponding stepper motors and enable precise control of their movements. In addition, OPC UA [3] to exchange production data with other systems was used. OPC UA is a communication protocol that enables the secure and standardized exchange of information between different industrial devices and software platforms. This capability was used to transmit real-time data about the current movement step and therefore the position of each motor. 3.2 Stepper Motor Driver: TB6600 The stepper motor drivers used in the printing system are TB6600 devices. It provides the necessary power and signals to drive the stepper motors that are responsible for the movements of the print head and axes. The TB6600 driver supports various modes of operation, including full-step, half-step and micro-step modes, which determine the degree of resolution and smoothness of the motor movements. The TB6600 driver is connected to the SIMATIC S7-1200 PLC. The PLC sends commands to the driver, which specify the required motor movements in parameters such as speed and direction. The driver then converts these commands into electrical signals that control the stepper motors as required. 3.3 Stepper Motor: NEMA11 & NEMA17 Two types of stepper motors were used: The NEMA11 stepper motor is used in the printhead. It is a compact and lightweight motor that provides accurate positioning. The NEMA11 motor is suitable for applications where space is limited but precise movement is required. The NEMA17 stepper motor is used in the system’s linear axes. It is larger than the NEMA11 and offers higher torque. The NEMA17 motor handles the translation of the printer head and ensures smooth and precise movement along the axes. 3.4 Edge Device: Raspberry Pi The Edge Device [6] in the printing system is a Raspberry Pi computer. The Raspberry Pi acts as a communication interface between the PLC and the Node-RED interface. It receives production data from the PLC via OPC UA and processes it into maintenancerelevant information via Node-RED. It also has a GPIO (General Purpose Input/Output) interface that can be used to connect it to external devices and sensors. For example,

322

C. Madritsch et al.

a temperature sensor is connected to the Raspberry Pi to monitor the temperature of a stepper motor and therefore detect anomalies to be able to react to defects at an early stage. Using the OPC UA protocol, the Raspberry Pi can connect to the PLC and receive real-time data about the current step and position of each motor. This data is then further processed using Node-Red, allowing the user to remotely monitor and control the printing system.

4 Mechanical Design The printer head as well as the linear translation mechanism (“gantry”) was fully developed by the project team. It makes use of sophisticated mechanisms to enable the control of the printing needles as well as the movement of the printer head itself. 4.1 Printer Head The printer head, as shown in Fig. 2, is the core of the mechanical development of the printing machine. It features a mechanism, which allows for all six needles to be controlled in a way that all possible needle configurations can be achieved. The most important requirement for the printer head was, that it should be lightweight. This keeps the size of the gantry, as well as the accelerated mass in the system small and thus ensures fast and precise operation. A lightweight construction was achieved by restricting the number of actuators in the printer head. Instead of actuating each of the needles individually, they are linked by a camshaft mechanism. The two sections of the camshaft can be controlled separately, thanks to two ratchet mechanisms. In order to get the starting position for the needles, two hall-sensors were integrated into the camshaft’s two separate parts.

Fig. 2. Printer Head Design

Work-in-Progress: “Smart Print Automation” Remote Lab and Cloud

323

4.2 T-Bot Gantry In order for the printer head to be able to print text with an array of only six needles, it needs a way of precisely moving up and down as well as left-to-right. Therefore, a two-dimensional gantry was developed (see Fig. 2). The gantry used for this system is called “T-Bot” and is made up of a cross-sled, which connects to linear axes. The two stationary motors are connected via a single timing belt and differentiate this kind of gantry from traditional systems. A “T-Bot” also features little moving mass that needs to be accelerated with each translation. However, the control of such systems is not entirely trivial. The individual axes of the T-Bot can be controlled by the combined movement of the stationary motors.

5 Software There are two software parts, one program on the PLC to move the motors for the X-Z axis and a motor to adjust the needles of the print head. The other software part is a Node-RED program that processes the data from the PLC. 5.1 PLC The PLC program was created in TIA Portal V17 and programmed using FBD (Function Block Diagram). For each motor a technology object was created in which the individual parameters for each motor are stored to enable precise travels. In order to print the FH logo, the three motors had to be coordinated with each other so that the line to be printed was always set before the printing movement was carried out. This interaction was made possible with the help of command tables. For each motor, the motion sequence was stored in a command table, consisting of forward and backward rotation of the motors and waiting times. When PLC program is executed each command table gets executed at the same time in order to guarantee a synchronous movement of the motors. 5.2 Edge Device The programs on the edge device were installed using Docker [7]. To use the functions of an edge device, the programs Node-RED, influxDB [10], Portainer [11] and Grafana [9] were installed. Portainer is used to manage all the different Docker Containers, influxDB can store the production data and Grafana can be used to visualize data in graphics. Node-RED is the communication tool between the PLC and the Raspberry PI. To enable OPC-UA communication between these two devices, an OPC-UA server must be created on the PLC in which all variables that can be accessed from other devices are defined. Each variable gets an individual Node-Id, which is required to access a specific variable. The Node-Id can then be used in the Node-RED program to read from a variable or to write something into the variable. Figure 3 shows a program section from the Node-RED program. In this figure it is shown how to read data periodically from the PLC into the Node-RED program. This can be used similarly for writing to the PLC. The data can then either be stored using influxDB or can be visualized using Grafana.

324

C. Madritsch et al.

Fig. 3. Read data from PLC part

6 Conclusions and Future Work The Smart Print Automation project is complete and can be used “stand alone”. The next part of the project will enable the system to be used as a remote lab. Currently the interface to our existing remote lab infrastructure is both configured and programmed. Students will be able to interact with the PLC part of the application by remotely using the TIA software development system, and with the edge device [8] part of the system by accessing the software running on the Raspberry Pi. One additional benefit will be the use of a Could Connector. This interface will make the data collected from the edge device available and usable (e.g. machine learning) through a simple web interface. The timeframe for completing these tasks is currently set for the end of the winter term, so that students will be able to use the system remotely next year. In the meantime, the system can be used offline and on-site during lab exercises in the Smart Automation lecture. This mobile technology showcase gives an outlook to what edge devises, edge computing and smart automation will be capable in future applications.

References 1. Sujatha, M., et al.: IoT and machine learning-based smart automation system for industry 4.0 using robotics and sensors. J. Nanomater. 2022(4) (2022) 2. Klinger, T., Kreiter, C., Pester, A., Madritsch, C.: Building a remote laboratory for advanced experiments in transmission line theory. In: International Conference on Interactive Collaborative Learning, Dubai, UAE (2019) 3. OPC Website. https://opcfoundation.org/about/opc-technologies/opc-ua/. Accessed 10 July 2023 4. Node-RED Website. https://nodered.org/. Accessed 28 June 2023 5. Siemens PLC Website. https://www.siemens.com/global/en/products/automation/systems/ industrial/plc.html. Accessed 4 July 2023 6. Lea. P.: IoT and Edge Computing for Architects: Implementing edge and IoT systems from sensors to clouds with communication systems, analytics, and security, 2nd edn. Packt Publishing (2020) 7. Docker Website. https://www.docker.com/. Accessed 10 July 2023 8. Madritsch, C., Langmann, R.: Education and Training for Automation 4.0 in Thailand – ETAT. In: IEEE Global Engineering Education Conference (EDUCON), Tunis, Tunisia (2022)

Work-in-Progress: “Smart Print Automation” Remote Lab and Cloud 9. Grafana Website. https://grafana.com/. Accessed 20 June 2023 10. Influxdb Website. https://www.influxdata.com/. Accessed 3 July 2023 11. Portainer Website. https://www.portainer.io/. Accessed 30 June 2023

325

Mobile Application Facilitating Agricultural Monitoring D. Delioglani(B) and A. Karakostas DRAXIS Environmental S.A., 54-56 Themistokli Sofouli Street, 54655 Thessaloniki, Greece [email protected]

Abstract. Mobile technology is rapidly advancing and has already applications in various sectors, including agriculture. The use of such technology brings numerous benefits to this specific sector, ranging from increased productivity to better decision-making concerning agricultural activities. In Europe, activities of this kind are monitored by the Common Agricultural Policy (CAP) which concerns, in particular, the aspects of farming, environmental protection, and food security. According to CAP, farmers are obliged to comply with certain requirements when carrying out their agriculture activities in order to increase operational efficiency and to prevent any irregularities that may occur. However, compliance with CAP is quite challenging and requires a lot of time due to the high numbers of farmers and, at the same time, the small numbers of inspectors. The mobile application presented in this paper aims to increase the operational efficiency of CAP monitoring by making it a less time- and resource- consuming procedure. Keywords: Mobile · Agriculture · Monitoring

1 Introduction The Common Agricultural Policy (CAP) is a set of policies and programmes set by the European Union (EU) for any aspect concerning agriculture [1]. It aims to ensure food production, support agricultural markets, promote development in rural areas, as well as protect the work of entrepreneurs of the agricultural sector in the EU member states. A new version of the CAP, CAP 2023-27, was launched on 1 January 2023, which addresses various challenges faced by EU’s agricultural sector, including sustainability, climate change, biodiversity conservation, and socio-economic resilience. Monitoring compliance with CAP is not an easy task, since addressing all the needs and requirements set by EU’s set of policies and programmes is quite challenging. The Land Parcel Identification Systems (LPIS) provide detailed digital geometries of agricultural reference parcels to aid the management of the CAP and are currently being released as open access data in numerous areas [2]. Farmers use the LPIS to declare their cropping practices, including specific environmental measures where relevant, in an annual aid application. The combined use of detailed reference parcel databases with satellite data facilitates near-real-time information gathering for a vast number of agricultural parcels. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 326–333, 2024. https://doi.org/10.1007/978-3-031-56075-0_31

Mobile Application Facilitating Agricultural Monitoring

327

However, information derived from the abovementioned processes do not suffice. In particular, in-situ data also need to be collected in order to train and validate the information extraction process. This can be challenging due to data unavailability until late in the growing season and data may also not be considered as the ground-truth. Furthermore, traditional in-situ ground-truth collection lacks the scale and possibility for automated integration into big data analyses and is prone to sampling errors. Nowadays, paying agencies make an effort to monitor and take decision in more parcels than one at a time. To this end, various data sources need to be taken into consideration, such as images from Sentinel satellites, Very High Resolution (VHR) Satellite imagery, Unmanned Aerial Vehicle (UAV) imagery, Street-Level Images (SLIs) and geotagged photos. Even by having in disposal all this information, learning how to demonstrate the results may be difficult both for inspectors and for farmers. End-users should understand that the procedure of field inspections is not mandatory and only when the available data do not lead to a conclusive result or when the authorities foresee that they will not be effective, should they be carried out. However, numerous disputes take place every year following the outcome of the eligibility checks on the farmers’ declarations. This paper describes the work that was conducted from the development of a mobile application aiming to improve CAP monitoring. Specifically, this application enables farmers and paying agencies to better and more efficiently monitor the compliance with CAP, by allowing them to upload information about the parcels of their interest (e.g., size, crop type, etc.), receive various indicators about the crop monitoring, and upload geotagged and timestamped photos of the parcels. An offline mode is also offered, allowing users with limited or no internet access, a frequent issue in rural areas, to utilise the application without having an active internet connection. All the data entered into the system during this mode are uploaded to the database of the mobile application, when the internet connection is restored.

2 Materials and Methods Collecting and analysing the requirements of a software solution are essential processes before initiating its technical development. That was also the case for this mobile application which primarily targets inspectors of paying agencies and farmers. To this end, the functional requirements of the application were collected from real end users of the agricultural sector. These acted as a basis for the design of the technical requirements and the system architecture of the application. The finalised functional and technical requirements of the mobile application are presented in the sub-sections below. It is worth noting that since the application is currently being pilot-tested, additional requirements may be presented which will be implemented in the final version (final system) of the mobile application in case they are deemed necessary. 2.1 Functional Requirements The application can be used by two user categories, namely CAP inspectors and farmers. The functionalities are similar for both, with the difference lying on the amount of

328

D. Delioglani and A. Karakostas

information to which each use category has access, i.e., farmers can only see information regarding their parcels, while inspectors can see parcel information for more than one farmer. The functional requirements were collected from end users through questionnaires and interviews. Based on acquired information, it was identified that when utilising the mobile application, the users should be able to: 1. Have a personal account to the mobile application, in order to upload and monitor parcel information; 2. Update the information of their personal account when necessary; 3. View all their parcels on top of a map, in order to have a more general overview of them; 4. View all their parcels also in a list, in order to see additional information about them; 5. Upload documents and photos through their mobile for a specific parcel, in order to showcase actions that have been performed on the field; 6. View the results of the following services per parcel, in order to monitor the status of each of them: a. Layers on top of the map, in order to monitor the vegetation status; b. Crop type identified by the crop classification service, in order to know if something is wrongly declared; c. Crop type that has been declared, in order to compare it with the identified one; d. Historical images, in order to see the results for the parcel; e. UAV image for a respective parcel, in order to be informed for the reason of the inspection or to help in better decision making regarding the parcel’s compliance to the rules; f. Information about the displayed legends, in order to understand what is displayed into the map. 7. Have at their disposal an application with fast performance; 8. Have at their disposal an application that can be used even without internet connection. 2.2 Technical Requirements and System Architecture The mobile application requires a robust technical foundation to fulfil the functional needs of both inspectors and farmers. It was developed using a client-server architecture, where the application serves as the client and interacts with the server-side components. The backend database is implemented using PostgreSQL, providing reliable data storage and management capabilities. PHP is utilised as the server-side language to handle logic and data processing, ensuring efficient and secure communication between the application and the server. The user interface of the application is developed using Vue.js, a powerful JavaScript framework for building dynamic and responsive frontend components. Geospatial data management is efficiently handled by a GeoServer, providing storage, retrieval, and serving capabilities for geographic data. Apache serves as the web server, responsible for hosting the application and handling HTTP requests, ensuring smooth connectivity and communication between the mobile app and the various server-side components. The application integrates with external services, and APIs provided by these services are utilised to fetch data, such as the NDVI (vegetation index), and UAV images. The

Mobile Application Facilitating Agricultural Monitoring

329

application handles the received data, performs necessary processing and analysis, and presents the results in a user-friendly manner within the interface of the app. Table 1 presents the data that are handled by the mobile application. Table 1. Data inputs and outputs of the mobile application. Data Inputs

Data Outputs

Parcel Photos: Users can upload photos of their parcels

Notification Alerts: Users receive real-time notifications when UAV photos of the parcels are uploaded. These notifications enable users to stay informed and take necessary actions

Shapefiles: Users can upload shapefiles Geolocation: The application requests users’ geolocation information when they upload a parcel photo. Geolocation data helps in associating the photo with the correct parcel location

Visualisation Outputs: The application displays visualisations, including maps with parcel overlays and representation of the NDVI

System Components The different system components of the mobile application are: • • • • • •

Web Servers: APACHE used as a web-server; Database Servers: PostgreSQL is utialised as back-end database; Source Code Frameworks: PHP with Laravel framework; Spatial Servers: GeoServer is used as spatial server; API Gateways: Laravel Sanctum; Reverse Proxies: APACHE along with NGINX Firewall Proxy are used to enhance security and performance.

The mobile application relies on Docker containers for its system components. These include the callisto-api container, which runs a PHP (Laravel) application served by Apache. It handles the logic and interacts with the database. The callisto-postgres container hosts the PostgreSQL database for data storage, while the callisto-geoserver container serves geospatial data and drone images through APIs. Together, these components enable efficient data processing, storage, and retrieval, enhancing the functionality and user experience of the mobile application. Block Diagram A block diagram representing the user-facing elements of the mobile application is presented below (Fig. 1). The available components interact with the Vue.js framework, which provides a flexible and reactive frontend development environment. The Vue.js framework communicates with the Apache server, responsible for handling HTTP requests and serving the frontend application to users. The Apache server acts as an intermediary layer between the frontend and backend components. The API represents all the backend components and handles the business logic and data processing of the mobile application. It interacts with the PostgreSQL database,

330

D. Delioglani and A. Karakostas

where data related to parcels, users, and other application entities are stored. The PHP Backend component retrieves data from the database and provides it to the frontend components as needed. Additionally, a GeoServer is integrated into the architecture to serve spatial data, such as maps and geospatial information, to the frontend. It interacts with the PostgreSQL database to access and retrieve relevant spatial data. It is important to note that the system architecture is built upon the utilisation of Docker. With Docker, each component can be encapsulated in a lightweight and isolated container, ensuring consistent execution environments in order to offer seamless deployment across different systems. Docker’s flexibility enhance the agility of managing the application’s infrastructure.

Fig. 1. Block diagram of the mobile application.

Overall, this block diagram showcases the flow and interaction of components in the delivery of the mobile application, from the user interface components to the underlying backend infrastructure, enabling an efficient user experience. User Interface The user interface of the mobile application consists of several components that enable users to interact with the system efficiently. Each component leverages a frontend library (vue.js) and technologies to provide a seamless user experience. The main components of the user interface and their corresponding frontend libraries/technologies are as follows: BaseMap: The BaseMap component, integrated into the user interface as the first page of the application, enables users to visualise their parcels on a map provided by the vueopen-layer library. It leverages the mapping capabilities of vue-open-layer to display a base map and user parcels. In addition, users are able to select a parcel in order to view additional information. Notifications: The notification component provides users with real-time updates related to their parcels. It is implemented using capacitor notification system, vue.js and vuetify

Mobile Application Facilitating Agricultural Monitoring

331

(CSS framework) to create a visually appealing and informative section within the user interface. Dashboard: The dashboard component serves as the main control centre for users. Users are able to change their password, delete or edit their profile information. The component employs Vue.js, along with Vuetify framework, to build a dynamic and responsive dashboard interface. Photo Upload, Parcel Upload: The file – parcel upload components allow users to upload shapefiles or photos related to their parcels. It utilises a capacitor camera functionality in order users to select and upload files from their devices. The component also uses Vue.js with Vuetify framework. NDVI Index The NDVI Index component allows users to view the NDVI for their parcels. It provides a visual representation of vegetation health and growth based on colours. Users can access the NDVI for their parcels and view the document provided by the administrator of the application (technology provider), which offers insights (info) about the corresponding parcel. The component also uses Vue.js with Vuetify framework. System Security The security of the mobile application aims to protect sensitive data and ensures user privacy. A couple of security measures have been implemented in order to avoid risks and maintain a secure environment. Authentication and Authorisation: The mobile application employs robust authentication mechanisms to verify the identity of users. User credentials are securely stored and transmitted using encryption techniques. Role-based access control is implemented with the use of Sanctum (Laravel library) to grant appropriate permissions to different user roles, such as farmers, inspectors and administrators (solely used by DRAXIS). Secure Communication: All communication between the frontend and backend components, as well as external services, is performed over secure channels using HTTPS protocols. This ensures the confidentiality of data during transit. Data Protection: Sensitive data, including user information, parcel details, and geolocation data, are stored in the PostgreSQL database. The database is configured with strict access controls and encryption to prevent unauthorised access and ensure data integrity. NGINX Proxy Firewall: The mobile application uses a NGINX proxy firewall as an additional layer of security. The NGINX firewall acts as a reverse proxy, handling incoming requests and filtering them based on predefined security rules. It helps protect the application from common web-based attacks, such as database injection and distributed (DDoS) attacks. Geolocation Privacy: The application requests geolocation information from users only when necessary and with their explicit consent. Geolocation data is securely transmitted and stored, according to privacy guidelines.

332

D. Delioglani and A. Karakostas

3 Results The mobile application presented in this paper offers various benefits to both user categories. On the one hand, through the mobile application, farmers are able to monitor their parcels status and assist them on their compliance with the CAP standards. Furthermore, farmers are able to upload geotagged and timestamped photos in order to prove their compliance with the standards whenever it is required. On the other hand, inspectors are empowered by a tool that provides them a depository of data that guides them for more targeted on-the-spot checks and help them to better determine the eligibility of farmers for CAP payments (Fig. 2). The application is available on PlayStore, making it accessible to any stakeholder who owns an Android smartphone.

Fig. 2. The mobile application.

4 Conclusions This mobile app aims to facilitate the procedure of CAP monitoring by making it lesstime and -resource consuming. Even though it targets primarily two user categories, i.e., farmers and CAP inspectors, additional stakeholders could also benefit from its capabilities (e.g., policy-makers, scientific community). At the time of submitting this paper, the application is pilot tested by real end users in rural areas of Greece and Cyprus. The feedback they will provide will play a key role in the final development stages of the mobile application.

Mobile Application Facilitating Agricultural Monitoring

333

Acknowledgments. This mobile application was developed by DRAXIS Environmental S.A. within the CALLISTO project which has received funding from the European Union’s Horizon 2020 research and innovation pro-gramme under grant agreement No 101004152.

References 1. European Commission. https://agriculture.ec.europa.eu/common-agricultural-policy/cap-ove rview/cap-glance_en. Accessed 13 June 2023 2. European Court of Auditors: The Land Parcel Identification System – A useful tool to determine the eligibility of agricultural land – but its management could be further improved. In: Special report No 25, Publications Office (2016)

Open-Source Based Remote Control of Thermo-Optical Plant uDAQ28/LT ˇ c´ık(B) and Katar´ına Z´ ˇ akov´ J´ an Sefˇ a Institute of Automotive Mechatronics Faculty of Electrical Engineering and Information Technology, Slovak University of Technology in Bratislava, Ilkoviˇcova 3, 841 04 Bratislava, Slovakia {jan.sefcik,katarina.zakova}@stuba.sk

Abstract. With the emergence of hardware reusability and the popularity of open source tools, this paper proposes a method of building an online laboratory utilizing tools that are completely built using open source software to teach control theory. This approach is then verified by implementing a real educational device (uDAQ28/LT) into this online laboratory setup.

Keywords: remote control open source software

1

· thermal-optical system · pysimCoder ·

Introduction

Many online laboratory setups rely on proprietary software tools for simulation, generating code, communication with real hardware, and creating block diagrams or user interfaces. Our idea is to replace this software with the small and lightweight open source tool pysimCoder [4], which can handle communication with real systems, block diagram design, and real-time simulation. We utilized a way to communicate and control lab devices on the Linux operating system remotely in real time. Until now, this was done using an online laboratory developed at our institute using MATLAB and Simulink software. When running a remote simulation, it takes more time to actually build and run the scenario. This time can be reduced by replacing Simulink with pysimCoder. Then we want to generalize the process of integrating and remotely controlling real laboratory hardware using software entirely based on open source.

2

Experimental System

The uDAQ28/LT (Fig. 1) thermo-optical system is an experimental laboratory device designed and developed for the purpose of system control education. It is suited to tasks including data collection, experimental system identification, signal processing, and various types of control. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2024  M. E. Auer and T. Tsiatsos (Eds.): IMCL 2023, LNNS 937, pp. 334–341, 2024. https://doi.org/10.1007/978-3-031-56075-0_32

Remote Control of the uDAQ28/LT

335

The system is built on proprietary hardware based on an Atmel ATMega8 16AI 8-bit AVR RISC microcontroller. Users have a choice of using the analog RS232 interface or the USB 2.0 interface to interact and communicate with the system. The serial channel’s baud rate required for correct communication with the device is 250,000 bit/s [1].

Fig. 1. The uDAQ28/LT, thermo-optical system.

Inputs: The uDAQ28/LT thermo-optical system has three settable inputs, each of which is represented by a separate internal component: – a light bulb, – a light-emitting diode (LED), – a fan. The main source of light and the primary heating element are provided by the bulb. The LED can serve as a potential second light source for the optical channel, or it can also be utilized as a disturbance signal in a control loop. The fan is used to lower the temperature in the heated chamber of the device, or it can also be utilized in a fan speed control loop. The operating voltage range for all input variables is 0 to 5 V. Outputs: The uDAQ28/LT plant is available in two different versions, depending on the number of system outputs. In general, the system has at most seven output variables: – – – –

the the the the

ambient temperature (only certain models), temperature in the heated space directly or filtered, light intensity directly or filtered, electric current, and the speed of the fan.

The temperature in the heated part of the system is measured by the PT100 sensor, and the ambient temperature is obtained by the ADT75 sensor. The system’s operating temperature interval is < 0, 100 > ◦ C. The light intensity variable value is provided by the system’s second LED, which is connected

336

ˇ c´ık and K. Z´ ˇ akov´ J. Sefˇ a

as a photo-diode light sensor/detector. There is no specific SI unit or physical quantity to which measurements of light intensity are calibrated. Only that the system is measuring more light intensity is indicated by a higher value. The filtered values of the uDAQ28/LT variables are the results of an embedded firstorder filter with a fixed time constant of 20 s, which cannot be modified.

3

Communication Interface

The uDAQ28/LT system uses a proprietary communication protocol defined by the manufacturer of the system. In general, the protocol is very simple. It is based on two string commands that are transferred via a serial channel. The way of interacting with the system via serial channel is customized for the Linux operating system. Writing values to system inputs is done by sending an input command, which must match the following format: ‘‘SA,B,C\n’’, where: – S is required first character of the input string; – A,B,C are integers in range from 0 to 255 separated by comma and representing input values for bulb, fan, and LED respectively; – \n or “Line Feed” is mandatory ending character of the input command. The system’s output data can be accessed from the uDAQ28/LT serial port following each write operation. It is not possible to repeatedly read data from the device serial port without a prior writing operation, because the read operation is destructive and empties the device’s output buffer. The information from the uDAQ28/LT system output text string is formatted as ‘‘T fT I fI AMP RPM\n’’ or ‘‘ambT T fT I fI AMP RPM\n’’ based on the system’s version. The following describes the significance of each value in the output string: – – – –

ambT is ambient temperature value (only certain models), T, fT are temperature and its filtered value in the heated space of the system; I, fI are light intensity and its filtered value; AMP, RPM are electric current drawn by the fan and speed of the fan.

An example output text message from the uDAQ28/LT serial port could look as follows: ‘‘225 869 860 1833 1820 2514 8360\n’’. The raw numbers in the output string have specific boundaries. According to the uDAQ28/LT manufacturer, the temperature, light intensity, and fan current values are in range < 0, 4096 >. Fan speed value is in range < 0, 10000 >, and ambient temperature value is in range < 0, 1024 >. These values have to be adjusted to the correct boundaries, according to the device manual.

4

Simulation Software

The pysimCoder [4] is an open-source rapid prototyping tool that can be used to create code that runs in real time and may be utilized in a number of scenarios.

Remote Control of the uDAQ28/LT

337

The application itself is built upon the PySimEd project, the qtnodes project, and the extended Python control library. The behavior and execution of the software resemble those of other applications like Simulink or Xcos. Although real-time code generation is the primary function, pysimCoder is also feasible to model and run straightforward block diagrams and simulations in either continuous or discrete time thanks to the built-in graphical block editor. Since September 2022, pysimCoder has been able to construct both single-block and multi-block subsystems, and it replaces the old XML-based file format with a new, easier-legible JSON-based block file storage format [4]. 4.1

Application Interface

The pysimCoder has an easy-to-use graphical user interface. The application has two primary windows (Fig. 2), one of which shows a library of potential building blocks that users can define and use in line with their needs, and the other of which is a workspace for designing block diagrams. The workspace window also contains controls for starting the simulation as well as for real-time code generation, which may be later executed outside of the pysimCoder application.

Fig. 2. The pysimCoder graphical user interface has a block library and workspace.

4.2

Platform Specification

The tool is developed for the Linux OS, but it can be installed on a Windows PC thanks to the Windows Subsystem for Linux (WSL2), available in Windows 10 and Windows 11 OS, using a Docker image of pysimCoder, which can run in a browser. Currently, real-time code can be generated for embedded systems supported by the Apache NuttX RTOS, the Nucleo-H745ZI-Q (STM32H7) board, the BRIKI board, and the Linux operating systems on PC and Raspberry Pi (either using a preempt RT patch or not) [5]. Since the source code for library blocks is written in C, the application’s modular design makes it possible to specify and implement more development boards or microcontrollers.

338

4.3

ˇ c´ık and K. Z´ ˇ akov´ J. Sefˇ a

PysimCoder Block for uDAQ28/LT

Previous MATLAB and Simulink implementations of uDAQ28/LT [2,7] served as the base for the new block for pysimCoder. In contrast to earlier versions of system implementations, a single C language file defines functionality and block operations, including device initialization, I/O activities, and secure device serial port termination in pysimCoder. Four methods are invoked in the C file’s main function during the simulation: init, which initializes the block and the system; inout, which performs input and output operations on the system and the block; modify, which changes the block’s parameters and internal state; and end, which ends work with the block. Initialization: During the initialization operation, the serial port of the device is opened, the proper baud rate is set, and the serial channel is properly configured for writing and reading text commands. The uDAQ28/LT system requires a non-standard baud rate of 250 kbit/s for proper communication, which is set using the BOTHER flag of the termios2 structure in the Linux operating system (see Listing 1.1). 1 2 3 4

tty . c_cflag &= tty . c_cflag &= tty . c_cflag |= tty . c_ispeed =

~ CBAUD ; tty . c_cflag |= BOTHER ; ~( CBAUD