Information Technology and Systems: ICITS 2023, Volume 2 (Lecture Notes in Networks and Systems, 692) 3031332601, 9783031332609

Citation preview

Lecture Notes in Networks and Systems 692

Álvaro Rocha Carlos Ferrás Waldo Ibarra   Editors

Information Technology and Systems ICITS 2023, Volume 2

Lecture Notes in Networks and Systems Volume 692

Series Editor Janusz Kacprzyk , Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Fernando Gomide, Department of Computer Engineering and Automation—DCA, School of Electrical and Computer Engineering—FEEC, University of Campinas— UNICAMP, São Paulo, Brazil Okyay Kaynak, Department of Electrical and Electronic Engineering, Bogazici University, Istanbul, Türkiye Derong Liu, Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, USA Institute of Automation, Chinese Academy of Sciences, Beijing, China Witold Pedrycz, Department of Electrical and Computer Engineering, University of Alberta, Alberta, Canada Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Marios M. Polycarpou, Department of Electrical and Computer Engineering, KIOS Research Center for Intelligent Systems and Networks, University of Cyprus, Nicosia, Cyprus Imre J. Rudas, Óbuda University, Budapest, Hungary Jun Wang, Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong

The series “Lecture Notes in Networks and Systems” publishes the latest developments in Networks and Systems—quickly, informally and with high quality. Original research reported in proceedings and post-proceedings represents the core of LNNS. Volumes published in LNNS embrace all aspects and subfields of, as well as new challenges in, Networks and Systems. The series contains proceedings and edited volumes in systems and networks, spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, Aerospace Systems, Automation, Manufacturing, Smart Grids, Nonlinear Systems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution and exposure which enable both a wide and rapid dissemination of research output. The series covers the theory, applications, and perspectives on the state of the art and future developments relevant to systems and networks, decision making, control, complex processes and related areas, as embedded in the fields of interdisciplinary and applied sciences, engineering, computer science, physics, economics, social, and life sciences, as well as the paradigms and methodologies behind them. Indexed by SCOPUS, INSPEC, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science. For proposals from Asia please contact Aninda Bose ([email protected]).

Álvaro Rocha · Carlos Ferrás · Waldo Ibarra Editors

Information Technology and Systems ICITS 2023, Volume 2

Editors Álvaro Rocha ISEG University of Lisbon Lisbon, Portugal

Carlos Ferrás Facultade de Geografía e Historia University of Santiago de Compostela Santiago de Compostela, Spain

Waldo Ibarra Departamento de Informática Universidad Nacional de San Antonio Abad del Cusco Cusco, Peru

ISSN 2367-3370 ISSN 2367-3389 (electronic) Lecture Notes in Networks and Systems ISBN 978-3-031-33260-9 ISBN 978-3-031-33261-6 (eBook) https://doi.org/10.1007/978-3-031-33261-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023, corrected publication 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This book is composed by the papers written in English and accepted for presentation and discussion at The 2023 International Conference on Information Technology & Systems (ICITS’23). This conference had the support of Universidad Nacional de San Antonio Abad del Cusco (UNSAAC), Information and Technology Management Association (ITMA), IEEE Systems, Man, and Cybernetics Society, and Iberian Association for Information Systems and Technologies (AISTI). It took place in Cusco, Peru, April 24–26, 2023. ICITS’23 is an international forum for researchers and practitioners to present and discuss the most recent innovations, trends, results, experiences, and concerns in the several perspectives of Information Technology & Systems. The Program Committee of ICITS’23 was composed of a multidisciplinary group of 261 experts and those who are intimately concerned with Information Systems and Technologies. They have had the responsibility for evaluating, in a ‘doubleblind review’ process, the papers received for each of the main themes proposed for the conference: A) Information and Knowledge Management; B) Organizational Models and Information Systems; C) Software and Systems Modeling; D) Software Systems, Architectures, Applications, and Tools; E) Multimedia Systems and Applications; F) Computer Networks, Mobility, and Pervasive Systems; G) Intelligent and Decision Support Systems; H) Big Data Analytics and Applications; I) Human– Computer Interaction; J) Ethics, Computers & Security; K) Health Informatics; L) Information Technologies in Education; and M) Media, Applied Technology, and Communication. ICITS’23 received 362 contributions from 31 countries around the world. The papers accepted for presentation and discussion at the conference are published by Springer (this book) and by RISTI and will be submitted for indexing by WoS, EI-Compendex, SCOPUS, and/or Google Scholar, among others.

v

vi

Preface

We acknowledge all of those that contributed to the staging of ICITS’23 (authors, committees, workshop organizers, and sponsors). We deeply appreciate their involvement and support that was crucial for the success of ICITS’23. Cusco, Peru

Álvaro Rocha Carlos Ferras Sexto Waldo Ibarra

The original version of the book was revised: New chapter 49 has been included as the last chapter. The correction to the book is available at https://doi.org/10.1007/978-3-031-33261-6_50

Contents

Information Technologies in Education The Application of Multi-integration Teaching Mode Based on Cloud Computing in Computer Professional Teaching . . . . . . . . . . . . . Maohong Zhang, Jing Zhao, Hong Zhang, and Fuqiang Yang

3

Impact of Fourth Industrial Revolution-Related Technologies on Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olutayo O. Oyerinde and Ada M. Dienga

15

Improving Student Placement in Schools: The Case of a School Group in Portugal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sónia Sousa and Jorge Coelho

29

Intelligent Tutoring System for Teaching and Learning Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Francisco Niño-Rojas, Diana Lancheros-Cuesta, Martha Tatiana Jimenez-Valderrama, Sergio Eduardo Gómez Ardila, and Gelys Igreth Mestre Carrillo A Serious Game for Recycling Education in Peruvian Children . . . . . . . . Paul Tocto, Guillermo Dávila, and Victor Ayma University Degree Completion as an Indicator of Effective Institutional Quality Management in Higher Education . . . . . . . . . . . . . . . Claudio Ruff, Marcelo Ruiz, Paola Juica, Bastián Gutiérrez, and Alexis Matheu Scientific Research and Writing Skills in Engineering Students . . . . . . . . Rosa Núñez-Pacheco, Elizabeth Vidal, Aymé Barreda-Parra, and Eveling Castro-Gutierrez Topic Modelling for Automatically Identification of Relevant Concepts Discussed in Academic Documents . . . . . . . . . . . . . . . . . . . . . . . . . Segarra-Faggioni Veronica, Ratté Sylvie, and Jong De Frank

39

49

61

75

85

vii

viii

Contents

Covariational Reasoning in an Approach to the Concept of Definite Integral Using GeoGebra Within an Economic Context . . . . . . . . . . . . . . . Mihály André Martínez-Miraval, Martha Leticia García-Rodríguez, and Daysi Julissa García-Cuéllar

97

The Mobile Application in Learning for People with Special Hearing Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Javier Sánchez-Guerrero, Andrés Haro-Velasteguí, Sandra Carrillo Ríos, and Jaime Ruiz Localizing a Mathematics Tutoring System to Spanish in Latin-America . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Ivon Arroyo, Andrew Euredjian, Luisa Perez Lacera, Danielle Allessio, Waldo Geremías, and Beverly P. Woolf Educational Data Mining to Forecast Essay Score. A Case Study About ENEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Rian Bessa Lopes, Leonardo Pio Vasconcelos, Igor Garcia Ballhausen Sampaio, Leandro Miranda, José Viterbo, and Daniela Gorski Trevisan The Impact of Digital Game Narratives on Environmental Awareness: An Efficiency Assessment of Game Narratives Created by Young Adults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Tanira M. H. Suandique, Pedro Beça, and Monica Aresta Hybrid PBL and Learnability for Computer Science and Multimedia Degrees: How to Trigger Soft Skills Strategies for Personal and Professional Development? . . . . . . . . . . . . . . . . . . . . . . . . . 157 Dulce Mourato and Paula Amaro Mobile Learning and English Reading Abilities in Postgraduate Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Kevin Mario Laura-De La Cruz, Lesly Joselin Apayco-Zavala, Miliam Quispe-Vargas, Silvia Milagritos Bazan-Velasquez, Cecilia Claudia Montesinos-Valencia, and Bertha Silva Narvaste Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision Through Unlearning Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Juan-Gabriel Cegarra-Navarro, Anthony Wensley, and Jorge Cegarra-Sánchez Media, Applied Technology and Communication Applying Machine Learning Techniques to Box Office Forecasting . . . . . 189 Rian Bessa Lopes and José Viterbo

Contents

ix

Attitudes, Use, and Knowledge of ICT and Student Satisfaction as Predictors of the Level of Learning Achievement . . . . . . . . . . . . . . . . . . . 201 Ygnacio Tomaylla-Quispe, Mónica Mogrovejo-Chicani, Julia Arpi-Titi, and Olger Gutierrez-Aguilar Influential Factors in Extraversion Mediated by Reputation in Social Networks in University Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Olger Gutierrez-Aguilar, Ygnacio Tomaylla-Quispe, Aleixandre Brian Duche-Pérez, and Sandra Chicana-Huanca Machine Learning for the Strengthening of Ecuadorian Sign Language Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Jeneffer Barberán-Moreira, Miguel Mantuano-Casual, Willian Ocampo-Pazos, Angel Sabando-García, and Fausto Orozco-Iguasnia Organizational Models and Information Systems Towards the Evaluation and Continuous Evolution of Open Government Data Portals: A Framework Proposal . . . . . . . . . . . . . . . . . . . 243 Castelar Costa Junior, Flavia Cristina Bernardini, Guido Silva Vaz, José Viterbo, Vanessa Nunes, and Claudia Cappelli IT Governance in Brazilian Public Healthcare Sector . . . . . . . . . . . . . . . . . 255 Roberto Luiz Silva dos Santos, Sandro Luís Freire de Castro Silva, Sandra Regina Freitas da Silva Morgado de Gois, Jose Geraldo Pereira Barbosa, and Antonio Augusto Gonçalves Virtual Learning Networks in the Learning of a Foreign Language: Remote Education Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 Jehovanni Fabricio Velarde-Molina, Stefany Juliana Noa-Copaja, Kevin Mario Laura-De La Cruz, Silvia Milagritos Bazan-Velasquez, Domingo Nicolás Pérez-Yufra, and Giomar Walter Moscoso-Zegarra Understanding the Success Factors of Research Software: Interviews with Brazilian Computer Science Academic Researchers . . . . 275 Erica Mourão, Daniela Trevisan, and José Viterbo Software Systems, Architectures, Applications and Tools Server-side Adaptive Federated Learning over Wireless Mesh Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Felix Freitag, Lu Wei, Chun-Hung Liu, Mennan Selimi, and Luís Veiga Mobile Development with Xamarin: Brief Literature, Visualizations and Important Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Renato M. Toasa, Paúl Francisco Baldeón Egas, Henry Recalde, and Miguel Gaibor Saltos

x

Contents

AI-enchanced Crowdsourcing as an Element of Information Systems Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Vaidas Giedrimas, Jurgita Lieponiene, Birute Ragalyte, Henrikas Sinickas, Jurgita Paulaviciene, Audrius Valinskas, Edgaras Sriubas, and Alma Paukstiene User’s Appreciation About Software Prototyping: A Case Study . . . . . . . 319 Pedro-Saul Delgado, Wilmer Bermudez, Brian Pando, and Ronald Ibarra Predicting the Number of Pedestrians per Street Section: A Detailed Step-by-step Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Christophe Guyeux Information Technology Monitoring in Healthcare: A Case Study . . . . . . 351 Vasco Ramos, Carolina Marques, Hugo Peixoto, and José Machado Quantum Computing and Lawful Interception Applications . . . . . . . . . . . 363 Kourtis Michail-Alexandros, Xilouris George, Chochliouros Ioannis, and Kourtis Anastasios Log4j Vulnerability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Pedro Ferreira, Filipe Caldeira, Pedro Martins, and Maryam Abbasi Integrating Security and Privacy Mechanisms with Fast Health Interoperability Resources (FHIR), a Scoping Review . . . . . . . . . . . . . . . . . 387 João Pavão, Rute Bastardo, and Nelson Pacheco Rocha An Approach for Visualizing SVG Sprites Systems . . . . . . . . . . . . . . . . . . . 397 Nikolay Buhtiyarov and Daniela Borissova A Software System to Support Student Engagement in Academic Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 Darina Dicheva, Keith Irwin, and Christo Dichev Virtual Reality and the Cognitive Screening of Older Adults: A Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Rute Bastardo, João Pavão, Bruno Gago, and Nelson Pacheco Rocha Machine Learning Applications that Might Support Home-Based Screening of Parkinson’s Disease: A Scoping Review . . . . . . . . . . . . . . . . . . 431 Catarina Abreu, Gonçalo Costa, João Tiago Oliveira, Vlada Hanchar, and Nelson Pacheco Rocha Threat Modeling to Design a Decentralized Health Data Management Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 Christina Erler, Shanshan Hu, Alexa Danelski, Wilhelm Stork, Ali Sunyaev, and Martin Gersch Software Application to Assist the Publishing Sector: A Tool in MS Excel Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 Daniela Borissova, Zornitsa Dimitrova, and Nina Keremedchieva

Contents

xi

Web Application with Facial and Emotional Recognition for the Control of Human Talent at the “España” Institute of Ambato-Ecuador . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 Fernández Villacrés Gustavo Eduardo, Navas Olmedo Walter Humberto, Reyes Bedoya Donald Eduardo, Pazmay Pazmay Pablo David, and Arcos Naranjo Gissela Alexandra Application of Sonification Method in Teaching Information Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Andrey Vishnevsky and Nadezda Abbas Analysis of Oversampling Techniques and Machine Learning Models on Unbalanced Spirometry Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Roberto Castro Izurieta, Nadia N. Sánchez-Pozo, Juan S. Mejía-Ordóñez, Juan González-Vergara, Luz Marina Sierra, and Diego H. Peluffo-Ordóñez On the Use of Social Robots for Rehabilitation: The Case of NAO Physio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Larissa Rodrigues da Costa, Jaelson Castro, Cinthya Lins, Judith Kelner, Maria Lencastre, and Óscar Pastor Automating Risk Stratification Processes in Obstetric Emergency: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519 Aline Silva de Lima, Lucas dos Reis Cardoso, Elaine Flavio Rangel Seixas, Mônica da Silva, Flavio Seixas, Luciana Salgado, and José Viterbo A Theoretical Journey on Health and Education: Main Aspects and Advances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 Diego Bustamante Yánez, Alex Criollo Rodríguez, Iruma Alfonso González, and Hector F. Gomez A Design of a Video Game Applying a Layered Architecture Based on the Unity Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 Aguas Luis, Recalde Henry, Toasa Renato, and Salazar Elizabeth A Complementary Approach for Smart Contract Design Using DasContract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 Cristóbal Marinkovic, Julio Fenner, Óscar Ancán, and Carlos Cares A Customized and Low-Cost Antivirus Solution Focused on User Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 Aguas Luis, Camero Rosa, Toasa Renato, and Arico Lisbeth Storefront Business Model Enhancement by Using Georeferenced Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 Jean Carlo Jativa, Pilar Urrutia-Urrutia, Fernando Ibarra-Torres, and Félix Fernández-Peña

xii

Contents

Correction to: Information Technology and Systems . . . . . . . . . . . . . . . . . . Álvaro Rocha, Carlos Ferrás, and Waldo Ibarra

C1

Information Technologies in Education

The Application of Multi-integration Teaching Mode Based on Cloud Computing in Computer Professional Teaching Maohong Zhang , Jing Zhao, Hong Zhang, and Fuqiang Yang

Abstract With the development of Internet + and cloud wisdom, the teaching mode of universities has changed greatly. Among them, online and offline hybrid teaching method is widely used in college teaching. Through analyzing the current teaching methods, the application of multi-integration teaching mode based on cloud computing in Computer Professional Teaching is put forward. Under the condition of existing teaching, using cloud computing technology, combining with the characteristics of cloud service platform, and constructing a learning environment. From the three main links of project driven theoretical teaching, cascading experimental teaching, and independent content expansion teaching, a cloud computing diversity auxiliary teaching platform of “Principles of Computer Organization” is built, to improve students’ interest in learning initiative, and to cultivate students’ logical analysis and reasoning ability. Through analysis and verification, this teaching mode integrates the concept and advantages of MOOC, effectively changes the traditional teaching mode of pure classroom teaching and the phenomenon of students’ cramming in listening, makes up for the shortcomings of the online and offline hybrid teaching, and improves the training mode in the computer professional education system, and achieves efficient teaching effect and teaching quality. Keywords Cloud wisdom · Multi-integration · Cloud computing · MOOC

1 Introduction In recent years, in order to realize the informatization, networking and intelligence of education, the utilization rate of hardware resources is extremely low, which has been unable to meet the actual demand of teaching. Cloud computing services have changed the development of education informatization [1]. Cloud computing has the characteristics of stronger computing power, M. Zhang · J. Zhao (B) · H. Zhang · F. Yang Shandong Women’s University, Jinan 250300, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_1

3

4

M. Zhang et al.

stronger data security, easier group cooperation ability and strong compatibility. Integrating cloud computing technology into the teaching platform not only reduces costs, but also benefits sharing and effectiveness of teaching. Teaching cloud is the migration of cloud computing technology in teaching, and is the infrastructure of future teaching. It serves students in the form of computing resources, and provides corresponding education resources for teachers and various educational institutions. Based on the training objectives of computer science and technology professionals, focusing on cultivating students’ problem-solving ability, and taking teachers and students as the main body, the teaching reform and Practice Research of computer professional courses is carried out from the aspects of teaching content, teaching methods, teaching practice and quality evaluation [2, 3], Based on constructivism learning theory, humanistic learning theory and ubiquitous learning theory, and a multi-integrated teaching model based on cloud computing is put forward, its core based on classroom teaching and network quality courses, integrating MOOC concepts and advantageous elements, and reflecting multiple characteristic values [4], which is helpful for the development and extension of hybrid teaching mode. Cloud computing provides more advanced technology and services for multi-integration teaching, which plays an obvious role in promoting students to carry out various advanced cognitive activities and cultivating group wisdom.

2 Cloud Computing With the rapid development of computer science and network technology, as an emerging production, cloud computing has brought revolutionary impact to the IT industry [4]. Technical logic and business structure of the whole society have been reconstructed [5], and cloud computing has changed traditional single computer function. At present, the development of cloud computing technology is gradually mature and widely used, cloud computing service mode is one of the characteristics of cloud computing and traditional IT, cloud computing platform saves a great amount of cost for users, and reduces the investment cost for enterprises.

2.1 Cloud Computing Concept Cloud computing is the product of the integration of computer, information technology and Internet, also known as grid computing. Cloud computing is a service mode of payment on demand, which can provide dynamic, easy to expand and virtualized resources through the Internet. At present, the widely accepted concept of cloud computing is defined by the National Institute of Standards and Technology (NIST) [6]: Cloud computing is a way to pay by actual usage. This approach provides users with available, convenient, on-demand network access, and enters the configurable computing resource sharing

The Application of Multi-integration Teaching Mode Based on Cloud …

5

pool, Which can be provided quickly, just put it into little management work, or small interactions with service providers. Cloud computing provides users with a new experience. The core of cloud computing is to coordinate many computer resources. Therefore, users can obtain unlimited resources through network, and resources obtained are not limited by time and space [7].

2.2 Analysis of Cloud Computing Model The core idea of cloud computing is service. Therefore, the service realized through cloud computing is called cloud service. Cloud computing mode can be divided into public cloud, private cloud, hybrid cloud and industry Cloud (community cloud) according to deployment mode [8]. Among them, public cloud is a cloud that provides services for external customers, and all its services are for others to use, rather than for their own. Private cloud is a cloud used by an enterprise itself, and all its services are not used by others, but for the internal use of the enterprise. Hybrid cloud is a kind of cloud based on the combination of public cloud and private cloud. Industry cloud (community cloud) is a customized public cloud for specific industries. Large enterprises have more deployment of private cloud, public cloud, and hybrid cloud, while small enterprises mostly adopt deployment of public cloud, private cloud and industry Cloud (community cloud). Cloud computing model can be divided into IaaS (infrastructure as a service), PaaS (platform as a service) and SaaS (software as a service) according to the service model [9], which correspond to the “hardware”, “platform” and “software” in traditional IT respectively. Since PaaS is in the middle of the cloud computing industry, IaaS providers can provide PaaS services by relying on hardware and technology advantages and SaaS providers can also provide PaaS services for large customers by relying on customer relations. Pure PaaS providers are not competitive. Therefore, the opportunities for IaaS and SaaS are greater in the cloud computing industry.

2.3 Teaching Platform Architecture Based on Cloud Computing The teaching platform based on cloud computing provides rich teaching resources for teachers and students, and teachers can use the cloud platform to design diversified teaching activities flexibly and effectively. Through the real-time tracking and recording of learning behaviors on the cloud platform, and the analysis of learning data, teachers can timely master students’ learning progress, understand students’

6

M. Zhang et al.

Fig. 1 Architecture diagram of teaching platform based on cloud computing Platform user

Cloud computing service platform

SaaS

Cloud computing basic platform

PaaS

Infrastructure platform

Iaas

mastery of various knowledge points, and achieve targeted teaching. Architecture diagram of teaching platform based on cloud computing (see Fig. 1).

3 Multi-Integration Teaching Mode The multi-integration teaching mode, as according to professional training objectives, and formulate demand for training professional talents, formulate specific subject teaching plans, integrate curriculum system, eliminate outdated knowledge, Integrate content that is repeated with other courses, Improve gold content of courses and innovate teaching methods; Multiple integration teaching mode is also a thinking mode of integrating multiple teaching methods. Diversified texts are integrated into the teaching process of professional courses, and fully integrates MOOC, SPOC, online and offline hybrid teaching methods. Therefore, the design takes classroom as the main line, cloud computing as the platform, It combines online and offline hybrid teaching method, a diversified teaching mode of “MOOC + SPOC + online discussion + offline confirmation summary” is carried out, and drive students’ autonomy by project, and realize a diversified teaching mode of course learning that runs through before class, during class and after class. Evaluates teaching results through practice, and improves teaching effects, and improves teaching methods and improves teaching scheme so as to improve students’ application ability and comprehensively enhance their professional quality.

4 Multi-integration Teaching Mode Based on Cloud Computing With the rapid development of computer network, the application of cloud computing is born. In the environment of cloud computing, cloud computing is introduced into professional teaching, and the traditional teaching mode is changed, and realized a new online classroom teaching mode of cloud computing assisted teaching. The cloud computing platform is combined with the multi-integrated teaching mode to

The Application of Multi-integration Teaching Mode Based on Cloud …

7

realize personalized information teaching, assist teachers and students in teaching and learning, promote the communication between teachers and students, and improve the teaching efficiency and learning quality. And there is no need to consider the storage space of teaching resources. Multi-integrated teaching mode based on the cloud computing, with the characteristics of both open and hybrid teaching mode, combines traditional classroom teaching with network teaching organically. This paper takes “Principle of Computer Organization” as an example to formulate Multi-integrated teaching mode based on the cloud computing. The specific design of the whole course includes three stages: teaching guidance, main activity, teaching improvement and reflection, to provide students with accurate learning resources, cultivate independent learning ability, and improve the teaching quality and effect of computer professional courses [4].

4.1 Construction of Multi-integration Teaching Model Based on Cloud Computing The teaching goal of computer professional courses is to organically combine professional theoretical knowledge with practice and how to guide students to apply professional knowledge in practical learning and work has always been the focus of computer teaching. In the practice teaching of professional courses, let students understand the application field of computer courses, guide students to analyze and solve problems in a targeted way, and cultivate students’ ability to discover problems, analyze problems, and solve problems. The course of “Principles of Computer Organization” mainly describes the working principles of computer hardware and the principles of numerical calculations, which occupies an important position in the training program of computer science and technology. Through the course learning, students can not only master the basic design techniques of computer systems, but also cultivate their ability to discover, analyze, and solve problems. However, the content of “Principles of Computer Organization” course in theoretic and abstracted, teaching information coverage, and Teaching content is updated quickly, and it is seriously out of touch with practice, which makes the application not specific enough. Therefore, the course gives students and teachers a dilemma of “difficult to learn and difficult to speak”. This phenomenon makes “Principles of Computer Organization” curriculum development bottleneck, and it has also made the development of computer science and technology professional encounter inevitable embarrassment [10]. Therefore, how to make the students of computer science and technology master the knowledge of this course has become an urgent problem. At the same time, due to insufficient capital investment in many schools and insufficient attention to this course, the teaching resources and teaching platform are not perfect. As a result, the teaching method of this course is still based on traditional

8

M. Zhang et al.

classroom teaching, focusing on theory rather than practice, classroom interaction ability between teachers and students is weak, students’ initiative is not high, learning effect is poor. Based on the above reasons, combined with the actual job requirements and expectations for the design ability of computer graduates, the teaching mode of “Principles of Computer Organization” is reformed. Under the existing teaching conditions, the use of cloud computing technology in the field of education informatization [11], from the three main links of project-driven theory teaching, cascading experimental teaching, and independent content expansion teaching Curriculum teaching reform. Combining the characteristics of cloud service platform to construct a learning environment, and construct a multi-integrated teaching model for the course “Principles of Computer Organization” under the cloud computing-assisted teaching platform to assist teaching, and improve students’ initiative learning interest, and cultivates students’ comprehensive quality such as logical thinking and analysis ability.

4.2 Teaching Scheme of “Principles of Computer Organization” (1) Design of Teaching Content and Teaching Scheme Formulate the teaching syllabus, teaching content, learning resources and teaching activities for undergraduate students majoring in computer science and technology. First, use heuristic and exploratory teaching methods to stimulate students’ innovation ability, cultivate students’ collaborative ability, and further improve their ability to discover, analyze and solve problems; Second, use practice to cultivate students’ operational ability, comprehensively improve students’ comprehensive quality, accumulate students’ innovative ability, test students’ learning effects, feedback teaching quality through teaching evaluation, and further improve teaching scheme. Third, based on the cloud computing platform to explore and construct the teaching model of course “Principles of Computer Organization”; analyze the characteristics of college students in the new era, design and rectify the existing classroom teaching mode, and rationally integrate cloud computing technology with the existing teaching mode. Four, arouse the enthusiasm of students’ study. Teachers can improve students’ initiative learning ability through guidance, make them think actively and participate in the scope of knowledge autonomously, and improve their personal expression and confidence. (2) Design of Teaching Resources. The design of teaching resources mainly considers network resources and classroom resources. Network resources students mainly obtain learning resources through Zhi

The Application of Multi-integration Teaching Mode Based on Cloud …

9

Huishu, MOOC online resources, icourses and other network resources. On this basis, in the teaching design, in the teaching design, the introduction questions are added to guide students to seek answers through help of online resources. Comprehensive teaching design is carried out for the course “Principles of Computer Organization”, which sets 43 knowledge points in 64 class hours. The teaching design is updated at any time according to students’ learning situation. Before class, teachers release learning tasks according to knowledge points, students preview before class, and watch videos online. Students can use the cloud platform to study independently, take online tests, ask questions, interact and discuss with teachers and classmates online. According to the students’ viewing situation, spot check the students and ask them questions, which are included in the students’ usual grades. On this basis, introductory questions are added to classroom teaching to guide students to further in-depth study and analysis through online resource learning. Classroom teaching is a deterministic learning of online learning resources through in the classroom. According to online learning and classroom learning, teachers further adjust the teaching plan, assign corresponding homework to extend students’ knowledge, improve their learning ability and innovation ability, and assign preview tasks for the next class. (3) Design of Teaching Methods First, the positioning method is from surface to point and then from point to surface; Second, it organically combines task-driven, autonomous teaching and hierarchical module construction, and adopts heuristic, case, autonomous and cascading teaching methods [12]. Third, it adopts the task-driven theory to organize teaching and evaluate the teaching effect, and constructs an offline and online hybrid teaching model based on cloud platform technology, which can effectively improve the teaching quality. Four, we should pay attention to the students’ dominant position, promptly discover the problems encountered by students in the learning process, let students find their own position, display and develop their potential, reflect themselves, strengthen their confidence in learning, and improve the comprehensive quality of teaching. (4) Design of Teaching Evaluation Based on Multi-integration Construct a multi-directional evaluation mechanism of theory and practice. The evaluation is mainly about the mastery of theoretical knowledge and practical skills. Therefore, in the final evaluation of students, in addition to the evaluation of the final examination paper, it also adds evaluation elements such as academic paper or report evaluation and work argumentation defense.

10

M. Zhang et al.

The evaluation elements of academic papers or reports are mainly carried out in the form of academic papers. Adopt group discussion to demonstrate academic arguments, mainly through academic papers or reports. In the early stage of learning, teachers will give course content knowledge points, and students are free to choose to form groups, about 4 to 6 people in each group, and then in the process of learning this course, members of each group collect materials on the selected knowledge points, discuss the knowledge points of the demonstration, and finally comment and summarize in the form of published academic papers or reports. This process not only enables students to master the knowledge points of the course, but also trains students’ ability of essay writing ability. Before the end of the course, a final defense of the thesis or work prepared by the students is conducted, which mainly examines the students’ mastery of the course content [13], the level of understanding and the extension of the course content, and other knowledge points, and it trains students to have a direct experience of how to face the defense. In addition, in the evaluation method, the subject of evaluation has been changed. The original subject objects of evaluation is teachers’ teaching, and now it has become a subject of diversified evaluation mainly composed of teachers and students. The implementation of the specific plan is arranged in the defense stage before the end of the class. Teachers formulate defense evaluation standards. Teachers and students participate in the defense scoring at the same time. During the defense process, for the controversial problems, the discussion type on-site is adopted to deepen the students’ understanding of the knowledge they have learned, at the same time, arouse the enthusiasm of students learning [14], logical thinking ability and expression ability. In the final examination, it examines students’ mastery of theoretical knowledge, with a full score of 100; the evaluation elements of academic papers or reports evaluation and works argumentation and defense are the evaluation of students’ practical ability, which is regarded as the assessment of the usual performance of the course, with a full score of 100; The total score of this course is 100, The ratio of final exam score to usual score is 1:1.

4.3 Construct Cloud Computing Platform Model Under the Multi-Integrated Teaching Mode Combining the characteristics of cloud service platform to construct a learning environment, and construct a multi-integrated teaching model for the course “Principles of Computer Organization” under the cloud computing-assisted teaching platform to assist classroom teaching, and improve students’ initiative learning interest, and cultivate students’ autonomous learning ability, and better master the knowledge points of the course. The multi-integration teaching model based on cloud computing (see Fig. 2).

The Application of Multi-integration Teaching Mode Based on Cloud …

Design of Teaching Content

Theory Teaching

Network teaching

Teaching based on cloud platform

Practical applications

Experimental teaching

Comprehensive experimental simulation teaching

Knowledge extension

classroom teaching

Classroom discussion teaching

11

Fig. 2 Multi-integration teaching model based on cloud computing

Online teaching based on cloud platform

Classroom discussion teaching

Improve teaching scheme

Project-driven teaching

Teaching Evaluation

Teaching feedback

Fig. 3 The implementation process of multi-integrated teaching mode based on cloud computing

5 Teaching Plan Implementation and Experimental Results 5.1 Implementation of the Teaching Plan In order to better promote the implementation of “multi-integration” teaching method, the implementation process of the multi-integration teaching mode based on cloud computing is shown in Fig. 3.

5.2 Experimental Results According to the analysis of the practical teaching effect of one class in one semester, it is found that students’ interest in learning “Principles of Computer Organization” has significantly increased, and students have a high recognition of the multi-integration teaching model based on cloud computing. First, the analysis of questionnaire results shows that students’ learning interest [15], learning mentality, thinking ability and cognitive ability have changed significantly. In order to understand the preference of students to the traditional teaching mode and the multi-integration teaching mode based on cloud computing, a questionnaire survey was conducted on the students in this class and the upper higher class (traditional teaching mode). The results of the questionnaire survey are shown in Table 1. Second, analysis of the test results, there are more than 80% of the students, and their scores have been significantly improved. The average score of the entire class has been greatly improved, which compared with those of traditional teaching methods. As shown in Table 2.

12

M. Zhang et al.

Table 1 The preference of students to traditional teaching mode and multi-integration teaching mode based on cloud computing teaching mode

knowledge reception learning interest

learning mentality

thinking ability

cognitive ability

traditional teaching mode

30%

66%

40%

60%

multi-integration teaching mode based on cloud computing

97%

96%

86%

90%

Table 2 Comparison of course assessment scores between traditional teaching mode and multi integration teaching mode based on Cloud Computing teaching mode

score of the subject > 90

score of the subject > 80

average score

pass rate

traditional teaching mode

0

3

69.60

98.04%

multi-integration teaching mode based on cloud computing

16

25

84.99

100%

Third, students’ autonomous learning ability is basically in direct proportion to their grades. Sampling data information is randomly selected for analysis and research. The multi-integration teaching mode based on cloud computing has a certain impact on teaching quality. From the data analysis information table, it can be seen that autonomous learning ability is basically directly proportional to the grades. As shown in Table 3. Table 3 Academic records

Sname

Number of visits

Learning duration (minutes)

Final test score

Jiajia Li

221

521

78

Long Zhang

197

497

71

Hao Sun

230

501

74

Yingying Bi

366

627

93

Lina Zhang 43

127

35

Chen Liu

78

421

62

Ying Liu

86

449

67

Xing Li

167

498

73

Yi Sun

333

601

91

Yan Zhu

267

563

80

Jun Zhang

258

531

80

Tao Zhang

90

463

68

The Application of Multi-integration Teaching Mode Based on Cloud …

13

Through comparative analysis of experiments, the learning interest and cognitive ability have been greatly improved. The multi-integration teaching mode directly affects the change of students’ learning attitude and cognitive ability, and stimulates students’ knowledge internalization ability. However, through questionnaire survey and test analysis, there are also some students whose learning interest and learning style have not changed much in the whole learning process, and their learning autonomy and enthusiasm still need to be further improved. The multi-integration teaching mode based on cloud computing has significantly improved the teaching effect, basically realizing the approach of the “studentcentered” education concept, achieving resource sharing, teachers and students interaction.

6 Conclusions With the development of computer science and technology and information technology, especially the emergence of big data and artificial intelligence, the demand for students’ professional ability in the new era is higher and higher, and the requirements are more and more diversified. Therefore, Cloud computing technology has been fully utilized in the field of education and teaching, and Cloud computing technology and education cloud are organically integrated, Deepen the application of cloud computing in the field of education and teaching, create a virtual education and teaching environment, and provide a resource sharing platform for teachers and students. Further innovation and improve computer professional training program, effectively improve the learning effect of students’ fragmentation time. On the other hand, the multi-integration teaching mode based on cloud computing has changed the single traditional teaching method and made up for the deficiency of “emphasizing theory and neglecting practice” in traditional teaching. It adopts diversified teaching methods and combines a hybrid online and offline teaching mode, to project drive students’ autonomy, to practice the teaching achievements inspection, and improve the quality of teaching, and improve the teaching methods, to further improve the teaching methods, and cultivate students’ autonomous learning and ability to solve practical problems. The multi-integration teaching mode based on cloud computing can truly reflect the quality and effect of students’ knowledge and ability learning and cultivation, enhance students’ learning initiative, improve students’ ability to discover, analyze and solve practical engineering problems, and effectively improve students’ innovation and entrepreneurship ability.

14

M. Zhang et al.

References 1. Zhao L, Nie X (2016) Analysis of the teaching effect of “multimedia technology” based on moodle platform under cloud computing. J Xinyang Agric For Univ 26(2) 2. Wang X (2015) Inspiration for the survival of developers in the cloud era. http://www.scienc enet.cn/ 3. Zhu X (2018) Research on the judgment rule of inducement method patent infringement in the environment of cloud computing. Zhejiang Normal University 4. Han R (2015) Research on the construction of multi-integration university-level resource sharing course. Liaoning Normal University 5. Ruan Q (2017) Design of expressway construction information platform based on cloud computing. Chang’an University 6. Tang Z (2016) Research on 3D visualization technology of medical image based on cloud computing. Inner Mongolia University of Science and Technology 7. Zhao B (2019) Research on cloud computing security risk and security technology. Comput Knowl Technol 15(2):27–28 8. Hai J (2015) Research on user data storage security in cloud environment. Heilongjiang University 9. Zhang S (2018) Optimized pricing and purchasing strategy of IaaS cloud service. Shandong University 10. Xu L (2011) Research of teaching reform and practice of “principles of computer organization” in application type undergraduate course. J Langfang Normal Univ (Nat Sci Ed) 11(3) 11. Zhou L (2014) Research on resource construction of “cloud classroom” *– a case study of Zhuhai no. 5 middle school. Educ Inf Technol (11) 12. Wang X (2014) Research on the path of integration of science and education in key agricultural universities—a case study of Huazhong agricultural university. Huazhong Agricultural University 13. Zhang L, Liu X (2012) A preliminary study on teaching ideas of display design course in higher vocational colleges. J Henan Finan Taxation Coll 26(4) 14. Zhong W (2019) Research on the middle-level teaching mode of Chinese composition teaching in primary school. Charming China (48) 15. Yang L (2015) A survey on english affectionate teaching in secondary schools. Shandong Normal University

Impact of Fourth Industrial Revolution-Related Technologies on Higher Education Olutayo O. Oyerinde

and Ada M. Dienga

Abstract The advent of fourth industrial revolution (4IR) has brought a variety of opportunities and changed the ways things operate in various spheres of life. This has not left the education sector behind. This study aims at assessing the impact of the 4IR related technologies on higher education with engineering education in South African Universities as case study. This chapter considers the 4IR holistically and attempts to highlight the impact of its associated technologies on engineering education. In this study, data were obtained through qualitative (survey with open-ended questions) and quantitative (surveys with closed-ended questions) research methods. The study shows that the engineering curriculum needs to be overhauled, the mode of teaching engineering education needs to be adapted to take advantage of the 4IR, and that mode of assessments must be reviewed to align with the 4IR era. Consequently, there is urgent need for more proactive response from the educational sector than the more gradual societal evolution and subsequent response from educational institutions observed in earlier industrial revolutions. Efforts need to be geared towards the formulation of education systems that equip graduates with the skills that are in demand. Keywords 4IR · Higher Education · Engineering Education · Teaching and learning · Assessment · Learning management system

O. O. Oyerinde (B) School of Electrical and Information Engineering, University of the Witwatersrand, Johannesburg 2050, South Africa e-mail: [email protected] A. M. Dienga Engineering Council of South Africa, Bruma, Johannesburg 2198, South Africa e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_2

15

16

O. O. Oyerinde and A. M. Dienga

1 Introduction From time immemorial people have been dependent on technology. These technologies differed from one era to the other; people used the technology they had available to make their lives easier while trying to enhance and perfect it. The previous three industrial revolutions took place at a time when the development of science and technology was embryonic in the Western world. Driven by the Second World War, the arms race, and the Cold War, this development happened at a high speed, culminating in the fourth revolution. This fourth industrial revolution (4IR) has been described as the advent of a world in which people transit between digital domains and offline reality employing connected technology to facilitate and manage their lives, powered by artificial intelligence and other related technologies. Different stages of industrial revolutions concerning production are described in [1] and [2] as 1st Industrial Revolution, in which water and steam power were used to mechanize production, while the 2nd Industrial Revolution dealt with mass production made possible by electric power; the 3rd Industrial Revolution hinged on automated production via semiconductors and information technology, and the current or 4th Industrial Revolution has to do with more customized production and a union of technologies that blurs the lines among the physical, digital and biological spheres. Klaus Schwab, in [3], describes the 4IR as a new “technological revolution that will fundamentally alter the way we live, work, and relate to one another. Each of these revolutions brings significant change to society, the economy, and the world itself [4]. Until the 4IR, various preceding industrial revolutions spanned roughly a century; however, technology has advanced rapidly to accelerate this new revolution. As each of the technologies become ubiquitous, they will essentially change how we produce, communicate, move, generate energy, consume, and relate to one another. One of the 4IR-related technologies that stand out is Artificial Intelligence (AI) and it can be programmed to provide automation for low-value tasks in the 4IR-driven industries, thereby freeing up engineers to perform higher-value tasks. In an education environment, an AI-based agent could be used to learn students’ behavior and suggest appropriate courses for them. In addition, an AI-based agent could be employed to track attendance, monitor classroom activities and the behavior of both lecturers and students. This technology could also be used to scan the contents available online and recommend appropriate content for students in line with the course and the different levels of understanding of the student group. More than this, information obtained from an AI-based agent could be used to decide on a suitable teaching methodology and for framing of course contents. In terms of assessment, the technology could be used to analyze students’ results and provide insightful information about their performance. Some other aspects of 4IR that focus profoundly on related technology include automation, machine learning interconnectivity, and real-time data. In other instances, 4IR also covers the Internet of Things (IoT), in which things (devices) or people around the world are connected to or through the Internet by which collection and sharing of data are made easy.

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

17

1.1 Evolution of Education 1.0 to Education 4.0 In the same vein as with the industrial revolution, the education system has evolved over the years. As 4IR emerges, also comes the Education 4.0. In [5] Makrides describes various evolutions in education systems as follows: Education 1.0: This was teacher-centered in which the teacher transmits knowledge as the sole leader in the classroom. During this era, no technology was used in the lecture room and the students were passive recipients of knowledge. Education 2.0: During this era, communication and collaboration began to grow. Students memorized knowledge for examination instead of mastering knowledge for appropriate applications. This era was characterized by the invasion of technology and social networking. Education 3.0: This era shifted to a student-centered approach, in which the teachers serve as an advisor, coordinators, and practice guides to students. The students get involved in personal research and the Flipped-classroom method applies. Self-study by students is made possible with the aid of technology that is available everywhere. Education 4.0: In this era, it is expected that co-creation and innovation will be at the center. New technologies such as AI, machine learning, IoT, cloud computing, smart manufacturing, and blockchain are expected to be introduced on a large scale directly impacting every section of humankind. The flipped classroom is expected to be applied wherever and whenever an interactive practical exercise takes place face-to-face. Students are required to study from home, with the designed learning plans being called creativity plans. The mode of education will be enabled by the availability of technology that is easily accessible and free. According to some studies focusing on engineering skills tied with Industry 4.0 by 18 different countries that are signatories to the Washington Accord of International Engineering Alliance, new skills hinged on the 4IR-related technologies are expected to be in high demand [6].

1.2 Overview of Relevant Works In his book [4], Schwab opined that the 4IR is essentially different from the three past revolutions as it is distinguished by a range of emerging technologies that are binding the digital, physical, and biological worlds together. Hence, it has a wide range of impacts on almost every facet of life, economies, discipline, and industry. In his view, Marr in [7] describes the 4IR as an exponential change that disrupts the traditional way human beings work, live, and interact with one another, because of the adoption of cyber-physical systems, the IoT, and the Internet of Systems (IoS). He postulated that as smart technologies are implemented in industries and workplaces, connected machines will network together, visualize the entire production chain, and ensure

18

O. O. Oyerinde and A. M. Dienga

decision-making is autonomous. In line with Schwab’s line of reasoning, Marr also suggests that this revolution is expected to impact all facets of life. Hence, education and training mechanisms need to be revised to effectively prepare people for the flexibility and critical thinking skills that will be needed in the future workplace. In [8], it was noted that the advancement in digital transformation and rising mutual connections lead to new challenges for various industries. According to Joi Ito, the Director of MIT Media Lab, the 4IR is revolutionary because it makes organizations have the capability to capture data from the physical world, processes it digitally, and returns informed action to the physical world [9]. This continuous and cyclical transmission of information, referred to as the physical-to-digital-to-physical loop, allows organizations to respond in real-time to changes in the ecosystem. Penprase in [9] suggests that considerable changes concerning the curriculum of science and technology will be needed to permit students to develop capacities in the speedily evolving areas of data science, genomics, robotics, AI, and nanomaterials. With the development of online instruction and expanded usages of AI, the authors in [10] suggest that new guidelines should be developed that will provide a theoretical basis for digital education. The authors suggest that the curriculum must assist students to cope with the complicated issues in the interlink between online spaces and the philosophical dimensions of AI, which may likely exceed human intelligence. One state-of-the-art idea exploring emerging sequencing of higher education is the Stanford 2025 plan, which foresees many mechanisms in which students can lengthen their education over an extended period [11]. Reasoning how the education curriculum will be impacted, the authors [12] pose some thought-provoking questions that demand answers. Some of these key questions include: “How will the lecturers teach?” “In what mode will the students learn? (Method)” “What are the course contents to be taught by the lecturers?” “What do the students need to learn?” “What do industry and society expect the students to learn? (Content)” “What will the learning platform be?” “What will be the lecturers’ role?” “What will be the students’ responsibilities?” “What are the characteristics of the students/lecturers?” Gleason in [12] argues that education as being pitched by the traditional citadel of learning is still crucial, but it is very critical that strong collaboration with industry and governments is robust. It is well known that a big lecture theatre with one tutor/ lecturer, in which information is passed from the lecturer to more than 200 learners seated in a classroom, is an outdated mode of learning for student because the skills for the automated is not being delivered via this type of pedagogy. Xing et al. in [13] highlight that machines outdid humans in terms of mechanical tasks starting from the first industrial revolution to the third industrial revolution. This resulted in a shift in the tasks associated with human labor in the service industry, from mechanical tasks to cognitive tasks. However, with the arrival of AI, 4IR technologies are in a position to perform better than humans in cognitive duties. Butler-Adam in [14] posited that one of the implications that arise for higher institutions of education because of the advent of 4IR related with curricula, teaching, and learning instead of being about robotic tutors. He reasoned that to succeed as a member of society and as an employee in the age of the 4IR, numeracy, literacy, and a solid grasp of how the world operates is critical. In their works, Manda and Dhaou

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

19

[15] attempt to understand both the opportunities and challenges that are confronting developing countries as they make efforts to adopt the digital transformation agendas that are driven by 4IR. In their works, South Africa was employed as a case study to capture various challenges confronting developing countries. From their study, the authors were able to showcase the various opportunities and challenges of the 4IR in the global South. While looking at the implications of the 4IR on education, Xing and Marwala in [16] argue that a new form of the university must emerge in which teaching, research, and service are performed differently. Specifically, they indicated that the 4IR-based university will be inter-disciplinary with virtual classrooms and laboratories, together with virtual libraries and virtual teachers, without degrading educational experience but rather augmenting it. In [17], the focus is shifted to the implications of the transformations that are associated with 4IR on engineering education globally and its value and importance. It highlights some of the challenges and implications that engineering education will be faced with in line with issues highlighted by Schwab in [5]. In [18], the authors, while investigating how to effect needed changes in education curricula, emphasized that flexible curricula and teaching methods for diverse contexts that are a departure from a teleological view of ‘skills’ are needed for higher education to be able to deliver education to learners, and to prepare them for the expectations and challenges of the 4IR. A unified Theory of Acceptance and Use of Technology (UTAUT) was employed for the study on the disruptive effects of technological innovations, the 4IR, in [19]. The authors employed face-to-face, semi-structured interviews in a bid to extract the views of selected key stakeholders in the education institution to understand their readiness and acceptance of the 4IR in the sector. The outcomes of their study suggest that the education institutions seem not yet ready for 4IR. In [20], the influence of the Covid-19 pandemic was studied as a spur for digital transformation in the education institution in South Africa. The study took advantage of the impact of the Covid-19 pandemic that made most of the governments all over the world impose tight lockdown. Consequently, learning in higher institutions migrated to online platforms. The authors attempted to monitor the rate at which the 4IR-related technologies were employed in the education sector during the lockdown. The findings, based on the data obtained from secondary sources, suggest that in South Africa, during the period, a variety of 4IR-related technologies were employed by various tertiary institutions during which teaching and learning were moved from face-to-face to remote (online) mode. Most of the reviewed works presented above focused mainly on the general impacts of 4IR on socio-economic welfare, human and broader education space. Some of the studies that emphasized the specific impact of 4IR on education talk broadly about this, although none of the studies specifically focused on the impact of 4IR on engineering education. The only study that focused on the effect of 4IR on engineering education [17], merely performed a literature review rather than basing the study outcomes on relevant primary data. Similarly, the study presented in [20] is based on literature review and secondary data. Hence, there is a need to pragmatically study the impacts of 4IR-related technologies on engineering education in general while using primary data from South African universities. This is to come

20

O. O. Oyerinde and A. M. Dienga

up with well-informed findings that could influence the needed policy change in the engineering education sector. In this study, the following are the research questions we attempt to answer. 1) RQ1: What optimum teaching and learning approach should be adopted in engineering education to take advantage of the 4IR-related technologies? 2) RQ2: In what ways can the technology (4IR) be incorporated into the teaching of the engineering curriculum without widening the gaps of the digital divide and socio-economic issues prevalent among diverse students? 3) RQ3: In what ways should the engineering curriculum be overhauled to equip engineering students with new skills associated with 4IR? 4) RQ4: What impact does the technology has on the mode of students’ assessment?

2 Methodology The study, approved by the Engineering Council of South Africa (ECSA)’s Research Ethics Committee (RES_FOR_001) dated 16th July 2020, involves some engineering academics and students. The selected universities include the South African universities that offer Bachelor of Science (BSc.) Engineering degrees such as the University of Cape Town (UCT), University of the Witwatersrand (WITS), University of Pretoria (UP), Stellenbosch University (SU), University of KwaZulu-Natal (UKZN), North West University (NWU), Nelson Mandela University (NMU), and University of Johannesburg (UJ). Different types of approaches exist in the literature for various research endeavors. These include experiment, survey, case study, action research, grounded theory, ethnography, archival research just to mention a few [21, 22]. The adopted mixed method in this chapter combines elements of qualitative and quantitative viewpoints and data collections concurrently or sequentially, by so doing draws from the strengths and minimizes the weaknesses of both qualitative and quantitative methods [23, 24]. Though some researchers believe that mixed method is incompatible [25], in that qualitative and quantitative methodologies are drawn from different epistemological assumptions, however, [26] argue that the use of mixed methods through the combination of different data sources helps uncover different views, perception, and experiences. In the following, the components of the mixed method employed for this study are presented.

2.1 Survey-Based on Open-Ended Questionnaires The open-ended questionnaire surveys were designed to extract general views, from the group of stakeholders, on the impacts or the foreseeable impacts of the 4IR on engineering education, with South Africa as a case study. The engineering students’ perspectives were sought through the open-ended questions that focused on 4IRrelated tools/technologies applicable to teaching and learning in the present time as

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

21

well as those that will be applicable in the future, the impacts of 4IR on teaching and learning in engineering generally, and the impact of 4IR on assessments in engineering education. For the lecturers, the open-ended questions ranged from impacts of 4IR related technologies in their teaching activities to the general impacts of 4IR on the engineering education curriculum, teaching and learning in engineering, and assessments in engineering education.

2.2 Survey-Based on Closed-Ended Questionnaires The closed-ended survey questions were designed to obtain the stakeholders’ perceptions on the impacts or foreseen impacts of 4IR on the engineering education curriculum, engineering education pedagogy (teaching), corresponding assessment methods, and expected skills for new engineering graduates entering industries as trainee engineers. The questionnaires were made available between November 2020 and mid-January 2021. 280 students completed the survey from UP, UKZN, NWU, WITS, and UJ. Out of the 280 student participants, 26 students did not specify their universities. In terms of engineering lecturer participants, only 44 lecturers participated in the survey from WITS, UCT, UKZN, NWU, NMU, and UJ.

3 Results This section presents results based on both closed- and open-ended questionnaires.

3.1 Closed-Ended Survey Questionnaires-Based Results The well-known Relative Agreement Index (RAI) is used in the analysis of the responses obtained from the various stakeholders. The RAI determines the relative agreement of all respondents to the questionnaires. The Relative Agreement Index (RAI) is computed, in the form of percentage measures, as: R AI =

n 

E i Pi ,

(1)

i=1

where i = response scale index, n = total number of response scale (description), in this case n = 5, E i = response stem integer, Pi = percentage of respondents selecting ith perception.

22

O. O. Oyerinde and A. M. Dienga

Explicitly, the RAI can be written as R AI = E 5 P5 + E 4 P4 + E 3 P3 + E 2 P2 + E 1 P1 .

(2)

By assigning response stem integer E i and percentage of respondents to response point description as follows, Strongly Agree Percentage, P5 : E 5 = 5, Agree Percentage, P4 : E 4 = 4, Neither agree or Disagree Percentage, P3 : E 3 = 3, Disagree Percentage, P2 : E 2 = 2, Strongly Disagree Percentage, P1 : E 1 = 1, then, RAI assumes the following form: R AI =

5P5 + 4P4 + 3P3 + 2P2 + P1 . 100

(3)

It is assumed in this study that a value of RAI greater than 3.0 indicates a positive perception of all the respondents to a particular question while a value of RAI lower than 3.0 is assumed to fall in the range of negative perception. Figure 1 to Fig. 3 depicts the responses of the engineering students to some of the close-ended questions. Figure 1(a) shows that a relatively higher percentage of the students are of the positive perception that employing a virtual reality platform for lecture delivery will enhance students’ attentiveness. The students’ responses in Fig. 1(b) suggest that a higher percentage of them think that employing the 4IRrelated application platforms will go a long way in enhancing students’ learning experiences in remote locations. The responses in both Fig. 2(a) and Fig. 2(b), with RAI values of 4.0822 and 3.7894 respectively, suggest that a higher percentage of the students believed the 4IR related virtual laboratory should be incorporated into engineering education because of its potential to equip engineering students with skills related to emerging technology, including their creations and implementation. The responses in Fig. 3(a) with an RAI value of 4.2573, suggest that most of the students are inclined towards amendment of the engineering education curriculum by incorporation of 4IR technologies. Further, a high number of the students would prefer that the mode of assessment be amended from general close-book to openbook-based assessment as illustrated by the responses in Fig. 3(b). Figure 4 and Fig. 5 depict the responses of the 44 engineering lecturers to some of the close-ended questions. In Fig. 4(a), Fig. 4(b), and Fig. 5(a), a marginally high number of engineering lecturers agreed that deploying a virtual reality platform into engineering education would enhance collaboration between teachers and students and that for either teaching mode (online or face-to-face), that virtual laboratory should be encouraged because it would allow students to practice with the tools and equipment before trying with the real-life instruments; and that virtual laboratory would equip students with skills around emerging technologies, as well as familiarize them with their creations and implementations. However, in the responses display

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

(a)

23

(b)

Fig. 1 Students’ responses to: a “Presentation of engineering courses in a virtual reality platform will enhance student’s attentiveness,” RAI = 3.2815; b “Virtual Realities technology can improve both learning process and outcomes for online engineering education for students in remote places,” RAI = 3.6032

(a)

(b)

Fig. 2 Students’ responses to: a “The use of Virtual Laboratory should be encouraged in engineering education,” RAI = 4.0822; b “Virtual Laboratory will equip students with skills around emerging technologies, as well as being familiar with their creation and implementation,” RAI = 3.7894

(a)

(b)

Fig. 3 Students’ responses to: a “There is a need to incorporate into engineering curriculum, various engineering discipline-based applications of 4IR technologies such as machine learning, AI, data science, cybersecurity, etc.” RAI = 4.2573; b “Open-book mode of assessments should be adopted for engineering students.” RAI = 3.6940

24

O. O. Oyerinde and A. M. Dienga

(a)

(b)

Fig. 4 Lecturers’ responses to: a “Virtual reality technology has the potential to greatly enhance collaboration between teachers and students, both in distance learning and classroom-based teaching,” RAI = 3.2953; b “The use of Virtual Laboratory should be encouraged in engineering education,” RAI = 3.6590

(a)

(b)

Fig. 5 Lecturers’ responses to: a Virtual Laboratory will equip students with skills around emerging technologies, as well as being familiar with their creation and implementation,” RAI = 3.4775; b “There is a need to incorporate into engineering curriculum, various engineering discipline-based applications of 4IR technologies such as machine learning, AI, data science, cybersecurity, etc.” RAI = 3.9322

in Fig. 5(b), most lecturers believed the engineering education curriculum must be overhauled to fit into the 4IR era by incorporating various engineering disciplinebased applications of 4IR technologies such as machine learning, AI, data science, cybersecurity, etc.

3.2 Closed-Ended Survey Questionnaires-Based Results To obtain unconfined opinions of the stakeholders in engineering education– students and lecturers – a few open-ended questions were designed. These questions aimed at getting the stakeholders’ views on the impact of 4IR on teaching and learning, assessment, and the engineering education curriculum. Word clustering analysis is applied

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

25

to all the responses from the open-ended questionnaires. The results of these analyses are illustrated in Fig. 6 and Fig. 7. In Fig. 6(a), the majority of the students stated they intend to USE automated/intelligent systems for linking the theory to the practical aspects of their subjects. The majority of the students, as illustrated in Fig. 6(b), believed the main impact of 4IR would be to make delivery of education easy to every student at the comfort of their own home, and that it will make “education a more personal experience for students (making students to be actively involved) rather than making students to be just observers or passive”. Further in Fig. 6(c), the students believe that 4IR-related technology will “ensure that more focus is placed on assignment-based assessments”. On the part of the engineering lecturers (Fig. 7), the 4IR related technologies will “make the engineering curriculum evolves to cater for blended teaching and learning”, “allow students to engage with real-life problems in real-time”, and lecturers have to upgrade themselves at a faster pace”. In terms of teaching and learning in engineering education, the lecturers believed that 4IR will “brings the workplace into the classroom”, “make students have greater access to their lecturers”, “develop student learning abilities”, “allow for more effective asynchronous teaching and learning which improves the flexibility of engineering education and students access to it”, “bring a complete reset of how we teach engineering”. In terms of assessment in engineering education, the lecturers believed that 4IR will “make assessments to be re-engineered to reflect the new needs”, “allow for the application of case study and application assessments”, “ensure applicationoriented assessments which test student problem-solving ability”, and “it will be quick and customized for individual students.”

(a)

(b)

(c)

Fig. 6 Students’ responses to: a “As a student, how do you wish to use automated/intelligent systems in the future?”; b “According to you, kindly state the main impacts of 4IR on teaching and learning in engineering education (mode of teaching deliveries)”; c “According to you, kindly state the main impacts of 4IR on assessments in engineering education”

26

O. O. Oyerinde and A. M. Dienga

(a)

(b)

(c)

Fig. 7 Lecturers’ responses to: a “In your opinion, kindly state the main impacts of 4IR on engineering education curriculum”; b “In your opinion, kindly state the main impacts of 4IR on teaching and learning in engineering education”; c “In your opinion, kindly state the main impacts of 4IR on assessments in engineering education”

4 Discussion and Conclusions By using a parallel mixed research design approach, this study explored and examined the perceptions of engineering students and engineering lecturers on the impact of 4IR-related technologies on engineering education in South Africa. The findings show that there is an urgent need for a more practical response from the educational sector to tap from what 4IR provides vis-à-vis engineering education. Consequently, higher education institutions must be more responsive in updating their curricula to meet the rapid pace of scientific together with technological advancement. It is paramount to employ blended learning with the use of virtual reality, automated technologies, and some other 4IR-related technologies for efficient course delivery and enhancement of student experiences. Further, stakeholders in engineering education will need to invest in the relevant infrastructures, and make funding available for the previously financially poor students to ensure that the introduction of 4IR technologies into the teaching of the engineering curriculum is realizable without widening the gaps of the digital divide and socio-economic issues prevalent among diverse students. It is very crucial to begin the gradual introduction of some of these 4IR-related technologies that can easily achieve the required engineering skills and competencies that are urgently needed. The training of the engineering lecturers to be able to handle various implications of the 4IR needs to be prioritized. It should be noted that a better responsive curriculum would require urgent faculty development and curriculum overhauling. Hence, this will result in the development of students that can think and reinvent themselves in the ever-changing world into which they will graduate. Curricula should assist the students to build capacity for ethical reasoning, develop the ability to recognize both societal and human impacts, and comprehend the impact of 4IR technologies on people. This is necessary because the 4IR-driven workplace needs students that possess creative insights, can collaborate in diverse teams and can navigate global cultural differences. In terms of assessment, an open book mode of assessment that emphasizes inventive applications

Impact of Fourth Industrial Revolution-Related Technologies on Higher …

27

of a subject’s learning objectives to reach a solution should be introduced in the engineering education mode of assessment. Other assessments such as applicationoriented, auto-graded, tailored assessments towards individual students’ growth and ability to reflect on what was learned, and quick and customized assessments for individual students need to be considered. To circumvent the problem of academic integrity of unsupervised open-book assessments, proctored open-book assessments will need to be employed. Lastly, collaboration among engineering lecturers, departments, faculties, universities, and research institutes will also be needed to exploit the associated features and advantages of 4IR-related technologies. The onus rest on the engineering education policymakers to ensure that the engineering education takes advantage of the 4IR-related technologies and equally turns their corresponding advantages to factors that enhance the engineering education outcomes.

References 1. Taalbi J (2019) Origins and pathways of innovation in the third industrial revolution. Ind Corp Chang 28(5):1125–1148. https://doi.org/10.1093/icc/dty053 2. The Editors of Encyclopaedia Britannica (2021) Industrial Revolution. Encyclopedia Britannica, pp 1–4. https://www.britannica.com/event/Industrial-Revolution. Accessed 26 June 2021 3. Peters MA (2017) Technological unemployment: educating for the fourth industrial revolution. Educ Philos Theory 49(1):1–6. https://doi.org/10.1080/00131857.2016.1177412 4. Schwab K (2017) The fourth industrial revolution. In World Economic Forum, Geneva. ISBN 1944835008. Accessed 26 June 2021 5. Makrides GA (2018) The evolution of education from education 1.0 to education 4.0: is it an evolution or a revolution? In: Proceedings of MASSEE international conference on mathematics MICOM 2018 conference, Nicosia, Cyprus, pp. 1–26, 18–23 6. Samanta S (2020) Industry 4.0: The Global Impact on Engineering Skill Development & Employment After 2020. Adamas University, West Bengal, India. http://adamasuniversity. ac.in/industry-4-0-the-global-impact-on-engineering-skill-development-employment-after2020. Accessed 26 June 2021 ´ 7. Slusarczyk B (2018) Industry 4.0 – are we ready? Pol J Manag Stud 17(1):232–248. https:// doi.org/10.17512/pjms.2018.17.1.19 8. Ito J (2018) Technology: challenged vs. prepared, how ready are organizations to embrace industry 4.0 technology? The Fourth Industrial Revolution is here—are you ready? Deloitte Insights 15–19 9. Penprase BE (2018) The fourth industrial revolution and higher education. In: Gleason N (ed) Higher education in the era of the fourth industrial revolution. Palgrave Macmillan, Singapore, pp 207–229. https://doi.org/10.1007/978-981-13-0194-0_9 10. Alias RA, Hikmi SN, Puteh M, Tapsir SH (2017) Higher Education 4.0: Current Status and Readiness in Meeting the Fourth Industrial Revolution Challenges, Redesigning Higher Education Towards Industry 4.0, A. Selamat [Ed], 99 pages, Kuala Lumpur, Malaysia 11. Stanford University (2013) Learning and Living at Stanford-An Exploration of Undergraduate Experiences in the Future. Accessed 26 June 2021 12. Gleason NW (2018) Singapore’s higher education systems in the era of the fourth industrial revolution: preparing lifelong learners. In: Gleason N (eds) Higher education in the era of the fourth industrial revolution. Palgrave Macmillan, Singapore, pp 145–169. https://doi.org/10. 1007/978-981-13-0194-0_7

28

O. O. Oyerinde and A. M. Dienga

13. Xing B, Marwala L, Marwala T (2018) Adopt fast, adapt quick: adaptive approaches in the South African context. In: Gleason N (ed) Higher education in the era of the fourth industrial revolution. Palgrave Macmillan, Singapore, pp 171–206. https://doi.org/10.1007/978-981-130194-0_8 14. Butler-Adam T (2018) The fourth industrial revolution and education. S Afr J Sci 114(5/6):1. https://doi.org/10.17159/sajs.2018/a0271 15. Manda MI, Dhaou SB (2019) Responding to the challenges and opportunities in the 4th Industrial revolution in developing countries. In: Proceedings of the 12th international conference on theory and practice of electronic governance (ICEGOV2019), Melbourne, VIC, Australia, pp 244–253. https://doi.org/10.1145/3326365.3326398 16. Xing B, Marwala T (2017) Implications of the fourth industrial revolution on education. The Thinker 17:10–15 17. Fomunyam KG (2019) Education and the fourth industrial revolution: challenges and possibilities for engineering education. Int J Mech Eng Technol 10(8):271–284 18. Castrillon G, Menon K (2019) Reimagining curricula for the fourth industrial revolution. Indep J Teach Learn 14(2):6–19 19. Oke A, Fernandes FAP (2020) Innovations in teaching and learning: exploring the perceptions of the education sector on the 4th industrial revolution (4IR). J Open Innov Technol Market Complex 6(2):1–22 20. Mhlanga D, Moloi T (2020) COVID-19 and the digital transformation of education: what are we learning on 4IR in South Africa? Educ Sci J 10(7):180. https://doi.org/10.3390/educsci10 070180 21. Crotty M (1998) The foundation of social research: meaning and perspective in the research process, 1st edn. Sage Publications, London 22. Saunders M, Lewis P, Thornhill A (2009) Research methods for business students, 5th edn. Pearson Education, Harlow 23. Creswell JW, Plano Clark VL, Garret AL (2008) Methodological issues in conducting mixed methods research design. In: Bergman MM (ed) Advances in mixed methods research: theories and applications. Thousand Oaks, Sage Publications, pp 66–83 24. Borrego M, Douglas EP, Amelink CT (2009) Quantitative, qualitative, and mixed research methods in engineering education. J Eng Educ 98(1):53–66 25. Ahmed V, Opoku A, Aziz Z (2016) Research methodology in the built environment: a selection of case studies, 1st edn. Routledge - Taylor and Francis Group, UK, US 26. Bryman A (2008) Social research method, 3rd edn. Oxford University Press, Oxford

Improving Student Placement in Schools: The Case of a School Group in Portugal Sónia Sousa and Jorge Coelho

Abstract Student placement is a stressful task, mainly due to last minute enrollments. In this paper we describe how we converted a manual and tedious process in a mainly automatic one. This step is crucial to improve quality since it allows to deal with last minute enrollments with ease and focus on fine tuning the results. Thus, the main outcome of this new approach is an improvement in the quality of the results contributing to the improvement of global school management and we believe that can also play a relevant role in promoting student success.

1 Introduction Every year, student placement in schools is a problematic task. Mainly because of the complexity of the process and the little amount of time available to accomplish it. In this paper we present the work carried on in the Agrupamento de Escolas de Gaia Nascente in order to make this process mainly automatic and describing its advantages. The Agrupamento de Escolas de Gaia Nascente is a group of 10 public schools where 7 of them are a mix of pre-school and primary school (1st to 4th grades), 2 other schools provide teaching from 5th to 9th grades and 1 additional school that provides teaching from 7th to 12th grades. Until now, when a new academic year started to be planned, new students (enrolling for the first time in pre-school and 1st grade) were placed in schools and classes manually in a slow and error prone process. In the last academic year of 2021/2022, 314 new students enrolled in the pre-school and 144 new students enrolled in the 1st grade. The assignment of those new students to one specific school and class is done accordingly with the current law [2–4, 8] (in portuguese). This law defines deadlines and criteria to sort students. For example, age, siblings studying in the same school, need for speS. Sousa Agrupamento de Escolas Gaia Nascente, V. N. Gaia, Portugal e-mail: [email protected] J. Coelho (B) School of Engineering (ISEP), Polytechnic Institute of Porto (P.PORTO), Porto, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_3

29

30

S. Sousa and J. Coelho

Fig. 1 Impact of last minute enrollments

cial education, etc. Due to some specificities of the whole process it is possible that schools outside our group send us new students to add to this process in the eve of the deadline for the conclusion of the process. This is due to the outside schools not being able to find a place to those students and the resulting need to transfer them to our group of schools. The problem with last minute enrollments is graphically described in Fig. 1 where each external school Si sends xi candidates to our group of schools. Each time this happens a huge pressure is put on the staff assigned to this task since they have to sort all the students again by following the criteria described in the law (sometimes starting the whole process from the beginning) in a short period of time. Obviously this is error prone and a very stressful process that eliminates any possibility of refining the results. Fine tuning is a manual process based on criteria that have the goal to create better classes. Making this procedure lighter and sharing it with other schools will allow that the candidates are placed in a shorter period of time and students without any available place can be sent to other schools in a timely manner. Even if this process in its automatized version is not shared with other schools and thus, we keep receiving last minute candidates, it is possible to quickly verify the availability of a place for them, sort students accordingly to the law and fine tune the whole process without the burden of a substantial quantity of manual work. School management and its different areas is a topic of several previous works [1, 6, 9–13] but to the best of our knowledge none of the previous works could be used in our scenario due to the specificity of our context and goals. With the pursuit of excellence in our internal processes and to present the best possible solutions to our community and students, we developed the work described in this paper and analyzed its impact in the quality of the results. The remaining of this paper is organized as follows, in the next section we present details on the sorting procedure for the candidates and then we explain briefly the implementation of our application. Then, we evaluate our solution and discuss its impact in our work and finally we conclude and present future work. From now on, the terms student and candidate appear with the same meaning.

Improving Student Placement in Schools: The Case of a School Group in Portugal

31

2 Sorting Criteria Explained The sorting of students is done with a set of criteria defined by the law and briefly explained next. We focus on students enrollment in the pre-school and in the first grade. For students enrolling in pre-school, the law defines the main criteria as the age of the candidate, the priority is higher for older students thus, students must turn: First priority: 5 years old until 31st of December of current year. Second priority: 4 years old until 31st of December of current year. Third priority: 3 years old until 15th of September of current year. Forth priority: 3 years old between 15th of September and 31st of December of current year. To untie students, the following criteria is used: 1. Student has special educational needs. 2. Parents are minor and students. 3. With siblings or other children belonging to the same household attending the intended school. 4. Students from low income families entitled to receive support from Scholar Social Action (ASE), whose guardians reside in the same area as the school. 5. Students from low income families entitled to receive support from Scholar Social Action (ASE), whose guardians work in the same area as the school. 6. Guardians live in the same area of the school. 7. Older (age in years, months and days). 8. Guardians work in the same area of the school. 9. Other criteria defined locally by the school. For students enrolling in the first grade the criteria is almost the same as the last enumeration. Also, they must turn 6 years old until 15th of September of the current year otherwise they will be placed only if there are remaining places to fill after all the older students have been placed. The criteria is as follows: 1. Student has special educational needs. 2. Was a student of pre-school in the same group of schools. 3. With siblings or other children belonging to the same household attending the intended school. 4. Students from low income families entitled to receive support from Scholar Social Action (ASE), whose guardians reside in the same area as the school. 5. Students from low income families entitled to receive support from Scholar Social Action (ASE), whose guardians work in the same area as the school. 6. Guardians live in the same area of the school. 7. Older (age in years, months and days). 8. Guardians work in the same area of the school. 9. Other criteria defined locally by the school.

32

S. Sousa and J. Coelho

Table 1 Sample of the spreadsheet given as input to our application Code Name Address Id Stu001 Stu002 Stu003 ···

Student name 1 Student name 2 Student name 3

Address 1 Address 2 Address 3

11111111 11111112 11111113

··· ··· ··· ···

The candidates’ data is obtained by accessing to a web application provided by the Ministry of Education named Portal das Matrículas1

3 Implementation In this section we describe how we implemented our application and what features it provides. We start by describing how we get and process the data of candidates that will allow to apply the criteria defined in Sect. 2. Then we describe the algorithm for sorting students along with related technicalities and finally we describe how data is exported and what it is used for.

3.1 Obtaining Data Automatically Data is obtained by interaction with an application made available by the Ministry of Education that stores all the data from the enrollments and is able to output that data in a spreadsheet. That spreadsheet has a total of 204 columns of data and as many lines as the students that enrolled. One sample of such type of file is presented in Table 1. Our application processes that spreadsheet and extracts only the information needed for this task which consists of 29 of those columns, stores them in appropriate data structures (dictionaries) and applies the algorithm presented in the next section.

1

https://portaldasmatriculas.edu.gov.pt/.

Improving Student Placement in Schools: The Case of a School Group in Portugal

33

3.2 Sorting and Distribution of Students To explain our approach we introduce several definitions. Definition 1 (Set of schools). We define the schools of our group by a set S = S1 , . . . , Sn were each Si correspond to one of our schools. Although we have a well defined number of schools, we prefer to implement our solution as general as possible. Definition 2 (Classes). For each school Si from the set S we define the following properties: Ci = Ci1 , . . . , Cik as the set of classes for school Si where each Cit has cit p available places. Definition 3 (Candidates). We define E = e1 , . . . , em as the set of candidate students to enrollment in one of our schools. Definition 4 (Candidates’ properties). For each candidate ei we have a set of properties Pei = p1 , . . . p j that gather all the data necessary for the application of the sorting criteria. We now define Algorithm 1 for the placement of students in schools. Candidates and Candidate’s properties are drawn from the spreadsheet presented in Sect. 3.1 while schools and classes data is extracted from local files (spreadsheets also) that can be customized to a particular group of schools. Our implementation is developed using Python2 its spreadsheet data translation libraries3 and custom comparators. However in this paper we focus on the high level algorithm that defines the main process and further details along with the complete implementation can be found in our GitHub repository4 . The algorithm executes by distributing students by schools, starting by checking their preference list and verifying if there are students that don’t fit in they preferred school then it will try to assign them to their next preference and so on, always obeying to the criteria defined in Sect. 2. This happens until we reach a fixed point, meaning that we don’t have further changes in the list of student placement (described in the algorithm by A). Thus, the algorithm is unable to put additional students in any of the schools/classes and so it stops. Eventually there may be students that were not placed anywhere. Also note that the law defines that classes including students with special education needs should be shorter than normal. That is the reason why the number of places for students in schools depends on the students enrolled and this is included in the algorithm by a updating P at each step.

2

https://www.python.org/. https://openpyxl.readthedocs.io/. 4 https://github.com/safsousa/. 3

34

S. Sousa and J. Coelho

Algorithm 1 Candidate distribution Let S = S1 , . . . , Sn be the set of schools Let P = p1 , . . . , pn be the set with the total number of available places for each school such that pi = pi1 p + . . . + pik p Let E = e1 , . . . , em be the set of candidates. Let A = A1 , . . . , An be the set of lists of students where each Ai corresponds to each Pi and each Ai is initialized with ∅ 1: for each e j ∈ E do 2: Given that the first school preference of e j is Sk add e j to Ak . 3: end for 4: while A does not reach a fixed point do 5: for each A j ∈ A do 6: Reorder students accordingly with the criteria defined 7: end for 8: Update the P with the new limits for classes’ sizes. 9: for each S j ∈ S do 10: if length(A j ) > P j then 11: Remove the excess of students from A j and place them in A'j 12: Given the next school preference from student e j ∈ A'j 13: Place the student in that school. 14: end if 15: end for 16: end while Table 2 Partial sample of the sheet for school i Code Name Birth date Stu001 Stu002 Stu003 ··· Stu030

Postal code

···

Student name 1 Student name 2 Student name 3

2000-01-01 2000-01-01 2000-01-01

1234-101 1234-102 1234-103

··· ··· ···

Student name 30

2000-01-01

1234-130

···

3.3 Exporting Results After sorting all the students a spreadsheet workbook is created such that it contains x + 1 sheets, such that we have one sheet for each school we are placing students in and one additional one containing students that hadn’t been placed anywhere due to lack of space. In Table 2 one can find a partial sample of one of these sheets for a given school. This workbook is then handled to the team for further manual work and use in the final enrollment process.

Improving Student Placement in Schools: The Case of a School Group in Portugal

35

Fig. 2 Time distribution under last minute enrollment conditions

4 Evaluation To evaluate our work we explain and analyze two different dimensions, the problem with last minute enrollments which was the main initial motivation and fine tuning of the results which is an effect of the automatization of this process.

4.1 Last Minute Enrollments Typically we assign one person working on this process for a period of one week which means that the person spends 35 h on this task. In case we have the need of restarting the whole process in the eve of the deadline (which often happens) and using as reference the 35 h needed by one person to complete the task, it means we need to compress it in 8 h by, ideally, adding more human resources. Assuming it is a completely parallelizable task (which is not the case) we need to assign 5 persons to complete this job in a timely manner. This has consequences since we need to have people trained for this task and without any other relevant activity in that moment in order to assign them to this task without further collateral effects. In reality what happens is that the work is assigned to people available to help without further criteria, usually creating a big pressure on them and making them work overtime frequently creating burnout situations and even resulting, in the past, in cases of resignation. As it is easy to understand this also puts pressure on the management team to find a better solution. As a result of this work we are able to reduce a process that takes several hours to less than one hour. In Fig. 2 a relation between the time (in hours) needed to accomplish this task is presented. Finally the remaining time can now be used to fine tune the results which is a task now seen as of maximum importance as explained in the next section.

36

S. Sousa and J. Coelho

Fig. 3 Time distribution under last minute enrollment conditions considering fine tunning

4.2 Fine Tuning of the Results Although not being the main subject of this paper one very important aspect that this work enables is the staff to focusing on fine tuning the results. This is of the highest importance for us. Our group of schools is underperforming for some years according to official rankings. Thus, we are deeply committed in applying methodologies that are known to have a positive impact in students outcomes. The organization of the classroom in a student-centered methodology is seen as a solution with positive outcomes [7]. In a generic sense one can say that the classroom environment is of high importance for student success. Elements of the learning environment such as guaranteeing that students feel well in the classroom are important to this goal. Identifying situations of conflicts between students are typically done by schools of origin (in case of students that were before in pre-school) and can be addressed in the student placement phase by, for example, placing conflicted students in separate classrooms. It is also known that an inclusive classroom composition [5] is a relevant factor for student success since it encourages the practice of peer help increasing learning interaction, self-esteem, mutual respect, solidarity and acceptance of diversity. It is now possible for the team to apply such type of methods in fine tuning the results by guaranteeing that students are placed accordingly with the law and that classes are composed with the best practices described in current related research. In Fig. 3 the new distribution of tasks is presented. Note that, when there are available resources, the fine tuning can be done in parallel by more than one person by, for example, distributing one school per person.

5 Conclusion and Future Work In this work we presented a problem that is common in schools in Portugal which is related with student placement in the early years of school. The particularities of this process makes it quite hard to accomplish manually but this has been the usual procedure for several years. Here we present, as far as we know, the first automatized solution for this problem which we make publicity available through our GitHub repository and discuss its advantages. The most visible outcome is that

Improving Student Placement in Schools: The Case of a School Group in Portugal

37

it is now possible to solve this problem correctly in a timely manner and as a second outcome, schools have now time to apply recent research in student placement in classes to improve the classroom environment and hopefully increase the success rate of their students. As future work we hope to work with other school groups to keep improving the developed application, study its use in other school years and promote the creation of other useful tools based on the automation of particular processes.

References 1. Casey D (2013) Educational management systems in secondary education in Hordaland County, Norway. In: Passey D, Breiter A, Visscher A (eds) Next generation of information technology in educational management. Springer, Heidelberg, pp 151–162 2. da Secretária de Estado Adjunta e da Educação G, de Estado da Educação S (2018) Despacho normativo n. 6/2018, de 12 de abril. Diário da República 72:10277–10282 3. da Secretária de Estado Adjunta e da Educação G, de Estado da Educação S (2020) Despacho normativo n. 5/2020, de 21 de abril. Diário da República 78:79–97 4. da Secretária de Estado Adjunta e da Educação G, de Estado da Educação S (2021) Despacho normativo n. 10-b/2021, de 14 de abril. Diário da República 72:4–18 5. Flecha R (2015) Forms of Classroom Arrangement: Streaming, Mixture, Inclusion. Springer, Cham, pp 21–30. https://doi.org/10.1007/978-3-319-11176-6_3 6. Ishizue R, Sakamoto K, Washizaki H, Fukazawa Y (2018) Student placement and skill ranking predictors for programming classes using class attitude, psychological scales, and code metrics. Res Pract Technol Enhanced Learn 13(1):7. https://doi.org/10.1186/s41039-018-0075y 7. Keiler LS (2018) Teachers’ roles and identities in student-centered classrooms. Int J STEM Educ 5(1):34. https://doi.org/10.1186/s40594-018-0131-6 8. do Ministro EaG (2022) Despacho n. 4209-a/2022, de 11 de abril. Diário da República 71:2–4 9. Shah M (2014) Impact of management information systems (MIS) on school administration: what the literature says. Procedia Soc Behav Sci 116:2799–2804. https://www.sciencedirect. com/science/article/pii/S1877042814006764. 5th World Conference on Educational Sciences 10. Song S, Isican I, Fu Y (2022) Research and implementation of student management system based on java EE technology. In: Hung JC, Yen NY, Chang JW (eds) Frontier computing. Springer, Singapore, pp 1621–1625 11. Strickley A (2011) A baseline for a school management information system. In: Tatnall A, Kereteletswe OC, Visscher A (eds) Information technology and managing quality education. Springer, Heidelberg, pp 62–74 12. Tulowitzki P, Gerick J, Eickelmann B (2022) The role of ICT for school leadership and management activities: an international comparison. Int J Educ Manag 36(2):133–151. https://doi. org/10.1108/IJEM-06-2021-0251 13. Zeng Y, Boontasorn N (2022) Student management system based on intelligent technology of internet of things. In: J Jansen B, Liang H, Ye J (eds) International conference on cognitive based information processing and applications (CIPA 2021). Springer, Singapore, pp 866–876

Intelligent Tutoring System for Teaching and Learning Mathematics Francisco Niño-Rojas, Diana Lancheros-Cuesta, Martha Tatiana Jimenez-Valderrama, Sergio Eduardo Gómez Ardila, and Gelys Igreth Mestre Carrillo

Abstract Nowadays, it is evident to find a diversity of student profiles in the classroom, mainly in the first semesters of academic programs at the university level. One of the areas where it has been possible to determine greater diversity is in the basics of mathematics and physics, because in the same classroom there are students from different disciplines, with diverse preconceptions, abilities and other characteristics; and with the presence of various heterogeneous scenarios where they can apply their acquired knowledge. This represents a great challenge when it comes to addressing didactic strategies to allow delivery of teaching adapted to these particularities in students. This article presents the design of an adaptive model in a virtual learning environment for an intelligent tutoring system to achieve the personalization of guided and adapted routes for mathematics learning. Keywords Intelligent tutoring system · virtual environment · adaptive learning · Frame Model · Mathematics

F. Niño-Rojas (B) · D. Lancheros-Cuesta · M. T. Jimenez-Valderrama · S. E. Gómez Ardila · G. I. Mestre Carrillo Universidad de La Salle, Bogota, Colombia e-mail: [email protected] D. Lancheros-Cuesta e-mail: [email protected] M. T. Jimenez-Valderrama e-mail: [email protected] S. E. Gómez Ardila e-mail: [email protected] G. I. Mestre Carrillo e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_4

39

40

F. Niño-Rojas et al.

1 Introduction Artificial Intelligence (AI) is a discipline that has been developed from the idea of adapting the functioning of computer systems so that they develop operations that emulate, with different levels of approximation, brain abilities [1]. Nowadays, the advancement of technology has allowed the development of systems and platforms with artificial intelligence. One of the systems developments derived from artificial intelligence is Intelligent Tutoring Systems (ITS). To define an “Intelligent Tutoring System” it is convenient to go back to the meaning of each of the words that make up the concept: First, “System” refers to a set of components that interact in an articulated way to achieve a goal [2]; “Tutor”, in the context of education, is associated with its Latin origin “tuieri”, which means to point out, indicate, direct, guide [3]; and the word “intelligent” has been defined as relative to the subject that has intelligence, and this as the ability to adapt to an environment. Likewise, the term intelligence in the field of ITS refers to the ability to perform actions “that would be considered intelligent if they were carried out by humans” [4]. An “Intelligent Tutor System” could then be defined as a computer system with the capacity to adapt an educational instruction to certain needs of the student. An ITS is a type of Educational Adaptive System. Our interest, rely on those systems that may provide students with different learning elements (materials, content, tools, services, recommendations, etc.) appropriate to their needs and personal characteristics. This tendency is known as adaptive learning [5]. Research efforts in this area have addressed two adaptation orientations with respect to: a) the capabilities that a computer system has to deliver or produce suitable materials to users (also known as adaptivity), and b) the means or resources that a user has to customize the functionalities and properties of a system (also known as adaptability) [6]. The first approach focuses on the machine or system (that is, its adaptability) and the second focuses on the user (that is, on the customization or adaptability that the system allows according to the user’s preferences). Both processes focus on improving the user experience of a system by increasing their efficiency and satisfaction with respect to personal activities and system functionalities, respectively. Higher education in Colombia faces important challenges, among which the expansion of coverage levels and lowering academic dropout rates stand out. Coverage rates do not exceed 50% and dropout rates remain worrying [7]. By level of education, for the year 2018, the annual dropout rate for university programs was 8.79%, for technological ones at 10.75% and for professional technicians at 17.41% (Fig. 1). In the country, it has been detected that 37% of all dropout students abandon their university studies in the first semester and 16% in the second semester, that is, more than half of the dropouts are concentrated in the first two semesters and one of the causes associated with this phenomenon is poor academic performance [9]. The academic entry conditions can become a determining factor in predicting the chances of dropping out of a young person enrolled in a higher education program.

Intelligent Tutoring System for Teaching and Learning Mathematics

41

Fig. 1 Annual dropout rate according to level of training 2018. [8]

In this sense, in the area of mathematics, historically, there is evidence of a high rate of loss in university students enrolled in the first academic spaces in the area of mathematics. The difficulties that usually arise are related to concepts, procedures and attitudes. In relation to concepts, difficulties related to real numbers, equations, inequalities and functions are presented. In relation to the procedures, the greatest difficulties arise with the understanding of the mathematical text and the transposition of a problem into mathematical language. At the attitudinal level, there are difficulties related to low interest in the course, little participation and a tendency to distraction generally related with digital content [10]. On the other hand, it is also necessary to establish that one of the causes attributed to low performance corresponds to the traditionalist approach in the teaching of this discipline, that is, teaching by content, where algorithms are prioritized and mechanical procedures are repeated [11]. In large part, this is associated with its rigid nature, the lack of methodological innovation in the classroom, and the lack of contextualization in the teaching of content. Therefore, a methodological and didactic change is necessary that is flexible, that allows to recognize the particularities or personal difficulties of the students and to make use of technological tools that facilitate the delivery of adapted materials between theoretical concepts and contextualized procedures in an interactive way. The technological advances of the last decades and the way in which they have influenced and changed the way we relate to each other, have also had an impact on education. From there, the need arises to implement or create methodologies adapted to a technician society, and especially, new learning modalities that allow teaching and learning to be guided in the best possible way, supported by technological mediation and aiming at a formation of critical people in a changing society according to the needs of the 21st century [11]. In the teaching and learning of mathematics, it is not only necessary to keep in mind the fundamental concepts and procedures necessary to solve a problem, but it is also essential to keep in mind the difficulties, preconceptions and learning rhythms that each student has, for which It is necessary to call for methodologies that enable the design of adaptation strategies and the use of technological tools such as Intelligent Tutorial Systems (ITS) that serve as support for the appropriation of conceptual and symbolic structures typical of mathematics.

42

F. Niño-Rojas et al.

ITSs are flexible, interactive and adaptive environments for learning. Flexible because they can be adjusted to the rhythms of each student, allowing different navigation routes; Interactive because the channels for communication can allow ideas to cross; and adaptive because they adjust to the needs of each user [12]. These systems have been developed from technologies based on artificial intelligence, it is a system that can be expressed through hypermedia, but its architecture allows non-linear sequences, provides them with the possibility of multiple teaching methodologies and supports them with knowledge bases for feedback with the ability to adapt an educational instruction to certain needs of the student [13]. This article shows the development of an adaptive model in a virtual learning environment with adaptive characteristics that includes artificial intelligence techniques to strengthen basic mathematics skills in students entering the Universidad de la Salle in Bogotá Colombia and achieve the personalization of a teaching process and improvement in the development of skills and competencies. The article shows in Sect. 2 the review of related works, in Sect. 3 the methodology carried out, in Sect. 4 the results and finally some conclusions are mentioned.

2 Related Works Taking into account that in recent years there has been an increase in the development of research projects that seek to design, analyze, and measure technological platforms with artificial intelligence components used in education, the analysis of related works is described below.

2.1 Methods or Strategies in the Teaching of Mathematics In the teaching of mathematics, various didactic strategies are used that are characterized by active learning and learning that places the student at the center of the learning process. According to [14], some strategies used by teachers are: problem solving, active case analysis, educational games, among others. Focusing on the learning of mathematical foundations, which is part of the topics necessary to successfully tackle the learning of differential calculus, didactic strategies are commonly developed, and few have been the works that incorporate the use of artificial intelligence and specifically systems smart tutoring.

2.2 Adaptive Learning In recent decades there has been an interest in providing students with different learning elements (materials, content, tools, services, recommendations, etc.) that

Intelligent Tutoring System for Teaching and Learning Mathematics

43

are suitable for their needs and personal characteristics. This tendency is known as adaptive learning [5]. Research efforts in this area have addressed two adaptation orientations with respect to: a) the capabilities that a computer system has to deliver or produce suitable materials to users, and b) the means or resources that a user has to customize the functionalities and properties of a system [6]. The first approach focuses on the machine or system (that is, its adaptability) and the second focuses on the user (that is, on the customization or adaptability that the system allows according to the user’s preferences). Both processes focus on improving the user experience of a system by increasing their efficiency and satisfaction with respect to personal activities and system functionalities, respectively.

2.3 Intelligent Tutorial Systems Artificial Intelligence (AI) is a discipline that has been developed from the idea of adapting the functioning of computer systems so that they develop operations that emulate, with different levels of approximation, brain abilities [1]. One of the favorite topics of researchers in this discipline is that of “intelligence”, associated with aspects such as “learning”, understood as the ability to make subsequent decisions based on previous experiences. Prior to this decision making, the machine must “memorize” the experiences, so that this information is contrasted with a new one and from a series of logical comparisons the machine “decides” between different options. One of the computer developments derived from artificial intelligence are the Intelligent Tutoring Systems (STI). To define an “Intelligent Tutoring System” it is convenient to go back to the meaning of each of the words that make up the concept: First, “System” refers to a set of components that interact in an articulated way to achieve an end [2]; “Tutor”, in the context of education, is associated with its Latin origin “tuieri”, which means to point out, indicate, direct, guide [15]; and the word “intelligent” has been defined as relative to the subject that has intelligence, and this as the ability to adapt to an environment. Likewise, the term intelligence in the field of ITS refers to the ability to perform actions “that would be considered intelligent if they were carried out by humans” [4]. An “Intelligent Tutor System” could then be defined as a computer system with the capacity to adapt an educational instruction to certain needs of the student. An STI is a type of Educational Adaptive System.

2.4 Intelligent Tutoring Systems for the Teaching and Learning of Mathematics Research around STIs has often been related to cognitive aspects of learning mathematics. The first STIs were developed to investigate the appropriation and mastery of specific mathematical topics [16], one of the first STIs was called “Buggy” because

44

F. Niño-Rojas et al.

the system was based on identifying and addressing the errors that children made when performing mathematical operations [17]. In recent years, there has been little research in Colombia on the design and development of ITS for the teaching and learning of mathematics. However, proposals have been made to improve the understanding of mathematics. For example, in Florencia - Caquetá, the development of an STI was proposed to support the teaching of mathematics, specifically it sought to carry out a deepening of the concepts of algebra, since this area is considered important in the “development of the thinking of the student”. reasoning and analytical thinking of students. The foregoing with a view to improving the results of the Saber 11 tests, which in the 2010–2014 period were not favorable and tended to be below the national average [18]. On the other hand, on the international scene, research carried out in Mexico, Spain and the United States reveals evidence of the great impact on the development of mathematical and attitudinal skills in students. In Mexico, an STI was developed that managed to adapt to different learning styles, based on the identification of the difficulties presented by undergraduate students in learning mathematics. These difficulties were evidenced through a diagnostic test that allowed inferring some characteristics that the ITS should have in design and development. With the implementation of the STI, the possibility of improving the academic and attitudinal performance was studied, not only of undergraduate students, but also of engineering, business sciences and health students. This STI is based on a categorization of multiple intelligences developed in students, and focuses specifically on teaching algebra, arithmetic, equations, exponents, geometry, and trigonometry [19]. At the University of Valencia, in Spain, it was possible to implement the Intelligent Tutorial System HBPS (Hypergraph Based Problem Solver) for the solution of algebraic problems capable of supervising the resolution of algebraic arithmetic word problems. This STI offers the flexibility of using several letters in the algebraic representations or translation of the statement, identifies errors, verifies algebraic expressions, offers help and checks the validity of the result. This research is carried out with a group analysis and a case study, where it is possible to show that the STI can be a useful instrument in teaching this type of problem. However, from the case study it is revealed that the performances are not very satisfactory when some students execute the actions in said environment. For this reason, they suggest keeping in mind in future updates a “higher level of adaptability to individual characteristics” of the student [20]. Less recent, but just as important, is the STI Pump Algebra Tutor (PAT) [21], also oriented to the solution of algebraic problems associated with the real context, contemplates the idea that the application of problems related to reality makes mathematical knowledge more meaningful. PAT offers help to the student during the procedure that is carried out when solving an exercise and suggests clues about what would be the most favorable path when asking for help. PAT uses a Bayesian model to monitor learning during the solution of a problem, identifying the strengths and weaknesses of students, and based on the data collected, it adapts and suggests a problem path for each student.

Intelligent Tutoring System for Teaching and Learning Mathematics

45

2.5 Authoring Tools for Cognitive Tutoring (Cognitive Tutor Authoring Tools - CETAT) Some ITS are developed from a multi-agent architecture where a lot of development work is needed to get to the final version. However, several research centers have joined forces to develop CTAT, a tool that allows educational researchers with limited programming experience to create intelligent tutoring systems. These systems provide guidance to students as they solve problems and provide just-in-time, context-sensitive help [22]. CETAT allows the implementation of an interface for the student and an interface for the tutor (teacher): while the student is presented with different routes or alternatives for solving an exercise or problem, the teacher is presented with an interface to program said routes or algorithms. In the design of the application, Anderson, one of its authors, proposed a model in which what is required by the student (help, recommendations, tutorial support, resources, among others) is articulated with the rules designed by the teacher [23]. As can be seen, the adaptation process in educational platforms requires modeling from the student, the teacher, and the context.

3 Methodology The model designed was established based on the following phases: Phase 1. Collection of information: This phase consisted of the systematic collection and analysis of information on intelligent tutorials in mathematics, for this an exploration will be carried out in different sources of information that allows the collection of relevant data and functionalities of the tutorial systems. smart devices used in the teaching of basic mathematics to analyze them against a series of review criteria. For this, the following activities were carried out: Activities a. Definition of review and analysis criteria. b. Selection of related works according to the review criteria. c. Design of a correlation matrix to identify relevant aspects of the information collected. Phase 2. Design of the adaptation process: This phase had the purpose of designing an adaptive model (domain, student and instructor) to design and develop the intelligent tutor system that can be integrated into a virtual learning environment in order to contribute to the teaching and learning of basic mathematics for engineering students entering the University. Activities a. Definition of the domain model related to the topics and competences of mathematics in the first semester. b. Definition of the student model based on the difficulties and skills of learning mathematics. c. Definition of the instruction model that brings together the instructions, aids and recommendations to elaborate the learning routes of the training moment.

46

F. Niño-Rojas et al.

4 Results In order to achieve the expected adaptation objectives, the architecture of an STI, based on an educational adaptive system model, requires the following components: a domain model, a tutoring model (based on rules and recommendations) and a student model (Fig. 2). Considering the above, a computational architecture with the adaptive model is designed. This can be seen in Fig. 3. As can be seen, the initial layer has the data model, there is the student profile with the basic characteristics such as name, identification, engineering program. In the same way, this profile contains

Fig. 2 Architecture of an STI

Fig. 3 SIM computational architecture (intelligent tutorial system for the teaching and learning of mathematics)

Intelligent Tutoring System for Teaching and Learning Mathematics

47

Fig. 4 Structure Designed

the preferences, tastes, and difficulties that it presents in the area of mathematics. In this same layer is the profile of the teacher, which stores their basic data, the course and the methodology that they carry out. Finally, in this layer the context model was determined, which contains the data of the mathematical exercises that allow determining the levels of difficulty that the student has. Figure 4 shows an example of the structure designed in each of the exercises. The second layer of the architecture shows the system services, there you have the configuration of the course, the query of topics by the student and the query of exercises. Finally, in the architecture, the adaptive services are in layer 3, where the personalized learning routes and the adaptation of contents are located, taking into account the adaptation model.

5 Conclusions The definition of profiles in the adaptive models allows the adaptation of contents taking into account the difficulties of the students. What is exposed in this article is part of the development of the design phases of an intelligent tutoring system so that students who enter the engineering faculty can reinforce their skills and solve the shortcomings that they bring from schools.

References 1. Ocaña-Fernández Y, Valenzuela-Fernández LA, Garro-Aburto LL (2019) Inteligencia artificial y sus implicaciones en la educación superior. Propósitos Represent 7(2):536–568. https://doi. org/10.20511/pyr2019.v7n2.274

48

F. Niño-Rojas et al.

2. sistema - Definición - WordReference.com. https://www.wordreference.com/definicion/ sistema. Accessed 6 Sept 2022 3. Definista, ¿Qué es un Tutor? Su Definición y Significado 2021, Concepto de - Definición de. https://conceptodefinicion.de/tutor/. Accessed 6 Sept 2022 4. Woolf BP (2007) Building intelligent interactive tutors: student-centered strategies for revolutionizing e-learning. Morgan Kaufmann Publishers Inc, San Francisco 5. Brusilovsky P (2001) Adaptive hypermedia. User Model User-Adapt Interact 11(1–2):87–110. https://doi.org/10.1023/A:1011143116306 6. Chen SY, Magoulas GD: Adaptable and Adaptive Hypermedia Systems. IGI Global. https:// www.igi-global.com/book/adaptable-adaptive-hypermedia-systems/www.igi-global.com/ book/adaptable-adaptive-hypermedia-systems/3. Accessed 9 Sept 2022 7. Melo-Becerra LA, Ramos-Forero JE, Hernández-Santamaría PO (2017) La educación superior en Colombia; situación actual y análisis de eficiencia. Desarro Soc (78)59–111 8. SPADIES - Sistema para la Prevención y Análisis de la Deserción en las Instituciones de Educación Superior. Accessed 6 Sept 2022 9. Persona con capacidad excepcionalL: Ministerio de Educación Nacional de Colombia. https:// www.mineducacion.gov.co/1621/article-82790.html. Accessed 9 July 2021 10. Análisis de las dificultades que presentan los estudiantes universitarios en matemática básica 11. Enseñanza de las matemáticas básicas en un entorno e-Learning: un estudio de caso de la Universidad Manuela Beltrán Virtual 12. Sistemas Tutoriales Inteligentes, un aporte de la inteligencia artificial para la mediación pedagógica—Semantic Scholar 13. Mulwa C, Lawless S, Sharp M, Arnedillo-Sanchez I, Wade V (2010) Adaptive educational hypermedia systems in technology enhanced learning, p 73. https://doi.org/10.1145/1867651. 1867672 14. Estrategias para el aprendizaje y la enseñanza de las matemáticas. Scielo. Accessed 6 Sept 2022 15. Bernuy AE, García VM (2006) Collaboration model in e-learning for universities based on agents. In: Kumar D, Turner J (eds) Education for the 21st century—impact of ICT and digital resources. Springer, Boston, pp 267–271. https://doi.org/10.1007/978-0-387-34731-8_29. Accessed 22 Aug 2013 16. Corbett AT, Koedinger KR, Anderson JR (1997) Intelligent tutoring systems. In: Helander MG, Landauer TK, Prabhu P (eds) Handbook of Human-Computer Interaction, 2nd edn, Completely Revised Edition in M 17. Abowd GD, Dey AK, Brown PJ, Davies N, Smith M, Steggles P (1999) Towards a better understanding of context and context-awareness. In: Proceedings of the 1st international symposium on handheld and ubiquitous computing, London, UK, UK, pp 304–307. http://dl.acm. org/citation.cfm?id=647985.743843. Accessed 22 Aug 2013 18. Hernández JAD, Rengifo YSP (2015) Los sistemas tutores inteligentes y su aplicabilidad en la educación. Horiz Pedagógicos 17(2):104–116 19. Aguilar RMR, González JLMC, Campos ALL (2013) Diseño de un sistema tutorial inteligente. Apertura 5(1), Art no 1 20. Somoza JAG-C (2014) La enseñanza de la resolución algebraica de problemas verbales mediante un sistema tutorial inteligente, Universitat de València. https://dialnet.unirioja.es/servlet/ tesis?codigo=91016. Accessed 9 Sept 2022 21. Koedinger KR, Anderson JR, Hadley WH, Mark MA (1997) Intelligent Tutoring Goes To School in the Big City, ene. https://doi.org/10.1184/R1/6470153.v1 22. Aleven V, Kay J, Mostow W (2010) Intelligent, tutoring systems. In: 10th International conference on ITS 2010 23. Guzmán DS, Guzmán CR Aplicación de tutores inteligentes en la enseñanza de física en el tema: conversión de unidades físicas, p 11

A Serious Game for Recycling Education in Peruvian Children Paul Tocto, Guillermo Dávila, and Victor Ayma

Abstract Peru has one of the lowest recycling rates compared to other emerging countries and the Latin American region. One of the main reasons is the lack of environmental education. The present study aimed to implement a serious game to teach children to recycle from their own homes. The game, developed following the guidelines of the MDA framework and using the Unity programming language, consists of 3 modules in which recycling concepts are reviewed, such as classification, collection, and reusability. To verify the usability of the game and its efficiency for learning, 93 children between six and seven years old were surveyed before and after using the game. Data analysis was performed using an ANOVA test. The results show that the game is fun, easy to use, and allowed its users to significantly improve their recycling knowledge. Keywords Serious game · Recycling · Learning · Environmental knowledge · Usability

1 Introduction The world is facing global climate change that has been increasing over the years due to human activities that generate the emission of greenhouse gases that are harmful to the environment [1]. This negative effect is accentuated in emerging countries, where policies and technologies are absent preserving the environment [2]. Recycling is a practice that has proven to have a positive effect on mitigating environmental damage [3]. Because of this need, it is essential to seek better education on recycling, especially in emerging countries. P. Tocto · G. Dávila (B) · V. Ayma Facultad de Ingeniería, Universidad de Lima, 15023 Lima, Peru e-mail: [email protected] G. Dávila · V. Ayma Instituto de Investigación Científica, Universidad de Lima, 15023 Lima, Peru © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_5

49

50

P. Tocto et al.

According to statistics from the Peruvian Ministry of Environment, the tons recycled and tons produced of solid waste from 2014 to 2016, showed that the recycling rate varied between 0.2% to 0.7%. Countries such as Mexico reach rates of 10%; and cities such as Santiago de Chile, 12% [4, 5]. Peru’s very low performance can be explained, among other things, by a lack of environmental knowledge or education among the Peruvian population [6]. Therefore, inculcating recycling education in the population would help the recycling rate to increase in the upcoming years. To contribute to better recycling in emerging countries such as Peru, the present study implemented and validated the efficiency of a serious game to teach children to recycle from their own homes. Previous research proposed the development of serious games to promote environmental education. These are a learning-based approach in which people could acquire skills or knowledge in a real or controlled environment and can also offer a game experience that promotes skills or knowledge. Among the research focused on fostering environmental awareness, Rossano et al. implement serious play to provide children with the knowledge and skills needed to promote and enhance environmental awareness proving the usability of the serious game and effectiveness perceived by the students [7]. In addition, research was found to prove the effectiveness of teaching serious games to children and adolescents, where the permanent feedback and the active participation of the participants were relevant in the first stage of evaluation of a serious game. [8]. Serious games focused on teaching waste sorting have also been developed [9]. From the review of articles, we found several types of research related to recycling that uses technology, either to raise awareness or to educate the public, which have favorable results on user learning with small samples, for this reason, the present research aims to implement a serious game that teaches children about recycling. [10, 11] The research was first developed by elaborating a state of the art on research related to activities that encourage recycling with emphasis on serious games, then the serious game was designed using the MDA framework. Based on the design, the serious game was developed in the Unity game engine, to finally evaluate the game in first and second-grade children, taking a test before playing and after playing. The objective of this research is to implement a serious game to solve a local problem about recycling and create a solution validated as efficient, contributing to the area of applications of serious games for sustainability. The present research aims to implement a serious game.

A Serious Game for Recycling Education in Peruvian Children

51

2 Theoretical Foundations 2.1 Recycling The United States Environmental Protection Agency (EPA) defines recycling as the process of collecting and processing waste into new products [12]. There are several ways to recycle, including the separation of materials such as glass, plastics, paper, and metals [11]. In general, waste classification is carried out according to a colour code established in agreement to a technical standard. In the case of Peru, the colour code at the municipal level allows the separation of materials into green, black, brown, and red containers [13]. Thus, paper, cardboard, plastics, textiles, wood, leather, packaging, and metals are placed in green containers; similarly, waxed paper, ceramics, cigarette butts, and sanitary waste are separated in black containers; on the other hand, food waste, pruning waste and leaf litter are placed in brown containers; and batteries, lamps, luminaries, expired medicines, and pesticide packaging are separated in red containers.

2.2 Serious Games Serious games are designed with formative objectives to transmit knowledge and enhance player’s skills in protected environments - real or simulated. [14]. The design of this type of game requires players to experience sensations identical to those of a real environment, allowing them to learn and make decisions without fear of making mistakes. The game should be fun so that potential learning in the game does not become tedious or borig. It should provide progressive challenging scenarios to engage the player and motivate him to continue playing and learning [15].

2.3 MDA Framework Mechanics-Dynamics-Aesthetics, widely known as MDA framework, is a useful tool in game design that serves to analyze games, bridging the gap between game design and game development [16]. In essence, MDA decomposes the analysis of a game into three elements: Mechanics, Dynamics, and Aesthetics. In the broadest sense, mechanics are the objects and their associated properties that make up the game, such as the characters, their positions and their actions during the game. Dynamics, on the other hand, are the flow of states that originate from the interactions of the mechanics during the execution of the game. Finally, aesthetics refer to the feelings that can be evoked by players in the game; the MDA framework recognizes 8 aesthetics

52

P. Tocto et al.

(sensation, fantasy, narrative, challenge, companionship, discovery, expression, and submission) that may or may not be present in the game.

3 Serious Game Design For the design of the serious game model, the MDA framework was used, where the following elements were identified: • Mechanics – Collection Module: The objects in the game can be collected by the player, which produce an increase or decrease of points. – Sorting Module: Objects when deposited in a recycling bin will add points if the player did it correctly, otherwise points will be subtracted. – Usage Recognition Module: Recycling garbage cans in the scene will either add points or subtract points if the player has matched them with the correct usage. • Dynamics – Collection Module: The player will be able to move a character around the scene, also the player will be able to see the name of the objects when interacting with them. – Classification module: The player will be able to drag objects around the scene and will be able to see their name as they are dragged. – Usage Recognition Module: The player will be able to move the recycling bins around the scene and see the name of the object. • Aesthetics: typified as a game of challenge, sensory, and expression. It is challenging because it seeks the fulfilment of goals that guarantee learning about recycling. It is sensory because there are components - e.g., music - that seek a pleasurable experience for the player. Expression, as it seeks to amuse the player while he/she learns what recycling is. The game presents Remi as the main character, who is a child concerned about the current pollution and the negative effects on the environment, so he wants to teach other children to recycle and consequently mitigate this problem. The game consists of 5 scenes in which the player will be able to interact: Start, Collection Level, Sorting Level, Usage Recognition Level (reusability), and Scoreboard. Each of these levels has music to make the game more enjoyable and attractive. Remi, the main character, explains the objectives to accomplish before starting the level. The game will start when the player presses the “Play” button.

A Serious Game for Recycling Education in Peruvian Children

53

3.1 Collection Module Before starting the level, a brief audio explaining the problem to be solved through recycling is played.. Once the game has started, two indicators constitute the goals that the player must achieve: Time (60 s) and Score. Concerning the MDA framework, the mechanics are based on the collection of objects, which when collected add or subtract points to the player, and these objects are scattered throughout the game area. The dynamics in this level consist of the player’s movement with the keyboard arrows throughout the game area and the ability to visualize the name of an object by placing the mouse pointer over it. The main objective of the level is to collect as many objects as possible in the shortest time, for subsequent sorting in the next level. The presented objects that the player can collect are presented in Fig. 1: paper sheet, box, glass bottle, plastic bottle, banana peel, and a can. Figure 2 visualizes the scene presented to the player of the collection level.

Fig. 1 Possible objects to be collected in the game Fig. 2 Collection Module

54

P. Tocto et al.

Fig. 3 Sorting Module

3.2 Sorting Module The objective of the level is to identify the materials to be recycled and their respective coloured trash can. Before starting the level, a brief audio explaining how to classify using the recycling bins. Once the game has started, there are two challenges that the player has the goal to overcome: Time (60 s) and Score. In this level, the player will be able to visualize the objects collected in the previous level and must classify them according to the corresponding recycling container. For the MDA framework, the mechanics of the game is to obtain or subtract points by relating an object to a recycling bin. The dynamics in this level consist of the player dragging the object on the screen to relate it to a recycling bin and visualizing the name of the object with a label by placing the mouse pointer over it. The main objective of the level is to correctly match the collected objects to the corresponding recycling bin. The recycling bins in this level are as follows: metal – yellow colour, organic – brown colour, paper and cardboard – blue colour, glass – grey colour and plastic – white colour. Figure 3 shows the scene presented to the player at the classification level.

3.3 Usage Recognition Module The main objective of the level is to correctly match the recycling bins with the corresponding reusability image. At the beginning of the level, a brief audio explains the second use that can be given to each material. Once the game has started, there are two challenges that the player must overcome: Time (60 s), Score. While in the classification level the game shows the corresponding colours for each trash can, in this level the player must associate the material to its new potential use, dragging the trash can to the image of the second use proposed. Regarding the MDA framework, the mechanics of the game is to obtain points by correctly associating the recycling can with an image of reusability; or to subtract points in case the association between the can and the potential reusability is not correct. The

A Serious Game for Recycling Education in Peruvian Children

55

Fig. 4 Usage Recognition Module

dynamics in the present level consist of the player dragging the recycling bin on the screen to match it with the reusability image of his choice and displaying the name of the recycling bin with a label by placing the mouse pointer over it. The recycling bins in the present level are as follows: glass bottle pots, plastic bottle pots, paper or cardboard crafts, organic compost and pencil cans. Figure 4 visualizes the scene presented to the uses level player.

4 Results 4.1 Experimental Design Recycling with Remi is a serious game, whose objective is to teach Peruvian elementary school children to recycle based on the colour code established in the Peruvian Technical Standard NTP 900.05. [13] The game was implemented for Web environments using the Unity video game engine. Currently, the game is hosted on the Unity Play server and can be played through various Web browsers, such as Google Chrome, Mozilla Firefox, and Microsoft Internet Explorer. To test the efficiency of the serious game in the acquisition of knowledge about recycling, in this work experiments were conducted with the participation of a group of 93 children (boys and girls) from the first (38 children) and second (55 children) grade of the primary school of the Lomas de Santa Maria school, located in the district of Chaclacayo, in Lima, Peru. Researches based in serious games related with environmental learning, carried out the evaluation in groups of 12 and 20 users [11, 14], and a research based on a serious game to evaluate group decision making skills, evaluated his game with a group of 18 users [16], being the sample proposed (93 users) greater than previous investigations. The ages of the children ranged from 6 to 8 years old. In addition, it is important to highlight that the experiments were conducted with parental consent, granted through a letter of informed consent.

56

P. Tocto et al.

During the experiments, each participant accessed the serious game through a Web browser available on their personal computers, and under the supervision of the teacher in charge of each section. Since the serious game does not consider levels of play, exposure to it had an average duration of 25 min per participant. To quantify the children’s knowledge acquisition, it was proposed to measure their levels of knowledge about recycling before and after exposure to the game through the ANOVA test conducted from an online questionnaire, which is hosted on the Google Forms server. The questionnaire was prepared based on the contents recorded in the Peruvian Technical Standard NTP 900.058. Formally, the children were asked to mark the correct alternatives to the following questions: • • • • •

What material can be recycled in the yellow container? What material can be recycled in the brown container? What material can be recycled in the blue container? Which material can be recycled in the white container? What material can be recycled in the grey container?

All questions were accompanied by the following alternatives: metal, food waste, paper/carboard, plastic and glass.

4.2 Results The results show that after using the serious game, the indicator of knowledge about the yellow container increases significantly since the ANOVA test obtained an F = 55.99 (greater than 3.89). The average number of hits before the game was 0.08; while after the game it rose to 0.52 (see Table 1). The indicator of knowledge about the brown container significantly increases, as evidenced by the ANOVA test presented in Table 2 with F = 8.27 (greater than 3.89). The average number of correct guesses before playing was 0.48, which increases after the game to 0.69. The indicator of knowledge about the blue container increases significantly, as the ANOVA test shows an F = 5.92 (greater than 3.89). Table 3 shows that the initial average hit rate of 0.53 increases to 0.70 after the game.

Table 1 Summary table and ANOVA of the knowledge question about the yellow container Groups

Count

Sum

Average

Variance

Before: Knowledge about the yellow container

93

7

0.08

0.07

After: Knowledge about the yellow container

93

48

0.52

0.25

Origin of variations

F

Critical value for F

Between groups

55.99

3.89

A Serious Game for Recycling Education in Peruvian Children

57

Table 2 Summary table and ANOVA of the knowledge question about the brown container Groups

Count

Sum

Average

Variance

Before: Knowledge about the brown container

93

45

0.48

0.25

After: Knowledge about the brown container

93

64

0.69

0.22

Origin of variations

F

Critical value for F

Between groups

8.27

3.89

Table 3 Summary table and ANOVA of the knowledge question about the blue container Groups

Count

Sum

Average

Variance

Before: Knowledge about the blue container

93

49

0.53

0.25

After: Knowledge about the blue container

93

65

0.70

0.21

Origin of variations

F

Critical value for F

Between groups

5.92

3.89

Table 4 Summary table and ANOVA of the knowledge question about the white container Groups

Count

Sum

Average

Variance

Before: Knowledge about the white container

93

26

0.28

0.20

After: Knowledge about the white container

93

47

0.51

0.25

Origin of variations

F

Critical value for F

Between groups

10.39

3.89

There was a significant increase in the indicator of knowledge about the white container (see Table 4), as evidenced by the ANOVA test that calculates an F = 10.39 (greater than 3.89). The initial average number of correct answers (0.28) showed an improvement (0.51) sufficient to be significant. Finally, Table 5 shows that the indicator of knowledge about the grey container also increases significantly. In this case, the ANOVA test shows an F = 32.76 (greater than 3.89). The average number of hits before the game was 0.12, a value that increased to 0.47 after the game.

58

P. Tocto et al.

Table 5 Summary table and ANOVA of the knowledge question about the grey container Groups

Count

Sum

Average

Variance

Before: Knowledge about the grey container

93

11

0.12

0.11

After: Knowledge about thegrey container

93

44

0.47

0.25

Origin of variations

F

Critical value for F

Between groups

32.76

3.89

5 Conclusions For the present research, a serious game called "Recycling with Remi” was developed and validated, which aims to contribute to children’s educational training in recycling. It was demonstrated that after using the game, the participating children have an improvement in knowledge about the use of garbage containers of all colours. The improvement is significant in terms of knowledge about the use of yellow, brown, blue, white, and grey bins. In addition to the technological and theoretical contributions, the present study opens new avenues for research. The multiplayer approach taken in previous studies [17], can be adopted in a new version of this game to exercise collaborative work, for example, by having an entire city recycle in the game. Also, the game could be adapted to other regulations, since the game is only applicable to Peruvian regulations. A limitation that occurred when evaluating the serious game was access to schools and participants, which was only in one school. The sample could be expanded in the future to more schools and grades to strengthen the results obtained. Future studies can also evaluate the effectiveness of this game in older populations of infants, with different sociodemographic characteristics. Likewise, to ensure the acquisition of knowledge, it is proposed to conduct a second survey after a few months to give greater validity to the technique applied to the present research. Additionally, metrics based on the player’s performance within the game are proposed, for example, the number of correctly classified elements, resolution time, score of hits and mistakes. Acknowledgements The authors wish to extend their profound gratitude to Professors Rosario Guzman, Paul Miller Tocto and Edwin Escobedo for their invaluable assistance throughout the development of this manuscript.

References 1. Fekete H, Kuramochi T, Roelfsema M, den Elzen M, Forsell N, Höhne N, Gusti M (2021) A review of successful climate change mitigation policies in major emitting economies and the potential of global replication. Renew Sustain Energy Rev 137:110602

A Serious Game for Recycling Education in Peruvian Children

59

2. Amin A, Liu X, Abbas Q, Hanif I, Vo X (2021) Globalization, sustainable development, and variation in cost of power plant technologies: a perspective of developing economies. Environ Sci Pollut Res 28(9):11158–11169 3. Schwarz A, Ligthart T, Bizarro D, De Wild P, Vreugdenhil B, Van Harmelen T (2021) Plastic recycling in a circular economy; determining environmental performance through an LCA matrix model approach. Waste Manage 121:331–342 4. Margallo M, Ziegler-Rodriguez K, Vázquez-Rowe I, Aldaco R, Irabien A, Kahhat R (2019) Enhancing waste management strategies in Latin America under a holistic environmental assessment perspective: a review for policy support. Sci Total Environ 689:1255–1275 5. Minam. https://sinia.minam.gob.pe/informacion/estadisticas 6. Limas R, Meoño J, Mougenot B (2017) Willingness to accept for municipal waste recycling: a case study from Lima, Peru. In: 30th international business information management association conference-vision 2020: sustainable economic development, innovation management, and global growth, IBIMA 2017. International Business Information Management Association, IBIMA, pp 1366–1375 7. Rossano V, Roselli T, Calvano G (2017) Multimedia technologies to foster ecological skills. In: 2017 IEEE 17th international conference on advanced learning technologies (ICALT), pp 128–130. https://doi.org/10.1109/ICALT.2017.76 8. Puerta LNZ, Álvarez MCG (2018) A methodological proposal to learn to program through the development of video games. In: 2018 13th iberian conference on information systems and technologies (CISTI), pp 1–6. https://doi.org/10.23919/CISTI.2018.8399326 9. de Oliveira Barreto A, Melo LB, Inácio LS (2021) Using gamification practices as pedagogical support in a non-governmental institution. In: 2021 16th iberian conference on information systems and technologies (CISTI), pp 1–4. https://doi.org/10.23919/CISTI52073.2021. 9476465 10. Idrobo M, Saenz M, Marceles K, Chanchi G, Vidal MI, Burbano C (2018) Recycling: a serious game focused on the classification of waste. In: 7th international congress, WITCOM, Mazatlán, Mexico, November 5–9, 2018, Proceedings. https://doi.org/10.1007/978-3-030-03763-5_20 11. Gaggi O, Meneghello F, Palazzi C, Pante G (2020) Learning how to recycle waste using a game. In: Proceedings of the 6th EAI international conference on smart objects and technologies for social good (GoodTechs 2020). Association for Computing Machinery, New York, pp 144–149. https://doi.org/10.1145/3411170.3411251 12. Environmental Protection Agency. https://www.epa.gov/recycle/recycling-basics 13. Norma Técnica Peruana 900.058. https://www.minam.gob.pe/gestion-de-residuos-solidos/ norma-tecnica-peruana-de-colores-ntp-900-058-2019/ 14. Castellano G, Carolis B, MacChiarulo N, Rossano V (2019) Learning waste recycling by playing with a social robot. In: Conference proceedings - IEEE international conference on systems, man and cybernetics. https://doi.org/10.1109/SMC.2019.8914455 15. Linehan C, Lawson S, Doughty M, Kirman B (2009) Developing a serious game to evaluate and train group decision making skills. In: MindTrek 2009: proceedings of the 13th international MindTrek conference: everyday life in the ubiquitous era, pp 106–113. https://doi.org/10.1145/ 1621841.1621861 16. Hunicke R, LeBlanc M, Zubek R (2004) Computer Science. https://users.cs.northwestern.edu/ ~hunicke/MDA.pdf 17. Neto N, Escudeiro P, Galasso B, Esdras D (2020) Development of an inclusive multiplayer serious game for blind and deaf. In: 2020 15th iberian conference on information systems and technologies, pp 1–6. https://doi.org/10.23919/CISTI49556.2020.9140906

University Degree Completion as an Indicator of Effective Institutional Quality Management in Higher Education Claudio Ruff , Marcelo Ruiz , Paola Juica , Bastián Gutiérrez , and Alexis Matheu

Abstract This study aims to analyze the characteristics of the universities and the university students that could explain the timely and effective graduation of the students of a cohort. The research studies the retention and university degree completion rates as universal concepts, for which it is used the experience of Chile. For this purpose, the study is focused on university bachelor’s degrees through a longitudinal follow-up of nine years (2011–2020) of an entering cohort. The materials and methods used are based on a quantitative correlational analysis of true cohort completion rates. The analysis is conducted with an R software in its RStudio interface. The main results are that only 16% of students complete their bachelor’s degree during the formal duration, and approximately 54% if analyzed up to three years later. Furthermore, alarming figures are presented, in which many of the current study programs do not surpass 30% of effective degree completion during the period studied. The same results can be found in evening or distance programs. It is argued that these results must be considered in the particular Chilean context since this country is one of the Latin American countries belonging to OECD that spend the most on higher education. In this sense, higher education institutions must question where to invest in the formative processes. It is concluded that it is crucial to enhance the public debate on tertiary education effectiveness and question the real impact of tuition-free education as state policy. Keywords University degree completion · Degree completion rates · Educational path

C. Ruff · M. Ruiz · P. Juica · B. Gutiérrez · A. Matheu (B) Institutional Analysis Center, Universidad Bernardo O’Higgins, Av. Viel 1497, 830993 Santiago de Chile, Chile e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_6

61

62

C. Ruff et al.

1 Introduction One of the current challenges facing higher education is to consolidate and install components that ensure the degree processes under parameters defined by quality management. This implies implementing a series of institutional coordination mechanisms that conceive the degree process as a permanent objective, sustained under the perspective of continuous improvement and evolution [1]. In this context, it seems fundamental to seek a deep understanding of this process, whose multidimensionality requires considering, as part of the comprehensive analysis elements such as teaching culture, learning experiences, student progression, among others [2]. This vision overcomes the idea that considers as sufficient evidence of evaluation of this work the mere rendering of accounts or the mere projection of statistical data, but, instead, promotes the design of a quality system based on the formative nature that recognizes the degree stage as one of the culminating steps of the teachinglearning processes of students [3]. Thus, it is necessary to promote ponder about the degree, respecting the independence and autonomy of each student and assuming that this is a task of institutional responsibility that involves all its strata and not only teachers, so it is necessary to establish criteria and articulate instruments that make up an effective and efficient coordination system [4]. The research aims to analyze the characteristics of the universities and the university students that could explain the timely and effective graduation of the students of a cohort.

2 Literature Review 2.1 Critical Factors of the Degree Process. Several positions contribute to analyzing the elements that constitute this problem. Among those, the ones proposed by Royo [5], who determines critical factors of this process in his analysis, namelyLack of institutional orientation to students about the process; Deficient teacher training that prevents new approaches to the curriculum of teach-ing and learning methodologies; Lack of unification of criteria regarding the degree, adhering to a unitary and inte-grated project; Lack of control and transparency over the whole of the parts of the de-gree process and Lack of coordination and collaboration among the teachers who share this process. These weaknesses are interrelated and impact the quality of the entire system and prevent any future improvement of the system, making it unavoidable to implement a management process that addresses these problems. This requires changes in the governance structure of the institutions, which must be able to assume the degree as a shared and joint project.

University Degree Completion as an Indicator of Effective Institutional …

63

At this point, several questions arise related to this process, such as, for example, those that aim to determine whether the current quality assurance mechanisms have been able to guarantee the effectiveness of timely graduation, or whether they continue under a bureaucratic management system that not only slows down the actions of the actors involved but also has a direct impact on the students. Likewise, in the national context, questions are raised about the transparency of the results, explicit in the absence of data on timely graduation in Chile and the transcendental question of what is happening with the promotion of tuition-free education versus timely graduation. It is necessary to clarify some concepts related to the study problem, such as the definition of timely graduation. Indeed, according to the terms specified by the National Accreditation Commission [6], timely graduation is measured considering the information of the last five cohorts of a major and is defined as the degree that is carried out with a maximum term of one additional year to the theoretical duration of the major. Based on the above definition, it is significant to emphasize how fundamental it is to have at the country level, clear records and standards that support a strategic analysis of this background which, in the words of the Organization for Economic Co-operation and Development [7], constitutes one of the measures that evidence the degree of development of a nation by producing and training a qualified and professionally skilled workforce.

2.2 University Dropout and Retention and Its Impact on Degree Completion The explosive increase in coverage that the expansion of enrollment in higher education in Chile has undergone in recent decades, which, according to the latest studies on the subject, has reached figures in the order of 47.1% in “Immediate Access,” two points less than the sustained rate of increase that was maintained, until 2015, at 49.1% [8] has generated a high impact on the development of the Chilean university system. However, it is not possible to ignore that, despite the increase in enrollment, other factors, such as those related to retention and university dropout, affect the evaluation of the entire formative system [9]. Regarding the concepts of retention and dropout, the former is understood as the percentage of enrolled students who persevere in their studies during the first year of their majors; and dropout is understood as those groups that abandon their majors during the course of their studies. It is possible to point out that the studies carried out by the Higher Education Information Service of the Ministry of Education [9], determine that in Chile, out of the 25 universities with the highest enrollment, only 16 of them manage to exceed retention rate of 80%, while the student dropout reaches 27.8%.

64

C. Ruff et al.

The situation described has worsened in these times of pandemic since in 17 university institutions there was a flight due to desertion or freezing of their studies of approximately 9 thousand students (11%). In turn, 29,902 debtors of the Stateguaranteed loan (by its Spanish acronym, CAE) were unable to meet their payments because they were unemployed. In similar numbers, 24,682 requested reductions in their installments [10]. All of the above has substantial repercussions on the costs associated with professional training, which, when not completed, frustrates the expectations and quality of life students who, without a professional degree, find it very difficult to compete in the professional market. Thus, contributing to the promotion of the values that today support the educational reforms in Chile, such as the promotion of quality education, more just, comprehensive, and equitable, overcoming the significant social gaps that mark the differences given by the social disadvantage of the most vulnerable sectors [11].

2.3 Timely Graduation According to experts on the subject, the low percentages of timely graduation are due to multiple factors, such as the poor training base, the lack of flexibility of the programs, the lack of institutional support, or other factors such as the need that many young people have to work and study at the same time, the excess of courses (in the order of seven per semester) compared to four courses that in most developed countries are taken per semester, the existence of prerequisites, the poor foundations of school education, among others [12]

2.4 Tuition-Free Higher Education in Chile In recent years, higher education in Chile has undergone a complex transition from a self-financing regime to one governed by tuition-free education, where the vision of the role of the State has changed profoundly, transmuting from being classified as a merely subsidiary entity to being demanded as benefactor [13]. In this scenario, since 2014, a plan was launched to open the debate on the issue of tuition-free education in higher education. The measures began to be implemented in 2016 and are reflected in the Universal Free Education Law that has been in force since 2018. It established that since education is a right, it must be available to all people considering their skills, abilities, and merits without any kind of discrimination [14]. It is important to highlight that some of the requirements established for accessing tuition-free education state that all students from households that can prove that they belong to the 60% of the country’s lowest-income bracket will be entitled to it. In addition, it should be considered that if the student already has the benefit of free tuition, this is lost at the moment of changing to an institution not attached to it [15].

University Degree Completion as an Indicator of Effective Institutional …

65

Additionally, one of the most important requirements when it comes to maintaining the free tuition is that once the student is already studying in higher education, the time of these cannot exceed the nominal duration of his/her major degree. Consequently, it is highlighted that only in 2019 a total of 17,310 students lost the benefit of free tuition due to this cause and that, in the last five years of the 789 thousand students who have been beneficiaries of free tuition, 68,766 of them have lost this benefit for exceeding the nominal duration of their majors. Moreover, 74.6% of those who lost the benefit had access to a state loan to continue financing their studies [16]. According to Rodrigo Valdés (former Finance Minister of President Bachelet), one of the objectives of the law was to eliminate co-payment and subsidize access to education for the poorest sectors of the population. Therefore, the State became the entity that assumed the burden and the financial weight in tuition and fees. The effect of this law was, effectively, to put an end to co-payment by transferring the debt to the State.

3 Materials and Methods This study has a quantitative approach. Its general objective is to analyze the characteristics of the universities and the university students that could explain the timely and effective graduation of the students of a cohort. The specific objective of this research is to analyze the following variables of a cohort: gender, age range, type of institution, institution, day, length of major, accreditation of the institution, years of accreditation, and year of graduation, in order to achieve the general objective. In order to achieve the goals proposed, the authors developed three research stages: (1) to analyze the state of the art of degree completion and quality assessment in tertiary education. Also, in this stage, the databases for the analysis were identified. (2) To analyze the data quantitatively, considering the variables previously described. (3) To compare the studies from the stage (1) with the results obtained by the authors, aiming to develop the discussion and conclusions sections of this paper. Student progression in tertiary education is analyzed in the concept of academic trajectories. The concept describes measurable characteristics of a cohort of students during their curriculum at the tertiary level or between two different educational levels, in which three dimensions stand out: temporal component, efficacy, and academic performance. The following study is aimed at effectiveness in the indicator that links it to the effective completion of the curricular program. The indicator studied is calculated at the international level from two perspectives: (1) the graduation rate, which is the percentage of graduates according to the age range of the population, and (2) the one used in this study: the completion rate, which is the percentage of students in a cohort that completes tertiary education in a given year.

66

C. Ruff et al.

The data used in this study are the enrollment and graduation data of higher education in Chile from 2011 to 2020, information available from the Ministry of Education, with more than 15 million records. Variables were generated from the fourth year to the ninth year of graduation, plus the number of enrolled students of the cohort, type of institution, course of study, and years of accreditation of the institution. The information is of a census nature since all available records are used. This study used two regression models applied to the variables of gender, age range, type of institution, institution, day, length of major, accreditation of the institution, years of accreditation, and year of graduation for the students of the 2011 cohort described and filtered according to what has already been mentioned in this document. The year of graduation of the students of the 2011 cohort and the graduation or not of the students in the ninth year of training were considered as dependent variables, introducing in the models all the other variables previously mentioned as independent variables, to show which are the characteristics of the institutions or the students that explain in a more significant way the timely and effective graduation of the students of the cohort analysis. The forward method (linear regression in the first model and logistic regression in the second) was used to adjust the model for the selection of variables, analyzing the significant eigenvalues during the construction of the model. Likewise, Fisher’s statistical model was applied to evaluate the stoppage of the algorithm and the selection of the independent variables for the final model. Only students who continued their educational trajectory in the same institution of origin were considered in the models.

4 Results From the contextual point of view, the OECD report Education at a Glance (2019), which analyzes the scenario in this area of the 36 member countries, introduces a concept that has only been analyzed in that year, the concept of completion rate (percentage of students who access and educational program and graduate from it after a specific number of years). The OECD study shows that, in Chile, based on 2017 data, only 16% of those who enter university graduate within the formal duration of their major, and even in OECD countries, it reaches only 40%. Even three years later, the indicator in the case of Chile reaches 54% Using the methodology suggested by the OECD, the study results that follow the 2011 cohort for new years since university entrance describe the present Chilean situation by several variables. In addition, there are no significant differences in the indicator by type of institution. Firstly, Table 1 confirms the OECD data, given that in the fifth year of study, the great majority of the majors in Chile have a graduation rate of 15.94%. In addition, it is observed that three years later, 53.64% of the students graduate.

University Degree Completion as an Indicator of Effective Institutional …

67

Table 1 Completion rate of undergraduate students, 2011 cohort, up to 9 years later, graduates in any institution and any degree program Indicators

Enrolled

Graduates year 4

year 51

year 62

year 7

year 83

year 9

CRUCH* State Universities

37,409

4.00%

12.33%

48.69%

42.04%

52.70%

58.28%

Private Universities

93,041

4.54%

18.04%

22.35%

44.54%

53.13%

56.87%

CRUCH Private Universities

27,52

3.62%

13.75%

97.52%

44.67%

56.61%

62.41%

157,97

4.25%

15.94%

15.94%

43.97%

53.64%

58.17%

Grand total *

Council of Rectors of Chilean Universities (by its Spanish acronym, CRUCH) Source: own elaboration

Table 2 Completion rate of undergraduate students, 2011 cohort, up to 9 years later, graduates in the same institution and in any degree program Cohort 2011 Graduates in any major

Graduates Enrolled

year 4

CRUCH State Universities

29,702

2.25%

Private Universities

81,129

CRUCH Private Universities Grand total

year 5

year 6

year 7

year 8

year 9

9.72%

24.14%

37.89%

46.05%

50.78%

2.88%

15.35%

28.01%

36.49%

41.47%

43.67%

23,096

1.22%

10.23%

23.90%

38.22%

46.67%

50.73%

133,927

2.46%

13.22%

26.44%

37.10%

43.38%

47.01%

Source: own elaboration

The first analysis for discussion is the real impact of free tuition on the most vulnerable students, given that the higher education law restricts the benefit to the years of the actual duration of a degree program. The above table opens a clear question about the effectiveness of the policy. If the indicator is restricted to the same institution, the results drop significantly by 10 points in general (Table 2), which includes another element of analysis to the tuition-free element, given that students can only have a change of institution or major degree to keep the benefit.

1

First reference OCDE. Timely graduation. 3 Second reference OCDE. 2

68

C. Ruff et al.

Table 3 provides other elements of analysis, such as the indicators of timely graduation of the Chilean tertiary system in the daytime, which corresponds to approximately 27% by cohorts, where private universities have 29.45% in the indicator, slightly higher than the CRUCH universities. In the eighth year, the results are similar. Furthermore, the decrease of the indicator at the eight-year in the distance and evening courses stands out, which fuels the discussion of the effectiveness of studies in these modalities. The analysis by training programs shows that only one major exceeds 60% of graduates in the ninth year, Medicine, which confirms the importance of the variables of entry behaviors in student retention [17]. In addition, only 16 majors (13.56%) of the 118 with more than 50 enrolled in the cohort exceed 50% at the ninth year, while 32.3% of the majors do not exceed 30%. Finally, this first analysis by institution provides us with other important elements; in the first place, only 12% of the institutions exceed 60% retention in the ninth year. Most of these institutions are aimed at entry profiles with students from the highest income quintile or selective regional universities. Also, similar behaviors are observed in most institutions up to the third year of the formal duration of the degree Table 3 Completion rate of undergraduate students, 2011 cohort, up to 9 years later, graduates in the same institution and in any major by daytime Cohort 2011 Graduates in any major

Graduates Enrolled year 4

Distance Learning 119 (Private Universities)

13.45%

year 5

year 6

year 7

year 8

year 9

16.81%

17.65%

19.33%

21.85%

21.85%

Daytime learning

117,774

2.19%

13.06% 27.04%

38.62%

45.44% 49.36%

CRUCH State Universities

28,92

2.06%

9.48% 24.05%

38.01%

46.35% 51.82%

Private Universities

66,033

2.62%

15.63% 29.45%

38.98%

44.56% 47.51%

CRUCH Private Universities

22,821

1.11%

10.16% 23.88%

38.33%

46.83% 51.61%

Other

1,59

5.85%

17.17%

29.50%

34.03%

36.10%

37.30%

Private Universities

1,59

5.85%

17.17%

29.50%

34.03%

36.10%

37.30%

Semi-attendance

166

18.67%

32.53%

36.75%

37.95%

39.16%

39.76%

Private Universities

166

18.67%

32.53%

36.75%

37.95%

39.16%

39.76%

Evening learning

14,278

4.01%

13.84%

21.07%

25.06%

27.48%

28.93%

CRUCH State Universities

782

9.46%

18.29%

27.49%

33.25%

34.91%

35.81%

Private Universities

13,221

3.56%

13.52% 20.61%

24.50%

26.91% 28.35%

CRUCH Private Universities

275

16.36%

28.73%

33.82%

Grand total

133,927

Source: own elaboration

10.18%

25.09%

37.45%

2.46% 13.22% 26.44% 37.10% 43.38% 47.01%

University Degree Completion as an Indicator of Effective Institutional …

69

Fig. 1 Correlation of study variables. Source: own elaboration

programs, and 33% of the universities do not manage to exceed 40% four years after the formal graduation year.

4.1 Statistical Results of the Analyzed Models Figure 1 shows two relevant but predictable correlations, first, between the accreditation status and the years of institutional accreditation, with a positive correlation of 0.52; and second, a positive correlation between the age range and the school day. The adjustment of the free models is started without considering any independent variable to select the one with the highest significance or the one with the highest value for Fisher’s statistic, being those listed in Table 4 the most significant of the model. Thus, the following variables are concluded as explanatory variables of this model around the year of graduation: years of accreditation, gender, daytime, institutional accreditation, and age range, in order of relevance for the model. This first model opens important questions, in the case of the variable with the highest incidence in this model, corresponding to the years of accreditation, as observed in contingency the more years of accreditation, the higher the average duration of the years of the degree, which does not imply whether there is a better degree, a concern that will be answered in model 2. This variable provides us with an element to understand the actual effectiveness of the training programs. That is why it should be considered if timely graduation is an appropriate indicator or if it is necessary to move towards more modern definitions of the concept, such as the one explained in the case of the OECD, 2019.

70

C. Ruff et al.

Table 4 Most significant variables of the model Step

Chosen variable

Df

Step 1

Years of accreditation

Initial model: Year of degree ~ 1

Gender

Model: Year of degree ~ Years of accreditation

Step 2

1

1 Step 3

Daytime

Institution accreditation

Age range

1,33E + 15

1,868,662

1,30E + 15

1,865,893

F Value

Pr(>F)

4593,333

< 2,2E-16

2800,01

< 2,2E-16

2,3E + 09

1,28E + 15

1,863,485

609,262

< 2,2E-16

2,9E + 08

1,28E + 15

1,863,180

307,583

< 2,2E-16

Model: Year of degree ~ Years of accreditation + Gender + Daytime + Institutional accreditation 6

Final model

2,70E + 13

AIC

Model: Year of degree ~ Years of accreditation + Gender + Daytime 1

Step 5

4,51E + 09

RSS

Model: Year of degree ~ Years of accreditation + Gender 4

Step 4

Sum of Squares

1,55E + 09

1,26E + 15

1,861,544

276,301

< 2,2E-16

Model: Year of degree ~ Years of accreditation + Gender + Daytime + Institutional accreditation + Age range

Notes: DF - Degrees of freedom; RSS – Residual Sum of Squares; AIC - Akaike Information Criterion Source: own elaboration

In the case of the variable study period, it is important to note that the years of graduation of the periods with the lowest graduation rates (distance and evening programs) are those with the lowest average number of years of study. This paradox raises the question of whether the duration of the degree programs as currently measured is an indicator of effective quality (see Table 5). In the case of gender, women last an average of 6.21 years compared to 6.43 for men. In this sense, the logistic regression model 2 (Table 6) highlights that the same Table 5 Contingency table of the study day variable, model 1

Daytime

Average number of years of graduation

Distance learning

5.24

Daytime

6.42

Other

5.37

Semi-attendance

4.85

Evening

5.52

Grand total

6.31

Source: own elaboration

University Degree Completion as an Indicator of Effective Institutional …

71

variables of the previous model are repeated so that the combined interpretations of both provide a comprehensive view of the effectiveness of the degree indicators. Regarding the variable years of accreditation, again the main variable of the model contingency. Table 7 clarifies the question posed in the first model analyzed, that is, 7-year universities present the highest percentages of graduation at the 9th year, but this does not occur in the times of timely graduation. Under the information provided, the following question or discussion appears: The training processes of universities accredited for 6 and 7 years, finally, have better indicators of total retention, and not of graduation or formal duration. This can be interpreted given the characteristics of the students entering these universities (i.e., entry behaviors) or a supposed higher demand for their training programs. When the same analysis is carried out with the gender variable (Table 8), the situation is not the same; female students have lower study duration rates than male students and have much higher graduation rates than male students. Table 6 Adjustment of forward 2 model, for students graduated in the same institution of origin versus accredited or not accredited students. Step

Chosen variable

Df

Step 1

Years of accreditation

Initial model: Graduated ~ 1

Step 2

Gender

Step 3

Daytime

1

Deviance 183,179

AIC 183,183

F Value

Pr(>F)

3323,524

< 2,2E-16

Model: Graduated ~ Years of accreditation 1

180,416

180,422

2074,249

< 2,2E-16

Model: Graduated ~ Years of accreditation + Gender 4

178,012

178,026

457,171

< 2,2E-16

Model: Graduated ~ Years of accreditation + Gender + Daytime

Final Model

Notes: DF - Degrees of freedom; AIC - Akaike Information Criterion Source: own elaboration.

Table 7 Contingency table of the variable years of accreditation, model 2 Years of accreditation

Graduated in the 4th year

Graduated 5th year

Graduated 6th year

Graduated 7th year

Graduated 8th year

Graduated 9th year

0

8.51%

18.79%

27.54%

31.75%

34.22%

36.00%

1

4.38%

12.95%

21.91%

29.76%

35.58%

40.17%

2

4.10%

12.40%

18.51%

22.05%

23.81%

24.69%

3

4.74%

18.86%

31.01%

39.89%

44.68%

47.62%

4

4.17%

17.01%

33.35%

44.47%

51.50%

55.29%

5

2.70%

16.97%

33.77%

47.38%

55.42%

59.51%

6

4.28%

12.30%

27.04%

41.87%

51.13%

56.78%

7

3.41%

11.68%

28.65%

47.21%

58.14%

64.12%

Source: own elaboration

72

C. Ruff et al.

Table 8 Contingency table of the gender variable, model 2 Gender

Graduated in the 4th year

Graduated 5th year

Graduated 6th year

Graduated 7th year

Graduated 8th year

Graduated 9th year

Male

3.98%

12.63%

23.40%

33.96%

40.62%

44.63%

Female

4.29%

19.09%

35.63%

47.39%

53.97%

57.51%

Source: own elaboration

In short, it could be pointed out that this analysis confirms the results presented by OECD data [7] regarding the degree completion of university students in Chile. In fact, only 16% of those students who enter a program graduate within the formal duration of their major. Also, the results revealed that variables of entry behavior, such as income, are essential to understanding study retention. Concretely, the institutions that exceed 60% of retention by the ninth year of a cohort comprise only 12% of the institutions analyzed. Those institutions are the ones that have a prominent quantity of students from the highest income quintiles or the ones that correspond to selective regional universities. Finally, it could be stated that the most significant variables in the results are years of accreditation, gender, daytime, the institution of accreditation, and age range.

5 Conclusions This study agrees with the proposal of Pérez et al. [17], who state that the entry behavior variables are crucial in understanding the implications of students’ retention in tertiary education. Despite considering the completion rate as an indi-cator of institutional efficiency [7], this research proposes that the entry behavior variables should also be taken into account. Another consequence of deepening the study results is that the Chilean state loses a significant proportion of the investment it has made in ensuring tuition-free education for its most vulnerable students. In fact, Chile is one of the coun-tries that, in the last decades, has made a significant investment in education, concerning the state cost in higher education, the amount reached 3,102 million dollars, i.e., 3.9% of all fiscal expenditure made by the State was allocated to higher education of which 38% was in support of financing tuition-free education [12]. Consequently, Chile is one of the countries that spend more than any other Lat-in American country belonging to the OECD on higher education, in relation to its gross domestic product (GDP), which is worrying if one considers that the benefits associated with tuition-free education are lost when students do not comply with timely and formal degree projections. The study provides information that does not exist in official agencies on the educational processes and the real indicators to measure their impact. It is a pioneer in the descriptive and correlational delivery of variables of student graduation rates,

University Degree Completion as an Indicator of Effective Institutional …

73

daytimes of formative processes, undergraduates majors, gender, accreditation, type of institutions, and study programs, providing the public and academic debate with the relationships between the variable graduation with retention, entrance and graduation profiles, institutional accreditation and opens a debate regarding tuition-free education, use of public resources and the real use of effectiveness indicators. The idea proposed above means that the actual duration of majors and the number of degree completion per year offer only partial information on the problem studies. Thus, it is important to analyze the terminal efficacy of tertiary education programs or institutions. The figures, which also coincide with the results of the OECD 2019 and other studies analyzed, state that close to 54% of the students who enter a university study program manage to complete their studies, thus questioning the effectiveness of the entire system. This effectiveness is even more questionable in evening or non-face-to-face programs and a large percentage of daytime programs. Future studies could compare the most significant variables analyzed in this research between institutions of the same region. Hence, each variable and its impact on degree completion could be studied deeply. Furthermore, this research could be translated into a ranking that aims to provide quality assurance. Finally, the study provides another aspect to the effectiveness of free tertiary education as public policy, since it reaffirms the existing doubts about the effect of state coverage of university education on the effective equality of opportunities for the most vulnerable students: a phenomenon that is accompanied by process of deterioration of the quality provided by the institutions. This new aspect tells us whether State spending has the expected reward for students, given that only one out every two students manage to finish their studies. Acknowledgements This research was funded by the Institutional Analysis Center of the Bernardo O’Higgins University. The authors declare that they have no conflicts of interest.

References 1. Del Valle Lobos, C.C.: A look at university evaluation processes. In: Book of abstracts CIVINEDU, 2nd International Virtual Conference on Educational Research and Innovation, p. 79. Adaya Press (2018) 2. Ortigosa, E, Martín Garzón, E., Ortigosa, P, Casanova, J, Romero, L.: Control and Improvement of the Coordination Between Subjects of a University Degree (2015) 3. Ganga-Contreras F, Muñoz M, Codjambassis A (2018) Actions developed in the field of teaching management: a preliminary approach to a university department in Chile. Space Magazine 39(25):23–47 4. Carvajal CM, González JA, Tassara CA, Álvarez MS (2018) Over-duration: an approach quantitative. Training Col. 11(3):19–28 5. Royo JP (2012) Ten principles for a quality management system specifically designed for the coordination and internal improvement of university degrees. REDU, Teaching Magazine Univ. 10(3):49–69

74

C. Ruff et al.

6. Comisión Nacional de Acreditación, CNA [National Accreditation Commission (NAC)].: Clarification to some queries about the use of forms for the accreditation of professional careers and degree programs (2010) 7. OECD.: Education at a Glance, OECD Indicators. OECD Publishing, Paris. https://doi.org/10. 1787/f8d7880d-en (2019) 8. Consejo Nacional de Educación Superior (CNED) [National Council for Higher Education (NCHE)].: Ministry of Education, Trends Report on Higher Education Statistics by sex. https:/ /www.cned.cl/indices-educacion-superior. Accessed 8 Sep 2020 9. Gil FCC, Gil EBC (2019) Academic reading and writing in relation to graduation rates in university education. Magazine RedCA 2(4):127–138 10. Ingresa, Sistema de Créditos de Estudios Superiores [Enter, Higher Education Credit System], Public Account. https://portal.ingresa.cl/?s=pandemia. Accessed 8 Sep 2020 11. Arrau F, Loiseau V (2019) Desertion in Higher Education in Chile 12. DIPRES, Dirección de Presupuesto Gobierno de Chile [Chilean Government Budget Directorate].: Higher Education Budget. https://www.dipres.gob.cl/598/w3-search.php. Accessed 8 Sep 2020 13. Gonzalez L, Espinoza O (2016) Gratuity in higher education in Chile: ups and downs and challenges. In: Breakpoint. The new Constitution and citizenship. Equitas Foundation, pp. 75– 99 14. Biblioteca del Congreso Nacional de Chile, BCN [Library of the National Congress of Chile, LNC].: Higher education and gratuity. https://www.bcn.cl/leyfacil/recurso/educacion-superiory-gratuidad. Accessed 8 Sep 2018 15. Biblioteca del Congreso Nacional de Chile, BCN [Library of the National Congress of Chile, LNC].: Law 21091, On Higher Education. https://www.bcn.cl/leychile/navegar?idNorma=111 8991. Accesesed 8 Sep 2018 16. Corporación de Investigación, Estudio y Desarrollo de la Seguridad Social [Corporation for Research, Study and Development of Social Security]. https://www.ciedess.cl/601/w3-article8264.html. Accessed 8 Sep 2021 17. Pérez AM, Escobar CR, Toledo MR, Gutiérrez LB, Reyes GM (2018) Prediction model of first-year student dropout at Bernardo O’Higgins University. Educ Res 44:56–88

Scientific Research and Writing Skills in Engineering Students Rosa Núñez-Pacheco , Elizabeth Vidal , Aymé Barreda-Parra , and Eveling Castro-Gutierrez

Abstract Current education demands that university students possess a series of skills, among which scientific research and writing skills are essential for the future professional. The field of engineering is not exempt from this academic demand. The purpose of this study was to compare the results of the pre- and post-test on the research skills and scientific writing of students in an engineering program. The study is a quasi-experimental type of pretest and post-test design in the subject “Writing and Research Reports”. The participants were 94 students, 82 (87.2%) males and 12 (12.8%) females. The Self-rated skills for research-based learning (AHABI) instrument was applied and statistically significant differences were found in the three factors: process, management and development of information, and the total scale with a positive increase in research skills at the end of the semester. It is concluded that scientific writing courses have a positive impact on the development of research skills. Keywords Investigative skills · Engineering · Higher Education

R. Núñez-Pacheco (B) · E. Vidal · A. Barreda-Parra · E. Castro-Gutierrez Universidad Nacional de San Agustín de Arequipa, Arequipa, Perú e-mail: [email protected] E. Vidal e-mail: [email protected] A. Barreda-Parra e-mail: [email protected] E. Castro-Gutierrez e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_7

75

76

R. Núñez-Pacheco et al.

1 Introduction The development of research skills is one of the basic processes at any level of education [1] especially at the higher level. Research skills comprise a set of knowledge, skills and values that allow the conscious assimilation of the scientific method and its application in the generation of knowledge and the solution of problems [2, 3]. The mastery of research skills is not only the responsibility of researchers, but also of teachers, who must develop complex learning in their students according to the current and future society [1, 4]. Rojas and Aguirre [5] explain that it is not only necessary to talk about research skills, but also about competencies and attitudes, which include knowing how to observe, synthesize, abstract, communicate and think critically. The teaching of research skills involves the active participation of teachers through the application of didactic strategies and methodologies that awaken the scientific interest of students and at the same time provide the theoretical bases necessary to obtain adequate results [6]. The main purpose of the development of these skills is mainly focused on the writing of a scientific article, which must meet certain requirements and standards for its subsequent publication. The study of research skills and the effects thereof is not a recent topic. Salazar et al. [4] conducted a study on the application of these skills in higher education and concluded that improvements in the curricula should be considered, in which priority is given to scientific research and teacher preparation. For the correct learning of research skills, the teacher must be able to guide the student so that he/she can act independently, which is why this type of course falls into the category of nontraditional education [6–8]. One of the experiences on the teaching of research skills corresponds to the study by Llulluy-Nuñez et al. [7], who report on the incorporation of a program to enhance the research skills of students in an engineering school. There is also the case described by Carlín et al. [8] who refer to a call for a university contest of scientific articles to encourage research activity in students through the presentation of talks, posters and papers. The students received training given by research professors. In addition, scientific writing workshops were held to develop and strengthen research skills [9, 10]. In engineering geology, Walton [11] paid special attention to the improvement of student performance and the factors predicting the effectiveness of peer reviews to help students improve their scientific reports. As Pozzo [10] indicates, engineering education at the scientific level is closely related to the development of research, which, at the same time, requires specific research skills and mastery of academic writing. For this reason the mastery of formal and effective academic writing that allows establishing scientific communication is established as an emerging need, applicable to undergraduate students as well as to engineers already graduated, This means that engaging with scientific literature is a skill of essential importance for students and researchers who are part of scientific degree programs [12].

Scientific Research and Writing Skills in Engineering Students

77

On the other hand, engineering writing is a competency that is often considered a crucial skill for engineers to develop, but is often underestimated in the undergraduate or graduate curriculum [13]. It should be considered that the teaching of scientific writing in engineering is not the same as in the humanities or social sciences [14]. Engineering students are taught to differentiate the variety of styles and genres that exist such as technical report, scientific article, essay, etc. [15]. The desirable characteristics of engineering writing are: simple sentences, precise word choice, correctness in grammar and punctuation, and a linear organization. These characteristics allow for effectiveness in areas that professionals consider important: precise and unambiguous content, fast and predictable reading; and attention to detail [16]. Scientific research and writing skills can be learned and improved through specific activities and resources. Since writing and reading scientific articles appropriately are key competencies, the importance of developing correct scientific writing in students cannot be ignored. There is a lot of research on the development of such skills. [13, 17–19]. In the same line, we can mention the work of Vidal and Castro [20], who report the experience in the development of technical writing for engineering students based on the cognitive process of the writing theory of Flower and Haye [21] and the sociocultural perspective. It should also be noted that in scientific writing in engineering, the use of specialized software such as LaTeX [22, 23], or the use of open source code tools such as AcaWriter [24]. Finally, Vidal and Toro [25], emphasized the specialized information search stage based on information literacy standards, which is the set of skills that allow locating, evaluating and effectively using the necessary information. Developing these skills allows students to master the content and expand their research, become more selfdirected and assume greater control over their own learning. The current research aims to compare and analyze the research skills of engineering students before and after participating in a writing course.

2 Methodology 2.1 Objective The objective of this research was to compare the results of the pre- and post-test on the research skills and scientific writing of students in an engineering program.

78

R. Núñez-Pacheco et al.

2.2 Design and Participants The study is of a quasi-experimental type with pretest and post-test design. The teaching–learning process of the subject “Writing and Research Reports” was evaluated, which has three main components: Planning, Writing and Reviewing, taking as a basis the proposal of Hayes and Flower [26] as a cognitive process. In the planning phase, emphasis was placed on the search for information and the elaboration of an outline. During the writing phase, each of the parts of the scientific article were developed. The IEEE format for conferences is used, considering each of the sections and the recommendations both at the writing level and the use of citations and references, tables, figures, etc. The sample consisted of 94 students, 82 (87.2%) males and 12 (12.8%) females, aged between 16 and 41 years (M = 21.02 and SD = 3.87). The sample was nonprobabilistic and intentional, and consisted of students enrolled in the course “Writing and research reports”, which is carried out in the third semester of the Systems Engineering course at a public university in Peru. The participants gave their informed consent and agreed to participate voluntarily and confidentially in the study. They completed the research skills scale during the regular classes of the subject in their respective sections: A (40.4%), B (24.5%) and C (35.1%). In the pretest, the questionnaire was completed by 94 students; and in the post-test, by 65 students.

2.3 Instrument The self-rated skills for research-based learning (AHABI) instrument was used. The questionnaire was validated by Cobos et al. [27] in engineering students. It consists of 20 items, according to a Likert scale (1 means strongly disagree and 5 means strongly agree). It is composed of three factors: 1) processing scientific information, 2) handling scientific information, and 3) developing scientific information. Cronbach’s alpha, in the validation of Cobos et al. [27], reaches values of 0.91 for the total scale, 0.891 for factor 1, 0.711 for factor 2 and 0.687 for factor 3. In this study Cronbach’s alpha is 0.923 for the total scale, 0.872 for factor 1, 0.845 for factor 2 and 0.715 for factor 3.

2.4 Procedure Data were collected during the first academic semester of 2021. An online questionnaire was applied to obtain information about the students’ research skills. The pretest was applied at the beginning of the course, and the post-test at the end of the course, after 17 weeks, which is the time the academic semester lasted.

Scientific Research and Writing Skills in Engineering Students

79

2.5 Data Analysis For data processing and analysis, a descriptive analysis was performed, represented in tables with measures of central tendency and standard deviation. One-factor ANOVA was applied to compare the means of each factor of the questionnaire. For the difference between pretest and post-test scores, the Kolmogorov–Smirnov test was applied to determine the normality of the sample. Given the results obtained, the nonparametric Wilconxon test was used to observe the magnitude of the change before and after the development of the subject. A significant value of p < 0.05 was considered for comparison. To calculate the size, Cohen’s d was used with values from 0.2 to 0.3 indicating a small effect, around 0.5 a medium effect and greater than 0.8 a large effect (Cohen, 1998). The SPSS 23 statistical package was used for data analysis.

3 Results Table 1 describes the mean and standard deviation of the scale items answered at the beginning of the semester. Only three items obtained a mean above 3.5 (items 2, 9 and 10). In general, the averages are located in a midpoint that ranges between 2.51 and 3.33. The mean and standard deviation obtained by the students in the total scale and in the three factors is as follows: 48.40 (9.48) in the total scale, 25.35 (5.30) in “Process scientific information”, 21.35 (4.93) in “Handle scientific information” and 14.83 (3.23) in “Develop scientific information”. The scores decrease as the complexity of the skills increases; the third factor, develop scientific information, has the lowest mean. The results obtained at the beginning of the course allow the teacher to focus the teaching on this factor to improve the students’ research skills. The scores decrease as the complexity of the skills increases; the third factor, developing scientific information, has the lowest mean. The results obtained at the beginning of the course allow the teacher to focus the teaching on this factor to improve the students’ research skills. Table 2 describes the comparison of the participants’ results in terms of research skills with their corresponding factors evaluated by means of the Wilconxon test. Statistically significant differences are found in the three factors: process, information management and development, and the total scale with a positive increase in research skills at the end of the semester.

80

R. Núñez-Pacheco et al.

Table 1 Perception of research skills and scientific writing Items

Mean

Standard deviation

1. I management research articles of a theme drawn from scientific journals, databases, etc

2.82

0.85

2. I recognize a scientific paper in document Wikipedia, Rincón del vago, etc

3.74

1.08

3. I know what is literature review

2.51

0.97

4. I identify scientific journals

3.28

1.01

5. I recognize database of scientific journals

2.97

0.94

6. I identify the structure of a scientific research article

3.12

1.00

7. I use scientific techniques to organize information

2.76

0.99

8. I analyze the main ideas of a scientific article

3.31

0.94

9. I reflect as I read a scientific article

3.63

0.93

10. I interpret data, graphics, etc. from a scientific article

3.63

0.97

11. I summarize scientific information

3.12

0.87

12. I critically discuss the research article

3.02

0.92

13. I make conclusions after reviewing scientific literature

3.16

0.86

14. I prepare an abstract or essay on a research topic

2.81

0.94

15. I use references according to rules of scientific writing in a text that I elaborate, whether it is an abstract or an essay

2.94

1.10

16. I write key words in English for a research topic

2.66

1.11

17. I identify a new research topic in the literature review

2.73

0.82

18. I am able to orally communicate the results of a scientific literature review

2.91

0.97

19. I elaborate keywords of a research topic

3.10

0.79

20. I bring my ideas in the development of a research topic

3.33

0.77

Table 2 Pre- and post-test measurement of students’ research skills Factors

Pretest M

Post-test DS

M

DS

Wilcoxon

Effect size

Z

d

Process scientific information

25.8

5.07

31.49

5.33

−5,656

1.1

Handle scientific information

21.58

5.15

29.14

5.1

−6,091

1.49

Develop scientific information

14.88

3.46

19.95

3.73

−6,116

1.42

Total

62.01

12.37

80.58

13.39

−6,200

1.35

p < 0.001

Scientific Research and Writing Skills in Engineering Students

81

4 Discussion and Conclusions The current study showed that program participants improved their performance in research skills and scientific writing after participating in the “Research Writing and Reporting” course. A positive increase was found in the three skills evaluated: process, management and development of information. The results obtained allow us to affirm that scientific writing courses are of utmost importance in the training of future engineers and in the development of research skills, which need greater support, especially in the first years of university. The results are in agreement with other studies related to training and reinforcement programs in research skills, such as the work of Pozzo [10] and Walton [11]. It should be considered that the engineering area is eminently technological and students have skills in handling systems and devices. Therefore, research skills should be taught taking into account the complementary training of students and their other skills. Similarly, it is necessary to promote the teaching of research skills with the use of specialized writing software [22–24], since some research has shown the little use of ICT in the training and development of research skills in careers such as those with a computer science profile [28]. It should also be considered that while there are multiple studies on the impact of research on undergraduate students in the social sciences and medical sciences, in the engineering disciplines, it is more limited. This can be attributed to several reasons: the greater emphasis on mathematics and science in the early years of the engineering curriculum, a strictly sequential degree path, and possible lack of flexibility in program requirements [29]. Finally, it can be stated that there are undoubted benefits resulting from research skills development activities, especially in a context where scientific production is increasingly relevant and demanding, becoming a highly relevant skill for higher education and for the future of undergraduate and graduate students [30]. Future research will examine in depth the impact of writing courses in engineering careers, and will also investigate the most appropriate methodologies for their teaching. Acknowledgements To the Universidad Nacional de San Agustín de Arequipa for the funding granted to the project “Transmedia Gamification and Video Games to promote scientific writing in Engineering students”, under Contract No. IBA-IB-38-2020-UNSA.

References 1. Rodríguez-Torres Á, Posso-Pacheco R, De la Cueva-Constante R, Barba-Miranda L: Herramientas metodológicas para el desarrollo de habilidades investigativas en los estudiantes: Una praxis necesaria. Olimpia. Revista de la Facultad de Cultura Física de la Universidad de Granma 15(59), pp. 119–132 (2018)

82

R. Núñez-Pacheco et al.

2. Martínez D, Márquez D (2014) Las habilidades investigativas como eje transversal de la formación para la investigación. Tendencias Pedagógicas 24:347–360 3. Machado E, Montes de Oca N: Las habilidades investigativas y la nueva Universidad : Terminus a quo a la polémica y la discusión. Humanidades Médicas 9(1) (2009) 4. Salazar J, Cáceres M, Moreno J (2019) Preliminares para la elaboración del estado de la cuestión sobre habilidades investigativas y su aplicación en los procesos de enseñanza y aprendizaje. Rev Metrop Ciencias Apl 2(3):192–197 5. Rojas C, Aguirre S (2015) La formación investigativa en la educación superior en América Latina y el Caribe: una aproximación a su estado del arte. Eleuthera 12:197–222 6. Medina S: Estrategias didácticas y adquisicion de habilidades investigativas en estudiantes universitarios. J Bus Entrepreneurial Stud 4(1) (2020) 7. Llulluy-Nuñez D, Neglia L, Vilchez-Sandoval J, Sotomayor-Beltrán C, Andrade-Arenas L, Meneses-Claudio B: The impact of the work of junior researchers and research professors on the improvement of the research competences of Engineering students at a University in North Lima. Proc LACCEI Int Multi-conference Eng Educ Technol 2021-July:1–6 (2021) 8. Carlín E, Carballosa A, Herrera K (2020) Formación de competencias investigativas de los estudiantes en la Universidad de Guayaquil. Univ y Soc 12(2):8–16 9. Serrano M, Pérez D, Solarte N, Torrado L (2018) La redacción científica como herramienta para cualificación del estudiante de pregrado. Ciencia, Docencia y Tecnología 29(56):208–223 10. Pozzo M: Workshop: improving research and writing skills in engineering education. IEEE World Conference on Engineering Education (EDUNINE), pp. 1–3. (2021) 11. Walton G: Writing skills development in an engineering geology course through practice and feedback on report submissions using a rubric. J Geosci Educ 68(1) (2019) 12. Hubbard K, Dunbar S: Perceptions of scientific research literature and strategies for reading papers depend on academic career stage. PLoS One 12(12) (2017) 13. Zerbe E, Berdanier CGP, Buswell NT, Melo JMM: Validating a short form writing attitudes survey for engineering writers. In: 126th ASEE Annual Conference and Exposition (2019) 14. Flores M (2014) La competencia comunicativa escrita de los estudiantes de ingeniería y la responsabilidad institucional. Innovación educativa 14(65):43–60 15. Batson K: An approach to teaching academic writing to international graduate students in colleges of engineering. In: ASEE Annu Conf Expo Conf Proc 2018-June (2018) 16. Conrad S (2017) A comparison of practitioner and student writing in civil engineering. J Eng Educ 106(2):191–217 17. Berdanier CGP (2021) Linking current and prospective engineering graduate students’ writing attitudes with rhetorical writing patterns. J Eng Educ 110:207–229 18. Sologuren-Insúa E (2021) Escritura académica en tres escuelas de ingeniería chilenas: La familia ‘Informe técnico’ como macrogénero discursivo en el área de Ingeniería Civil Informática. Educ Siglo XXI 39(1):107–130 19. Navarro F, Uribe F, Lovera P, Sologuren E: Encuentros con la escritura en el ingreso a la educación superior: representaciones sociales de los estudiantes en seis áreas de conocimiento. Ibérica: Revista de la Asociación Europea de Lenguas para Fines Específicos (38), 75–98 (2019) 20. Vidal E, Castro E: Developing technical writing for engineering students under a paper conference format. In: CEUR Workshop Proceedings, pp. 77–86 (2020) 21. Flower L, Hayes JR (1981) A cognitive process theory of writing. Coll Compos Commun 32(4):365–387 22. Sotomayor-Beltran C, Barriales ALF, Lara-Herrera J: Work in progress: the impact of using LATEX for academic writing: a Peruvian engineering students’ perspective. In: EDUNINE 2021 - 5th IEEE World Engineering Education Conference: The Future of Engineering Education: Current Challenges and Opportunities, Proceedings. pp 2019–2022 (2021). 23. Sarkar D: Introducing LaTeX to the freshmen engineers: technical writing with a difference. In: 2021 First-Year Engineering Experience (2021) 24. Knight S, et al.: AcaWriter a learning analytics tool for formative feedback on academic writing. J Writ Res (12),141–186 (2020)

Scientific Research and Writing Skills in Engineering Students

83

25. Vidal E, Toro Y: Information literacy for lifelong learning: an experience with personal learning environments. In: Proceedings - International Conference of the Chilean Computer Science Society, SCCC (2020) 26. Hayes JR, Flower LS: Identifying the organization of writing processes. In: Gregg, L., Teinberg, E. (eds.) Cognitive Processes in Writing. Erlbaum, Hillsdale/Nueva York (1980) 27. Cobos F, Peñaherrera M, Ortiz AM (2016) Design and validation of a questionnaire to measure research skills: experience with engineering students. J Technol. Sci Educ 6(3):219–233 28. Zambrano Acosta JM, Estrada Molina O, Beltrón Cedeño CC, Zambrano Beltrón LA (2017) Habilidades investigativas -en relación al uso de las TIC- a desarrollar en estudiantes de carreras con perfil informático. Revista Cognosis 2(1):3–16 29. Kaul S, Ferguson CW, Yanik PM, Yan Y.: Importance of undergraduate research: efficacy and student perceptions. In: ASEE Annu Conf Expo Conf Proc 2016-June (2016) 30. Sulisworo D, Rahayu T, Akhsan RN: The students’ academic writing skill after implementing blended learning using facebook. Information Technologies and Learning Tools 56(6) (2016)

Topic Modelling for Automatically Identification of Relevant Concepts Discussed in Academic Documents Segarra-Faggioni Veronica, Ratté Sylvie, and Jong De Frank

Abstract A combination of natural language processing and topic modeling identifies topic terms a collection of documents. Latent Dirichlet Allocation (LDA) is an algorithm widely used to infer topics that the document belongs to, on the basis of words contains in it. This research applied LDA to identify topics automatically from academic documents as a way of validating the relevant concepts discussed in the curriculum literature. The experiment involved academic documents about Knowledge Building. We apply some techniques to prepare the data, after data training and validation. Topic modeling was used to identify the topic terms that can be used in the knowledge-building dialogue. Those concepts are meaningful to both the teacher or students because they provide a visualization of the content coherence. Keywords Knowledge building · topic modeling · LDA · NLP

1 Introduction Currently exist a large amount of information input by studying literature about Knowledge Building field that provides opportunities for collaboration between students and teachers, it also allows for lifelong learning [1, 2]. The selected information is a list of books and articles suggested by the teacher, we will use the term “authorative sources”. Students read the authorative sources and connect to, build-on the information to their basic knowledge [3]. S.-F. Veronica (B) Universidad Tecnica Particular de Loja, Loja, Ecuador e-mail: [email protected] R. Sylvie Ecole de Technologie Superieure, Montreal, QC, Canada e-mail: [email protected] J. De Frank Aeres Applied University Wageningen, Wageningen, The Netherlands e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_8

85

86

S.-F. Veronica et al.

In the educational context, the interest of researchers has increased due to the use of topic models. Topic modeling is a statistical method that efficiently explores a large set of documents to find common topics of co-occurrence of words throughout the collection and annotate documents. LDA is an unsupervised technique that automatically creates topics based on words in documents. Previous studies applied LDA to retrieve topics from student online discourse in relation to topical structures of relevant expert texts and recommends online posts most relevant to each topic such as [2]. With the purpose of determining the topic terms in the authorative sources, we applied a combination of natural language processing (NLP) and Latent Dirichlet Allocation (LDA) to identify topic terms in a collection of academic documents as a way of validating the relevant concepts discussed in the curriculum literature. According to [4], LDA algorithm is simple to implement and find useful hidden topics in the collection of documents. This paper presents an automatic learning model based on natural language processing techniques (NLP) and machine learning techniques to identify relevant concepts discussed in academic documents. This paper is organized as follows: Sect. 2 describes related work about Latent Dirichlet Allocation. Section 3 presents the materials and method applied to this study. Section 4 describes the experimental results and discussion.

2 Related Work Natural language processing and machine learning techniques are applied to discover knowledge patterns in a group of documents. According to [5], text mining discoveries focus on representation, text level, and world level. Some authors have proposed working methods for document classification [6, 7]. Researches have employed classical methods based on topic modeling techniques such as Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), and probabilistic latent semantic analysis (PLSA). LSA is one of the most used methods for word meaning representation because it is a useful computational text analysis tool [6, 8]. LSA is a document based vector space model, and it applies to information retrieval and cognitive science [9]. In fact, LSA extracts salient topics by examining word co-occurrence [10]. Several researches [10, 11] have applied LSA to evaluate the students essays. However, some disadvantages of LSA are shown in [10, 12]. LSA is less efficient for synonym detection, concept categorization, selection preferences, and analogy. On the other hand, previous studies applied LDA in an educational context [10, 13–15]. Latent Dirichlet Allocation is a probabilistic method used for topic modeling that was proposed by [16]. Topic modeling is a statistical method that efficiently explores a large set of documents to find common topics extracted by LDA using the collection and annotate

Topic Modelling for Automatically Identification of Relevant Concepts …

87

Table 1 Summary of topic modeling related work in the educational context Author

Study case

Dataset

Molavi M. et al. (2020)

OER topic extraction approach

Lectures from Coursera and Khan Academy

Chen, W. et al. (2017)

Text categorization model based on LDA-KNN

Chinese corpus of Fudan University

De Jong, F. et al. (2017)

Retrieves topics from student online discourse

Wikipedia

documents. The objective of a topic model is to create interpretable document representations that predict topics from the occurrence of words belonging to a collection of documents [17]. In this research work, the algorithm selected is Latent Dirichlet Allocation (LDA). LDA is used in natural language processing and machine learning to solve the issue of semantic similarity measurement in traditional text classification [7]. As shown in [16], LDA is a three-level hierarchical Bayesian model (document, word, topic). LDA gets the semantic relatedness between terms through topics and regards each document as a distribution over a set of topics. That means that, LDA extracts a number of topics for the entire collection and defines the words that belong to those topics. In LDA, it is essential to determine the number of topics that occur in the collection of documents. According to [18], the optimum number of topics is between 4 and 8 topics. In summary, LDA recognizes the coherent topic by finding the pattern of co-occurrence of words [7, 13]. The advantages of using LDA for topic modeling are the relative simplicity to implement and find useful hidden topics [4]. Topic modeling is a statistical method to detect a group of words related to common topics in a semantically related collection of documents [10, 19]. Table 1 shows a summary of the related works on topic modeling in the educational context:

3 Proposed Work 3.1 Dataset In this study, the dataset refers to the curriculum literature which have been suggested by the teachers. They are authoritative knowledge sources such as the selection of chapters and scientific literature for courses (scientific and specialist literature). In this research, the curriculum literature includes 8 documents (4 book chapters and 4 scientific journal articles) related to knowledge building discourse. All curriculum literature is written in English and their size varies between 11 and 44 pages.

88

S.-F. Veronica et al.

3.2 Methods The first step is to apply Natural Language Processing techniques to prepare and clean data, remove common words called stop-words; and, lemmatize the dataset to remove inflectional endings and return the base form of a word [20]. Next, we applied latent Dirichlet allocation (LDA) to extract one or more topics in the curriculum literature. Each document is split into separate words and organized by their frequencies. We used the LDA package in Python’s Gensim library to conduct this experiment. Considering the study of [18], we made several tests to choose the best representation of the topics through the topic’s words. We started doing 4 through 20 topics. We found that the best representation used 8 topics.

3.3 Topic Modeling on the Curriculum Literature The topic terms generated by topic modeling from authoritative sources (curriculum literature) contribute significantly to productivity and creativity in the knowledgebuilding discourse. Topic models provide one or more topics from a text collection. In this approach, we extract the relevant concepts discussed in the curriculum literature. The main idea of applying the latent Dirichlet allocation model (LDA) on the curriculum literature is to extract the hidden topics that occur in a collection of documents and their relationship with each other that is part of a particular activity or specialist field (See Fig. 1). LDA is one example of the topic modeling techniques that assumes each document is viewed as a mixture of topics [13, 16]. In topic modeling, it is essential to determine the number of topics in order to decide how many topics to pick up from a document. [18] propose a metric to find the optimal number of topics to pick up from a document. In the model implementation, we define the number of topics (eight topics) to generate the document-topic distribution from our collection of the documents. Next, each document is, in turn, split into separate words and organized by their frequencies.

Fig. 1 Latent Dirichlet allocation model

Topic Modelling for Automatically Identification of Relevant Concepts …

89

Every word is assigned to a temporary topic using Dirichlet distribution. Then, it gets the essential features and specifies the number of top N-words.

4 Experimental Results and Discussion 4.1 Topic Modeling: Selecting Topic Terms LDA model finds the semantic relatedness among different concepts in each topic. Therefore, the topic information obtained is a group of topics, each topic categorized by a series of words. After that our models is pre-trained, we can visualize the LDA model applying LDAvis. LDAvis is an interactive topic model visualization that shows the most common topics and the relevant terms in each one of the topics. According to [21], LDAvis helps users interpret the topic-term relationships in a LDA model. LDAvis has two core functionalities: to select a topic to discover the most relevant terms for that topic, the ability to select a term to reveal its conditional distribution over topics. We applied LDA to identify the knowledge building topics involved in the curriculum literature. To conduct this experiment, we used the LDA package in Python’s Gensim1 library. Considering the study of [18], eight topics were identified with the most relevant concepts according to each curriculum literature. Finally, the output from the model is eight topics; each one is categorized by a series of words (see Table 2 and Table 3). The words learning, knowledge and education are very commonly used words in the context of learning discourse.

4.2 Topic Modeling: Visualization In order to visualize our LDA model, we applied the pyLDAvis2 package in Python, which allows deep inspection of a topic-term relationship into the distribution of topics [21]. We can observe the estimated models in an inter-topic distance map based on the 30 top most relevant words for each topic. Topics are associate with a group of words that occur frequently in the corpus [2]. Figure 2 shows the inter-topic distance map of the curriculum literature 1: “Bibliographical learning - within the new lifelong discourse”(Chapter 8 in Illeris, K. (2019). Contemporary Theories of Learning). This chapter sets out Illeris’ overall theory and all students were likely to have read it.

1 2

https://pypi.org/project/gensim/ https://pypi.org/project/pyLDAvis/

90

S.-F. Veronica et al.

Table 2 Example of the most relevant terms for topics according to each curriculum literature Topic

CL1

CL2

CL3

CL4

1

knowledge life

process understanding

learning transformation

reflection teacher

2

course individual

concept area

adult mind

student supervisor

3

learning process

people situation

place thing

course question

4

skill framework

school learning

faculty principle

example phase

5

research development

scheme

people work

student supervisor

6

employment training

understanding theory

school time

quality skill

7

experience individual

change situation

work authority

example phase

8

skill course

resistance teacher

faculty evaluation

problem teacher

Table 3 Example of the most relevant terms for topics according to each curriculum literature Topic

CL5

CL6

CL7

CL8

1

process action

model activity

idea building

organisation group

2

discourse experience

group organization

problem information

work student

3

learning education

methaphor participation

time database

knowledge process

4

schema thought

school community

discourse advancement

learner mistake

5

understanding experience

activity work

state creation

reflection teacher

6

pedagogy ideology

world artifact

classroom information

student supervisor

7

learning process

group level

problem information

level skill

8

analysis life

school community

discourse community

example phase

We can see in Fig. 2 the connection between topics and the prevalence of the words. On the left panel, there is a global view of the topic model. These topics are represented in the form of bubbles. The larger the bubble, the more frequent is the topic in the documents. Distance between the topics is an approximation of semantic relationship between the topics. In addition, a topic model with a low number of

Topic Modelling for Automatically Identification of Relevant Concepts …

91

Fig. 2 Topic modeling of the curriculum literature 1, Bibliographical learning – within the new lifelong discourse

topics will have big non-overlapping bubbles. For example, bubble 1 is overlapping bubble 3 (closer in the distance) than the non-overlapping topic. Next, the bubble 2 and bubble 5 are not overlapping with other topics. The bubble 4 is overlapping bubble 8. Finally, bubble 6 and bubble 7 are overlapping. Then, on the right panel, the horizontal bar chart represents the individual relevant terms for the currently selected topic on the left. The 30 most salient terms in all topics in the curriculum literature are shown on the right of Fig. 2. Figure 3 shows the most relevant terms for topic 1 of curriculum literature 1 “Bibliographical learning - within the new lifelong discourse”. On the left panel, topic 1 is the red bubble (See Fig. 3). We can see that topic 1 is overlap with topic 3 that means they shares common words. On the right side, there are words such as learning, process, experience, option, form, place. The red shaded area describes the frequency of each word in topic 1. There are terms of topic 1, which are shared with other topics (e.g., topic3: learning, training, society, education). For instance, terms such as learning, process, experience provide great insight into the dataset related to “lifelong learning discourse”. Figure 4 shows the essential words for topic 4. The most frequent terms for topic 4 are plotted on the right panel in Fig. 4. Terms such as “knowledge, market, society” are in red in the horizontal bar graph. The red bar is the number of times the word appeared in the document within the topic 4, while the blue bar is the overall appearance in all the other topics. Hovering over the specific words (in the right panel), only the topic containing the words are visible. We can see in Fig. 4 that bubble 4 is overlapping bubble 8. Therefore, this topic share the term “knowledge” with topic 8.

92

S.-F. Veronica et al.

Fig. 3 Most relevant terms in topic 1 related to “lifelong learning discourse”

Fig. 4 Most relevant terms in topic 4

To observe the inter-topic relationship, we now shift our focus to topic 6. It is overlapping topic 7 and they have the common word “modernity”. The most frequent term for topic 6 is “employment” (see Fig. 5). And the most frequent terms for topic 7 are “beck, change”. Finally, in the topic 8, we can observe the relevant word is skill (see Fig. 6). This topic share the term “course” with topic 2.

Topic Modelling for Automatically Identification of Relevant Concepts …

93

Fig. 5 Most relevant terms in topic 6

Fig. 6 Most relevant terms in topic 8

5 Discussion In the educational context, the interest of researchers has increased due to the use of topic models. Topic models such as latent Dirichlet allocation (LDA) recognizes the coherent topics by finding the pattern of co-occurrence of words.

94

S.-F. Veronica et al.

In this approach we extract the relevant concepts discussed in the curriculum literature. The main idea of applying LDA on the curriculum literature is to extract the hidden topics that occur in a collection of documents and their relationship with each other. As shown in the visualisation of inter-topic distance map (See Fig. 2), the areas of the circles are proportional to the relative prevalence of the topics in the corpus [21]. LDA is one example of the topic modeling techniques that assumes each document is viewed as a mixture of topics. In this research, the selected topic terms can be used to validate relevant words with a connecting, knowledge building contribution to student knowledge development during the discourse [2].

6 Conclusion The application of NLP and topic modeling was elementary in the process of this study because it allowed to identify relevant terms from the corpus and eliminate terms and characters that did not provide meaning, these processes are important and should always be carried out prior to the use of an algorithm. The main contribution of our study is to identify topics automatically from academic documents as a way of validating the relevant words in books that are discussed in the students’ dialogue. Those words can be synonymy or different variants to express the same concept. LDA was applied to identify knowledge-building topics involved in the curriculum literature. LDA model is trained with fitted parameters to find the specific topics (i.e., topic terms used in discourse, the knowledge-building dialogue). Topics are associated with a group of words that occur frequently. The resulting group of words can also be interpreted as lexical fields, groups of words whose meanings depend on each other; together, they form a conceptual structure that is part of a particular activity or specialist field. In future, we intend to apply recent development in deep learning methods. Acknowledgements The research team would like to thank Prof. Frank de Jong from University of Applied Sciences in the Netherlands, for contributing with the data. In addition thanks to UTPL, especially to Tecnologıas Avanzadas de la Web y SBC Group.

References 1. Andone D, Mihaescu V, Vert S, Ternauciuc A, Vasiu R (2020) Students as OERs (open educational resources) co-creators. In: 2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT), pp 34–38, Los Alamitos, CA, USA. IEEE 2. de Jong F (2020) Knowledge in-(ter)-action: responsive learning as knowledge building. Aeres Hogeschool

Topic Modelling for Automatically Identification of Relevant Concepts …

95

3. Balyan R, McCarthy KS, McNamara DS (2017) Combining machine learning and natural language processing to assess literary text comprehension. In: Proceedings of the 10th International Conference on Educational Data Mining, EDM 2017, pp 244–249. International Educational Data Mining Society 4. Crain SP, Zhou K, Yang SH, Zha H (2012) Dimensionality reduction and topic modeling: from latent semantic indexing to latent dirichlet allocation and beyond. In: Mining Text Data, pp 129–161. Springer US, Boston, MA. https://doi.org/10.1007/978-1-4614-3223-4_5 5. Gómez MM, et al (2005) Text Mining using Comparison of Semantic Structures. Technical report 6. Cvitanic T, Lee B, Song HI, Fu K, Rosen D (2016) LDA v. LSA: a comparison of two computational text analysis tools for the functional categorization of patents. In: ICCBR, pp 41–50, Atlanta, Georgia 7. Chen W, Zhang X (2017) Research on text categorization model based on LDA—KNN. In: 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pp 2719–2726, Chongqing, China. IEEE 8. Landauer TK, Foltz PW, Laham D (1998) Introduction to latent semantic analysis 25:259–284 9. Padó S, Lapata M (2007) Dependency-based construction of semantic space models. Comput Linguist 33(2):161–199. https://doi.org/10.1162/coli.2007.33.2.161 10. Chen Y, Yu B, Zhang X, Yu Y (2016) Topic modeling for evaluating students’ reflective writing: a case study of pre-service teachers’ journals. In: LAK’16 11. Wiley J et al (2017) Different approaches to assessing the quality of explanations following a multiple-document inquiry activity in science. Int J Artif Intell Educ 27(4):758–790. https:// doi.org/10.1007/s40593-017-0138-z 12. Baroni M, Dinu D, Kruszewski G (2014) Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. In: Association for Computational Linguistics, Baltimore, Maryland, USA 13. Yan E, Zhu Y (2018) Tracking word semantic change in biomedical literature. Int J Med Inf 109:76–86. https://doi.org/10.1016/j.ijmedinf.2017.11.006 14. De Jong F, et al (2017) Making a difference: analytics for quality knowledge-building conversations the overall focus of the symposium. In: International Conference on Computer Supported Collaborative Learning, pp 711–718, Philadelphia, Pennsylvania, USA 15. Sekiya T, Matsuda Y, Yamaguchi K (2015) Curriculum analysis of CS departments based on CS2013 by simplified, supervised LDA. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge - LAK ’15, pp 330–339, New York, New York, USA. ACM Press 16. Blei DM, Ng AY, Jordan M (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022 17. Amami M, Pasi G, Stella F, Faiz R (2016) An LDA-based approach to scientific paper recommendation. In: Métais E, Meziane F, Saraee M, Sugumaran V, Vadera S (eds) Natural Language Processing and Information Systems. Springer International Publishing, Cham, pp 200–210. https://doi.org/10.1007/978-3-319-41754-7_17 18. Deveaud R, SanJuan E, Bellot P (2014) Accurate and effective latent concept modeling for ad hoc information retrieval. Document Numérique 17(1):61–84. https://doi.org/10.3166/dn.17. 1.61-84 19. Risch J (2016) Detecting Twitter topics using Latent Dirichlet Allocation. Technical report, Uppsala University, Uppsala, Sweden 20. Bird S, Loper E (2006) NLTK: The Natural Language Toolkit. Technical report 21. Sievert C, Shirley KE (2014) LDAvis: a method for visualizing and interpreting topics. In: Workshop on Interactive Language Learning, Visualization, and Interfaces, pp 63–70, Baltimore, Maryland, USA. Association for Computational Linguistics. LNCS Homepage http:// www.springer.com/lncs. Accessed 21 Nov 2016

Covariational Reasoning in an Approach to the Concept of Definite Integral Using GeoGebra Within an Economic Context Mihály André Martínez-Miraval , Martha Leticia García-Rodríguez , and Daysi Julissa García-Cuéllar

Abstract Definite integral is usually associated with area problems and it is approached through a process of approximation with Riemann sums; this process can be particularly complex for a considerable number of students enrolled in nonscience careers, hence it is important to develop intuitive notions in these students, and one option is to do so through the use of digital technologies. This study aims to analyze the covariational reasoning of university students who go through the stages of partition, product, sum and limit, in an approach to the concept of definite integral through the use of GeoGebra in economic context tasks. The theoretical approach of covariational reasoning and a qualitative methodology were considered. Behaviors associated with mental actions of a covariational reasoning up to the level of coordination of values in the stages of partition and sum, and smooth continuous covariation, associated with the use of the notion of limit at infinity, were identified. The use of digital technologies and the fact that the problem considered an economic context related to spending, raised the interest of the students to learn a new mathematical concept. Keywords Definite Integral · Covariational Reasoning · GeoGebra

M. A. Martínez-Miraval (B) · D. J. García-Cuéllar Pontificia Universidad Católica del Perú, Av. Universitaria 1801, San Miguel 15088, Perú e-mail: [email protected] D. J. García-Cuéllar e-mail: [email protected] M. L. García-Rodríguez Instituto Politécnico Nacional CICATA-Legaria, Calz Legaria 694, Col. Irrigación, Miguel Hidalgo, 11500 Ciudad de México, CDMX, México e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_9

97

98

M. A. Martínez-Miraval et al.

1 Introduction According to the literature, it is important to approach the concept of the definite integral due to its various applications in several fields of Mathematics, Engineering and Physics, involving the calculation of lengths, areas and volumes [1], and also due to its relationship with other integral calculus concepts, for example, antiderivatives, integral function, Fundamental Theorem of Calculus, among others [2]. The concept of definite integral may be introduced by a process of approximation to the area of a region through the use of rectangles of equal base, involving simultaneous changes between the values of the different quantities. The use of dynamic geometry systems makes it possible to relate different objects so that simultaneous changes between different variables can be appreciated, this provides the dynamism needed to develop in students intuitive notions related to the concept of definite integral, thereby supporting students’ learning to be more complete [3]. The use of GeoGebra applets designed with sliders allows approximations to the area of a region bounded by the graph of a positive function on a given interval and the X-axis, either by comparing the area of a region with the values of the approximation when using extreme left, right and midpoint Riemann sums [4], or by analyzing the error of the approximation approaching zero [5], even, the use of GeoGebra is complemented with mathematical operations involving the approach and reduction of sums, and the application of the limit to validate conjectures about the process [6]. When the study of the definite integral is approached through the use of Riemann sums in approximation processes, some researches decompose it into the components: product, sum, limit and function [7]. No matter which decomposition is performed, the coordination of changes in the values of various quantities is observed at each stage, therefore the concept of covariation emerges, and the analysis of the covariational reasoning of students who solve tasks involving these dynamic phenomena of approximation becomes relevant [8]. This review of the literature on the definite integral and the university career of the research subjects motivated the realization of this research, which aimed to analyze the covariational reasoning of university students who go through the stages of partition, product, sum and limit, in an approach to the concept of definite integral through the use of GeoGebra in tasks of economic context.

2 Theoretical Framework Covariational reasoning can be described in terms of “conceptualizing the values of individual quantities as variables and then conceptualizing two or more quantities as varying simultaneously” [9]. The way in which students coordinate the changes between the values of two or more quantities it becomes visible through the behaviors

Covariational Reasoning in an Approach to the Concept of Definite …

99

Table 1 Mental actions related to each level and associated behaviors Level

Behaviors

Smooth continuous covariation

MA6: Extend the simultaneous coordination idea between the base measure of each rectangle and the sum of the areas of the inscribed or circumscribed rectangles when the number of rectangles tends to infinity

Coordination of values MA4: Construct a new element representing the coordination of the individual values of two quantities, such as a point (a; b) where “a” indicates the number of rectangles and “b” expresses the sum of the areas of such rectangles Gross coordination of values

MA3: Express verbally how they coordinate changes between the number of rectangles and the sum of their areas in terms of increases or decreases

No coordination

MA1: Perform arithmetic operations to give meaning to the values displayed in GeoGebra, such as the sum of areas of rectangles

Note: Adapted from [8]

they show when solving tasks, such behaviors are associated with mental actions that present a certain hierarchy related to levels of covariational reasoning [10]. The theoretical construct of [9], which emphasizes quantitative reasoning, multiplicative objects, and coordination of changes in the values of quantities, provides a table describing six levels of covariational reasoning and their respective descriptions. In order to identify the level of covariational reasoning of students, it is necessary to recognize what mental actions they put on the line when solving covariation tasks, related to the concept of the definite integral as the limit of a Riemann sum. To this end, the work of [8] is taken into consideration, such research presents a description of the behaviors associated with different mental actions when working a mathematical problem related to the definite integral (Table 1). This table shows only the mental actions identified in the present investigation.

3 Methods and Procedures This research followed a method of a qualitative and interpretative approach. [11] state that in this kind of methodologies the processes and procedures carried out by the students, and their descriptions of them. To perform the analysis of the students’ work, information was taken from a written record and from a semi-structured interview that took place while the students were solving the activity; this interview was videotaped in order to identify the behaviors associated with the mental actions of the students with the purpose of learning about their covariational reasoning. The study was carried out with two universitary students between 18 and 20 years old. The work of the student named Joaquín is presented, who completely developed the activity called Areas and Costs, in a two-hour session. Only in cases where

100

M. A. Martínez-Miraval et al.

differences are present and different mental actions are justified, Joaquín’s answers will be complemented with those of another student, whom we will call Andrea. The Activity The context is that a person, named Mr. Torres, wants to cover the floor of the bar area of his terrace with rectangular wood floorboards, and for this purpose he turns to a carpenter. Due to the difficulty of covering a non-linear top edge region with rectangular floorboards, the carpenter gives Mr. Torres options to perform the task: in the case where the floorboards is inscribed in the area, putty should be placed in the spaces not covered by the floorboards, however, if the floorboards is circumscribed, the excess floorboards should be covered with a granite like sheet, similar in color to the terrace floor. The costs for labor and the material are shown in Table 2. In the first part of the activity, the algebraic expression f(x) = −0.15 × 2 + 4 is provided, this expression models the top edge that limits the bar floor area, then students are asked to calculate the cost for placing a floorboard in each of the positions to know if they are able to determine the height of the floorboards (Fig. 1). In the second part of the activity, students explore the costs incurred by Mr. Torres for the placement of floorboards on his bar through a simulation in GeoGebra by manipulating sliders that allow a variation in the number of rectangular floorboards (Fig. 2). The simulation in GeoGebra was implemented in order to make conjectures about the total cost of the work and about the value of the area of the bar floor when the number of floorboards exceeds 200. Table 2 Costs submitted by the carpenter

Item

Cost

Labor

S/. 3000

Floorboards

S/. 100 per m2

Putty

S/. 30 per m2

Granite-like sheet

S/. 20 per m2

Fig. 1 First part of the activity and position of a floorboard on the bar floor

Covariational Reasoning in an Approach to the Concept of Definite …

101

Fig. 2 Second part of the activity and graphical view of the applet designed in GeoGebra

The complete work of student Joaquín is presented below, in addition, it is also mentioned the answer of student Andrea to question 2 of the second part of the activity.

4 Analysis Joaquín used different mathematical concepts to solve the question given in the first part of the activity (first moment): the notion of the image of a function, areas of geometric shapes and rule of three. Figure 1 presented the two positions in which a rectangular floorboard could be placed. The student recognized that when the floorboard is inscribed to the region, the height is obtained as the image at the right end of the base of the rectangle, and when the floorboard is circumscribed to the region, the height is obtained as the image at the left end of the same base. Joaquin was able to determine the area of the region not covered by the floorboard, as well as the excess floorboard, allowing him to use the costs shown in Table 2 in order to determine how much Mr. Torres would spend in both cases. The student calculates the cost of wood floorboard and putty using a rule of three, instead of multiplying the cost per square meter by the value of the area, which would make the student recognize that there is a functional relationship between them, and thus more easily recognize the elements of the equation shown in the applet in the second part of the activity. In the semi-structured interview, Joaquín’s opinion was asked about how he would calculate the heights of the floorboards if there were more than one, which led to the following answer.

102

M. A. Martínez-Miraval et al.

01 Researcher: If four rectangles were drawn with the same base measurement, how would you determine the area of each of the floorboards? 02 Joaquín: I find the height with the function and then multiply it by four. Although the heights would be different ... I would have to draw it.

The behavior shown by Joaquín both in the calculation of the first part of the activity and in his answer given during the interview, is associated with an MA1 (see Table 1) because the student only performs arithmetic operations in order to find the cost of placing a single wood floorboard, also his answer in line 02, evidences a lack of coordination between the changes in the number of rectangles and their heights. Performance of the first part of the activity places the student in the stages of partition and product, in which the student has to recognize the measure of the base of a rectangle and relate its height to the image at the right or left end of its base. The question on line 01 makes the student think that the recognition of measurements must be done for each rectangle in the regular partition. In the second part of the activity, Joaquín accesses the applet designed in GeoGebra. By increasing the number of wood floorboards with the sliders, Joaquín recognizes different quantities that vary simultaneously, expressing what he detected verbally during the interview (only the changes that Joaquín observed as he manipulated slider n in question 1 are presented in this part). 03 Researcher: If you increase the values of both sliders, what magnitudes vary and how do they vary? 04 Joaquín: As the divisions increase [he is referring to the regular partitions of the analysis interval generated by varying the number of rectangles] it is observed that: the width decreases [he is referring to the size of the base of the rectangle], the height increases [he is referring to the height of each rectangle], the area of the putty decreases, the price tends to increase, and the area of the wood floorboard also increases [he is referring to the sum of the areas of the wood floorboards].

The behavior shown by Joaquín in this interview excerpt, is associated with an MA3 (see Table 1) because the student does not mention individual values of the variables that change simultaneously, instead, he coordinates the changes between the number of wood floorboard and the variables described in line 04 when he says: “if the divisions increase…”, “the width decreases…”, “the price tends to increase…”, among other phrases, which denote that a general comparison is made between two quantities. The question in line 03 makes the student perform a regular partitioning process, and at the same time locates him in the sum stage of the approximation process, since he recognizes the changes in the sum of the areas of the wood floorboards, and associates to it, the sum of the areas of the putty and of the granite like sheets. Joaquín moved m and n until the value 200 (Fig. 3) to develop question 2. The researcher orally asked the second question, in order to analyze the conjectures that the student would make, when asking him to guess what would happen with the cost and the sum of the areas of the rectangles when considering values greater than 200. Here is an excerpt from the interview:

Covariational Reasoning in an Approach to the Concept of Definite …

103

Fig. 3 GeoGebra graphical view for values of m and n equal to 200

05 Researcher: If more than 200 wood floorboards could be placed, what would you say about the cost of Mr. Torres? 06 Joaquín: According to what is shown, in the limits of m and n [he is referring to the limit at infinity], the areas of the putty and the granite like sheet tend to be zero. This indicates that the prices will be S/. 4280, because the floorboards will cover all the spaces without the need for putty or the granite like sheet. 07 Researcher: What could you say about the sum of the areas of all the rectangular wood floorboards? 08 Joaquín: In the limits, the area has a limit of 12.80 m2 . Calculated it, through the side of n, by summing of the area of the wood floorboard and the area of the putty, for any value of n, and through the side of m, it is calculated, by subtracting the area of the wood floorboard and the area of the granite like sheet.

The behavior shown by Joaquín in this interview excerpt is associated with an MA4 (see Table 1) because the student, besides observing and relating, dynamically, the costs he would have to incur and the sum of areas of the rectangles, according to quantities of specific wood floorboards, he also provides additional information on the sum of areas of rectangles (12.8 m2 ) and the total common cost for both positions of the floorboards (S/. 4280). He considered in the equation of the cost, that the values of the area of the putty and the area of the granite like sheet would be equal to 0 (tendency to 0), a result obtained from his intuition when he saw in the graphical view of GeoGebra, that the wood floorboards tend to cover the whole region. Regarding the sum of areas of rectangles, Joaquín obtained the limit value by recognizing the information given by GeoGebra, which provides how the sum of areas of the wood floorboard, the sum of areas of the putty and the sum of areas occupied by the granite like sheet change, and he managed to determine the total area of the region with arithmetic operations. According to what Joaquín expressed in line 06, it is evident that he coordinates simultaneously the values of the cost and the sum of the areas of the rectangles, associated with the reduction and tendency to 0 of the sum of the areas occupied by the putty or the sum of the areas occupied by the granite like sheet, it can be stated that he shows behaviors associated with an MA6 (see Table 1) because he coordinates the simultaneous changes between two continuous variables, as the number of wood floorboards tends to infinity.

104

M. A. Martínez-Miraval et al.

The question on line 05 places the student in the limit stage of the approximation process by making a conjecture about the changes and trend of the areas of the putty and the granite like sheet as the floorboards imaginatively increase toward a limiting value, which can be interpreted as infinity. This would make it possible to introduce the notion of the limit of a Riemann sum to define the integral. On the other hand, the student Andrea gave the following answers in the interview: 09 Researcher: If more than 200 wood floorboards could be placed, what would you say about the cost and the sum of areas of the floorboards in both cases? 10 Andrea: I think that both costs in the cases will be found in a specific number of wood floorboards, because the area to be covered by putty and the area to be covered by granite like sheets will tend to be reduced to 0 m2 , the more the number of wood floorboards the closer it is. The area of the sum of wood floorboards tends to the limit of the area of the complete figure, but it will never be exact because of the spaces that will seek to approach 0 m2 , but never reach 0.

The statement made by Andrea in line 10, as well as Joaquín, shows a simultaneous coordination between the values of the cost and the sum of the areas of the rectangles, however, it is noticed that Andrea have a discrete thought, because she indicates, first, the possibility that there is a finite number of wood floorboards that makes the costs in both positions of the floorboards equal, and second, that even if these floorboards tend to infinity, they would not be able to cover a region of non-linear edge, due to the rectangular shape of them. These behaviors can be associated with an MA5 (see Table 1), and with the sum stage of the approximation process.

5 Discussion The variables identification involved in the process of approximation to the area of a region, and the recognition of the simultaneous coordination between the changes in their values, seems to be determined by the students’ covariational reasoning capacity supported by the dynamism generated by a dynamic geometry system such as GeoGebra. We agree with [3], when he states that the dynamism that can be generated with GeoGebra, by mobilizing diverse constructions, helps learning to occur interactively; this is observed in the present study, in Joaquín’s expressions related, first, to the magnitudes that varied when increasing the number of wood floorboards (line 04), and second, when he makes conjectures based on his intuition when hypothetically increasing the number of floorboards towards infinity (line 06). An MA1 was identified in the partition and product stages, because it was sought that the student would know how to determine the cost according to the position of a single floorboard, so that when this number was larger, he would not focus on the calculation, but on the interpretation of the changes between the different variables, which is why in these two stages an explanation of how the values of the variables change and relate to each other is not requested.

Covariational Reasoning in an Approach to the Concept of Definite …

105

An MA3 was identified in the sum stage, because making an approximation to the area of a region implies increasing the number of rectangles, this leads to relate the increases (or decreases) in the sum of the areas of the wood floorboards with the decreases in the sum of the areas of the putty and of the granite like sheets when the floorboards are inscribed (or circumscribed) to the region. On the other hand, the applet design made it possible for the student to coordinate the changes between the different sums of areas: of rectangles, putty or granite like sheets, creating a multiplicative object of them, evidencing an MA4 in the sum stage, as well as determining new values of this object, by pointing out that the sum of areas of the wood floorboard is 12.8 m2 , when the sum of areas of the putty and the granite like sheets tends to 0, placing the student in the transit towards the limit stage. An MA6 was identified in the limit stage, because to recognize that the sum of areas of rectangles can accurately cover a region, one must coordinate the changes of two continuous variables, and stop thinking that rectangles are being drawn, as what happened with Joaquín, who was sure that the sum of areas of the putty and the granite like sheet tended to zero, so that “… the woods will cover all the spaces without the need for putty or the granite like sheet” (line 06). Some difficulties appeared when it was necessary to explain how to determine the heights of the wood floorboards; these errors were described by [6] after the students calculated the area of a small number of rectangles. Likewise, the decision to work within an economic context that was familiar to Management students, as well as involving the costs of the areas not covered by the rectangles (putty) and the excess areas (granite like sheets) and linking it with questions that allowed activating the mental actions of the students with the objective of seeing how they transitioned from one stage to another in the process of approximation with Riemann sums; is similar to the work done by [7], who was interested in analyzing how students reason when working with the definite integral using Riemann sums in a physics context, and how they coordinate the product, sum, limit and function processes involved in that context. Regarding to Andrea’s work and her answer described in line 10, it was observed the problem of using rectangles to approximate the area of a region limited by a nonlinear curve, due to the perception that the number of rectangles can be increased, but the area of the region would never be covered. This problem is related, according to [12], to the way in which the student conceives infinity, which in Andrea’s case would be the notion of potential infinity, and not actual infinity as Joaquín conceives it. Finally, it is also important to mention that working with GeoGebra can reduce students’ ability to draw graphs of functions with pencil and paper, which should motivate them to reflect on the relevance of incorporating digital technologies in student work. That is, recognize why use them? at what time? and in what way?

106

M. A. Martínez-Miraval et al.

6 Conclusions The importance of using a dynamic geometry system when approaching a fundamental concept of Calculus, as is the case of the definite integral, lies in the fact that dynamic environments can be designed and this helps students to connect the simultaneous changes between the values of different quantities with their covariational reasoning, in this way, students activate their mental actions, both when making conjectures from hypothetical situations and when trying to justify them. These mental actions evolve according to the stage of the process in which the students find themselves. The use of familiar contexts among the research subjects, like the expense made by a person paying for a service, or the purchase of materials in monetary units per square meter, motivates them to want to learn a new concept by applying mathematics, leaving in the background its abstractness in the process of approximating the concept of the definite integral with Riemann sums. Furthermore, if students can be made to make sense of each element that varies simultaneously as the number of rectangles increases (sum of rectangle areas, regions not covered by the rectangles, regions of the rectangles that exceed the region, among others), this would lead to students enjoying learning Calculus topics. Acknowledgements The authors thank the Pontificia Universidad Católica del Perú and the Instituto Politécnico Nacional for the support provided in the development of this research.

References 1. Wagner J (2017) Students’ obstacles to using Riemann sum interpretations of the definite integral. Int J Res Undergraduate Math Educ 4(3):327–356 2. Jones S (2015) The prevalence of area-under-a-curve and anti-derivative conceptions over Riemann sum-based conceptions in students’ explanations of definite integrals. Int J Math Educ Sci Technol 46(5):721–736 3. Zengin Y (2018) Incorporating the dynamic mathematics software GeoGebra into a history of mathematics course. Int J Math Educ Sci Technol 49(7):1083–1098 4. Caglayan G (2016) Teaching ideas and activities for classroom: integrating technology into the pedagogy of integral calculus and the approximation of definite integrals. Int J Math Educ Sci Technol 47(8):1261–1279 5. Tatar E, Zengin Y (2016) Conceptual understanding of definite integral with GeoGebra. Comput Sch 33(2):120–132 6. Martínez-Miraval MA, García-Cuéllar DJ (2020) Estudio de las aprehensiones en el registro gráfico y génesis instrumental de la integral definida. Formación Universitaria 13(5):177–190 7. Sealey V (2014) A framework for characterizing student understanding of Riemann sums and definite integrals. J Math Behav 33(2014):230–245 8. Martínez-Miraval MA, García-Rodríguez ML (2022) Razonamiento covariacional de estudiantes universitarios en un acercamiento al concepto de integral definida mediante sumas de Riemann. Formación Universitaria 15(4):105–118. https://doi.org/10.4067/S0718-500620220 00400105

Covariational Reasoning in an Approach to the Concept of Definite …

107

9. Thompson P, Carlson M (2017) Variation, covariation, and functions: foundational ways of thinking mathematically. In: Cai J (ed) Compendium for Research in Mathematics Education, pp 421–456. National Council of Teachers of Mathematics, Reston, VA 10. Carlson M, Jacobs S, Coe E, Larsen S, Hsu E (2002) Applying covariational reasoning while modeling dynamic events: a framework and a study. J Res Math Educ 33(5):352–378 11. Bogdan R, Biklen S (2007) Qualitative Research for Education. An Introduction to Theory and Methods, 5th ed. Pearson 12. Roa-Fuentes S, Oktaç A (2014) El infinito potencial y actual: descripción de caminos cognitivos para su construcción en un contexto de paradojas. Educación Matemática 26(1):73–101

The Mobile Application in Learning for People with Special Hearing Capabilities Javier Sánchez-Guerrero , Andrés Haro-Velasteguí , Sandra Carrillo Ríos , and Jaime Ruiz

Abstract The insertion of Augmented Reality (AR) and Virtual Reality (VR) in mobile applications has been a very indisputable fact in recent times, much more if this is dedicated in the educational field of people with special abilities including hearing, which requires a differentiated treatment with this resource to take advantage of their visual difficulty. Therefore, it is transcendental to obtain information to know the existing relationship within the teaching-learning process. The objective of the research was the development of a mobile application to strengthen the learning of students with special auditory abilities of the Specialized Educational Unit Dr. Camilo Gallegos Domínguez. The project has a quantitative approach, which helped us to analyze the data and a qualitative approach to know the degree of assimilation shown by the students about the mobile application. We used the Technological Acceptance Model (TAM) and Design to know the perception of the students by means of a survey as an instrument. The results showed a high degree of satisfaction towards the use of AR and VR in the mobile application as a learning method. Keywords Augmented reality · Virtual reality · teaching-learning · mobile app

J. Sánchez-Guerrero (B) Faculty of Human Sciences and Education, Technical University of Ambato UTA, Ambato, Ecuador e-mail: [email protected] A. Haro-Velasteguí Business Administration, Polytechnic Higher School of Chimborazo ESPOCH, Riobamba, Ecuador e-mail: [email protected] S. C. Ríos · J. Ruiz Faculty of Systems, Electronics and Industrial Engineering, Technical University of Ambato UTA, Ambato, Ecuador e-mail: [email protected] J. Ruiz e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_10

109

110

J. Sánchez-Guerrero et al.

1 Introduction Hearing impairment is one of the most common but necessary social problems to take into account, it is defined as the reduction of receiving sounds, which can be in its entirety (cophosis), partially (hypoacusis), unilateral or bilateral. Due to this disease, people tend to suffer discrimination, making it difficult to be included in important areas such as education, socio-cultural and labor, due to the absence of oral communication with other people [1]. According to studies by the World Health Organization (WHO), 466 million of the world’s population suffer from hearing loss, among which 7.3% are children. In many cases, the solution to the recorded problem begins with the early discovery of the same, which help to improve in some way thanks to various resources that can be hearing aids, cochlear implants and today there is talk of devices that help in the process of learning sign language and the adoption of subtitles in the communicative process [2]. Virtual learning (e-learning) is becoming increasingly important in the learning process and professional preparation, in Malaysia a project has been designed to allow indigenous people educational inclusion. The objective set for the construction of the work is focused on the design, development and evaluation of different web applications for learning sign language in Malaysia (LSM). The application describes both E-Hearme, E-Isyarat, Virtual Malaysian Sign Language (website) and the Malaysian Sign Language Group in social networks. In the construction, the alphabet of sign language, numbers, words, fingerspelling and multimedia content, which are composed of the interface, the program, database, images and interactive files, for the present has been made use of open-source technology resource, among which we have HTML5, JavaScript, AJAX, jQuery, PHP and MySQL [3]. Currently the term m-learning continues to have more importance due to the number of smartphones and tablets and their ease of obtaining them, thanks to its great boom in technology, which has allowed it to be an accessible resource for the vast majority of the general population. The work focused on mobile learning makes a study on the help it can provide to the teacher by providing educational material with great advantage, this with the help of the internet. The proposed objectives within it are in the design of a Filipino sign language (FSL) reinforcement application for people with special hearing abilities. The content of the program is a dictionary, images and a series of questions with interactive activities that incorporate about 50 main FSL signs for the hearing impaired and non-hearing-impaired population, helping them to raise awareness of the environment. In the dictionary project you can see a multimedia video resource through a human guide, which presents some important words through sign language as an additional feature presents a reinforcement test with the signs interpreted through animated images in second dimension [4].

The Mobile Application in Learning for People with Special Hearing …

111

In the country, according to the National Council on Disability CONADIS, [5], of the 471.205 Ecuadorians with some type of registered disability, 14.12% represent people with special hearing abilities, but it turns out that no applications have been developed to help inclusion with society. In [6], on “Computer-assisted learning model for teaching communication of people with hearing impairment”, he mentions that information and communication technologies (ICT) are great allies of the educator to promote the teaching-learning process since these tools make it possible to promote a better quality of educational life and facilitate learning. The methodology applied in this work is focused on two stages: the first is called definition, which includes a conceptual study, analysis and the initial design of the project. The second corresponds to the development stage where a computational design, development and deployment of the resource is created. This methodology is known as MeISE. In order to obtain the results, the student’s t-test is used, which allows analyzing the results, where it is determined that the computer-assisted teaching-learning method helps the motivation of students with hearing impairment. The implementation has allowed the development of a system with interactive approach for children to maintain a meaningful learning through an essential tool in the performance of each class. At the same time that technology significantly helps the educator, there is also a demand for the correct use and manipulation of technology, which is why teachers are called to the evolution of schools 2.0 that will favor the teaching-learning process, thus cultivating a pleasant and favorable educational life for society. The objective of the work carried out was to develop a mobile application to strengthen the learning of students with special hearing abilities of the Specialized Educational Unit Dr. Camilo Gallegos Dominguez.

2 State of the Art There are many works related to this topic, of which a systematic review has been made to consolidate the theoretical foundation for the work done. [7] In the topic “Designing SmartSignPlay”, it describes the design of an interactive app (called SmartSignPlay) that allows the use of interactivity and the popularity of smartphones as a great advantage for the learning of ASL vocabulary and phrases by deaf children and their families, which includes the entire population of children with medium and complete hearing difficulties. Research findings have determined that the lack of communication resources that exist between parents and children, in cases results in the late building of the child’s language and social skills. The content of the mobile application consists of a vocabulary that is organized into lessons highlighting the frequency of a word shown through an animated avatar, where the interactive user draws the trajectory of the hand movement and selecting the correct hand shape.

112

J. Sánchez-Guerrero et al.

Many of these innovations can be applied in many areas of knowledge and the effects shown revealed a high compliance in the use of educational documents with AR, by the interaction of content that aroused interest and motivation in the subject matter, in addition there was progress in the teaching and learning process [8]. In other areas that are not as educational as physical education, the use of innovative technologies in mobile applications, it is common for people to use innovative technologies to motivate weight control and physical activity, among others [9]. According to [10], in the research with the theme “Mobile Phone Access to a Sign Language Dictionary”, a mobile application is developed as a model that helps deaf children and their parents in the search of ads defined in American Sign Language (ASL) through the smartphone camera that focuses on a main word printed in books, taking a picture to later click on a captured word for access to the definition itself. The topic of work is focused on the design of an “application made with augmented reality to support education and learning of the alphabet. For the construction of this project had as a previous work, to know the application that helps people with special hearing abilities through augmented reality and application focused on learning sign language. The method for the creation of the prototype was the modeling of 3D objects through Blender desktop software, then saved in the MySQL database, the established population was 30 people between students of the Autonomous University of Ciudad Juarez (UACJ) who suffer from deafness and people without any disability, the same that were selected randomly. As a result of the research, it is found that the population has shown an interest in the 3D resource and wish to have more interaction through it as it is an emerging technology [11]. The project theme “Tisekuae Xuta Xtaya: Help for deaf people”, details the advantages of the mobile application in the field of education, allowing children to learn Mexican Sign Language (LSM), which provides an important contribution in cognitive and socio-affective development, education, communication and personality. The Methodology implemented for the realization of the project was the User-Centered Design (UCD), which includes three important phases: study, design and evaluation. The study is related to the community of Teotitlán de Flores Magón, with children who have attended preschool, between 5 and 12 years old. The objective is to improve their cognitive capacity and present them as an introduction to a technological future helping the communicative ability and therefore the quality of life of people with special hearing abilities [12].

3 Methodology Due to the fact that for the present research information is required to help in the development of the mobile application for people with special hearing abilities, the project has a documentary modality because previous information was reviewed in primary sources, books, magazines, theses, etc. Field because the data collection was carried out in the place of the facts of the Specialized Educational Unit “Dr. Camilo Gallegos Dominguez”. It has a qualitative-quantitative (mixed) approach, since it

The Mobile Application in Learning for People with Special Hearing …

113

includes both a qualitative model that allowed us to evaluate the behavior shown by the students on the mobile application and a quantitative one, which helped us with the collection and analysis of the data obtained during the research process and therefore to obtain results. It is descriptive because the subject is aimed at the observation and description of certain attitudes towards the development of the research in terms of each of the variables established. A total of 17 students with special hearing abilities have been selected, corresponding to the 2nd and 3rd year high school students of the Specialized Educational Unit “Dr. Camilo Gallegos Domínguez”. For the present research the ADDIE methodology was used, which is an instructional model based on phases in the whole process that allows any user without previous experience to use it as a guide for the development of a program or course. The design is constituted by the phases of Analysis, Design, Development, Implementation, Evaluation and Evaluation [13].

3.1 Analysis It is necessary to have own models in three dimensions, so it was necessary to use the 3D software is Daz Studio 4.10 for the generation of models, free distribution that supports both Windows and MAC OS, we can find it on its official website (https:/ /www.daz3d.com/), the requirements to work are minimal since it only requires a Pentium III 700 MHz and a 128 MB card compatible with OpenGL. The origin of the program is designated to William Fetter in the decade of the 60’ that was popularized for the edition of videos, since then the procedure is generated for movies, video games. Android Studio 3.x development platform for the design of the mobile application as it allows a much more specific design thanks to the inclusion of Java Script language which is used in most apps, it was developed by Google to create application software for the Android platform, which officially started on December 8, 2014 with version 1.0.0, and is compatible with operating systems such as Microsoft Windows, macOS, Linux, Google Chrome OS, free program with apache 2.0 license. Sketchfab was also used which constitutes a web platform to share and visualize 3d content, it allows the demonstration of Augmented Reality (AR) and Virtual Reality (VR), the site accepts different types of formats such as: FBX, OBJ, DAE, BLEND, STL, we can also upload files such as ZIP, RAR or 7z, containing their textures, materials and meshes for a better presentation of the object (Fig. 1).

114

J. Sánchez-Guerrero et al.

Fig. 1 Design app

3.2 Design Navigation Map

3.3 Development For the process of generating animations, the character called Genesis 3 was kept, considered an avatar package with 2 genders, in the application Genesis Male 3 has been used, which has 100% human characteristics. After selecting the character, we worked on the animations of each of the letters of the Ecuadorian sign alphabet. The dactylological alphabet is an essential resource for the construction of the movements that generate the communication of signers. Dactylological Alphabet Also known as manual alphabet, it has 30 different symbols that refer to each letter of the Ecuadorian alphabet, helping to generate words through combinations, similar to a written language. It is shown in Fig. 2.

The Mobile Application in Learning for People with Special Hearing …

115

Fig. 2 Manual alphabet

3.4 Implementation Four tests were conducted in the computer lab with second and third year high school students respectively, these experiences were developed in the span of 4 weeks which consisted of using the smartphone to access the website (https://zumbanalui. wixsite.com/misitio), where the information of the application and the download of the same was provided. The students had to: read the content, download and install the application on the mobile, manipulate the menu, visualize the animations, experience AR and VR. The functionalities were guided through the TeamViewer software helped to project the smartphone screen. The teacher in charge was a key part of the communication as he was in charge of interpreting the instructions given through signs. All tasks were completed as the teacher and myself were in charge of checking each device running the application. When installed, the application is identified with the icon of the letter A in LSE as shown in Fig. 3.

3.5 Evaluation For the validation process of the created resource was focused on two aspects, the technological, for which the TAM model has been used and the aesthetic (Design),

116

J. Sánchez-Guerrero et al.

Fig. 3 Icon app

an AD-HOC survey of 4 categories was used, the data were tabulated using the SPSS statistical package with the Cronbach’s Alpha method, for obtaining results and verify the proposed hypothesis. According to Ventura, (2018), Cronbach’s technique created in the 1951s, proposes a coefficient α that helps to measure the level of reliability or obtain the mean that exists in relation to a certain number of variables. To determine the reliability of the instrument, Cronbach’s Alpha was applied, which establishes values between 0 and 1, which indicates that, if there is a value close to 1, the better the reliability of the instrument, on the contrary, if the value is less than 0.7, it is considered to have a high degree of inconsistency [14].

4 Results To validate the surveys applied, the SPSS statistical software and Cronbach’s Alpha statistic were used, obtaining values of 0.92 and 0.87 respectively, showing a high degree of reliability of the instruments used, as detailed in Tables 1 and 2. For the verification of the hypothesis, Pearson’s Chi-square was used, based on the logical model, which was established at the beginning of the research. With the research hypothesis: “The use of the mobile application will benefit the learning of people with special hearing abilities”. After analyzing the data using the SPSS statistical package, the Pearson Chisquare value corresponding to the TAM model was 21.87, while 17.91 for the design model is shown in Table 3 and 4, respectively. Table 1 TAM Model Reliability Data

Cases

Valids Excluded Total

N 17 0 17

% 100 0 100

Reliability statistics

Crombach's Alpha 0,922

N of elements 13

The Mobile Application in Learning for People with Special Hearing …

117

Table 2 Reliability data of the Design Model

Cases

Valids Excluded Total

N 17 0 17

% 100 0 100

Reliability statistics

Table 3 Chi-square test TAM model

Table 4 Chi-square Test Design Model

Crombach's Alpha 0,876

N of elements 14

Statistician

Value

df

Sig. Asint.(2 colas)

Pearson Chi-square Similarity ratio Linear-by-linear association N of valid cases

21,87 16,93 3,26 17

12 12 1

0,039 0,152 0,071

Statistician

Value

df

Sig. Asint.(2 colas)

Pearson Chi-square Similarity ratio Linear-by-linear association N of valid cases

17,91 9,00 3,35 17

4 4 1

0,01 0,061 0,067

Since, the value of the TAM model is 21.87 and 17.91 for the design model, it allows rejecting the null hypothesis (H0) and accepting the alternate hypothesis (H1), where it mentions that: The use of the mobile application DOES benefit the learning of people with special auditory abilities.

5 Conclusions and Future Works According to the data resulting from the research, it was possible to prove that the use of the mobile application with AR and VR encourages the learning process, this is evidenced by the large number of responses as a high level of satisfaction in the use and ease of manipulation of the resources presented. In the same way, the responses found when evaluating the design of the resource had a large number of responses as highly satisfactory towards the quality of content, usefulness and access to it. Finally, upon analysis, it is determined that the technology presented to the students has a high degree of acceptance through the reliability data obtained.

118

J. Sánchez-Guerrero et al.

Thus, it is concluded that the students of the Specialized Educational Unit Dr. Camilo Gallegos Dominguez see the use of AR and VR as an innovative and motivating process, by increasing their participation in a dynamic way, thus promoting self-learning. The research process developed, creates opportunities in the other areas of special capabilities, detected in the context and research environment, opening a range of options to provide real solutions through the use of devices that are being used daily and become inclusive. Acknowledgements The authors would like to thank the Universidad Técnica de Ambato (UTA), the Dirección de Investigación y Desarrollo (DIDE), the Facultad de Ciencias Humanas y de la Educación and the UODIDE-FCHE, for the support provided for this work through the research project “Espacios Virtuales en 3 Dimensiones, Recursos Digitales y Entornos Personales de Aprendizaje como herramientas de apoyo en la Neuro Didáctica”, code PFCHE19.

References 1. Guzmán D (2017) Guante Electrónico para Traducir de Lenguaje de Señas a Caracteres con Voz Artificial, Ambato 2. O. M. d. l. Salud (2021) Organizacion Mundial de la Salud. https://www.who.int/es/newsroom/fact-sheets/detail/deafness-and-hearing-loss. Accessed 20 July 2022 3. Karbasi M, Zabidi A, Yassin I (2017) Malaysian sign language dataset for automatic sign language recognition system. J Fundam Appl Sci 9(4S):459–474 4. Garcia MG, San Luis C, Samonte MJ (2016) E-tutor for Filipino sign language. In: 2016 11th international conference on computer science & education (ICCSE) 5. Conadis (2022) Estadisticas de discapacidad, Enero. https://www.consejodiscapacidades.gob. ec/estadisticas-de-discapacidad/. Accessed 30 July 2022 6. Sánchz P (2016) Modelo de aprendizaje asistida por computadora para la, La Paz 7. Chuan C-H, Guardino CA (2016) Designing SmartSignPlay: an interactive and intelligent American sign language app for children who are deaf or hard of hearing and their families. In: Companion publication of the 21st international conference on intelligent user interfaces, pp 45–48 8. Sánchez-Guerrero J, Salazar Mera J, Gavilanes López W, Sánchez Reinoso R, Tamami Dávila C (2018) Use of augmented reality AR in university environments. In: 2018 international conference on eDemocracy & eGovernment (ICEDEG), pp 291–297 9. Mocha-Bonilla J, Sánchez-Guerrero J, Jimenez L, Pilamunga F, Barona-Oñate R, Sánchez Guerrero A (2018) Analysis of the body composition index and basal metabolic rate through the mobile application eHealth-UTA. In: 2018 international conference on eDemocracy & eGovernment (ICEDEG), pp 386–391 10. Jones MD, Hamilton H, Petmecky J (2016) Mobile phone access to a sign language dictionary. In: Proceedings of the 17th international ACM SIGACCESS conference on computers & accessibility, pp 331–332

The Mobile Application in Learning for People with Special Hearing …

119

11. Cruz Sánchez VG (2018) Aplicación de apoyo visual para el aprendizaje del alfabeto dactilológico con realidad aumentada, México 12. Armas E, Cano R, Hernandez V (2018) Tisekuae Xuta Xtaya: Ayuda a las personas sordas. Avances En Interacción Humano-Computadora 3(1):10–13 13. Gamez I (2015) Los Modelos Tecno-Educativos, revolucionando el aprendizaje del siglo XXI, Editorial Lulú Digital 14. Soler J, Soler R (2012) Usos del coeficiente alfa de Cronbach en el análisis de instrumentos escritos. Revista Médica Electrónica 34(1):1–6

Localizing a Mathematics Tutoring System to Spanish in Latin-America Ivon Arroyo, Andrew Euredjian, Luisa Perez Lacera, Danielle Allessio, Waldo Geremías, and Beverly P. Woolf

Abstract We localized a Mathematics Intelligent Tutoring System that addresses motivational aspects of learning to the language and culture of a Latin-American country, Argentina. We analyzed its impact after three different schools using the software for seven weeks in three schools in Cordoba, Argentina. Results yielded a significant improvement in mathematics performance in Spanish (after 6 sessions) showing the feasibility of learning mathematics with MathSpring in Spanish as a supplement to classroom instruction, with curriculum/software localized to the region. Further analyses showed an affective impact, as students improved on their grit –perseverance and passion for long term goals, as affective digital characters instilled perseverance, speaking to them in their own local Spanish accent. Keywords Localization · Intelligent Tutoring Systems · Cultural Differences

1 Introduction Across the planet there are 7.5 billion people, of which just 1.5 billion (20%) speak English. The Center for Immigration Studies finds that 22% of residents in the United States now speak a language other than English at home (Ziegler&Camarota, 2019). Creating learning technologies for English speakers only seems a missed opportunity for educating people across the globe, at the same time that learning technologies are a key scalable solution to reaching students all over the world.

I. Arroyo (B) · D. Allessio University of Massachusetts Amherst, Amherst, MA, USA e-mail: [email protected] A. Euredjian · L. P. Lacera · B. P. Woolf Worcdester Polytechnic Institute, Worcester, MA, USA W. Geremías Universidad Blas Pascal, Cordoba, Argentina © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_11

121

122

I. Arroyo et al.

There has been increasing interest in expanding the reach of Learning Technologies, meaning exporting technologies for education that were generated in Countries that are considered “very highly developed” by the United Nations Human Development Index (HDI) to those who are not. Interest has been shown in the research community to create forums to discuss cross-cultural differences and the rational introduction of learning technologies in the developing world (e.g. [2], exploring the economic constraints, socio-cultural differences, political and other constraints that shape the implementation and the affordances for learning technologies in the developing world. Localization of learning technologies is the process of translating and adapting training content for a specific culture and language. Localization is not just straightforward translation; it involves additional efforts beyond the conversion of text from one language into another. It includes transforming imagery, tone, currencies and more to account for cultural and regional nuances. The value in localizing eLearning content is that it ensures that the learning materials and activities are comprehensible and culturally appropriate for the target audience, as well as reachable. Besides differences in socialization and cultural differences, well-intentioned introduction of learning technologies in developing countries can fail for mundane reasons such as teachers not willing to use the technology because of lack of comfort with technology, or simply lack of computers or poor WiFi/Internet connectivity, in sharp contrast to abundance of mobile devices. Such constraints cannot be ignored. Rather than blindly implanting technologies, based on a rationalized discussion of such issue and constraints, and possibilities for the immersion of learning technologies, visions and roadmaps of such technologies for the developing world and subsequent practical implementation for technology enhanced learning are necessary. The main research questions of this research study were three: RQ1) What does localization of a learning software created in the United States, to a latin-american country, entail and how to do it?; RQ2) Is the implementation of the localized software still effective after the localization, given the tutoring software’s math teaching goals?; RQ3) Does MathSpring still have a positive affective impact in their attitudes, given the growth-mindset training in MathSpring, thanks to the messages delivered by characters who train their perseverance and effort during problem solving?

2 MathSpring/CultivandoMatemáticas MathSpring Tutoring System. MathSpring is an online mathematics multimedia “intelligent tutoring system” developed by our team at UMass-Amherst supported by funding from the National Science Foundation in the United States [1]. MathSpring is game-like, in that students grow gardens that visually represent their mathematics progress, where plants grow depending on their effort excerpted at problem solving. It is multimedia in that it provides both audio and visual support to the student. It is intelligent in that it builds an internal model of the student, as would a good

Localizing a Mathematics Tutoring System to Spanish in Latin-America

123

teacher, and provides personalized content to each individual child, as well as remedial tutoring when needed, depending on this. For instance, MathSpring might have been set up to reach the goal of teaching a student Grade 7 mathematics, however, it might seamlessly move back to grade 6 and 5 material as needed, unnoticeably to the student. MathSpring is an affective tutoring system int that animated “learning companions” in it have an affective agenda: they provide emotional support and build their socio-emotional skills, instilling a growth mindset. Students’ emotions get assessed regularly (in between math problems) through prompts that ask students how they feel. Figure 1 shows MathSpring, and the learning companion empathizing with students, offering support when they become frustrated or anxious, delivering ~ 70 different messages after assessing a student’s emotional state [1]. Avatars also train students’ “growth mindset” (Dweck, 2002), encouraging students to consider mistakes as a natural part of learning and that intelligence is malleable. MathSpring provides detailed analyticat s and visualizations to teachers to inform adjustments to classroom instruction. We have ample evidence that MathSpring can also leave a positive affective impact on students through its animated characters who offer encouragement or praise on each problem, with students in schools in the United States. In controlled research studies, after using MathSpring, students in the United States have evidenced an increase in mathematics scores, both from before to after using it, as well as comparing groups of students who use MathSpring to control groups that did not use it. Also, we know that students’ academic emotions during problem solving (e.g. frustration, anxiety, interest) are positively impacted by the learning companion [1], at least when the software is in used in English.

3 Method 3.1 Implementation of Bilingualism in MathSpring We localized MathSpring to Argentine language and culture and called it ‘Cultivando Matemáticas’, available freely from anywhere at cultivandomatematicas.org. This involved more than translation of math content from Spanish to English. This section explains that localization from a software engineering perspective, as well as from a curricular perspective. Localization of Curriculum and Math Content. MathSpring is aligned to the Common Core Standards in the United States, and every math problem is associated to a CC standard, and teachers can view students’ performance by standard and by topic (which are groups of standards that span across grade-level). While Mathematics is a universal language, mere translation of problems was not enough in this localization effort. There were differences in notation (for instance, the decimal point in South America is a ‘.’ where the ‘,’ is used to mark the thousands place). There were differences in the curriculum in Argentina for 6th grade (our focus of this

124

I. Arroyo et al.

Fig. 1 The ‘Practice Area’ of CultivandoMatematicas. The items that are still in English are automatically translated when the language of the browser is set to “Spanish”, following the ResourceBundle mechanism explained later. All the content shown in Spanish is assigned from the Teacher Tools (by the teacher configuring the class/assignments) by selecting that this is a “Spanish” class. Students solve problems with support of the tools to the left, where the “Ayuda/Hints” button explains each problem-solving step (read aloud to the student) and other scaffolds (examples, videos). Math problems are either short answer, multiple choice, or check-all-that-apply. Lucas (right) is an affective learning companion, a pedagogical agent who empathizes with the student and instills the importance of effort and perseverance. All content is in English, and the two possible (male/female looking) characters.

study), in that some topics were taught in later grades. Last, there were differences in the content taught, as metric system taught is based on the meter, liter, and gram as units of length, capacity, and weight or mass (as well as conversions within it), while the United States focuses on the imperial system of measurement, where objects are measured in feet, inches, pounds, ounces. In addition, math problems and hints/ support have audio files associated to them, so a simple textual translation was not enough. The solution was to create a new set of math problems in selected topics of MathSpring for Argentina’s 6th grade curricula to Spanish, their hints/help used for problem-solving support, as well as messages provided by the affective avatars (learning companions), with audio files recorded with local Spanish-speaking personnel. The mathematics topics that the students practiced in the resulting CultivandoMatematicas.org were: 1) Fractions and equivalent fractions (conversions), 2) Basic operations with fractions (addition/subtraction), 3) Advanced operations with fractions (multiplication/division),

Localizing a Mathematics Tutoring System to Spanish in Latin-America

4) 5) 6) 7) 8)

125

Decimals (conversions from fractions to decimals and identification), Basic operations with decimals (addition/subtraction), Advanced operations with decimals (multiplication/division), Unit conversion using the metric system (weight, volume, distance), Unit conversion using time (hours, minutes, seconds) .

As mentioned before, these are topics/problem sets, collections of problems that are associated to a group of standards, Common Core Standards as in the United States. The topics/problem sets are selected by the teacher when configuring the class. Currently, there is a drop-down menu in the Teacher Tools, when teachers create a class for their students in CultivandoMatematicas, asking for whether the content for the class should be Spanish based, or English-based. Depending on the teacher’s selection, topics, problem sets and standards will be available in English or in Spanish. If the teacher chose that the class will be Spanish based, only topics that have math problems created in Spanish will be available for the teacher to select. The character should be chosen for the class also, different characters speak in different langauges. Multi-lingual MathSpring Software Implementation. The Multi-lingual implementation of the “wrapper” of MathSpring (any text that surrounds the mathematics content) uses a dictionary file. The file is a simple list of key-value pairs, where the key is a token and the value is the text to be used in the caller’s program. There may be multiple dictionaries. Each dictionary contains values for a single language. All dictionaries need to provide the same set of tokens. The dictionaries are delivered as part of the installation object (.war file). The architecture of MathSpring and its localized components are shown in Fig. 2. In java servlet/Java Server Page (JSP) applications, the ResourceBundle class provides a way to access separate dictionary files based a selected locale. Servlets and JSPs have access to the requester’s http request object which contains the current locale of the requester’s browser. For instance, within the Java code, the ResourceBundle.getString() method accepts a token and returns the text value. For instance, in a typical JSP scenario, e.g. the login page, the user at the browser requests a JSP page providing the current locale from the browser. The requested JSP opens the appropriate dictionary file for that locale. The View-creation code uses the ResourceBundle.getString() method to retrieve the text in the current language Finally, the View is returned to the browser for display. Another example is a typical servlet scenario, e.g. a request for a Teacher Report within the Teacher Tools: The user at the browser sends a request for a report an http request which contains the current locale from the browser. The requested servlet opens the appropriate dictionary file for that locale. The requested JSP opens the appropriate dictionary file for that locale. The Mode-code determines the next math problem to show the student. The Viewcreation code uses the ResourceBundle.getString() method to retrieve the text in the current language Finally, the View is returned to the browser for display. The typical case though is shown in Fig. 1, the MathSpring Problem Solving page, where the student makes a JSP request to go to the Problem-Solving page, as

126

I. Arroyo et al.

Fig. 2 The architecture of the MathSpring / CultivandoMatematicas web-based intelligent tutoring system, and its mechanism to retrieve content in Spanish

in the login page example. The requested servlet opens the appropriate dictionary file for that locale. The View-creation code uses the ResourceBundle.getString() method to retrieve the text in the current language. The View-creation code also uses the ResourceBundle.getString() method to store translated text message locally at the browser site as variables in the browser environment. These are used in conjunction with the local JavaScript code to achieve the multi-lingual presentation throughout the user’s session solving problems. Finally, the View is returned to the browser for display.

3.2 Participants In Fall 2019, 150 students attending three schools in Argentina used MathSpring in Spanish. The participants of this research study were 6th grade students (11–12 year old children) from three different schools in the city of Cordoba, Argentina. The schools in question will be referred to as School A, School B, and School C All three schools were private schools. School A had three classes participating in the study: the first class (6A) had 22 students, the second class (6B) had 18 students, and the third class (6C) had 18 students. School B had two classes participating in the study: the first class (6A) had 34 students and the second class (6B) had 37 students. School C also had two classes participating in the study: the 1st class (6 V) had 16 boys and the 2nd class (6 M) had 15 girls.

Localizing a Mathematics Tutoring System to Spanish in Latin-America

127

3.3 Measures to Answer the Research Question To understand the cognitive impact of the software, a Spanish 10-question mathematwas created to assess students’ varying math abilities. These mathematics problems were not the same as those within CultivandoMatematicas, but covered the content in the topics listed in Sect. 2.1. In general, there was one question per topic, with the exception of fractions, where two questions at different levels of difficulty (easier, harder) were created, as a few different topics (problem sets) covered that content. In order to understand the affective impact of CultivandoMatematicas, we decided to assess a students’ “Grit”, before and after using the software. Grit is defined as a “passion and perseverance for long-term goals” (Duckworth, & Eskreis-Winkler, 2015). It is a personality trait that encompasses the characteristics of people who do not let challenges stop them from achieving their goals. People who have grit can be seen working diligently towards their goals, not necessarily short-term goals, but long-term ones that can take years to complete. These people are usually found to be very successful not only in their work but their home lives as well. Although this does not necessarily mean that they have a high capacity for knowledge, it is orthogonal to cognitive capacity (Duckworth et al., 2015). Interestingly, Grit can be seen as potentially more important than intelligence itself. People who have a high grit tend to be people who have a passion for what they do, exert huge amounts of effort when they want a goal completed, they will not back down, and have high diligence in what they do. Interestingly, Grit has been considered mostly a trait, something that can be changed over a lifetime –as people age, or after very long periods of training. One English grit test survey (8-items) used to assess grit was translated to Spanish and administered to students. Selected items are in table 1. Students answered in a scale from 1 to 5, saying how likely the statement was like them. Table 1 Some items from the Grit Test, translated to Spanish, used to assess Grit on the 1st and last day of CultivandoMatematicas. Items with a * are to be reverse coded

Four (4) selected items from the 8-item GRIT pre/posttest survey, translated to Spanish 1. Las nuevas ideas y proyectos nuevos a veces me distraen de los que ya tenía de antes.* 2. Los contratiempos no me desaniman 3. He estado enfocado en una idea o proyecto por un tiempo, pero luego perdí el interés.* 4. Termino todo lo que empiezo 5. A menudo me he fijado un objetivo pero después decidí perseguir otro.*

128

I. Arroyo et al.

3.4 Procedure This study was conducted over a period of six weeks in which students from School B and School C used CultivandoMatematicas once a week in the classroom while students from School A used the system twice a week in the classroom, once in Spanish and once in English, as the school was bilingual. On the first day of class, the research team gave a basic introduction of the CultivandoMatematicas tutoring system and administered the mathematics test (pre-test). This allowed to assess students’ varying math abilities, and to use as a base from which to measure improvement with CultivandoMatematicas. We administered a pre-test and post-test to the students at the beginning and end of the seven beginning and end of the study respectively. Similarly, the last day, the same math post-test was administered to measure any improvement in the students’ mathematics skills after having used CultivandoMatematicas for six weeks. In two out of the three schools, there were not enough computers for each student to work on their own. For these cases, students that did not get their own computer were given a paper-based test, and later paired up with students that had taken the exam on a computer to work together. Each student, or pair of students depending on the ratio of computers to students, received login credentials that they would use for the entirety of the study. Students from School A (a school that taught ESL in the afternoons) received two logins, one for Spanish and one for English. Once finished with the pre-test, students used the remaining class time to begin using MathSpring and solved problems from the first topic. Each day between the first and last days, the students worked on math problems throughout the class period in CultivandoMatematicas. A new problem set was open to the students per day while problem sets from previous days were closed to make sure the students stayed on track. Students were expected to be finished or close to finished with a problem set by the end of each class period such that by the end of the study, all topics listed above would have been covered. In some classes two problem sets were opened with the expectation that the first would be completed by the students quickly and with time to spare to work on another problem set. The last day, the research team administered a post-test for each class. The post-test had the same questions as the pre-test in order to find out if students could correctly answer questions that they had struggled with or not known the answer to on the first day after having used MathSpring. Similar procedures to that of the pre-test were conducted where classes with an inadequate number of computers compared to the number of students had some students taking a paper exam. After completing the post-test, all problem sets were opened for the students to view their overall progress over the six-week period.

Localizing a Mathematics Tutoring System to Spanish in Latin-America

129

4 Results 4.1 Impact on Mathematics Problem Solving Ability The first day data on the mathematics tests describes starting points for each of the students with respect to students’ incoming math skill level (math pretest). This data tells us how well each student was able to understand and complete practice problems. The posttest data allows to gauge the amount of improvement or change in each student’s mathematical skills, in comparison to their scores at pretest time. The results from the pretest are displayed in Fig. 3. The average pretest score for all students across schools came out to approximately 43% correct answers. Students scored best on problems involving decimals and decimal operations at around 66% and scored worst on problems involving measurements and unit conversions at around 27%. Students also scored about 47% on problems involving fractions and around 38% on problems involving time. When comparing the means of all students’ pretests and posttests as well as the scores on each topic in the tests, it can be observed that the students improved over the course of the study while using CultivandoMatematicas. The overall test score average increased from 43 to 49% which, for such a short amount of time students were exposed to each topic, is a sizeable increase. Results of the paired samples t-test showed that the mean measurement of math problem solving ability differed between pretest time (M = 0.43, SD = 0.21, N = 141) to posttest time (M = 0.49, SD = 0.21, N = 141) at a 0.005 level of significance (t = 2.88, df = 140, p < 0.005). When looking at the topics individually, students were found to have improved most with problems involving time, going from 35 to 45% with problems involving fractions a close second in amount improved going from 46 to 54%. Students also improved a decent amount on the topic of measurement and unit conversions going from 26 to 32%. However, there was a lack of improvement on problems involving decimals which remained constant at 66.3%, which suggests there is room for improvement in that topic of decimals, either in the quality of instruction and/or in exposure time. As shown in Fig. 3, we found that the mean math problem solving performance at pretest and posttest time were significantly different (p < 0.005), showing a growth in overall math problem solving performance after the six weeks of the intervention.

Fig. 3 Paired samples t-test results and descriptive Statistics for the Math Pretest/Posttest Scores of Students in Argentina, before and after the six weeks of using CultivandoMatematicas.org

130

I. Arroyo et al.

When considering individual topics, we found that the fractions topic in particular had an important difference and significant growth (p < 0.001), while other topics had marginally significant growth –for instance, for measurement (p < 0.08), for time units’ conversion (p < 0.09). However, recall we mentioned there were more items to assess knowledge of fractions, unlike other topics, thus it is likely we were able to capture a less noisy metric of student ability for the “fractions” topic in particular in comparison to other mathematics topics. While there was an improvement in the math test in general, one of the largest improvements was in the problems about Fractions (Cohen’s d = 0.27, p < 0.002). Not surprisingly, students spent most time in the fraction math problems, as the fractions topic was the one made available to students first, and due to the personalization features of CultivandoMatematicas, they tended to spend more time in those first topics.

4.2 Impact on Students’ Motivation: Perseverance and Passion for Long-Term Goals Full data for one hundred and fourty four (144) students (pretests and posttests of Grit measurements) were collected from the participating students, after removing students who had not completed either the pretest or the posttest. A paired-samples t-test was conducted to compare for any potential changes in grit from pretest time to posttest time, as seen in Fig. 4. Results of the t-test showed that the mean measurement of grit differed between pretest time (M = 3.27, SD = 0.53, N = 144) to posttest time (M = 3.44, SD = 0.58, N = 144) at a 0.001 level of significance (t = 4.06, df = 143, p < 0.001). This implies that, on average, Argentina participants saw a increase in grit after the seven weeks of using CultivandoMatematicas.

Fig. 4 Paired samples t-test results and descriptive statistics of change in GRIT from the first day to last day of the six weeks of usage of CultivandoMatematicas.org

Localizing a Mathematics Tutoring System to Spanish in Latin-America

131

5 Discussion and Conclusions The evidence presented in this article suggests that it is both technically possible and practically feasible to implement the localization of an online mathematics intelligent tutoring system, such as MathSpring/CultivandoMatematicas, to a different language and culture, making it multi-lingual and multi-cultural. More importantly, we showed that the result of its use by students and teachers in the target country can still impact and benefit students, allowing both for cognitive and affective growth in the target children. Hundreds of students and their teachers successfully used CultivandoMatematicas, a mathematics intelligent tutoring system that personalizes instruction to individual students, and where affective pedagogical agents address their affective needs and train their attitudes towards problem solving, particularly when facing situations of discouragement and failure. From an emotional/motivational perspective, even though it is still disputed whether grit is this static personality-like characteristic of an individual, our results suggested that students’ Grit (perseverance and passion for long term goals) can improve after 7-weeks of using CultivandoMatematicas, after receiving affective digital agents that instill a “growth mindset” while solving problems in the software. Localization entailed more than translation, it involved changes in the curriculum covered by CultivandoMatematicas to match the realities of the region (such as pairing up students due to lack of computers), differences in curriculum, and characters and read-aloud features speaking to students in Spanish (in their local accents, from a few different areas within the country). We conclude that localizing and implementing online digital scalable tools such as CultivandoMatematicas, we can impart opportunities for individuals to keep growing and developing, beyond highly developed countries and into the developing world.

References 1. Arroyo I, Woolf BP, Burelson W, Muldner K, Rai D, Tai M (2014) A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. Int J Artif Intell Educ 24(4):387–426 2. Arroyo I, Zualkernan I, Woolf B.P (2013) Cross-cultural differences and learning technologies for the developing world. In: In: Lane HC, Yacef K, Mostow J, Pavlik P (eds.) Artificial intelligence in education. AIED 2013. Lecture notes in computer science(), vol 7926. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-39112-5_162 3. Duckworth AL, Eskreis-Winkler L (2015) Grit. In: Wright JD (ed.) International encyclopedia of the social & behaviour sciences, vol 2, no 10. Elsevier, pp 397–401 4. Dweck CS (2006) Mindset: the New Psychology of Success. Random House, New York 5. Ziegler K, Camarota SA (2018) 67.3 Million in the United States spoke a foreign language at home in 2018. center for immigration studies. https://cis.org/Publications. Accessed 11 Sep 2022

Educational Data Mining to Forecast Essay Score. A Case Study About ENEM Rian Bessa Lopes, Leonardo Pio Vasconcelos, Igor Garcia Ballhausen Sampaio, Leandro Miranda, José Viterbo, and Daniela Gorski Trevisan

Abstract The grade forecast is a very promising area of study in Educational Data Mining. We used the National High School Exam (ENEM) database, a national exam in Brazil, to study the effectiveness of essay grade forecast using machine learning algorithms. Different features configurations were explored in order to compare its utility. Considering the mean absolute error, Random Forest (103.59) followed by Neural Network (105.75) performed the best results. KNN regressor (108.94) and Linear Regression (109.18) algorithms done a less relevant generalizations. The prediction of essay grades did not perform as well as the prediction of objective subjects. Different skills are required for good grades in the argumentative essay and its forecast could be used as a tool for improving students’ outcomes by predicting poor performances. Keywords regression models · educational data mining · ENEM

R. B. Lopes (B) · L. P. Vasconcelos · I. G. B. Sampaio · L. Miranda · J. Viterbo · D. G. Trevisan Instituto de Computação – Universidade Federal Fluminense (UFF), AV Gal. Milton Tavares, Niterói, RJ, Brazil e-mail: [email protected] L. P. Vasconcelos e-mail: [email protected] I. G. B. Sampaio e-mail: [email protected] L. Miranda e-mail: [email protected] J. Viterbo e-mail: [email protected] D. G. Trevisan e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_12

133

134

R. B. Lopes et al.

1 Introduction From the early 2000 s, there was the popularization of machine learning, and its use increasingly frequent in the industry [4]. Given this rise and the vital role of education in economic development, an area of research standing out is Educational Data Mining [11]. Educational Data Mining (EDM) consists of using data mining techniques in educational databases [14]. One of the utilities implemented by EDM is the forecasting of grades as a strategy to improve education [19]. The Exame Nacional do Ensino Médio (High School National Exam) – ENEM is an exam that evaluates the performance of the students in Brazil. The ENEM has a very rich database, which contains information from variables related with students performances but it has few works exploring its potential. The purpose of the present study was to assess the effectiveness of machine learning algorithms’ application in essay grades forecast. We used four machine learning algorithms: Linear Regression, Random Forest, KNN regressor, and Neural Network. Each of the algorithms use diferrent machine learning approaches so it was possible to understand how well each of them learn better from educational data. Data were pre-processed and it was used four different features configurations. This work aims to answer the following research question: “How well can we predict the essay grade in the ENEM?” Carrying out the study, we investigate the ENEM data from 2019. The research question helps us to understand how well machine learning algorithms works in this specific subject of the exam. The main contribution of this work is to understand the effectiveness of a possible Educational Data Mining tool using machine learning. The essay grade may depends on different skills comparing to other subjects and its forecast is not well explored and could be used to improve students outcomes. This work is organized into seven sections, including this introductory section. Section 2 explains fundamental concepts for understanding the analysis performed. Section 3 reviews the related work similar to our area of interest and details the importance of this topic in the academic community. Section 4 presents the used materials and methods in the experimental analysis. Section 5 presents the results of this study. Section 6 discusses the results of the experiments. Finally, Sect. 7 is the conclusion of this article, as well as suggestions for future work.

2 Fundamental Concepts 2.1 Regression Algorithms Based on Machine Learning Regression algorithms based on supervised Machine Learning (ML) [3] are algorithms whose task is predicting a continuous number. This type of learning consists of presenting two types of variables to the algorithm. They are called independent variables or attributes represented by X = (x1 , x2 . . . , xn ), and the dependent variable,

Educational Data Mining ...

135

or response variable represented by Y = (y1 , y2 . . . , yn ). After training, the algorithm will be able to predict new Y values, based on new X values [10].

2.2 Educational Data Mining There is a strong relation between the quality of education and the economy of a country. Therefore many countries have been strongly investing in their education systems [11]. Data mining techniques brings to this endeavor a huge power of obtaining knowledge from educational data [14]. Educational Data Mining (EDM) allows us to develop many tools to reduce poor academic results, such as high dropout rates, absences, low grades, and failure. Early identification of students who have learning problems can be an important advance in personalizing education policy and allocating public resources [2]. One of the main topics in EDM is the use of machine learning to predict grades. In this context, Machine Learning is a strategy for action in educational management. As shown by [14], the students’ grade represents an extension of the achievement of long and short-term goals in the educational system. The grade can be presented as the final grade of a course, specific subjects, or an arithmetic average of several tests. As it represents an objective metric of academic results, grade prediction can feed intelligent tutoring systems and automated progress monitoring [14]. In other words, the prediction of grades is an important task to direct investment to improve education in general. It is possible to help in preventing evasion and poor performances by predicting students’ outcomes [23].

2.3 ENEM Exame Nacional do Ensino Médio (ENEM), or High School National Exam, is a standardized national exam, not mandatory, and it is one of the largest in the world in terms of the number of participants [17]. This exam has a great potential for studies and research, especially within the scope of EDM. ENEM measures the ability of candidates to master the set of scientific and technological principles about Secondary Education. The essay must be an argumentative text about a proposed theme with grades varying from 0 to 1000. [13].

3 Related Works In a review of works in Portuguese in the area of Educational Data Mining, [19], it was possible to identify the most used algorithms, such as J48, Neural Networks, Random Forest, and Naive Bayes. Authors also noticed that most works were dedicated to

136

R. B. Lopes et al.

performance evaluation in subjects - 15 (50%), or in specific courses - 10 (33.33%), and much less in schools - 2 (6.7%). Golino and Gomes [5] studied the use of algorithms - Learning Trees, Bagging, Random Forest, and Boosting, in the prediction of grades divided into classes (high and low) of students of the medical course of a private university. In work dedicated to EDM, Sorgatto et al. [20] used the INEP database of the School Census to predict grades through ENEM data by school. Several classification algorithms were used, considering that the ENEM score was categorized according to the statistical criteria for distributing values. Rodrigues et al. [17] also divided grades into categories and used classification algorithms. The data used were those of ENEM 2011, and the response variable was an arithmetic mean of the objective notes and the essay. Stearns et al. [21] also used two algorithms based on CART: AdaBoost and Gradient Boosting to predict the math score of the ENEM 2014 test. The attributes used were personal information, information from the school of origin, socioeconomic factors, all components of the ENEM registration form. Using data from ENEM 2016, Santos et al. [18] used the PCA technique to reduce the number of attributes, and the authors applied a Bayesian network to understand the most significant factors in the construction of the note. Our study selected Neural Network, Random Forest, Linear Regression, and KNN regressor algorithms to predict the essay grade. With these algorithms we can compare the result of the prediction of the essay grade with the prediction made in other subjects of other works. It is also possible to evaluate the predictive power between the algorithms used, especially the comparison with Linear Regression, which is the simplest [12].

4 Materials and Methods 4.1 Database To carry out this work, we used the database of Microdata from ENEM - National High School Examination of 2019, the most recent available at the time of this work, available by INEP. The data consists of 5,095,270 rows and 136 columns, with each row represented by one of the registered candidates. The data brings socioeconomic information from each candidate, which is answered when he or she enrolls, and which were used to make predictions in several works [5–7, 9, 17, 20]. In addition to the socioeconomic data, the columns also list data about the essay test, the objective test, the location of the test application, requests for specialized and specific resources to carry out the tests. There are also columns about requests for specific attendance, requests for specialized attendance, school data, and participant information [13].

Educational Data Mining ...

137

4.2 Pre-Processing We started from 135 columns and performed an attribute selection like in [8] and in [17] did. As criterion for deleting attributes was attributes with the same information in different attributes. Also irrelevant information for predicting scores such as the registration number was also deleted. Attributes such as the code of the test notebook, participant’s answers, or test template were also removed [15, 18]. The database is available at the link: https://bit.ly/34KgkwR. Finally, we did the hold-out evaluation, where we split the data into training data for fitting the model, and test data, where we have previously not seen [7] data by ML models. The division chosen was 70% for training data and 30% for test data, both randomly distributed [1].

4.3 Algorithms We used four machine learning algorithms to predict the scores: Linear Regression, Random Forest, KNN regressor, and MLP. We implemented all through the Scikitlearn module based on the Python language, whose names are, respectively, on the platform: Linear Regression, Random Forest Regressor, K-Neighbors Regressor, and MLP Regressor [16]. The Linear Regressor algorithm implements a traditional linear regression method by predicting coefficients to minimize the residual sum of the squares. Random Forest Regressor was trained with 250 trees with the limit of computational resources and processing time [10]. K-Neighbors Regressor implements a KNN-based regression. The k value was performed in the training set, obtaining the best results with 70 neighbors. We used uniform weights for neighbors [24]. The Regressor MLP implements a regression by a Multi-Layer Perceptron using Backpropagation. We used 25 neurons and one hidden layer. The activation function used was ReLU with 0.001 of initial learning rate and 200 maximum iteration limit [10].

5 Results In the present section, we elaborate on four different configurations for experiments resulting from the exploratory analysis of the database. The results presented the impact of each configuration algorithm quality and a comparative analysis of the different configurations.

138

R. B. Lopes et al.

Table 1 Configuration 1 Performance Algorithm RMSE Linear Regression Random Forest KNN Neural Network

MAE

R2

176.23

129.12

0.2034

169.17 177.02 170.77

124.71 128.99 125.48

0.2659 0.1962 0.2520

MAE

R2

176.24

129.13

0.2033

169.23 176.72 171.71

124.78 128.87 128.41

0.2654 0.1990 0.2437

Table 2 Configuration Performance 2 Algorithm RMSE Linear Regression Random Forest KNN Neural Network

5.1 Configuration 1 Initially, we split the database into train and test, which we will call Configuration 1. We gathered 1,000,256 rows and 95 columns, including the column with the essay grades (label). Table 2 shows the comparative result of the algorithms in this configuration. Random Forest obtains the best performance for three measured metrics. In worst cases, KNN has these results in R-squared and RMSE. And linear regression has the worst MAE (Table 1).

5.2 Configuration 2 Sequentially, the experiment continued by removing the columns with lower explanatory power: we called Configuration 2. We used the correlation between each attribute and the response variable in ranking form. The selection of attributes was also used by [6–8, 18] even though these justified the deletion of the columns due to the nature of the attributes. We applied a test to remove different percentiles of these columns in training set to establish an optimal value for the attribute extraction method. We analyzed between the 10th and 90th percentile, varying in every ten values of percentile. The range from the 60th to the 70th percentile demonstrated lower MAE in the performance of all algorithms based on the attribute extraction method. Thus, we adopted the 60th percentile for Configuration 2. Table 2 demonstrates the results of algorithms with Configuration 2. As can be seen, there was virtually no significant performance increase. Overall, all algorithms have some performance loss on all metrics, except KNN, which achieved a slight reduction in predictive error.

Educational Data Mining ... Table 3 Configuration Performance 3 Algorithm Linear Regression Random Forest KNN Neural Network

Table 4 Configuration Performance 4 Algorithm Linear Regression Random Forest KNN Neural Network

139

RMSE

MAE

R2

140.26 132.87 140.24 135.46

109.18 103.59 108.94 105.75

0.2236 0.3032 0.2238 0.2758

RMSE

MAE

R2

140.27 133.16 139.68 135.48

109.18 103.84 108.58 105.88

0.2235 0.3002 0.2300 0.2755

5.3 Configuration 3 In Configuration 3, we only the transformed target variable based on the ENEM Microdata [13]. As it turns out, the argumentative essay score contains objective criteria for assigning value 0 (annulled essay). Since those criteria prevent a grade from being attributed to the essay, we decided to remove the lines corresponding to those occasions. Thus, Configuration 3 only has students who have not had their annulled essays. The arithmetic mean of the argumentative essay score became 599.62, standard deviation 159.27, kurtosis -0.19, and skewness of 0.07. In addition, the scores started to have an amplitude of 40 to 1000 points. Table 3 shows the results Linear Regression. LR performed worse among all but almost imperceptibly after KNN, the most important difference being the MAE of RL 109.18 versus 108.94 of KNN. The best performances were from Random Forest and the Neural Network. Random Forest was the best of all metrics with the explanatory power of 30.32% versus 27.58% of the Neural Network.

5.4 Configuration 4 Configuration 4 was implemented, adding the Configuration 2 and Configuration 3 procedures. Configuration 4 represents both procedures for the attribute selection and target variable. Table 4 also shows no increase in the performance of algorithms except for KNN, which, as noted in Configuration 2, benefited from the attribute selection.

140

R. B. Lopes et al.

Fig. 1 Comparison of MAE between Configurations.

6 Discussion The analysis of the following chart helps us to understand the comparison between configurations. Figure 1 show MAE evolution according to each configuration, respectively. We can observe Configuration 2 has virtually no difference from Configuration 1. This suggest that eliminating columns with little explanatory potential cannot improve the performance of algorithms Configuration 3 is the only one that provided significant improvement to reduce predictive error. As can be seen, the elimination of students who had their essay grades zero reduced the skewness of the distribution of grades (before -0.77, then 0.07). As pointed out by Adjei et al. in [1], this improvement can be noted by the difference between RMSE and the MAE. We can conclude a greater explanatory power over the grades in Configuration 3. This same gain cannot be observed when implementing Configuration 2 or Configuration 4, compared to Configuration 1 and Configuration 3, respectively. It was also possible to identify that RF and MLP perform at all times better than RL and KNN. The present work suggests that in the educational database used and under the regression task, the RF and MLP algorithms perform better than a statistical model such as RL. The RF and MLP algorithms could have been benefited from the size of the database beyond the linear capacity of a statistical model could do [8]. Gomes et al. [9] used the 2011 ENEM database to also predict the essay score. According to the authors, the mean score was 545.21, a standard deviation of 146.45,

Educational Data Mining ...

141

Table 5 Comparison between works, algorithms and response variable Work

Database Year

Response Variable

Algorithm

RMSE

MAE

R2

Configuration 3

2019

Essay

Linear Regression

140.26

109.18

0.2236

Random Forest

132.87

103.59

0.3032

KNN

140.24

108.94

0.2238

Neural Network

135.46

105.75

0.2758

[9]

2011

Essay

CTREE

0.0331

[9]

2011

Essay

CART

0.1657

[7]

2011

Natural Science and its Technologies

CART

0.3252

[6]

2011

Mathematics and its Technologies

CART

[21]

2014

Mathematics and its Technologies

Gradient Boosting

65.90

0.3500

AdaBoost

72.66

0.1800

0.3842

and amplitude ranging from 40 to 1000 points in the training set. This distribution is closer to that obtained in Configuration 3, where zero values were eliminated. The skewness of the response variable measured by the authors in the training base was 0.09, close to 0.07 obtained in Configuration 3, and, far away from the distribution of Configuration 1 (-0.77), when zero values are not removed. Different response variables than the essay grade seems to be better explained by this database. Using the CART algorithm, Gomes & Jelihovschi in [8] predicted the arithmetic mean of the objective disciplines of the ENEM. The best R-square obtained by the authors was 0.4029 with the ENEM 2011 database. With this same database and algorithm, Gomes & Amantes in [7] predicted the Natural Science score and its Technologies using CART, obtaining the best R-square of 0.3252. Table 5 shows all comparisons. In Gomes et al [6], the authors predicted the grade of Mathematics and its Technologies of ENEM 2011 with the same algorithm, obtaining an R-square of 0.3842. Also, using the grade of mathematics in 2014 ENEM base, Stearns et al. [21] got the best R-square of 0.35 and MAE of 65.90 with Gradient Boosting. The best R-squared and MAE to predict the essay grade of the present study were 0.3032 and 103.59, respectively. Analyzing the Table 5 with respect to R-squared, none of the works on the ENEM could predict with more than 50% correctiveness. Although the essay grade forecast performed very poorly, As demonstrated by Chicco et al. in [25], positive R-squared are good predictions.

7 Conclusion This work carried out, through four different algorithms, the prediction of ENEM 2019 argumentative essay score based on the attributes of its microdata. Although the results were not so satisfactory, when compared to previous works with the same response variable and database, it performs better [7]. Others response variables in

142

R. B. Lopes et al.

the ENEM exam are apparently better explained by the set of attributes than the argumentative essay grade [6–8]. It was possible to conclude that Random Forest was the algorithm that performed best among all configurations, followed by Neural Network. Linear Regression and KNN performed almost identically. the selection of attributes was not effective in reducing the predictive error of any of the algorithms. The treatment of the response variable showed an important improvement in the prediction of the essay grades. The prediction of the essay grade might be an excellent tool for improving students performance, since these predictions could feed intelligent tutoring systems or even support teachers and education managers [22]. The difference in the forecast performance between other discipline shows us an importance to keep addressing this task, since it could be using for aiding support and intervention strategies in the education process [23] One limitation of the work was the computational power since some algorithms’ hyperparameters could improve results as pointed out by Tsiakmaki et al. [23] but depend on better computational resources. Another limitation pointed out by Gomes & Amantes [7], is that the database provided by the ENEM does not provide important variables already identified in other studies [2, 7]. As future work, the use of social attributes from other databases will be explored. Deep learning algorithms will be used in order to compare its performances to traditional ones.

References 1. Adjei S, Ostrow K, Erickson E, Heffernan N (2017) Clustering students in assistments: exploring system-and school-level traits to advance personalization. The 10th international conference on educational data mining, pp 340-341 2. Cornell-Farrow S, Garrard R (2020) Machine learning classifiers do not improve the prediction of academic risk: evidence from Australia. Commun Stat: Case Stud, Data Anal Appl 6:228– 246 3. Flach P (2012) Machine Learning: The Art and Science of Algorithms that Make Sense of Data. Cambridge University Press, Cambridge 4. Géron A (2019) Hands-on Machine Learning wth Scikit-learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media, Sebastopol 5. Golino H, Gomes C (2014) Four machine learning methods to predict academic achievement of college students: a comparison study. Revista E-Psi 4:68–101 6. Gomes C, Souza Fleith D, Maria C (2020) Predictors of students’ mathematics achievement in secondary education. Psicologia: Teoria E Pesquisa 36:e3638 7. Gomes C, Amantes A, Jelihovschi E (2020) Applying the regression tree method to predict students’ science achievement. Trends Psychol 109788 8. Gomes C, Jelihovschi E (2020) Presenting the regression tree method and its application in a large-scale educational dataset. Int J Res Method Educ 43:201–221 9. Gomes C, Lemos G, Jelihovschi E (2020) Comparing the predictive power of the CART and CTREE algorithms. Avaliação Psicológica 19:87–96 10. Hastie T, Tibshirani R, Friedman J (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media, Cham

Educational Data Mining ...

143

11. Kaliannan M, Chandran S (2012) Empowering students through outcome-based education (OBE). Res Educ 87:50–63 12. Kotsiantis S (2012) Use of machine learning techniques for educational proposes: a decision support system for forecasting students’ grades. Artif Intell Rev 37:331–344 13. Enem M (2019) Braslia. Disponivel Em: ¡Acesso Em, vol. 23, March 2019. http://portal.inep. gov.br/web/guest/microdados 14. Namoun A, Alshanqiti A (2021) Predicting student performance using data mining and learning analytics techniques: a systematic literature review. Appl Sci 11:237 15. Patil M, Hiremath B (2018) A systematic study of data wrangling. Int J Inf Technol Comput Sci (IJITCS) 1:32–39 16. Pedregosa F, Machine Learning in Python (2011) Others Scikit-learn: machine learning in python. J Mach Learn Res N 12:2825–2830 17. de Castro Rodrigues D, Dias de Lima M, da Conceição MD, de Siqueira VS, M. Barbosa R (2019) A data mining approach applied to the high school national examination: analysis of aspects of candidates to Brazilian universities. In: Moura Oliveira P, Novais P, Reis LP (eds) EPIA 2019, vol 11804. LNCS (LNAI). Springer, Cham, pp 3–14. https://doi.org/10.1007/9783-030-30241-2_1 18. Santos ATB, Paulino J, Silva MS, Rego L (2020) Educational data mining: a study on socioeconomic indicators in education in INEP database. In: Borah S, Emilia Balas V, Polkowski Z (eds) Advances in Data Science and Management, vol 37. LNDECT. Springer, Singapore, pp 51–65. https://doi.org/10.1007/978-981-15-0978-0_5 19. Santos R, Pitangui C, Vivas A, Assis L (2016) Análise de trabalhos sobre a aplicaçao de técnicas de mineraçao de dados educacionais na previsao de desempenho acadêmico. Anais Dos Workshops Do Congresso Brasileiro De Informática Na Educação, vol 5, p 960 20. Sorgatto D, Nogueira B, Cáceres E, Mongelli H (202) Predição de indicadores educacionais utilizando técnicas de aprendizado de máquina, July 2020 21. Stearns B, et al (2017) Scholar performance prediction using boosted regression trees techniques. In: ESANN 2017 proceedings 22. Hamal O, El Faddouli N, Harouni M, Lu J (2022) Artificial intelligent in education. Sustainability 14:2862 23. Tsiakmaki M, Kostopoulos G, Kotsiantis S, Ragos O (2019) Implementing AutoML in educational data mining for prediction tasks. Appl Sci 10:90 24. James G, Witten D, Hastie T, Tibshirani R (2013) An Introduction to Statistical Learning. Springer, Cham 25. Chicco D, Warrens M, Jurman G (2021) The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Comput Sci 7:e623

The Impact of Digital Game Narratives on Environmental Awareness: An Efficiency Assessment of Game Narratives Created by Young Adults Tanira M. H. Suandique, Pedro Beça, and Monica Aresta

Abstract Game narratives play a key role in games created to transmit knowledge and generate learning. The creation of digital game narratives by students demands not only imagination, but a set of rules and methods so that these artifacts meet the main requirements of digital games. The present work aimed to verify if digital game narratives created by students of the first year of higher education in the city of Pemba, Cabo Delgado province, northern Mozambique, contain indicators present in game narratives considered efficient in learning based on digital games. A total of 53 game narratives were created and collected in three distinct sections of learning workshops and subjected to content analysis. The existence of an interconnection, in the created narratives, between the rules, actions, characters and objectives of the games, showed that aspects of narrative design were present. Intrinsic fantasy was observed in most of the narratives illustrating that the gameplay aspects revolved around the environmental awareness content they were intended to illustrate. However, the results showed that the created game narratives created characteristics present in game narratives considered efficient in the field of learning based on digital games. Keywords Game Narratives · Digital Games-Based Learning · Environment Awareness · Narratives Design · Mozambique

T. M. H. Suandique (B) Aveiro University, Porto University, Lúrio University, Rua Do Breiner 401, Pemba, Mozambique e-mail: [email protected] P. Beça · M. Aresta Aveiro University, Campus Universitário de Santiago, Aveiro, Portugal © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_13

145

146

T. M. H. Suandique et al.

1 Introduction The results of serious game narrative creation activities in formal and informal environments are seen as positive for promoting knowledge acquisition and adding new behaviors in relation to different themes. The game’s narrative is defined as methods or styles used to tell the game’s story [1] and is seen as one of the key elements for the success of a serious game, be it educational, cognitive, and above all for promoting behavior change [2]. There is an extreme need for narratives to be as complete as possible to make the transmission of knowledge effective, as there is a difficulty in aligning the game narrative with the learning or educational objectives that are intended to be achieved [3]. Hence the growing need to create assessment tools for narrative creation processes [2, 3], illustrative studies of the main constituents of game narratives [4], and game narrative factors that influence immersion and great untapped potential in investigating how to create compelling narratives to support different types of learning [5]. However, there is a need to investigate the aspects mentioned above, in works of development or creation of narratives by those who do not have any knowledge in game narrative design, it proves to be a relevant activity for the validation of this type of practices within the scope of learning generation. Based on content analysis, the present study illustrates the efficiency assessment of digital game narratives created by young students of the first year of higher education in the city of Pemba. It should be noted that the narratives were created based on a material to support game narrative design (a toolkit to game design) [6, 7], and had a total focus on environmental awareness around marine plastic pollution.

2 Use of Narratives in Digital Game-Based Learning According to [8], narrative in digital game-based learning (DGBL) contexts can be defined as a series of events, tasks and feedback resulting from simulated real-world scenarios, which is used to frame the students’ investigation to learn in a systematic way through storytelling. Narrative has been shown to be quite effective within DGBL, making a huge contribution to the efficiency of learning generation. In this sense, DGBL can be understood as “…a competitive activity in which students set educational goals aimed at promoting the acquisition of knowledge, where games can be designed to promote learning or the development of cognitive skills, or even take the form of simulations that allow students to practice their skills in a virtual environment” [9, p. 156]. Within the field of game-based learning, narrative is often the story, setting and/ or structure that surrounds or is embedded in the learning environment [10]; when designed to be emergent, it can produce rich and engaging experiences, connecting to students’ previous experience for flexible thinking and distributed problem solving [11].

The Impact of Digital Game Narratives on Environmental Awareness …

147

In this sense, game narratives play a fundamental role in learning based on digital games, as this concept has its own definition attributed to this field of study. In addition to the conceptualization of the term game narratives within the DGBL, the goals are also clearly established as shown [12], one of the goals is to make the design of such resources personally meaningful so that students can use them to solve problems and face the challenges learning tasks, they are experiencing. Based on the above assumptions, it is possible to assume that the narrative has the means to provide an education of intentions and to alleviate the most rigorous pedagogical methods. However, it is recognized that, for this purpose, they must contain characteristics that provide efficiency.

2.1 Characteristics that Provide the Efficiency of Digital Game Narratives Game narratives significantly contribute to the emergence, motivation, engagement, and generation of knowledge in digital game-based learning [13]. However, these narratives must be surrounded by characteristics that consider them efficient. According to [14], the narrative can be integrated into the game as a broadly defined objective, or a localized incident or plot developed in the game. Whereas for [11] Game mechanics can also be understood as an activity composed of a set of actions. For [15] the construction of gameplay can be described in two layers: the “ludus” or game mechanics layer that involves rules and actions, and the narrative layer that comprises the scenario, plot and/or characters. and the author further adds that a game can convey a narrative and create an immersive narrative experience through “spatiality” or the game world, in which the story element is infused into a space through which the player navigates and interacts. Different studies [5, 10, 11, 13–15] illustrate characteristics or indicators present in narratives that make them more susceptible to generating knowledge, from this diversity, here we will focus on a crossing of some characteristics of game narratives focused on experience design or game mechanics (game rules and actions that players can practice in deferring these rules) and narrative architecture (intrinsic fantasy and characters), meeting and considering the assumptions of [10, 13, 15].

3 Method The present exploratory qualitative study was carried out in learning workshops attended by first-year students from two different universities in the city of Pemba, province of Cabo-Delgado, northern Mozambique. Adopting a convenience sampling technique, 53 students from two universities (one public and one private) participated

148

T. M. H. Suandique et al.

in this investigative work. 66% of the participants were male and the remaining 34% were female (n = 18), all aged between 18 to 25 years old.

3.1 Material To involve students in the process of creating game narratives for the purpose of environmental awareness, it was necessary to use support material. A material totally focused on the digital game design process, which illustrated the main elements that make up digital games (GCCS - Game Construction Cards Set) was used to support the process [16]. The material in question is composed of cards in physical format containing an explanation of each element that they contain in digital games and a narrative construction guide [7, 17]. After the use of the cards of elements of the game, the environmental content was then explored: marine pollution with plastics, a theme that was intended to make known to the participants of the study. Similar to the GCCS, the thematic content is also available in the form of physical cards, containing four categories, the first named “did you know” contains the definition of marine litter concepts, the second, “but after all” presents relevant facts around the garbage, the third “truth or lie” illustrates curiosities, popular facts around marine plastic pollution, the fourth and last category “join the good ones” presents a series of attitudes or behaviors that can be acquired in favor of reducing pollution. pollution of the seas. A third and final material was used in the creation of the narratives, a document to support the construction of the narrative that gives a possible orientation in the use of GCCS (RGDD - Rapid Game Design Document) (Fig. 1).

Fig. 1. Material used to create the game narratives.

The Impact of Digital Game Narratives on Environmental Awareness …

149

Fig. 2 The stages of the process of creating game narratives

3.2 Structuring of Activities The narrative creation activities took place in three stages: in the first, the students were introduced to the basic principles of narrative design, through the (GCCS Game Construction Cards Set). A relationship was made between various games known to them and the game elements presented in the game design content, to facilitate compression. After this exercise, the cards containing the environmental content were provided, and they were given time to understand the concepts, the facts, the curiosities surrounding marine plastic pollution, it should be noted that these first two activities were carried out in 90 min. After that, in the third stage, they were provided with the narrative creation guide (RGDD – Rapid Game Design Document). Using this document, students found it easy to compose their narratives. It should be noted that to reach the total number of participants, the activities of creating game narratives were carried out on three different days (Fig. 2).

3.3 Data Collection For the present study, data collection was based on collecting qualitative documents, for Creswell (Creswell 2010) “…during the research process, the researcher can collect qualitative documents. These can be public documents (eg newspapers, minutes of meetings, official reports) or private documents (eg personal diaries, letters, e-mails)” in order to carry out a qualitative analysis. A total of 53 digital game narratives were created by the study participants (first year higher education students from the city of Pemba, northern Mozambique), carried out in three different sections in October and November 2021 (Fig. 3).

150

T. M. H. Suandique et al.

Fig. 3 Narrative creation by study participants

3.4 Data Analysis The game narratives created within the scope of this research obeyed a digital game structure composed of twelve game elements: story, premise, characters, player, challenge, dramatic arc, objectives, resources, actions, conflict, rules, and outcome. This variety of gameplay elements illustrates that our participants could create games with an experience design leaning [11] and/or a narrative architecture [14]. The analysis of the game narratives developed by the study participants was based on content analysis [18], with an emphasis on efficiency characteristics in game narratives focused on experience design or game mechanics and narrative architecture., taking into account the assumptions of [10, 13, 19]. Specifically, we evaluated whether there is an integration between the rules and actions with the game world in which the story element is infused in a space through which the player navigates and interacts (intrinsic fantasy and characters).

4 Results After analyzing the game narratives developed by the study participants (n = 53), it was found that a small number (n = 11) would not add the other game narratives for the purposes of analysis. The exclusion criteria are divided into two: non-game narratives (n = 3), in this exclusion criterion two of the game participants only titled the games and presented a premise, the third and last that composes this group only titled their match. The second exclusion criterion was narratives without a focus on plastic pollution (n = 8), here the participants brought some common examples of games known to them (basketball, ludo, football, candy crush, mortal combat). The remaining game narratives (n = 42) were analyzed, and the results will be presented in this section according to the characteristics of efficiency in game

The Impact of Digital Game Narratives on Environmental Awareness …

151

narratives, starting with the characteristics focused on game mechanics and ending with those of narrative architecture.

4.1 Rules For [20] rules are definitions and instructions that players agree to accept during play. Every game has rules, even if those rules aren’t written down or taken for granted. while [8] adds to the rules the fact of determining win or lose criteria saying that “…the rules are designed or established to determine the conduct and the standard for both gaming behaviors and the win/loser state. Another approach that adds to the fact that all games have rules is presented by [11] where they say that rules constitute the internal and formal structure of games, so all games have them and are one of the defining qualities of games. The content analysis allowed us to divide the rules present in the developed narratives into two broad categories: those that clearly had them (n = 39) and those that did not, because they were confused with the game objectives (n = 3). Of the number in which the rules were clear, a subdivision was still observed, creating three subcategories of the rules: narratives with rules focused on environmental content (n = 29) for example: (“Don’t fish for fish” “You can only fish for garbage”), and or rules aimed at aspects of the system (n = 10) such as: (“You cannot play in a level without passing the previous one”).

4.2 Actions The rules specify what actions players can take to overcome challenges and achieve the game’s objective. Rules define not only what actions are permitted, but also which are prohibited, and which are required and under what circumstances [20]. Within the document that would help study participants to create game narratives (RGDD - Rapid Game Design Document), actions were explained to them as the factor that determine what the player can do to achieve the game’s objective [21]. The actions in the game narratives developed by the participants who were analyzed (n = 42), it can be said that most of them (n = 36) were clearly defined, that is, the actions had connections with the objectives. In the remaining ones (n = 6) the connection between the actions and the goals of the games was not observable. Another fact observed is that of the clearly observed actions there was a great inclination (n = 25) that were linked to the game contents (for example: “collect the largest amount of garbage”, “make the garbage categorization”). the rest (n = 11) were actions merely linked to movements (example: “the player can only move up and down”).

152

T. M. H. Suandique et al.

4.3 Characters Characters within narratives play a prominent role, for [13] strong characters can pull the student into the story through immersion. Characters with whom students interact have the potential to influence their attitudes and decisions. For [11] players not only manipulate characters they also feel like characters. A deeper look tells us that characters help frame and motivate interactions and actions with meaningful gameplay experiences [8]. The results of the characters are divided into two large groups: those that only pointed and named the players as characters (n = 10), and the rest that identified other characters besides the players (n = 32). From this larger group, a subdivision of the type of characters was made, having been created three categories: environmental characters (n = 8) such as: “sea lion”, “squid and shrimp”, “garbage”. The second category of characters with fun names (n = 4) such as: “plastic boss”, “the shark man”, etc., and the rest as common characters (n = 20).

4.4 Intrinsic Fantasy The existence of factors that drive motivation in software is seen as a lever in their use, it is believed that games with intrinsic learning integration are more educational and intrinsically motivating than those with extrinsic integration [15]. Intrinsic fantasies are internal to gameplay and there is a reciprocal relationship between gameplay and it, there is no separation between content (narrative) and gameplay [10]. Intrinsic fantasy is best suited for educational purposes as it intertwines pedagogical content with fantasy so that one cannot exist without the other. In these types of games, if a student is interested and motivated by fantasy, then he will be interested and motivated by learning content [13]. Establishing a clear relationship between the action, goals, rules, and characters, we can say that removing the games that did not have clear rules (n = 3), crossing with those that did not show the relationship between actions and goals (n = 6), it is observed that the remaining ones (n = 36), had a connection between the elements described above. Gameplay (e.g., “you lose if you don’t catch the plastic in 20 s”) in game narratives and fantasy observed a link.

5 Discussion The present work aimed to verify indicators of efficiency of game narratives, in narratives developed by students of the first year of higher education in the city of Pemba, northern Mozambique. It should be noted that for the creation of narratives, the students had a material to support the creation or design of narratives (Toolkit

The Impact of Digital Game Narratives on Environmental Awareness …

153

to game design) [7, 22]. Our analysis was based on efficiency characteristics in game narratives focused on experience design or game mechanics and narrative architecture, having carried out a content analysis in three aspects in the rules, actions, characters, and verification of the type of fantasy (exclusively to intrinsic). The existence of game rules in the narratives created by the participants of the study, especially the rules merely aimed at the objectives of the game and the content of environmental awareness (n = 39) shows that, above all, the narratives hold one of the main basic elements of design, as we are told [11], “…if you can’t identify the ground rules of a digital game you hope to create, you are out of touch with your own design.“ The existence of actions that differ from the rules and objectives in the narratives created (n = 36) is an efficient indicator that the actions were understood and that they will serve as facts as a limit to the number of ways in which the player can face the challenge [20]. Not all games need characters, but characters give life and warmth to a game and are essential to any game that includes a story [20], and as most constructed narratives (n = 36) had a story, the existence of characters illustrates that the study participants adhered to design principles, and this fact is highlighted by the existence of a variety of types of characters, from merely environmental content to fun ones.. The existence of intrinsic fantasy in game narratives is one of the factors that provides immersion, motivation, engagement and above all learning [13, 23], in the present study we can see that the constructed narratives had intricate fantasy. Because in most of the narratives (n = 36) the gameplay was totally focused on the environmental aspects. We can say that 69% of the evaluated narratives have an expected level of connection between the evaluated elements (existence of clear rules and actions focused mainly on environmental content, from which the objectives of the games can be perceived), 22% showed a moderate connection (the existence of clear but systemoriented rules and a mixture of actions aimed at environmental and movement content), 2% obeyed a weak link (existence of clear system-oriented rules and merely movement actions) and the remaining 7% do not illustrate a link between the four indicators evaluated (rules and not clear actions, as these were confused with the objectives of the game). However, the present study serves to illustrate that game narratives when created by those who do not have specific knowledge in design, with the help of appropriate narrative design material, can add indicators or characteristics that make narratives efficient.

6 Conclusion The use of narrative in learning based on digital games has gained more and more space, as the involvement of factors such as intrinsic fantasy contribute to the content that is intended to be known is entirely linked with the gameplay in games.

154

T. M. H. Suandique et al.

In the context in which the activities of narrative creation of digital games were carried out, the method is almost unexplored, a fact that makes it a medium that still lacks disclosure for future explorations to take advantage of all the benefits that this type of learning provides. Future studies intend to spread the method of creating digital game narratives to other audiences, with emphasis on primary and secondary education, and also to cross other distinct themes.

References 1. Qin H, Patrick Rau PL, Salvendy G (2009) Measuring player immersion in the computer game narrative. Int J Hum Comput Interact 25(2):107–133 2. Ardakan MA, Vahed Z (2019) A conceptual framework for cognitive game design analysis (CGDA). In: Proceedings of 2019 international serious games symposium, ISGS 2019, pp 81–88 3. Silveira IF (2019) Building effective narratives for educational games. In: Proceedings - 14th Latin American conference on learning technologies, LACLO 2019, no 978, pp 299–305 4. Khan A , Webster J (2018) Digital game narrative quality: developing a measure. In: International conference on information systems: transforming society with digital innovation, ICIS 2017 5. Dickey MD (2006) Game design narrative for learning: appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educ Technol Res Dev 54(3):245–263 6. Beça P, Aresta M, Santos R, Veloso AI, Pereira M (2019) Supporting the game construction process: development of artefacts in the context of a toolkit to game design, pp 99–110 7. Suandique TMH, Beça P, Aresta M (2021) Uma ferramenta de apoio a criação de jogos digitais para promoção da consciencialização ambiental A tool to support the creation of digital games to promote environmental awareness, no June, pp 23–26 8. Dai CP, Ke F, Pan Y (2022) Narrative-supported math problem solving in digital game-based learning. Educ Technol Res Dev (0123456789) 9. Erhel S, Jamet E (2013) Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness. Comput Educ 67:156–167 10. Dickey MD (2020) Narrative in game-based learning. In: Handbook of game-based learning. The MIT Press, London, pp 283–306 11. Salen K, Zimmerman E (2004) Rules of Play - Game Design Fundamentals. London 12. Dai CP, Ke F, Pan Y, Dai Z (2020) Work-in-progress-learners’ interaction with task narratives for math problem-solving in game-based learning. In: Proceedings of 6th international conference on immersive learning research network, iLRN 2020, pp 299–301 13. Naul E, Liu M (2020) Why story matters: a review of narrative in serious games. J Educ Comput Res 58(3):687–707 14. Jenkins H (2002) Design Game as Narrative Architecture. MIT Press, Cambridge; London 15. Ke F (2016) Designing and integrating purposeful learning in game play: a systematic review. Educ Technol Res Dev 64(2):219–244 16. Beça P et al (2021) G4N toolkit to game design: creating games for environmental awareness. Eser 17. Suandique TMH, Beça P, Aresta M (2022) Criação de Jogos digitais como meio de consciencialização ambiental: exploração de uma ferramenta de apoio a criação de jogos digitais por jovens adultos. Rev. Ibérica Sist. e Tecnol. Informação E47:226–238. ISSN 1646-9895 18. Bardin L (2011) Análise do Conteúdo - Edição revista e ampliada. Sao Paulo

The Impact of Digital Game Narratives on Environmental Awareness …

155

19. Ke F (2014) An implementation of design-based learning through creating educational computer games: a case study on mathematics learning during design and computing. Comput Educ 73:26–39 20. Adams E (2006) Fundamentals of game design, 2nd edn 21. Beça P, Aresta M, Orlet C, Santos R, Veloso AI, Ribeiro S (2020) Promoting student engagement in the design of digital games: the creation of games using a toolkit to game design. In: 2020 IEEE 20th international conference on advanced learning technologies, pp 98–102 22. Beça P, Aresta M, Santos R, Veloso AI, Gomes G, Pereira M (2019) Supporting the game construction process: development of artefacts in the context of a toolkit to game design. In: Videogame sciences and arts, Aveiro. Springer, Cham, pp 99–110 23. Habgood J, Ainsworth S (2011) Motivating children to learn effectively: exploring the value of intrinsic integration in educational games, no April

Hybrid PBL and Learnability for Computer Science and Multimedia Degrees: How to Trigger Soft Skills Strategies for Personal and Professional Development? Dulce Mourato

and Paula Amaro

Abstract How do transform students and their misperceptions about the application of theoretical subjects to day life and professional solutions? This question triggered a qualitative action research partnership between two professors, of two curricular units (Introduction to Physics and History of Science and Techniques), taught in Computer Engineering and Multimedia degrees at two Portuguese Higher Education institutions: Polytechnic Institute of Guarda (IPG) and Higher Advanced Technologies Institute (ISTEC) from Lisbon. Based on a PBL strategy, applied a descriptive and qualitative action research methodology, within the scope of participant observation and an explanatory scope, four concrete curricular approaches were selected and improved in order to obtain the most creative and revealing answers possible, which changed the knowledge interpretation actually obtained. The PBL perspectives were framed in the inductive selection, in one of the methodological paths and in the deductive option, in the other teaching proposal, which resulted in a general framework that could be adopted in probable new contexts. The action research carried out and the follow-up developed by both teachers, on group work performances, proposed inverse directions in innovative teaching– learning situations, from the particular to the general and from the general to the particular, complementing a learning set directly proportional to the activities developed. The results discussion culminated in the construction of an adaptable, innovative and developing hypothesis framework, that provides concepts such as Hybrid PBL, Learnability and Transdisciplinary that could potentiate new horizons for knowledge in the field of Sciences, Techniques and Technologies.

D. Mourato (B) Higher Advanced Technological Institute (ISTEC), Lisbon, Portugal e-mail: [email protected] P. Amaro (B) Polytechnic Institute of Guarda (IPG), Guarda, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_14

157

158

D. Mourato and P. Amaro

Keywords Hybrid PBL · STEM · Learnability · Education · Personal Development

1 Introduction Applying Curricula units and theoretical subjects to students’ daily life, and providing challenging professional solutions, in order to transform students’ misperceptions and stimulate creatively new ideas were the main questions that triggered an action research - a partnership between two professors, of two Curricular Units - Introduction to Physics (IP) and History of Science and Techniques (HCT), taught in Computer Engineering and Multimedia degrees at two Portuguese Higher Education institutions: Polytechnic of Guarda (IPG) and Higher Advanced Technologies Institute (ISTEC) from Lisbon. The main goals of this action research were to conceive a hypothesis framework through project ideas, prepared as a supplementary assessment on one hand and a free challenge focused on curricular unit programs on the other. Unfolding and effective application of concepts such as Hybrid Problem/Project Based Learning (PBL), Learnability and transdisciplinary in social, professional and soft skills development were the well-driven aims. The most difficult thing to notice in this study was the methodological approach, because, in qualitative terms, it did not fully satisfy the research requirements, shared in the common case study options, for example: How to conduct an original, dynamic research option, capable of being structured as an interactive and dynamic research instrument, with the possibility of being adapted to different teaching and learning scenarios? With that process in mind, the first step was to understand at which points our curricular unit subjects converge in IP and HCT and selected four of them: • Using the sample of Archimedes’ principle of the buoyant force and the screw invention (still used today as a method of irrigation) to explain how it affects several devices used in our daily life since then. Inspired by the famous phrase of Archimedes “Give me a lever and a place to stand, and I’ll move the world” the IP students could appreciate world examples of three types of levers, force concepts and torque. • Galileo Galilei and his spectacular engineering ideas: refracting telescope, optics studies transformation, the free fall studies and Astronomy discoveries. • Isaac Newton - three Laws of Motion – Inertia, Acceleration, Action and Reaction, the refracting telescope, the fluxions or the method of series and fluxions, and calculus provided methods for solving complex problems about orbits, curves and other issues. • Nikola Tesla and his inventions: alternating current (AC) motor, remote control, neon and fluorescent lights, wireless transmission, laser beams, x-rays, and robotics.

Hybrid PBL and Learnability for Computer Science and Multimedia …

159

The others specifics objectives were to present concepts, and contents and develop original practical ideas, complete additional assessments for Curricular Unit principal evaluation: create a laboratory of ideas, and explore students’ perceptions based on and perceive the amplitude of potential knowledge. Every part of this major puzzle (original hypothesis framework) as well as detailed the actuality of potential teaching–learning strategies such as Hybrid PBL and Learnability and how it is possible to create a transdisciplinary experience able to structure itself as an innovative method, promoting student interests and foster new skills and ways of thinking. These issues will be explained in the methods and methodology segment and the following sections.

2 Literature Review Throughout this literature review, it was possible to clarify what is the subject and the concrete application of Hybrid PBL/Learnability, in order to understand how the triggers and the challenges launched, demonstrating four scientists, their inventions and studies could change the way students elaborated their group researches, and how these concepts, processes and problems were presented in real life or virtual works, to promote new ideas and sediment future knowledge. In the first phase, the main points of Problem-Based Learning and stages of Project Based Learning description were identified to explain how they can be combined in the form of Hybrid Problem/Project Based Learning plus Learnability. It is very important for teachers and educators, in general, to understand the potentiality of Project Based Learning, as well as the weblog PBLWorks.com, created by Buck Institute for Education [1] that refers to and points to the several pedagogics features within which students gain knowledge, and associate skills by operating research to “answer an authentic, engaging, and complicated question, problem, or challenge”. In the article ‘A review of Problem-Based Learning applied to Engineering’ Rodríguez González and Fernández Batanero [2, p. 14] explain in detail that ProblemBased Learning is a methodological tool for teaching conduct and facilitating learners to solve some problems with several degrees of complexity (usually based on real STEM or life cases) using any resources, technologies or devices they think may be of use. It seems these authors use this kind of strategy as a subset or first step of Project Based Learning. Rodríguez González and Fernández Batanero [2, p. 17] refer also that teachers must be facilitators that explain general rules to the knowledge process “so far as possible, problems for the engineering disciplines should derive from real life or be a very close representation of it, involving the kind of applications which the students are likely to meet in the context of their future professional lives”. Like others, Markham [3, p. 47] specifies that Project Based Learning inspires new skills of future inquiry, and education features like collaboration, communication, connectivity and creativity must be designed to expand the curriculum to encompass

160

D. Mourato and P. Amaro

authentic issues and topics relevant to increase and develop each one performance in a global world. There are some basic features that combine, legitimate Problems with Projectbased learning and Learnability as described in DiBenedetto and Meyers [4] that included a conceptual framework for the knowledge, skills, and dispositions required identified as learning skills, life skills, career skills, social skills, knowledge competencies, incidental learning skills, dispositions, experiences, and interdisciplinary topics. The concepts of both methodologies Problem-Based Learning or Project Based Learning, in various types of research, appear indistinctly, without any a priori reflection or reference, however, in this study the two concepts complement each other in a hybrid way with Learnability [5] that is why in this case all PBL references were described as Project-based Learning. Mourato [5] established the approach between Hybrid PBL and Learnability using the ability to explore and ask questions about the subjects until one solution arrived. Another key Learnability concept used was interaction: “defined at first by the pioneers of technological accessibility in the creation of more intuitive and simple user interfaces, such as the level of ease, through which each user gains proficiency, whenever dealing with a new application or unknown software. More than learning by doing, users ‘gain’ the confidence to take risks without fear of making mistakes, in similar situations”. The option of qualitative research seems to be better adapted to this study, because according to DiBenedetto and Meyers [4, p. 3]the meaning is “socially constructed by individuals in interaction with their world” and uses a descriptive research method to better answer the research question as Johnson and Onwuegbuzie [6] elucidated. The grounded theory suggested by Charmaz [7] could be an appropriate method to choose when little is known about an original and particular topic, and it is important to have a fresh overview of perspectives or construct an original analysis of the data, as well as provide researchers with concrete tools, guidelines, and strategies to complete a successful qualitative research project and generate an explanatory theory that elucidates how or why a particular phenomenon is occurring, and predicts future occurrences on a topic of interest to the researcher [7–9]. However, only the action research method seems accurate for improving practice and solving PBL, because it contains action, evaluation, and critical reflection and – based on the evidence gathered – changes in practice are then implemented, according to Koshy [10]. Koshy [10, p. 9] defines action research as “a constructive inquiry in which the researcher constructs his/her knowledge on specific issues through planning, action, evaluation, refining and learning from experience”. Educational action research fits perfectly the main goal of this study, with limited existing data in the literature, the most appropriate framing of the study was through a qualitative research approach, in particular, due to the improvement of the quality of activities or teaching practice and the development and testing of the practical theories that guide one’s own practice, detailed by Feldman, Altrichter, Posch & Somekh [11].

Hybrid PBL and Learnability for Computer Science and Multimedia …

161

3 Methodological Environment After clarifying the methodological process followed, the teachers had to establish some rules a priori, namely, the creation of a schedule (Table 1) urgent planning to organize both curricular units (IP and HCT) additional assessments within the semester and the development of be carried out during classes, including offline and online tutorials. In one of the curricular units (HCT), with the work theme choice, a group selfassessment form (Table 2) was presented by the teacher to takes into account the development of the picked activities. The methodological particular variance, such as the explicit application of practical activities concrete assessment in a single curricular unit (Table 2), was purposeful and intended to assess the differences between eminently theoretical teaching and learning (HCT) and a theoretical-practical strategy with laboratory and concrete examples from Physical demonstration (detailed later in the presentation of the results). The evaluation rules of both curricular units were discussed in a pedagogical contract. In HCT 160 students enrolled at the beginning of the semester, and it was very explicit that special assessment group work was worth 40 percent in total if the students obtained a value greater than seven values (standard twenty points scale) in the unit final exam grade. In the case of the Introduction to Physics unit, the evaluation of 103 students was carried out as follows: the methods and teaching techniques to apply during the sessions are lectures, problem-solving, simulation and laboratory group work. Two types of evaluation took place: continuous and final assessment. Assessment just in time: nine small quizzes-tests (questions taken in classes - 50 percent), that gave a partial grade and group work for increment average (group work - 20 percent) and final exam (30 percent). Final assessment: written test (100 percent) or written test (80 percent) and group work (20 percent). Table 1 Chronogram of the Application Process and Research [12] Dates (2022)

Research Milestones

Goals to be Achieved

Mars–April

Identification and picking attractive curricular points for further assessment, in the first classes

Students selected groups and a range of approaches to the themes they carried out (participatory and democratic features)

May–June

Tutorial monitoring of the work developed by the students, inside and outside of the classroom. Samples to improve creativity using virtual labs were shown

Research and treatment of the themes with software and programs for the development of the mentioned subjects. For example, the creation of 3D printing of Archimedes’ endless screw

July

Implementation and delivery of work as a complement to the main assessment

Acquisition of knowledge and new perspectives on the learned concepts. Immediate contribution to social science (knowledge), technological and social change (practice)

162

D. Mourato and P. Amaro

Table 2 Assessment Criteria to be Taken into Account in the HCT Work [12] Preparation and Participation Evaluation (Scores 1–5)

Does it show evidence of prior knowledge or research beyond the notes, before carrying out the HCT work? Demonstrated detailed information about the problem settled and developed Did it accomplish the objectives established when the work was planned?

Contents knowledge Evaluation (Scores 1–5)

Did the students understand the material and develop related knowledge without problems or need additional explanations? Science and technologies were communicated effectively Did the students demonstrate knowledge of the contents and the use of appropriate language in a scientific way? The students placed the content in such a way that it can be challenged or receive feedback. Did the students participate in discussions by sharing their own opinions by adding content?

Self-knowledge as part of the group Evaluation (Scores 1–5)

Clearly defined learning objectives Demonstrated evidence of achievement of learning objectives Demonstrated evidence thorough reading of various resource materials Provided evidence that students understood the subjects that were required

Collaborative work and shared assistance Evaluation (Scores 1–5)

Worked to achieve individual and group learning goals Demonstrated effective interpersonal skills while performing work development tasks Did the students value other opinions (elements of the group or other exterior perspectives) about the effort developed and integrated them into collaborative work? Did the students work collaboratively? Their tasks in the development of the work, helped peers to progress in their learning?

Total Group Assessment

Evaluation (1–100)

Individual comments take into account the participation in the group and the learning knowledge acquired in the HCT curricular unit

Group member names Evaluation (Scores 1–20)

4 Findings The research question and global objective: How to trigger soft skills strategies for personal and professional development with Hybrid PBL and Learnability for HCT and IP students? - It provided some satisfactory answers although limited to the small

Hybrid PBL and Learnability for Computer Science and Multimedia …

163

size of the study. The answers reflect the good assessments’ average results and/or final evaluation grade improvement were proven. Highlighting students’ enthusiasm when discovering and researching STEM topics far away from the usual theoretical curriculum, until the application of this action research method. The other concern and first step were to determine specific objectives as concepts inherent to HCT presentation and the development of Physics activities, in a simplified way, using art, literature, and technologies to demonstrate the applicability of perceptions and scientific or technological laws to everyday moments. Bringing local traditions closer to the students, adapting the usual activities of Guarda region to the conceptual principles of Physics, that are inherent to their innovation pursuit contents and contribute to Physics literacy, for example, using the Rooster judgement, typical Carnival activity or the little Lamb Christmas recipes from Guarda (that leads to the creation of gastronomic specialities’) and Physics equation formulation. The idea is to give a chance to the poor animals that try to run away and escape from the cook, teaching them the ideal escape routes, including pendulum swing movements and optics effects, that could be compiled in a multimedia documentary, actual and realistic information or technological, digital and cultural artifacts. In order to show how the Hybrid PBL Learnability methodology could be easily applied in other contexts, was elaborated a mental map (Fig. 1) to refine the group work and include the subjects related to the four choices of authors introduced: Archimedes, Galileo Galilei, Isaac Newton, and Nikola Tesla. Using various multimedia documents, collaborative participation in forums, simulations, and personal interviews about their responsibility to present and future technologies, students understand that they have to go further, and research deeper to reach their goals. In the activities of demonstration in the IP curricular unit, various elements from the classroom were used to describe physical situations, such as projectile movements, so that students could transfer their knowledge to potential computer programs, consequently simulating the concepts involved. It was also possible to indicate a mathematical deduction method or descriptive equations to underlie the referenced physical situations. Also significant, was the use of simulation programs, such as ‘Physlet Physics’ and some laboratory devices, which serve to determine the value of gravity acceleration, projectile movement (range as a function of angle and initial velocity), levers, and induction electromagnetic radiation, in addition to other concepts covered. In HCT it was possible to divide the four teaching hours per week into a total of 60 h (two theoretical hours and two practical/laboratory hours) + 20 h of tutoring (offline and online). In all the practical/laboratory classes and in the requests for monitoring the group work, the students developed their activities and, in general, reviewed the particularities of the subjects taught, which they would not otherwise be able to explore so deeply. Following the four authors’ samples, some possible goals of group work emerged since the first class for HCT and IP: • Reproduce in whole or in a part particular event or specific technique of STEM – past, present, and future implications;

164

D. Mourato and P. Amaro

Fig. 1 Methodological Approach Hybrid PBL [12]

• An innovation or technology, placing it in time and demonstrating its progress; • Antecedents of forms of technology (for example Abacus, the first calculation instrument that allowed the evolution of other forms of processing, that preceded the current computer); • Chronology and the consequent historical and technological explanation (example: how the study of artificial intelligence began and its evolution to the present day, robot design and machine learning influence). • Critical appraisal of the origin of a technique and its repercussions on other devices or functionalities. All hypotheses of group work were allowed as long as fulfilling the objectives of the activities with common sense and good taste. Some of the group work developed in HCT classes are listed below: • An Android app that identifies fire sources in a house kitchen using Nikola Tesla’s electric principles;

Hybrid PBL and Learnability for Computer Science and Multimedia …

165

• Quiz-style game with augmented reality; • A 3D modelling application that aims to present the differences between geometric solids from ancient Greece to modern times, and an Archimedes 3D prototype of the never-ending screw. • A website about technology evolution, that aims to enumerate and present the origin of the first computers and original devices associated with them and their functionalities until today. • A guided tour in short film format (up to 15 min) on some curious aspects of technology using samples of old movies and multimedia effects; • An application whose purpose is to demonstrate scientific knowledge, through calculations or any other form of interaction with the user like the first Alan Turing Artificial Intelligence interaction. • An interactive timeline of the evolution of science and techniques from Hydraulic Civilizations to the present day. • Creation of a comic strip that clearly explains an evolution or a technology. The assessment of the group work depends if students achieving the set goals and using their creativity and imagination to reflect the knowledge and other values obtained. These points were very well described in Table 2 (Assessment Criteria to be taken into account in the development of the HCT work). It was possible to verify the potential versatility of the chosen Hybrid PBL/ Learning Methodology, which improved, in a very visible way, the positive evaluation, both in terms of final approval of the curricular units’ landscape and in terms of final values exponential increase obtained.

5 Final Notes This article was a risk and a pleasant surprise due to the results, somewhat unexpected in the HCT curricular unit (87 percent of students approved). The idea was to prove that this methodological framework was easily adapted in a transdisciplinary way to any curricular unit and that it can be reproduced, according to the main objectives listed. The adopted methodology contributes to greater literacy on Introduction to Physics and History of Science and Techniques, as well as STEM topics and technologies in general. Students improve their social, collaborative interactions, through Hybrid PBL, at an autonomous and combined construction of knowledge, privileging soft skills, multiculturalism, entrepreneurship, and creativity.

References 1. BIF Education, “What is Project Based Learning (PBL)?”. http://www.bie.org/about/what_pbl. Accessed 2 Sept 2022

166

D. Mourato and P. Amaro

2. Rodríguez González CA, Fernández Batanero JM (2016) A review of problem-based learning applied to engineering. EduRe J: Int J Adv Educ Res 3(1):14–31 3. Markham T (2012) Project-Based Learning: Design and Coaching Guide. HeartIQ Press, San Rafael 4. DiBenedetto CA, Myers BE (2016) A conceptual model for the study of student readiness in the 21st century. https://www.nactateachers.org/attachments/artcle/2390/9%20DiBedetto_ NACTA%20Journal%20Special%20May%202016.pdf. Accessed 2 Sep 2022 5. Mourato D (2022) Aprendizagem por Problemas, Identidade de Género e Learnability em projetos finais nas Licenciaturas em Engenharia Multimédia e Informática, ICITS’22 - RISTI, p 674, ISSN: 1646–9895, no E45. http://www.risti.xyz/issues/ristie45.pdf. Accessed 23 Dec 2022 6. Johnson RB, Onwuegbuzie AJ (2004) Mixed methods research: a research paradigm whose time has come. Educ Res 33(7):14–26 7. Charmaz K (2014) Constructing Grounded Theory, 2nd edn. SAGE, Newcastle upon Tyne 8. Birks MJ (2015) Grounded Theory: A Practical Guide, 2nd edn. SAGE, Newcastle upon Tyne 9. Merriam SB (2002) Qualitative Research in Practice: Examples for Discussion and Analysis, 1st edn. John Wiley & Sons, Hoboken 10. Koshy V (2010) Action Research for Improving Educational Practice: A Step-By-Step Guide. SAGE, Los Angeles 11. Altricher H, Feldman A, Posch P, Somekh B (2018) Teachers Investigate Their Work an Introduction to Action Research across the Professions, 3rd edn. Routledge, Oxfordshire, p 368 12. Mourato D, Amaro P (2022) Classroom and tutorials observations. Res J (No Press), 15

Mobile Learning and English Reading Abilities in Postgraduate Students Kevin Mario Laura-De La Cruz , Lesly Joselin Apayco-Zavala , Miliam Quispe-Vargas , Silvia Milagritos Bazan-Velasquez , Cecilia Claudia Montesinos-Valencia , and Bertha Silva Narvaste

Abstract English as a foreign language has gained popularity as a means of sustaining education by providing students with the option of learning English for specific objectives. The purpose of this study is to measure the level of English reading skills among doctoral students at a Postgraduate University in Peru, using M-learning. A pre-experimental examination led to the development of a hypothetical deductive research procedure. The process for collecting data involved administering a pre- and post-test. The findings of the posttest indicated that 60% of the students attained the level of remarkable accomplishment, 40% attained the level of expected achievement, and 0% were at the process and beginning level. From the findings collected, it can be concluded that Education Doctorate students were able to improve their English reading skills satisfactorily with M-learning application. Keywords Literal · Inferential · Critic · Postgraduate · Gamification

K. M. Laura-De La Cruz (B) Escuela de Posgrado Newman, Av. Bolognesi 987, Tacna, Peru e-mail: [email protected] L. J. Apayco-Zavala Universidad Peruana Cayetano Heredia, Av. Honorio Delgado 340, Lima, Peru e-mail: [email protected] M. Quispe-Vargas · S. M. Bazan-Velasquez Universidad Nacional Jorge Basadre Grohmann, Av. Miraflores S/N, Tacna, Peru e-mail: [email protected] S. M. Bazan-Velasquez e-mail: [email protected] C. C. Montesinos-Valencia Universidad Privada de Tacna, Av. Jorge Basadre Grohmann S/N, Pocollay, Peru e-mail: [email protected] B. S. Narvaste Universidad Casar Vallejo, Av. Del Parque 640, San Juan de Lurigancho, Peru e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_15

167

168

K. M. Laura-De La Cruz et al.

1 Introduction The current state of education has forced educational institutions at all levels to integrate the use of mobile services into the teaching and learning process [1]. Mobile learning, or M-learning, is a technological initiative that has transformed the learning process [2]. It consists of the use of mobile devices for learning; therefore, it requires some portable devices and a wireless network that allows users to access information [3] it does not require the student to be in a fixed location. Among the benefits it offers are portability, interactivity, contextual sensitivity, connectivity, and individuality [2, 4], but the success of its use for learning depends on teacher planning [5]. Reading in English is one of the most essential abilities for students to acquire [6], particularly for academic success [7, 8]. This ability is a cognitive relationship between the student and the written material [9]. It requires the student to evaluate the text and determine its meaning [6, 10], based on reflecting, questioning, analyzing, decoding, and connecting what has been read to prior knowledge [11]. Skimming and scanning are important ways to figure out how to read a book or what parts to read, especially in a foreign language, even though the skills needed to develop them vary from author to author [9]. The English language has become one of the most important requirements at the international level [12], as well as a necessity in academic and social settings. Advanced knowledge of English determines the economic and educational chances of a significant portion of the global population [13], but it is the capacity to read in English that provides access to a vast amount of information. Regarding this, Gamboa [14] suggests that this ability is one of the most complex and difficult for students when confronted with texts in English due to their lack of linguistic comprehension, and Cain et al. [8] add, lack of decoding comprehension. Similarly, De La Cruz et al. [15] emphasize the significance of learning English through gamification and the highly substantial improvement in learning, intensified teacher preparation, and identification of the needs of the student population in charge [16]. This fact about English reading skills highlights a problem that many countries, including Peru, face. According to the National Education Council, Peruvians read an average of 0.86 volumes every year, or less than one book in our native language [17]. This is a troubling reality that exacerbates the issue in a second language [18]. According to Matos [19], the lack of acceptable literary texts in English makes it difficult for pupils to increase and strengthen their reading comprehension. Similarly, Laura & Velarde [20] discuss the usefulness of using digital settings to investigate English reading comprehension in students of regular basic and higher education. Even though the significance of the ability to read in English is plain, there is an urgent need for additional research aimed at developing and enhancing it. In today’s worldwide society, there is a demand for professionals who can communicate effectively in English, which requires the ability to comprehend what is said and read. Therefore, studying and mastering the English language is crucial for the development of competent professionals and, consequently, the growth of society. If all professionals knew this, they would be more competitive and be able to aim for

Mobile Learning and English Reading Abilities in Postgraduate Students

169

higher positions, better living conditions, and better academic and job prospects if they wanted to go to graduate school overseas. M-learning is one of the methodologies and strategies professors could have used to build and improve students’ ability to read in English. According to Pisanty et al. [21], m-learning is an optional, cutting-edge method that strives to expand the scope of student learning. University More postgraduate students use mobile devices, which gives them access to many strategies and methods that will help them improve their English language skills.

2 Methodology The research is applied since it seeks to resolve a practical issue in the education sector. As Vara [22] notes, “the significance of applied research is pragmatic since its findings are immediately applied to the resolution of real-world problems”. The purpose of this study was to enhance English language learner’s reading skills through the application of a didactic technique. A single experimental group was employed in the pre-experimental design. This method was chosen because it is not possible to have a similar control group in which students are randomly assigned to a control group or an experimental group. The study population comprises 15 students enrolled in the Doctorate in Education Program at a university in the Southern region of Peru in academic year 2022. Several parameters were evaluated while deciding whether or not participants should be included. For the purposes of this study, a non-probabilistic convenience sampling method was utilized, as the sample population was comprised of Doctorate in Education students [22, 23]. “All research units are considered samples in the census sample,” says Ramirez [24]. The examination was used to collect data. This allowed for the evaluation of students’ comprehension of English-language literature. The investigation required three months: one lesson for the pre-test, twelve lessons for the experience (three weeks for each dimension), and one lesson for the post-test. The pedagogical exam was used to evaluate students’ English reading skills before (pretest) and after (posttest) the M-learning methodology was implemented. The literal, inferential, and critical meanings of the variable were used to make the test. Using the Kuder Richardson 2.0 formula, the reliability of the instrument was found to be 0.88, which shows how accurate the measurement was.

170

K. M. Laura-De La Cruz et al.

3 Results Table 1 illustrates the scores for the literal, inferential, and critical aspects. Demonstrating significant improvements following the implementation of m-learning, which statistically demonstrates the above, affirming that students identify vocabulary, and expressions of the text in English on a larger scale, which enables them to recognize vocabulary, expressions of the text in English with academic texts appropriate to their level of instruction. Table 2 provides the minimum and maximum scores obtained on the pre- and post-tests, revealing a substantial rise of five points for the minimum score and three points for the maximum score. Thus, the effectiveness of M-learning is demonstrated at the level of reading ability. Table 1 Level of reading skill in English, before and after m-learning application according to its dimensions Dimension

Statistics

Literal

Mean Median St. Desv Xmin Xmax

Inferential

Mean Median St. Desv Xmin Xmax

Critical

Mean Median St. Desv Xmin Xmax

Experimental group Inferential Statistics

Bilateral Sig

Pretest

Posttest

(n = 15)

(n = 15)

Student’s t

p < 0.05

5,60 5,00 1,183 4 8

8,20 9,00 1,612 6 10

T = -7.756 Gl = 14 P value P = 0.000

Significant improvement

(n = 15)

(n = 15)

Student’s t

p < 0.05

2,40 2,00 1,183 1 5

3,60 4,00 ,632 3 5

T = -4.583 Gl = 14 P value P = 0.000

Significant improvement

(n = 15)

(n = 15)

Student’s t

p < 0.05

2,27 2,00 1,223 1 5

3,40 3,00 ,737 2 5

T = -4.432 Gl = 14 P value P = 0.001

Significant improve-ment

Note: P valor (Dim 1: 0,000,002; Dim 2: 0,000,426; Dim 3: 0,000,569) Source: Written exam applied to the students in the sample

Mobile Learning and English Reading Abilities in Postgraduate Students

171

Table 2 The difference in the level of reading ability in English before and after the M-learning application Statistics

Experimental group Pretest

Posttest

Inferential Statistics

Bilateral Sig

(n = 15)

(n = 15)

Student’s T

P < 0.05

Mean Median St. Desv Xmin Xmax

10,27 10,00 2,840 7 16

15,20 16,00 2,455 12 19

T = -17.317 Gl = 14 P value P = 0.000

Significant improvement

IC al 95%

[ 8.69–11.84]

[13.84–16.56]

Note:unclustered data, P value (9,4112E-10) Source: Written exam applied to the students in the sample

4 Discussion and Conclusions Using M-learning, the goal of this study was to assess the level of English language reading ability of students enrolled in the Doctorate of Education program at Private University of Tacna. This result may be a result of the effective use of M-learning to reinforce the literal, inferential, and critical dimensions of English reading abilities. Based on the examination of the study’s findings, it is possible to infer that the presented hypothesis was accurate: M-learning raised the level of English reading skills. Mobile learning, according to Perez [25], is the capacity to use mobile technologies to enhance a student’s learning experience. It is applicable to numerous vocations. In the current study, the variety of various applications allowed for a greater degree of content modification and practice based on student level. As Yañez and Arias [26] point out, the use of mobile devices is on the rise due to their numerous benefits in the educational environment, which add to the competitiveness of the teaching process. In this sense, English language competency and the ability to comprehend English texts are extremely advantageous for academic, professional, and general purposes. The development in English reading skills can be attributed to the fact that the proposed applications were tailored to the level of the working group. This technique was presented, explained, and educated along, as no activity had ever been conducted with mobile applications tailored to their education. This study shows that doctoral students can learn the necessary reading and comprehension skills in a language, in this case English, if they are given, taught, and exposed to strategies that help them do so. As Campos [27] highlights, technological and instructional innovation support this learning. And it is essential to integrate this into independent language learning (English) for education Ph.D. applicants, whose academic and professional obligations prevent them from studying in institutions. This method permits the aforementioned individuals to learn and practice at their own pace. According to Cain [8], the design of learning environments enabled by mobile devices is becoming

172

K. M. Laura-De La Cruz et al.

more dynamic, engaging, simple, and manageable than a computer, with the aim of increasing and favoring teaching and learning processes. This makes it possible to come up with ways to help students improve their English reading skills using mobile devices. According to Rodriguez et al. [28], M-Learning strengthens the classroom-based English language learning process by incorporating technology and the Internet with all of its services, because mobile technologies are an appealing and simple tool that facilitates English language learning by fostering the development of listening, speaking, reading, and writing skills. According to Laura [29], the use of gamification technological tools has been demonstrated to improve students’ level of text comprehension in Regular Basic Education, indicating that the majority of students have a positive attitude toward the use of game-based learning and wish to implement such tools in other courses. On the other hand, it highlights the importance of fast and asynchronous feedback, which allows students to track their progress and, as a result, improve their English comprehension. Literature review and findings indicate that the English reading ability of professionals in our southern region and nation is crucial for understanding general English texts, technical English texts pertinent to their professional careers, and potential applications for various scholarships and professional exchanges. Each participant’s level was accommodated by the available mobile applications, which ranged from elementary to intermediate. The applications’ components were amusing and engaging, necessitating more active participation due to their ease of use and enjoyment. Mobile learning, according to Gómez and Lazo [30], promotes the development of networked information, develops creativity, and enables personalized learning, all while enhancing digital literacy, a vital ability in the digital world. However, after participants were introduced to the application, its characteristics, and the knowledge required to develop the proposed applications satisfactorily at each level, it became more dynamic and easier to improve this skill; this was due to the effective feedback and the usability of each application. Therefore, it is essential to employ mobile applications that support the development of skills, competencies, and the use of the English language to read various types of texts; each application is tailored to the context and requirements of the classroom.

References 1. Alsswey A, Al-Samarraie H, El-Qirem FA, Zaqout F (2020) M-learning technology in Arab Gulf countries: a systematic review of progress and recommendations. Educ Inf Technol 25(4):2919–2931. https://doi.org/10.1007/s10639-019-10097-z 2. Senaratne SI, Samarasinghe SM (2019) Factors affecting the intention to adopt m-learning. Int Bus Res 12(2):150. https://doi.org/10.5539/ibr.v12n2p150 3. Lai C, Hwang G (2014) Effects of mobile learning time on students’ conception of collaboration, communication, complex problem-solving, meta-cognitive awareness and creativity. Int J Mob Learn Organ 8(3):276–291

Mobile Learning and English Reading Abilities in Postgraduate Students

173

4. Ataeifar F, Sadighi F, Bagheri MS, Behjat F (2019) Iranian female students’ perceptions of the impact of mobile-assisted instruction on their English speaking skill. Cogent Educ 6(1):1–19. https://doi.org/10.1080/2331186X.2019.1662594 5. Kukulsa A (2007) Mobile usability in educational contexts: what have we learnt? Int Rev Res Open Distr Learn 8(2). https://doi.org/10.19173/irrodl.v8i2.356 6. Simbolon N, Marbun I, Simanjuntak EB (2020) Evaluating reading comprehension ability through the SQ4R model. Utopia y Praxis Latinoamericana 25(6):511–523. https://doi.org/10. 5281/zenodo.3987673 7. Altin M, Saracalo˘glu AS (2018) Effect of reading comprehension instruction enriched with cultural materials on English learning. Cogent Educ 5(1):1–14. https://doi.org/10.1080/233 1186X.2018.1475589 8. Cain K, Oakhill J, Bryant P (2004) Children’s reading comprehension ability: concurrent prediction by working memory, verbal ability, and component skills. J Educ Psychol 96(1):31– 42. https://doi.org/10.1037/0022-0663.96.1.31 9. Bojovic M (2010) Reading skills and reading comprehension in English for specific purposes. In: Bogdan S (eds) The international language conference on the importance of learning professional foreign languages for communication between cultures 2010, vol 17, Ljubljana, Slovenia, pp 1–5 10. Papastefanou T, Marinis T, Powell D (2021) Development of reading comprehension in bilingual and monolingual children—effects of language exposure. Languages 6(4):1–22. https:// doi.org/10.3390/languages6040166 11. Monroy J, Gómez B (2009) Comprensión Lectora. Mexicana de Orientación Educativa 6(16):37–42. http://www.seslp.gob.mx/pdf/comprension_lectora.pdf 12. Kumar T (2021) Cypr J Educ Speak Skill Among L2 Learn 16(1):411–422 13. Nget S, Pansri O, Poohongthong C (2020) The effect of task-based instruction in improving the English speaking skills of ninth-graders. LEARN J: Lang Educ Acquis Res Netw 13(2):208– 224 14. Gamboa ÁM (2017) Reading comprehension in an English as a foreign language setting: teaching strategies for sixth graders based on the interactive model of reading. Folios 1(45):159– 175 15. De La Cruz KML, Gebera OWT, Copaja SJN (2022) Application of gamification in higher education in the teaching of English as a foreign language. ICT with Intell Appl 256:323–341. https://doi.org/10.1007/978-981-16-5063-5_27 16. Laura K, Morales K, Clavitea M, Aza P (2021) Aplicación Quizizz y comprensión de textos en inglés con el contenido de la plataforma educativa “Aprendo en Casa. Revista Innova Educación 3(1):151–159. https://doi.org/10.35622/j.rie.2021.01.007 17. Lumbreras Editores. http://www.elumbreras.com.pe/content/la-problematica-de-la-compre nsion-lectora-en-el-peru. Accessed 09 Mar 2022 18. Escobar Aguirre J, Jiménez Murcia F (2019) Compresión lectora en inglés mediante el uso de herramientas interactivas en la básica secundaria. Aγαη 1(1) 19. Matos Juárez AS (2017) Lectura de textos literarios en inglés y comprensión lectora en los estudiantes de la especialidad Lengua – Inglés de la promoción 2012 de la Universidad Nacional de Educación Enrique Guzmán y Valle [Mather’s thesis]. Peru: UNE. http://repositorio.une. edu.pe/handle/20.500.14039/1424 20. Laura De La Cruz KM, Velarde Molina JF (2019) La Aplicación de un Software en Comprensión de Textos en Inglés para Estudiantes en Perú. Neumann Bus Rev 5(2):108–121. https:// doi.org/10.22451/3002.nbr2019.vol5.2.10042 21. Pisanty A, Enriquez L, Chaos-Cador L, García M (2012) “M-Learning en Ciencia” - Introducción De Aprendizaje Móvil en Física. Revista Iberoamericana de Educación a Distancia 13(1):129–155 22. Vara A (2015) Desde La Idea hasta la sustentación: Siete pasos para una tesis exitosa. Un método efectivo para las ciencias empresariales. Universidad de San Martín de Porres, Lima 23. Carrasco S (2007) Metodología de la investigación científica. Editorial San Marcos, Lima 24. Ramírez T (2004) Cómo hacer un proyecto de investigación. Editorial PANAPO, Caracas

174

K. M. Laura-De La Cruz et al.

25. Pérez O (2017) M-Learning en el aprendizaje de la mediación a los estudiantes del octavo semestre de la carrera de derecho de la Universidad Nacional de Chimborazo [Mater’s thesis]. Ecuador: Universidad Regional Autónoma de Los Andes. http://dspace.uniandes.edu.ec/bitstr eam/123456789/6553/1/PIUMCJ007-2017.pdf 26. Yañez J, Arias M (2018) M-learning, aceptación tecnológica de dispositivos móviles en la formación online. Revista Tecnología, Ciencia y Educación 6(10):13–34. https://dialnet.uni rioja.es/servlet/articulo?codigo=6775332 27. Campos J (2016) El uso de las TIC, dispositivos móviles y redes sociales en un aula de la educación secundaria obligatoria [Mater’s thesis]. España: Universidad de Granada. http://dig ibug.ugr.es/handle/10481/42209 28. Rodriguez Arce J, Juárez Pegueros J (2017) Impacto del m-learning en el proceso de aprendizaje: habilidades y conocimiento. RIDE. Revista Iberoamericana para la Investigación y el Desarrollo Educativo 8(15):363–386 29. Laura K, Franco L, Luza K (2020) Gamification for understanding English texts for students in a public school in Peru. Int J Dev Res 10(10):41787–41791. https://doi.org/10.37118/ijdr. 20319.10.2020 30. Gómez C, Lazo C (2015) Modelo de integración Educomunicativa de APPS Móviles para la enseñanza y aprendizaje. Rev Med Educ 46:137–153 (2015). https://doi.org/10.12795/pixelbit. 2015.i46.09

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision Through Unlearning Processes Juan-Gabriel Cegarra-Navarro , Anthony Wensley , and Jorge Cegarra-Sánchez

Abstract In knowledge management, both ophthalmology and neuroscience can provide research with useful metaphors with which to understand ways in which knowledge management may fall short of the mark. In the following paper, we discuss the nature of blind spots and peripheral vision. When we talk about blind spots or a failure of peripheral vision, we refer to things that people in general and knowledge workers, in particular, fail to perceive, even though they may appear obvious to others. Such failures may lead to misperceptions or misinterpretations of what is perceived. The current study suggests the use of processes that we refer to as ‘unlearning processes’ to alleviate the misperceptions or misinterpretations that arise from the existence of blind spots and misperceptions involving peripheral vision or its interpretation. In the paper, we will concentrate on the impact of blind spots leaving issues relating to peripheral vision to a subsequent paper. It is our contention that the use of such unlearning processes is likely to result in the achievement of an improved balance between rational, emotional, and spiritual knowledge for the individual and the organization. Such an improvement better serves the multifaceted interests it needs to serve in order to achieve success. We also postulate that a focus on avoiding blind spots and improving peripheral vision will enhance the ability of organizations to perceive “weak signals” and therefore trigger appropriate unlearning processes. Furthermore, by explaining the stimulatory effect of weak signals on unlearning processes, we reveal the potential positive effects of weak signals on the behaviour and thinking of individuals and, by extension, the ability of organizations to succeed in complex, dynamic environments. Keywords Blind spots · Peripheral vision · Weak signals · Unlearning processes

J.-G. Cegarra-Navarro (B) · J. Cegarra-Sánchez Business Administration Department, Universidad Politécnica de Cartagena, Murcia, Spain e-mail: [email protected] A. Wensley Accounting and Management Information Systems, Department of Management, University of Toronto Mississauga, Mississauga, Canada © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_16

175

176

J.-G. Cegarra-Navarro et al.

1 Introduction Much has been written about management’s perception and understanding of the strategic context within which firms operate. For example [1], describes the concept of strategic myopia which may be considered to nature of the needs that a company’s products or services satisfy. This results in the failure to detect or appropriately interpret signals that are relevant to the survival of the company because they do not seem to be relevant. For example, a company manufacturing electrically powered domestic drills may consider that the need it is satisfying is for domestic drills. However, if it considers that it is providing customers ways of creating holes it may recognize the development of better ways of creating holes in material, say, by using laser technology, as a threat. Thus, focusing too narrowly on the nature of the need the company is satisfying would lead if to fail to perceive threats to its survival. The phenomenon of managerial myopia has also been linked to the short-termism of innovation strategy [2]. The use of the term ‘myopia’ indicates the failure of mangers to perceive or inadequately interpret events in their peripheral vision. For example, several researchers also refer to peripheral vision loss as the inability to identify market trends and weak signals beyond companies’ current boundaries [3, 4]. Companies focus on knowledge that has proved valuable in the past and fail to recognize knowledge that is critical for establishing trajectories for future success. Adopting an ophthalmology perspective, we may consider that errors in perception may arise as a result of physical aspects of the eye, namely blind spots or from failures to appropriately interpret signals coming from the eye. The term blind spot refers to the part of the retina where the optic nerve interfaces with the retina [5]. Physically, this part of the eye is truly blind – that it has no receptors and hence light impinging on this part of the eye generates no signal whatsoever. We do not ‘see’ the eyes blind spot in our visual field because the brain ‘fill in’ the missing information using a variety of heuristics and by combining signals from both eyes. The eye is also able to focus on a relatively narrow part of our visual field. The rest of our visual field represents our peripheral vision. We can perceive objects in our peripheral vision much less accurately than objects in the focal part of our field of vision because the former are detected by a part of the retina much richer in rods and cones than the part of the retina which is responsible for peripheral vision. Given that signals in our peripheral vision are much weaker and less accurate we may either fail to perceive these signals in the first place or misinterpret them if we do perceive them. The concept of blind spots is usually understood as things that people do not see, for example, when we are driving there are blind spots or visual gaps in which we can lose sight of a vehicle coming from behind or from another lane, circumstances that pose a danger to our safety and the safety of others. In this context we may reduce the dangers of blind spots in a variety of ways. We may refocus our attention on object that would normally be in our peripheral vision or our blind spots by turning our head. We may make use of technologies such as mirrors, cameras, or short-range sensors to ‘fill in’ our blind spots. We may also deliberately train ourselves to ‘fill

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision …

177

in’ blind spots by constantly paying attention to all the vehicles behind us and hence tracking them before they enter our blind spots or intuitively learning the size of our vehicle so that we ‘know’ where the front or the back of the vehicle is with respect to other vehicles and objects around us. Finally, there is some indication that we may be able to improve our perception of the weak signals that are provided to us through our peripheral vision. To some extent we may be able to re-train our peripheral vision to be more sensitive. It is also worth noting that neuroscience provides us with the concept of ‘neuroplasticity’. The term neuroplasticity essentially refers to the ability of the brain to rewire itself following traumatic damage such as that which occurs as a result of a stroke or major disability. Thus, loss of sight may result in the parts of the brain which would normally process visual signals being re-tasked to process auditory signals. We respect to knowledge and knowledge management, when we talk about blind spots, we refer to things that people in general and knowledge workers, in particular, fail to perceive. They may be blind to the knowledge either because they simply don’t possess the knowledge or are aware of its existence but do not consider it relevant. More subtly, knowledge workers may misperceive knowledge they aware only aware of some aspects of the knowledge in question. When they misinterpret knowledge that may have complete or incomplete knowledge but they do not make an appropriate interpretation of the knowledge [6] (Zajac & Bazerman, 1991). Both misperception and misinterpretation of knowledge may lead to the phenomenon of knowledge hiding where knowledge is deliberately ignored [7, 8]. Interesting enough there is a parallel phenomenon with respect to visual perception where signals may be completely ignored even if they are within the focal visual field. In the context of knowledge and knowledge management knowledge workers may misinterpret the status of knowledge, as in the case of counter-knowledge or overinterpret weak signals [9, 10]. For example, doctors may rely on the results of one test to perform a diagnosis when they should use a combination of tests for higher reliability. In conclusion, when this study refers to managerial blind spot effects, we refer both to cases where knowledge workers do not possess (or ignore) weak signals that knowledge workers may indeed perceive and may indeed hear but not understand or understand incorrectly [9, 11]. To address management’s perceptions with respect to strategic issues, this paper provides guidance for identifying, articulating, and managing blind spots. In a later paper, we intend to provide a more detailed enquiry into peripheral vision and address ways of improving peripheral vision and the responses to signals detected by peripheral vision. The arguments presented in this paper also complement organizational theories of unlearning by providing a clearer definition of unlearning processes (i.e., awareness, relinquishing, and relearning) and lead to the development of a conceptual framework that sets out to address the blind spots that influence the management’s vision on strategic issues, such activities focus on the need to train the vision of knowledge workers. The paper is organized as follows: In Sect. 2, we give an overview of the types of blind spots that are likely to occur in a knowledge management context. In

178

J.-G. Cegarra-Navarro et al.

addition, we show the related studies on the connection between blind spots and the three fundamental fields of knowledge (i.e., rational, emotional, and spiritual). Section 3 describes the three unlearning processes adapted to overcome the weak signals that result from misinterpretations arising from the existence of blind spots. Section 4 presents the concluding remarks and provides recommendations for knowledge workers which propose adopting strategies both relating to adopting multiple perspectives and actively responding to weak signals. Finally, the study discloses its contributions and limitations with suggestions as to future research directions.

2 Different Types of Blind Spots Based on the literature coming especially from the cognitive sciences [12–14], we consider three fundamental fields of knowledge: rational, emotional, and spiritual. The ability of a human being to respond appropriately to the physical and social life environment depends on their achieving a balance between emotions, reason, and spirituality. That is, in other words, achieving a balance between emotions that move the human heart, the mind or reason, and the values or beliefs that reflect our spirit. Since each type of knowledge has its own characteristics and ‘logic’, achieving a good balance is a challenge for most knowledge workers [15]. The present study defines a “good balance” as having a state of harmony between heart, mind, and spirit. In the context of companies, we consider that achieving a balance or harmonization of emotional, spiritual, and rational knowledge leads to the satisfaction of the needs of organizational stakeholders and thus success is defined in a wholistic fashion. However, achieving this balance within the organization is a challenging task [16]. In particular, the so-called knowledge blind spots, which we refer to below, potentially disturb the harmonization process, by preventing the knowledge workers from having a complete vision and understanding of the company and its stakeholders and optimizing the instruments available to them [6]. #1: A rational blind spot occurs when reason dominates our decisions to the detriment of consideration of either emotions or values. Many knowledge workers experience emotional and social challenges and if they do not consciously take into account these emotional and spiritual perspectives, they are unlikely to achieve a balance. Furthermore, this prevents them from developing a holistic perspective of the environment within which the organization operates. They may only perceive a limited aspect of the reality that surrounds them [17]. In many decisions, knowledge workers confer superiority to reason because they believe that focusing on reason rather than on feelings and emotions provides a more consistent and defensible position to adopt. Furthermore, focusing on reason in preference to emotions or values is often considered to be a sign of maturity or personal ‘balance’. When this rational blind spot occurs, it may limit social integration, for example, when an individual focuses on the rational he (she) may fail to predict or understand the full richness of others’ social responses to his (her) actions or words [11].

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision …

179

#2: An emotional blind spot occurs when emotions prevent individuals from reacting rationally to signals from our environment. Over responding to emotions may result in an individual failing to perceive external signals and hence they become, essentially, blind and deaf. Indeed, emotions influence our spontaneous reactions, our way of thinking, our memories, the decisions we make, how we plan for the future, our communication with others and the way we behave. They are critical to establishing the value system, convictions and prejudices that guide our conduct and determine our ethical behaviour. In other words, the emotional state of people is an integral component of human well-being. Therefore, emotional blind spots with result in overemphasis on emotions may limit our happiness and our responses may be disruptive or inappropriate given the situation or setting [18]. #3: Spiritual blind spots occur when we focus on existing values, beliefs, or thoughts, and we are unable to envisage the impact of other values or value systems. The undue focus on particular values can limit self-criticism and prevent us from seeing existing problems or predicting future problems both theoretically and practically in society [4, 18] (Cegarra-Sánchez et al., 2022; Day & Schoemaker, 2004). It should be noted that unconsciously we always act based on our values and when we do not do so, we feel uncomfortable but may not fully understand why. Furthermore, the failure to appreciate that others may have different value systems may make it difficult to develop appropriate responses to challenges that can be explained to others and accepted by them.

3 Avoiding Weak Signals Created by Blind Spots Undue focus on particular perceptions is likely to weaken other perceptions and thus cause them to either be misperceived or not perceived at all. The “weak” in weak signals refers to the fact that ‘weak’ signals are perceived as being distanced from the core prior knowledge of the signal perceiver [19] (van Veen & Ortt, 2021). As noted above weak signals could be the consequences of blind spots. For example, team members can become narrowly focused and fail to adopt alternative perspectives when faced with weak signals related to rational or spiritual knowledge created by emotional blind spots [4]. [20] calls it such unbalanced approaches as “groupthink” and relates it to the tendency for members of high cohesion groups to lose their critical abilities and adopt narrow definitions of problems and decision situations. Based on [21], among the crucial factors that explain the emergence of ‘groupthink’, we can find the degree of cohesion and the degree of conformity of the members. Thus, the emotional desire to maintain a united team becomes more important than the quality of the decisions and weak signals relating to rational knowledge or spiritual knowledge are often ignored. Unlearning is a concept that has been traditionally related to the renewal of knowledge and the creativity of knowledge workers [22, 23]. It implies putting aside knowledge or counter-knowledge intentionally [24]. Given that prior research has demonstrated that the behaviour and thinking of individuals are affected by unlearning

180

J.-G. Cegarra-Navarro et al.

processes such as awareness, relinquishing and relearning [25–28], it is reasonable to think that knowledge workers may need to perform such processes in order to address blind spots and the resulting weak signals. This study proposes the three unlearning processes developed by [29] as one way to overcome these challenges. In the following paragraphs, we explain how these three unlearning processes can be used to overcome blind spots and the weak signals from the environment that they cause. While weak signals result in failure to make accurate predictions or appropriate decisions augmenting weak signals with additional relevant information improves the accuracy of predictions and interpretations based on the sources of such weak signals [19]. Drawing upon a reading of [30] and [19, 31] propose that the process of clarifying weak signals began when managers became aware of environmental developments, thereby turning a piece of information from the background noise into a signal. Therefore, awareness of the significance of scanning for new ways to think about the market, emerging technologies and new business models plays a key role in the acquisition of information to overcome the effects of blind spots [4]. Awareness can be supported by changing the lenses through which knowledge workers see the world, for example, allowing multiple perspectives, changing observer positions, or using technology tools [29]. Another negative consequence of blind spots is emotional instability, also known as “emotional imbalance”. It is important to note that what arouses an emotion is not the event itself, but the interpretation we make of it [32]. Therefore, the emotional imbalance caused by an interpretation, there is only one quick way to eliminate it, to get it out of our mind: another emotion, another stronger feeling, incompatible with the one we want to banish [33]. For example, it is not easy to rid ourselves of depression if we think that no one cares and that we are fighting a losing battle by ourselves. On this basis, we suggest that relinquishment is achieved by probing and learning another interpretation, which in turn helps to give up an incompatible or inappropriate emotion [4] (Day & Schoemaker, 2006). In fact, eliminating a particular interpretation involves not only developing our own alternative interpretations but also becoming aware of the interpretations of others [29]. As previously discussed, spiritual blind spots happen when an individual or a group adhere to extreme values and beliefs. In addition to compelling members of the group to accept these values and beliefs without questions, groups often attempt to compel others into accepting them. The existence of alternative values or beliefs is underrated or, indeed, denied completely. In these cases, there is no internal critical analysis of beliefs or values. They are considered to be self-evident. The elimination of choice with respect to beliefs and values reduces the complexity of achieving holistic balance and thus may appear to be attractive to individuals. This explains to some extent the attractiveness of adopting behaviours that lead to spiritual blind spots. Spiritual blind spots can be compared to people with color blindness, those who suffer from color blindness often have difficulty distinguishing some shades of certain colors, usually, these people require external aid to interpret different “close” colors in the color spectrum. Although everybody has the capacity to self-diagnose color blindness only a few are aware of such capacity and even fewer can transform

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision …

181

Fig. 1 Unlearning processes to avoid the emergence of blind spots

such a diagnosis into a treatment plan. In the end, a colorblind person requires some relearning and someone to teach him how to see colors well [34]. Figure 1 summarizes the above discussion, tensions between the attachment to outdated values, mixed emotions and powerful reasoning drive the spread of weak signals which manifest themselves as fear, pressure, uncertainty, or anxiety. As happens with mixed emotions, it is impossible to change an outdated idea, belief, or value, even a distorted one, without replacing it with an alternative idea or a more attractive belief [35]. Therefore, these challenges require a greater degree of clarification of weak signals through a collaborative unlearning system. This is for example, where help from an expert outsider can really make a difference in supporting other beliefs and considering different values and value systems [36, 37]. For example, when an arrogant co-worker sees someone who consults with others before making decisions he may become ashamed of himself and subsequently reconsider his arrogance [38]. As shown in Fig. 1, the three unlearning processes (i.e., awareness, relinquishing and relearning) are interconnected and act in parallel. For example, as a result of social pressure, one becomes “aware” that a certain habit is incorrect and automatically represses and replaces it with another one, according to the relinquishment process, and thus establishes the possibility for relearning and sharing new understandings with others [29].

4 Concluding Remarks The current study suggests some unlearning processes to counteract the deleterious effects of blind spots and increase the likelihood of achieving a balance between rational, emotional, and spiritual knowledge to ensure that the best interests of the company’s stakeholders are served. Therefore, this study contributes to the vast literature devoted to blind spots by proposing a conceptual framework which highlights three unlearning processes to overcome blind spots and put aside tensions between the attachment to outdated values, mixed emotions and powerful reasoning that inhibit knowledge workers from seeing the whole environmental picture.

182

J.-G. Cegarra-Navarro et al.

In the research literature relating to blind spots, a substantial amount of theorising has been presented dealing with the ways to overcome them [6, 17, 39, 40]. Although the overall idea is that managers need to ask themselves and others tough questions, this logical way of overcoming blind spots needs to take into account two distinct issues. On the one hand, the presence of individual tensions between the different kinds of individual, emotional and spiritual knowledge [18]. On the other hand, the presence of groupthink prevents members of groups from self-criticism [20]. In the case of groupthink, as with binocular vision where the eyes compensate each other, knowledge workers must establish the appropriate measures that limit their blind spots. As [21] point out, knowledge workers should try to decentralize the most important decisions and ask for immediate feedback once the decision is implemented, to correct any problems as soon as possible. In making important decisions, they can also ask for advice from someone completely outside the organization and with purely objective criteria. This person will help them identify our blind spots by reviewing the organizational strategy with them. If we are aware of one individual dominating a group it may help to either remove this individual or actively encourage others to participate in order to develop and investigate different perspectives, values and beliefs. These measures can best be used from the early stages of analysis or decision-making since this is likely to reduce the likelihood of commitment to inappropriate strategies thus reducing wasteful commitment of resources. This study makes another valuable contribution as it postulates “weak signals” either as a result of blind spots or as a trigger to unlearning. In fact, weak signals take place among knowledge workers and therefore what happens at the individual level is a growing awareness to find additional information specific to clarify the weak signal, which leads knowledge workers to relinquish or stop making the same old mistakes or errors again. Once an individual has got a better appreciation of the signal, relearning involves being able to lead by example. In other words, awareness at the individual level means paying attention to weak signals while relinquishing refers to probing alternative interpretations and relearning means believing and leading. It is important to note that the existing literature has pointed to weak signals as somewhat negative in that can pose a threat to our comfort zone [9, 11, 19, 30, 31]. However, from the perspective of this study, we can also see them as an opportunity to improve and strengthen our adaptation to the environment. Rather than ignoring such weak signals because they cause discomfort, they can stimulate the gathering of additional information which results in the identification of improved ways of achieving company objectives. For example, some employees may be considered to be ‘difficult’ because they question the appropriateness of particular corporate policies and decisions. However, these employees may be extremely valuable as they suggest alternative perspectives and solutions to problems and challenges. The ‘noise’ or weak signals generated by these individuals can inspire their fellow colleagues to look outside the box and expose these issues to the boss.

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision …

183

5 Conclusions Our study contributes to the research literature in the following ways. First, we respond to calls to examine the effects of blind spots on weak signals [4, 11]. Specifically, our study can help manage and diffuse tensions related to the attachment to outdated values, mixed emotions, and powerful reasoning. Second, by explaining the stimulatory effect of weak signals on unlearning processes, we reveal the potential positive effects of weak signals on the behaviour and thinking of individuals. Thus, we identify situations in which weak signals may have the potential to help improve the performance of organizations particularly in the areas of improving service quality and the reduction of costs. As previously noted, we consider this an important contribution to the efforts aimed at transforming weak signals into knowledge which can potentially lead to improved business performance. This paper is subject to certain limitations that need to be addressed. Firstly, this study is purely theoretical at this point and future research should develop empirical approaches that specifically focus on unlearning activities that encourage the three unlearning processes described above. Secondly, although this paper presents blind spots among knowledge workers which highlight the nature and influence of weak signals across different levels (i.e., individual, and organizational), the model presented is not exhaustive and assumes that other causes may affect weak signals. Therefore, further research needs to be conducted to investigate other causes which have not been included in this study and are likely to affect weak signals. Finally, despite an extensive literature review, it is likely that a more extensive review would pay significant dividends in extending the richness and breadth of ways to deal with weak signals.

References 1. Czakon W (2022) Strategic management and myopia: challenges and implications. Strateg Manag Myopia. https://doi.org/10.4324/9781003199151 2. Seo HJ, Kang SJ, Baek YJ (2020) Managerial myopia and short-termism of innovation strategy: financialisation of Korean firms. Cambridge J Econ 44(6):1197–1220. https://doi.org/10.1093/ cje/beaa023 3. Cegarra-Navarro A, Wensley JG, Garcia-Perez A, Sotos-Villarejo A (2016) Linking peripheral vision with relational capital through knowledge structures. J Intell Capital 17(4): 714–733. https://doi.org/10.1108/JIC-04-2016-0041 4. Day G, Schoemaker P (2006) Peripheral vision : detecting the weak signals that will make or break your company. Boston. 5. Chevaleraud JF (1986) Ojo y deporte. Masson, Barcelona 6. Zajac EJ, Bazerman MH (1991) blind spots in industry and competitor analysis: implications of interfirm (Mis)Perceptions for strategic decisions. Acad Manag Rev 16(1):37–56. https:// doi.org/10.2307/258606 7. Zhang Z, Min M (2021) Organizational rewards and knowledge hiding: task attributes as contingencies. Manag Decis. https://doi.org/10.1108/MD-02-2020-0150

184

J.-G. Cegarra-Navarro et al.

ˇ 8. Cerne M, Nerstad CGL, Dysvik A, Škerlavaj M (2014) What goes around comes around: knowledge hiding, perceived motivational climate, and creativity. Acad Manag J. https://doi. org/10.5465/amj.2012.0122 9. Day GS, Schoemaker P (2004) Peripheral vision: sensing and acting on weak signals. Long Range Plann 37(2):117–121 10. Thompson D (2008) Counter-knowledge. Atlantic Books, London 11. Day GS, Schoemaker P (2006) Peripheral vision: detecting the weak signals that will make or break your company. Harvard Business School, Boston 12. Damasio A (2010) Self comes to mind: constructing the conscious brain. Pantheon/Random House, New York 13. Fauconnier G, Turner M (2002) The way we think: conceptual blending and the mind”s hidden complexities. Basic Books, New York 14. LeDoux JE (1996) The emotional brain: the mysterious underpinnings of emotional life. Simon & Schuster, New York 15. Guthrie VL, Woods E, Cusker, C, Gregory M (2005) A portrait of balance: personal and professional balance among student affairs educators. Coll Student Aff J 24:110–127 16. Bratianu C, Bejinaru R (2019) Knowledge dynamics: a thermodynamics approach. Kybernetes 49(1):6–21. https://doi.org/10.1108/K-02-2019-0122 17. Kühl S (2020) The blind spots in theory U: the reconstruction of a (Change-) management fashion. J Chang Manag 20(4):314–321. https://doi.org/10.1080/14697017.2020.1744883 18. Cegarra-Sánchez, J, Cegarra-Navarro JG, Sánchez-Medina AJ (2022) Co-creating practical wisdom through rational, spiritual and emotional capacities. Kybernetes ahead-of-print, no. ahead-of-print. https://doi.org/10.1108/K-03-2022-0474 19. van Veen BL, Ortt JR (2021) Unifying weak signals definitions to improve construct understanding. Futures 134:102837. https://doi.org/10.1016/j.futures.2021.102837 20. Hart P (1991) Irving L. Janis’ Victims of Groupthink. Polit Psychol 12(2):247–278. https:// doi.org/10.2307/3791464 21. Cegarra-Navarro JG, Rodrigo-Moya B (2005) Learning facilitating factors of teamwork on intellectual capital creation. Knowl Process Manag. https://doi.org/10.1002/kpm.215 22. Fernandez V, Sune A (2009) Organizational forgetting and its causes: an empirical research. J Organ Chang Manag 22(6):620–634. https://doi.org/10.1108/09534810910997032 23. Martignoni D, Keil T (2021) It did not work? Unlearn and try again—unlearning success and failure beliefs in changing environments. Strateg Manag J 42(6):1057–1082. https://doi.org/ 10.1002/smj.3261 24. Cegarra-Navarro J-G, Eldridge S, Sánchez ALG (2012) How an unlearning context can help managers overcome the negative effects of counter-knowledge. J Manage Organ 18(2). https:/ /doi.org/10.5172/jmo.2012.18.2.231 25. Hislop D (2008) Conceptualizing knowledge work utilizing skill and knowledge-based concepts. the case of some consultants and service engineers. Manag Learn 39(5):579–596. https://doi.org/10.1177/1350507608098116 26. Tsang EWK (2005) Organizational learning and unlearning: transferring knowledge to greenfield versus acquisition joint ventures. Warwick.Ac.Uk 25 [Online]. http://www2.warwick.ac. uk/fac/soc/wbs/conf/olkc/archive/oklc6/papers/tsang.pdf 27. Tsang SA, Zahra EWK (2008) Organizational unlearning. Org Unlearn 61(10):1435–1462 28. Zhao Y, Wang X (2020) Organisational unlearning, relearning and strategic flexibility: from the perspective of updating routines and knowledge. Technol Anal Strateg Manag 32(11):1251– 1263. https://doi.org/10.1080/09537325.2020.1758656 29. Cegarra-Navarro JG, Wensley A (2019) Promoting intentional unlearning through an unlearning cycle. J Organ Chang Manag 32(1):67–79. https://doi.org/10.1108/JOCM-04-20180107 30. Ansoff HI (1975) Managing strategic surprise by response to weak signals. Calif Manage Rev 18(2):21–33. https://doi.org/10.2307/41164635 31. Ilmola L, Kuusi O (2006) Filters of weak signals hinder foresight: monitoring weak signals efficiently in corporate decision-making. Futures 38(8):908–924. https://doi.org/10.1016/j.fut ures.2005.12.019

Overcoming Knowledge Blind Spots and Enhancing Peripheral Vision …

185

32. Folz J, Fiacchino D, Nikoli´c M, van Steenbergen H, Kret ME (2022) Reading your emotions in my physiology? reliable emotion interpretations in absence of a robust physiological resonance. Affect Sci 3(2):480–497. https://doi.org/10.1007/s42761-021-00083-5 33. Ren Z, Zhao Z, Yu X, Zhang Z, Li X (2021) Modification of hostile interpretation bias and self-reported aggression in juvenile delinquents: a randomized controlled trial. Int J Clin Health Psychol 21(2). https://doi.org/10.1016/j.ijchp.2021.100226 34. Quackenbush TR (1997) Relearning to see : improve your eyesight naturally. North Atlantic Books 35. Sartori R, Costantini A, Ceschi A, Tommasi F (2018) How do you manage change in organizations? Training, development, innovation, and their relationships. Front Psychol 9. https:// doi.org/10.3389/fpsyg.2018.00313 36. Cegarra-Navarro J, Garcia-Perez A, Bedford BAD (2020) Contextual enablers and behaviour outputs for action of knowledge workers. Int J Econ Manag Eng 14(12):1185–1190 37. García-Pérez A, Cegarra-Navarro J, Bedford DAD, Thomas M, Wakabayashi S (2019) Critical Capabilities and Competencies for Knowledge Organizations. Bingley BD16 1WA: Emerald Publishing Limited [Online]. https://books.emeraldinsight.com/page/detail/Critical-Capabilit ies-and-Competencies-for-Knowledge-Organizations/?k=9781789737707 38. Gunnarsson NV (2020) The self-perpetuating cycle of shame and self-injury. Hum Soc 45(3):313–333. https://doi.org/10.1177/0160597620904475 39. Wiegand WA (1999) Tunnel vision and blind spots. Libr Q 69(1):1–32 40. Meissner P, Brands C, Wulf T (2017) Quantifiying blind spots and weak signals in executive judgment: a structured integration of expert judgment into the scenario development process. Int J Forecast 33(1):244–253. https://doi.org/10.1016/j.ijforecast.2015.08.002

Media, Applied Technology and Communication

Applying Machine Learning Techniques to Box Office Forecasting Rian Bessa Lopes and José Viterbo

Abstract The box office forecast is a big challenge due to the characteristics of the sector. A good forecast can be used as an investment tool or as a tool to promote public policies. In this work we used the public database of the Agência Nacional do Cinema (Brazilian Film Agency) - Ancine, and data taken from websites to gather predictive variables for the audience that each movie attracted. Statistical analysis, data cleaning and webscraping were used. Four different machine learning algorithms were used. In terms of F1 Score and with the use of hyperparameter tuning, the best algorithm was LGBM (0.7589) followed by Linear Regression (0.7361). MLP (0.7084) and SVM (0.6988) results completed the experiment. It was possible to show that the algorithms could learn from data and predict the box office performance of Brazilian movies relatively well. Keywords Box office · Brazilian film agency · Machine learning

1 Introduction The film industry is an important field for machine learning application. The sector produces a lot of data so it offers great opportunities to provide different possibles solutions. One of the major challenges in the sector is to identify which movie will be a box office success [3]. Forecasting the box office represents a major challenge due to the nature of the industry. Very few films are highly successful at the box office without, however, showing such a clear relationship with their attributes [2]. The box office prediction represents an important tool for investment in the film industry [3]. Many authors used R. B. Lopes (B) · J. Viterbo Instituto de Computação, Universidade Federal Fluminense (UFF), AV Gal. Milton Tavares, Niterói, RJ, Brazil e-mail: [email protected] J. Viterbo e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_17

189

190

R. B. Lopes and J. Viterbo

different algorithms in order to obtain fair predictions. IMDb and others database like Chinese and Indian film industry database were used for this purpose [3, 7]. In the present work we intended to address the box office forecast in a Brazilian database in order to understand its effectiveness, considering its industry unique characteristics [6]. To do so we used four different algorithms and different features configurations. We also used hyperparameters optimization. At the end we performed a hypothesis test to verify if the algorithms presented relevant differences in its effectiveness. The Brazilian film industry is recognized as a driver of employment and national culture [6], so the lack of studies in this market becomes a great opportunity to enrich both box office forecasting and machine learning literature. This work is organized as follows: Sect. 2 We explain the main aspects of the theme and algorithms. In Sect. 3 - we present works that addressed the theme of box office forecast published from 2018. In Sect. 4 - The methods and materials used to develop the work. In Sect. 5, the results and their discussion will be carried out. In Sect. 6, a conclusion is presented with the main contributions of the work.

2 Fundamental Concepts 2.1 The Film Industry The film industry has a great impact on a country’s economy, for this reason several researches using machine learning have been conducted all over the world [3]. The forecast of box office results of a movie could lead the entire industry to make better investments, in order to reduce the risks of the business [5]. The film industry in Brazil can be divided into three sectors: production, distribution and exhibition. Production and exhibition are characterized by an immense majority of small companies, which now survive by promoting State policies. The distribution sector is dominated by large international companies. Indeed, the dynamics of the Brazilian film market has enough particularities to give rise to specific research [6]. The forecast of the outcome of a movie production could benefit various segments of the industry. In the pre-production the forecast could be guiding the decision regarding budget, structure and cast. In the post-production phase, it could be guiding the entire pre-launch, distribution and exhibition strategy [5]. Incorporating databased decision-making processes into the film industry could even benefit agencies of all spheres in their respective public policies to promote the film industry. As the Inter-American Development Bank (IDB) document demonstrates, there are several successful examples with this measure, determining an increase in effectiveness and economy, consequently allowing more access to full citizenship [8].

Applying Machine Learning Techniques to Box Office Forecasting

191

2.2 Supervised Machine Learning Machine learning algorithms learn from data without explicitly been programmed for it. They seek to solve some business problem [15]. In supervised machine learning, the algorithm is presented to two sets of data. They will be called the attributes X = (x 1 , x 2 ..., x n ), and the response variable Y = (y1 , y2 ..., yn ). After training, the algorithm will be able to predict new values of Y, based on new values of X [17]. Four machine learning algorithms were used in this work. The first of them and one of the oldest is the Logistic Regression. Logistic Regression establishes a linear relationship between X and a logit function of Y, thus establishing prediction of categorical values [18]. The other algorithm used was the SVM (Support Vector Machine) which, in short, using a kernel function, transforms the data and establishes support limits to classify new observations [1]. The third algorithm is the Neural Network, or Multilayer Perceptron (MLP). The algorithm has one or more hidden layers preceded by an input layer and followed by an output layer. In each layer there will be one or more neurons that will make the nonlinear mapping between the input attributes and the output variable [20]. The fourth, Light Gradient Boosting Machine (LGBM) proposed in [19], is an optimization of the gradient boosting technique. The algorithm performs a grouping of weak classifiers (decision trees) to improve learning.

3 Related Works The problem of predicting results in movies has been addressed in different ways. In general, as demonstrated by Abidi et al. in [3] and Ahmad et al. in [4] the most used machine learning algorithms are Linear and Logistic Regression, Neural Networks, Support Vector Machine (SVM), Naive Bayes and models based on Decision Trees. Several databases have been used containing attributes such as release date, genre, cast, producer, distributor and budget [3]. Using data from the IMDb page (Internet Movie Database), Abidi et al. in [3] used a database with 651 films in English, and, from 4 variables, predicted the film’s popularity score. The algorithms used were GLM (Generalized linear model), Neural Networks, Decision Tree, Random Forest, Gradient Boosted Trees, with the best result attributed to GLM, with 0.479 of Root Mean Square Error (REQM) Ahmad et al. in [4] used metadata and comments from YouTube trailer reviews, as well as other financial information to predict revenue for 29 movies. The authors used Linear Regression, SVM, Neural Networks and Random Forest. Despite a reduced number of films worked, the authors considered the study reasonable, considering other works with similar values. Linear Regression and SVM performed better under many different configurations.

192

R. B. Lopes and J. Viterbo

Using data from social networks and a database with movie information, Ahmed et al. in [9] predicted the audience result of 5043 films divided into two classes: blockbuster and flop. The algorithms used were Gradient Boosting, Extreme Gradient Boosting and Random Forest and SVM in an algorithm voting system. The model proposed by the authors performed better than the base model, including reducing computational cost. On the other hand, Bristi et al. in [10], through a rating task, predicted the score divided into four categories, from the website IMDb, which measures the popularity of the film. The algorithms used were Bagging, Random Forest, J48, IBK and Naive Bayes. All algorithms performed above 98% of accuracy after statistical treatment of the database. Darapaneni et al. in [11] performed a binary classification task between success and failure taking into account box office and gross revenue. Six machine learning models were used, in addition to deep learning models. The best performance was Gradient Boosting, with an accuracy of 90%. Working with the IMDb database, which in addition to the data contained movie posters, Dhir and Raj in [12], predicted the IMDb popularity score divided into two classes. The algorithms used were SVM, Random Forest, Ada Boost, Gradient Boosting, KNN. The best performance was that of Random Forest, with an accuracy of 61%. Exploring the relevance of the world’s second largest film market, Jiang and Hao in [7] used 200 Chinese films to perform daily viewership prediction. The authors used Generative Adversarial Networks (GAN), a type of deep learning. The model proposed by the authors performed better under different data configurations when compared to the base model. As identified in previous works, none of them addressed the Brazilian market specifically. Consulting two databases of academic works, Scholar and Scopus, with an interval between 2015 and 2021, it was not possible to find works with Brazilian films.

4 Methods and Materials 4.1 Databases The database used is provided by the Brazilian Observatory of Cinema and Audiovisual - OCA. Created in 2008, OCA provides a variety of information about the audiovisual and cinematographic sector. OCA provides a section of the website with data open to the public for free use. The data we use is the Weekly Ticket Report, which is reported by the distributors [13]. The database contains originally 20 attributes including the audience. Others attributes are about applicant of the register of the movie and technical informations about the movie like the segment of the movie, genre and if it has received public funding or not [13].

Applying Machine Learning Techniques to Box Office Forecasting

193

Additionally, we also use information from the films from the AdoroCin- ema.com portal. The site gathers various information in Portuguese about movies. Through an webscrapping technique, we can enrich the attributes of the works present in the OCA database and enhance the predictive power of machine learning algorithms. The following attributes were obtained for each film: “Type”, “Director”, “Screenwriter” and “Cast” [14]. The use of more than one database to enrich this box office forecast was also successfully made in [11].

4.2 Pre-Processing Some columns were discarded as being redundant, for example the movie name column, which was present in both databases. Other columns were irrelevant because they contained only one type of value. It was also removed columns that implied the box office result, causing falsely high performance. Columns such as Rooms, and Copies distributed are directly proportional to the box office result, since larger audiences demand a greater presence of the film in theaters, therefore, a greater number of copies. We may, however, use this information for the first week in order to forecast the audience at the end of the total number of weeks the movie was shown. In [16] was also used information from the first week of the films’ exhibition to make the box office forecast in the Turkish market. We used a monthly time frame for this work then it was necessary to group rows in the database by film since it is weekly. We added up the audience of each film over the weeks. When grouping rows, the column “Exhibit Start Date” the value considered was the first, that is, the date of the first week in which the film was shown. The other columns are repeated over the weeks. In addition, we alternatively tested using information from the first week, such as Rooms, Copies and Income, in which case the information used referred to the first week. As for the database taken from the Adoro Cinema website, as it comes from a website, it tends to have many problems and needs more attention in the preprocessing [3]. The column which represents the name of the artists that make up the Cast was in string format. A separation of each actor/actress name was made and separated into three columns. Therefore, those who had three actors/actresses in their cast each would appear in a column. Then the treatment of label encoding was done, in which each name of an artist is represented by a unique numeric value, and replaced in the column. It is a strong feature of film bases many textual attributes, which motivates specific treatments of these types of attributes [11]. The study’s response variable, Audience, as can be seen in Fig. 1 is extremely concentrated in values close to zero, with very rare exceptions with very large audiences. This asymmetry can create complications for algorithms to learn, since there is an imbalance of examples with larger audiences. The Public variable had an average of 255,090, maximum of 11,303,177, minimum of 2, standard deviation of 872,719.16, asymmetry of 6.96 and kurtosis of 65.35.

194

R. B. Lopes and J. Viterbo

Fig. 1 Distribution of the Audience response variable

Since we proposed a classification task, the variable was discretized. The values were sorted and then divided so that we had equal amounts in each of the classes. The cutoff value is equal to the median which is equivalent to 6694 audience. In this way, the films that have a box office greater than this value fall into the 1 class, and the films with the lowest box office fall into the 0 class. The division into two classes was also used in [11].

4.3 Training and Testing Training and testing was done using a ten-partition cross-validation. We performed training and testing with four different machine learning algorithms. Logistic Regression, SVM and MLP, implemented by the Scikit-Learning package, and the LGBM, implemented by the lightgbm package. We use three metrics to evaluate the performance of the algorithms: Ac- curacy, Balanced Accuracy, F1 Score. We also measure the time in order to demonstrate the computational cost among the algorithms.

5 Results and Discussion Two executions were performed: one without using information from the first week of exhibition, namely the number of rooms, copies and income. Another execution used this information.

Applying Machine Learning Techniques to Box Office Forecasting

195

Table 1 Result No Data Use of the First Week Algorithm

Accuracy

Balanced Accuracy

F1 Score

Time (s)

LGBM

0.6561

0.6627

0.6561

0.0452

MLP

0.6494

0.6564

0.6489

0.3853

SVM

0.641

0.6476

0.6403

0.0208

LR

0.6376

0.6444

0.6384

0.0171

5.1 Without the First Week Information In this configuration there was no information about the first week of exhibition. Table 1 demonstrates the result of this configuration. As can be seen, the best performance is due to the LGBM algorithm, performing better in all metrics. The LGBM was followed by the MLP, SVM and lastly the Logistic Regression. We can notice that the time taken by the MLP (0.3853 s) is much higher than the other algorithms, especially considering the superior performance of the LGBM in much less time (0.0452 s).

5.2 With Information from the First Week In this configuration, we use the information from the exhibition of the works in the first week, in order to conclude if they promote the increase in performance by predicting the audience at the end of the entire exhibition period. The result can be seen in Table 2. We can see that there is indeed an improvement in all algorithms in all metrics. Table 3 shows the percentage gain using the first week data in terms of F1 Score. As can be seen, all algorithms have at least a gain above 5% in all metrics, except for LR with a 4.9% increment in F1 Score. The fact of having more attributes in the database increased the processing time of the algorithms, being the smallest increase in the LR (33.42%), and the largest in the LGBM (370.22%), which may suggest, in this lastly, scalability issues as we increase the number of attributes. Figure 2 allows us to observe that, although the LGBM performed better with or without the information from the first week, the MLP was the one that most Table 2 Results with First Week Information Algorithm

Accuracy

Balanced Accuracy

F1 Score

Time (s)

MLP

0.6932

0.6991

0.6934

0.5537

LGBM

0.6922

0.6985

0.6922

0.1742

LR

0.6759

0.6834

0.6774

0.0805

SVM

0.6710

0.6801

0.6699

0.0660

196

R. B. Lopes and J. Viterbo

Table 3 Relative Gain Between Treatments (Without and With First Week) in terms of F1 Score Algorithm

Accuracy

Balanced Accuracy

F1 Score

LGBM

0.0566

0.0549

0.0569

Time (s) 3.7022

MLP

0.0659

0.0642

0.0667

-0.5036

SVM

0.0544

0.0553

0.0580

0.8352

LR

0.0523

0.0554

0.0493

0.3342

Fig. 2 Comparison of Results With and Without Information Of the First Week

benefited from this information, having an increase of more than 6 0.5% of the F1 Score. LR, on the other hand, had the worst performance and the smallest increment with information from the first week.

5.3 Parameter Optimization Given the improvement in performance when using the configuration with information from the first week of exhibition, an optimization of the parameters of the algorithms used was implemented. Several combinations of different parameters were made in order to obtain an improvement in terms of F1 Score, where each combination was cross-validated with ten partitions. As can be seen in Table 4, there was an improvement in the performance of all algorithms. The best performance continued to be that of the LGBM, reaching 75.89% of F1 Score, followed by LR, which obtained 73.61%. Still with improvement, but to a lesser extent, the SVM went from 66.99% to 69.88% and the MLP from 69.34 to 70.84%. To obtain such improvements, the LGBM used a smaller number of trees, going from 100 to 20. The LR did not use a penalty.

Applying Machine Learning Techniques to Box Office Forecasting Table 4 Comparison of the result of the optimization of algorithms parameters in F1 Score

197

Algorithm

Without optimization

With optimization

LGBM

0.6922

0.7589

LR

0.6774

0.7361

MLP

0.6934

0.7084

SVM

0.6699

0.6988

factor, and used a smaller C parameter, which determines the regularization of the model, from 1 to 0.01. The MLP used an architecture with two hidden layers of 100 neurons and a momentum of 0.8. As for SVM, the C parameter was increased from 1 to 2, and the kernel used was poly.

6 Hypothesis Test To demonstrate the statistical relevance of the results, we developed a hypothesis test to ensure that the algorithms actually differ in terms of performance. Therefore, an Analysis of Variance (ANOVA) was performed, and the p-value was observed if it was less than 0.05. The null hypothesis proposed is that the performances do not differ. The results were analyzed from a cross-validation with 10 partitions, repeating the test 3 times. In Fig. 3 it is possible to see the distribution of the results. As can be seen, the SVM presents the greatest variation of results, while the LR presents the smallest. LGBM has the highest median among the algorithms. When we evaluate from the results of the ANOVA test the p-value was 0.1666, therefore too high to reject the null hypothesis with a significance level of 5%. The result suggests that it is not possible to rule out chance when comparing the performances of the algorithms.

Fig. 3 Distribution of F1 Score using 10 × 3 cross-validation

198

R. B. Lopes and J. Viterbo

7 Conclusion This work demonstrated the possibility of developing an audience forecast in film industry using Brazilian public data. It was also demonstrated the possibility of enriching the database using more than one data source through webscrapping tools. The particularities of the Brazilian movies were learned by the algorithms and we were able to reach an F1 Score up to 75.89% using the LGBM algorithm. The hypothesis test, however, did not demonstrate the possibility of guaranteeing that the algorithms differ in their performances with statistical relevance. The addition of information from the first week of the box office increased the learning of the algorithms, demonstrating the importance of the phase and what informations are available from which the forecast is made. The optimization of parameters of the algorithms proved to be efficient, and could lead to improvements in future works using different combinations of parameters. Further improvements can still be achieved using different machine learning or deep learning algorithms. The data used do not represent the entire complexity of the industry then different databases and techniques like natural language processing will be used to improve the results present in this work.

References 1. Géron A (2019) Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. O’Reilly Media, Inc., Sebastopol 2. Mbunge E, Fashoto S, Bimha H (2022) Prediction of box-office success: a review of trends and machine learning computational models. Int J Bus Intell Data Mining 20:192–207 3. Abidi S, Xu Y, Ni J, Wang X, Zhang W (2020) Popularity prediction of movies: from statistical modeling to machine learning techniques. Multim Tools Appl 79:35583–35617 4. Ahmad I, Bakar A, Yaakub M (2020) Movie revenue prediction based on purchase intention mining using YouTube trailer reviews. Inf Process Manage 57:102278 5. Bogaert M, Ballings M, Poel D, Oztekin A (2021) Box office sales and social media: a crossplatform comparison of predictive ability and mechanisms. Decis Support Syst 147:113517 6. Almeida C, Morelli Mendes C (2017) Indu´stria cinematográfica brasileira: uma análise do atual panorama. Passagens 8:173–187 7. Jiang L, Hao Y (2020) Applying machine learning to predict film daily audience data: system and dataset. In: 2020 3rd international conference on artificial intelligence and big data (ICAIBD). pp. 11–16 (2020) 8. Cerdeira P, Mendonça M, Lagowska U (2020) Polıticas pu´blicas orientadas por dados: os caminhos possıveis para governos locais (BID,2020) 9. Ahmed U, Waqas H, Afzal M (2020) Pre-production box-office success quotient fore- casting. Soft Comput 24:6635–6653 10. Bristi W, Zaman Z, Sultana N (2019) Predicting IMDB rating of movies by machine learning techniques. In: 2019 10th international conference on computing, communication and networking technologies (ICCCNT), pp 1–5 11. Darapaneni N, Bellarmine C, Paduri A (2020) Movie success prediction using ML. In: 2020 11th IEEE annual ubiquitous computing, electronics mobile communication conference (UEMCON). pp 0869–0874

Applying Machine Learning Techniques to Box Office Forecasting

199

12. Dhir R, Raj A (2018) Movie success prediction using machine learning algorithms and their comparison. In: 2018 first international conference on secure cyber computing and communication (ICSCCC). pp 385–390 13. Gov.br OCA - Observatório Brasileiro do Cinema e do Audiovisual. https://www.gov.br/anc ine/pt-br/oca,2022,5 14. AdoroCinema.com Adoro Cinema. https://www.adorocinema.com/,2022,5 15. Flach P (2012) Machine learning: the art and science of algorithms that make sense of data. Cambridge University Press, Cambridg 16. C¸ aglıyor S, O¨ ztaysi B, Sezgin S (2010) Forecasting box office performances using machine learning algorithms. In: International conference on intelligent and fuzzy systems, pp 257–264 17. Hastie T, Tibshirani R, Friedman J, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction. Springer, New York. https://doi.org/10.1007/978-0-387-848 58-7 18. LaValley M (2008) Logistic regression. Circulation 117:2395–2399 19. Ke G, Meng Q, Finley T (2017) Lightgbm: a highly efficient gradient boosting decision tree. In: 30th proceedings conference on advances in neural information processing systems 20. Ramchoun H, Ghanou Y, Ettaouil M, Janati Idrissi M (2016) Multilayer perceptron: architecture optimization and training. Int J Interact Multim 4(1):26–30

Attitudes, Use, and Knowledge of ICT and Student Satisfaction as Predictors of the Level of Learning Achievement Ygnacio Tomaylla-Quispe , Mónica Mogrovejo-Chicani , Julia Arpi-Titi , and Olger Gutierrez-Aguilar

Abstract This study examines the factors of use, attitudes, and knowledge that affect satisfaction with ICT in students of a regular basic training educational center with limited technological resources in Peru and South America. Two questionnaires were adapted, one on ICT and the other on satisfaction; a third instrument was the regular student report card. Data collection was carried out through an online form on a sample of 99 respondents. Partial least squares structural equation modeling was used to evaluate the proposed model and its fit. The study results show that both the perceived frequency of use and attitude and knowledge positively influence satisfaction with the help of ICT, although the latter does not significantly explain the school performance levels. Keywords Use · Attitude · Satisfaction · TIC · School performance

1 Introduction More than 9.9 million children and adolescents of school age in Peru were impacted by the COVID-19 pandemic (UNESCO). In March 2020, the Ministry of Education (MINEDU) implemented a distance education service program for preschool, primary, and secondary students called “Aprendo en Casa” that used multiple media Y. Tomaylla-Quispe Universidad Nacional de San Agustín, Arequipa, Perú e-mail: [email protected] M. Mogrovejo-Chicani · J. Arpi-Titi · O. Gutierrez-Aguilar (B) Universidad Católica de Santa María, Arequipa, Perú e-mail: [email protected] M. Mogrovejo-Chicani e-mail: [email protected] J. Arpi-Titi e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_18

201

202

Y. Tomaylla-Quispe et al.

such as the internet, television, and radio. That same year, the MINEDU reported more than 110,000 requests to transfer students from private to public schools. In this kind of environment, self-learning, low costs, adaptability, and flexibility are all things that make distance learning better than traditional classroom learning [1]. Using new information and communication technologies (ICT) simplifies communication between students and teachers through software and other educational applications. There are many antecedents to using ICT in education: primary, secondary, and higher technical and university training. ICTs are a set of devices derived from new tools (software and hardware) that support information and use communication channels that integrate the functions of storage, processing, and digitized data transmission. They have become a fundamental part of our social, educational, and cultural environment, and their mode of use varies according to each individual and the generation to which they belong. Children interact with technology in a context where they have had to live as “digital natives,” and the use of ICT is a routine for their daily activities structured around media such as social networks [2]. The Peruvian case of this study involves a student population at the secondary education level who used a distance learning platform or learning management system (LMS) for asynchronous sessions and the WhatsApp application for synchronous meetings with the teacher. The study’s objective is to determine the student’s attitudes, knowledge, and use of ICT and its implications for school performance.

1.1 Attitude Towards ICT The use of synchronous and asynchronous tools produces effects on the student’s perception and reactions; according to the conception of Krech and Crutchfield (1949), attitude is considered a “stable system of positive or negative evaluations, feelings, emotions, and favorable or unfavorable action tendencies concerning social objects” (p. 33). A proactive attitude towards using and applying ICT for educational purposes allows reflection on practices and activities, but without forgetting that the main protagonist of the learning process is the student [3]. The theoretical background shows that the attitude toward using technology in the classroom is positive, but there is a gap between what people know and what they do, especially when it comes to tools that are clearly geared toward a specific field of research [4]. Some studies have shown that more successful and motivated primary education students have more positive attitudes towards ICT [5].

1.2 Knowledge of ICT Knowledge of virtual technological tools is related to digital and audiovisual literacy, which favors the process of acquiring the knowledge needed to use ICT properly

Attitudes, Use, and Knowledge of ICT and Student Satisfaction …

203

(Fernández, 2013). The effectiveness of ICT tools is evident in the high level of basic skills students achieve in activities involving social networks. The higher the cognitive abilities in ICT management, the greater the use and activities in their social networks [6].

1.3 Use of ICT The main functions of ICT in regular primary education allow the digital literacy of students and teachers to function appropriately in today’s society. They make it possible to access information, communication, management, and data processing in the administrative and tutorial functions of the educational center. On the pedagogical level, its didactic use facilitates the teaching and learning processes through virtual communication through messages, forums, and accessible learning platforms such as Moodle and others. Used as a teaching medium through interactive software, it motivates, informs, trains, guides learning, and objectively evaluates learning [7]. Today, ICTs have a presence in the daily lives of adolescents. However, their excessive use negatively impacts young people’s academic, social, and family environments and could harm their school performance [8]. Greater use of ICT in education promotes greater autonomy in learning [9].

1.4 Satisfaction with ICT The perception of ease, usefulness, and attitudes towards using the computer influence intentions to use ICT and their satisfaction with the help of these tools [10]. They feel satisfied with their service and believe that there is nothing better than technology because it favors their learning strategies and constitutes a positive mediator that correlates the use of ICT with the satisfaction derived from its use [11]. Student satisfaction presents several dimensions related to technology in terms of its usefulness, interest in the subject, and as a tool for achieving learning objectives in the courses. Perceived satisfaction manifests itself in comfort, ease of use, and peer interaction [12].

1.5 Academic Performance The attitudes, knowledge, and use of ICT in the educational field are reflected in the level of learning achievement, which indicates to what extent a person has achieved specific goals and objectives from educational activities and processes in a virtual or face-to-face environment [13]. In quasi-experimental research that created a teaching strategy based on ICT for second-year high school students, studies have shown

204

Y. Tomaylla-Quispe et al.

that ICT helps students do better. In another study, students gained skills in critical thinking, curiosity, and cognitive improvement through theory and practice [14] when they used what they knew about math. Students are more likely to change their routines and habits to get information online if they do well in school and use digital devices daily. This is why a safe and controlled use model should be promoted [16]. On the contrary, training in ICT offered by some study centers has had little impact on student’s academic results. However, they improve when they use ICT in new ways and work together to learn digital skills [15]. Based on the above considerations, the following hypotheses were raised: H1: The perceived attitude positively influences the satisfaction perceived by students with ICT tools. H2: Perceived knowledge positively influences students’ perceived satisfaction with ICT tools. H3: Perceived use positively influences the satisfaction perceived by the students with the ICT tools. H4: The satisfaction perceived by students with ICT tools positively influences school performance.

2 Methodology At the end of 2021, the study was done in the Arequipa region of Peru, South America, in a public administration school for high school students. The total population of the public management educational center was 150 students, distributed across different levels—or cycles—of secondary education. Based on the criteria for the sample, 122 students who could connect to the Internet from a desktop computer or a cell phone were chosen. Students with poor final grades and very little regularity or participation in virtual sessions were excluded. The sample was a non-random selection of both sexes. Its main sociodemographic feature is that its members are in a middle-to-low social class and have some problems with the speed of their internet connection and the most up-to-date ICT. Over the course of four weeks, the students filled out a questionnaire on a Google form. In the end, 99 valid responses were given. Those that were not finished or did not show any difference were thrown out. In the first part of the survey, demographic information about the students is collected. The second part collects information about how they use, feel about, and know about ICT. These factors include the perceived ease of use (USE), perceived knowledge (CONOC) attitude (ACT), and satisfaction as a moderating variable (SAT), and the third part shows us the school performance (REND) of the students through their grade point averages. The items used for this study were adapted from the “Scale for the Study of Attitude, Knowledge, and Use of ICT” (ACUTIC) by Mirete Ruiz [17] and the modifications made by Martinez Daza [4]. Items for technological satisfaction were adapted from the Ramli study [12], and finally, the things on the school performance

Attitudes, Use, and Knowledge of ICT and Student Satisfaction …

205

variable arise from the exams and evaluation rubrics applied in 5 learning units that yielded the students’ average grades.

3 Results The participants’ demographic information says that women make up 59.6% of the data collected, while only 40.4% are men. In addition, most students are between 14 and 16 years old, representing 73.7% of the sample. Regarding the number of students per class and for each level of study, the majority had an average of 20 students per class from the first to the fifth year of secondary education. Regarding the learning modality, 100% of the students received the asynchronous sessions online through a virtual platform on the web. Synchronous communication between students and teachers was developed through the WhatsApp application. Regarding the measurement model’s evaluation, to measure each element’s reliability, its factorial load must be evaluated, which should be equal to or greater than 0.7 for the article to be considered reliable [18]. Similarly, the alpha and Cronbach’s compound values must be similar to or greater than 0.7. Furthermore, the meanvariance extracted (AVE) is defined as the grand mean value of the squared loadings of the items related to the construct and the standard measure to establish convergent validity. A value of 0.5 or greater for the AVE specifies that the construct accounts for more than half of the variance of its elements. As shown in Table 1, the AVE values are more significant than 0.5, which determines and establishes the convergent validity of the constructs (Table 2). To establish discriminant validity based on the Fornell and Larcker criterion [19], it is indicated that the square root of AVE must be greater than the rest of the correlations between the constructs, which is fulfilled in the present study. Concerning the Heterotrait-Monotrait (HTMT) relationship, a value lower than 0.85 [20] must be confirmed. According to Table 3, it can be deduced that the HTMT criterion is met, indicating that discriminant validity is adequately established. Regarding cross loads, the load of each indicator must be greater than the loads of the hands of their corresponding variables. Based on Table 4, we can see that Table 1 Assessment of the measurement model for reflective constructs Cronbach’s alpha

rho_A

Composite Reliability

Average variance extracted (AVE)

Attitude (ATT)

0.913

0.967

0.943

0.848

Knowledge (KNOW)

0.905

0.935

0.940

0.840

School performance (PER)

0.854

0.863

0.911

0.774

Satisfaction (SAT)

0.875

0.905

0.923

0.801

Use (USE)

0.903

0.925

0.939

0.836

206

Y. Tomaylla-Quispe et al.

Table 2 Fornell-Larcker criterion ATT Attitude (ATT) Knowledge (KNOW)

KNOW

PERF

SAT

USE

0.921 0.243

0.916

−0.203

−0.151

0.880

Satisfaction (SAT)

0.371

0.333

−0.177

0.895

Use (USE)

0.182

0.031

−0.020

0.363

0.915

PER

SAT

USE

School performance (PER)

Table 3 Heterotrait Criterion - Monotrait –HTMT ATT

KNOW

Attitude (ATT) Knowledge (KNOW)

0.262

School performance (PER)

0.223

0.184

Satisfaction (SAT)

0.391

0.358

0.205

Use (USE)

0.189

0.055

0.034

0.395

the cross-loading criterion is met except for the variables USE7, (0.442) and PER4, (0.341), which were eliminated from the construct because they had a load of less than 0.7. For the confirmatory factor analysis (CFA) of the variables for their principal component analysis, the Varimax rotation method with Kaiser normalization was used. There were 18 items subjected to the CFA, and according to the load and saturation indices, some adjustments were made to the model where USE7 (0.442) and PER4 (0.341) were eliminated for having a load lower than 0.70, finally leaving 16 items. The SPSS 23 Amos software was used to perform the goodness-of-fit indices to evaluate the model’s fit. According to the literature on the subject, the following values are indicated for a good fit. The Tucker-Lewis index “TLI” measures the comparative or incremental fit; this index tends to 1 for models with a perfect fit, considering values higher than 0.90 [21] to be acceptable; the Comparative Fit Index “CFI” (≥0.95) [22] and root mean square error of approximation “RMSEA” this is an indicator of absolute fit (0,05

108.9

The discrepancy between χ2 and degrees of freedom; (CMIN/DF)

RS Reputation in social networks

-0.079

-0.069

0.119

0.664

0.253

H3 NE Neuroticism->EX Extraversion

0.179

0.182

0.112

1.597

0.055

H4 NE Neuroticism->RS Reputation in social networks

0.418

0.419

0.108

3.864

0.000

H5 PC Creative thinking->EX Extraversion

0.104

0.106

0.096

1.083

0.139

H6 PC Creative thinking->RS Reputation in social networks

0.198

0.195

0.118

1.679

0.047

H7 RE Responsibility->EX Extraversion

0.307

0.298

0.097

3.159

0.001

H8 RE Responsibility->RS Reputation in social networks

0.202

0.201

0.113

1.796

0.036

H9 RS Reputation in social networks- >EX Extraversion

0.110

0.101

0.087

1.273

0.101

4 Conclusions According to the results obtained according to the tests of the proposed structural model, the results indicate that there are influential factors in extraversion in university students of the professional career of advertising and multimedia technologies, in the idea of their reputation in social networks as a variable. The principles of mediation are the most influential exogenous variables: responsibility and openness to experience. The variables openness to experience and responsibility positively influence extraversion; while neuroticism, creative thinking and responsibility positively influence reputation in social networks. Likewise, neuroticism, creative thinking,

Influential Factors in Extraversion Mediated by Reputation in Social …

221

and the idea of reputation in social networks would not influence extraversion. In the same way, openness to experience would not influence reputation in social networks.

References 1. Gutierrez-Aguilar O, Escobedo-Maita P, Calliñaupa-Quispe G, Vargas-Gonzales JC, TorresHuillca A (2022) The use of social networks, usefulness and ease of use, enjoyment through positive emotions and their influence on school satisfaction mediated by school achievement. In: Iberian conference on information systems and technologies, CISTI. https://doi.org/10. 23919/CISTI54924.2022.9820347 2. Gutierrez-Aguilar O, Torres-Palomino D, Gomez-Zanabria A et al. (2022) Personality and creative thinking in artists and the idea of their reputation on Social media. In: Iberian conference on information systems and technologies, CISTI 2022. https://doi.org/10.23919/CISTI54924. 2022.9820283 3. Laros JA, Peres AJdS, Andrade JMd, Passos MFD: Validity evidence of two short scales measuring the Big Five personality factors. Psicologia: Reflexao e Critica 31(1). https://doi. org/10.1186/s41155-018-0111-2 4. Gavrilescu M, Vizireanu N (2018) Predicting the Big Five personality traits from handwriting. Eurasip J Image Video Process 2018(1):https://doi.org/10.1186/s13640-018-0297-3 5. Enrique A (2004) Neuroticismo, extraversión y estilo atribucional en veteranos de guerra: una aproximación desde el estrés postraumático. Interdisciplinaria 21(2):213–246 6. Schmidt V (2010) Las bases biológicas del neuroticismo y la extraversión, ¿por qué nos comportamos como lo hacemos? PSIENCIA. Revista Latinoamericana de Ciencia Psicológica 2(1):20–25 7. Kaufman JC, Pumaccahua TT, Holt RE (2013) Personality and creativity in realistic, investigative, artistic, social, and enterprising college majors. Personal Individ Differ 54(8):913–917. https://doi.org/10.1016/j.paid.2013.01.013 8. Lebuda I, Karwowski M, Galang AJR, Szumski G, Firkowska-Mankiewicz A (2021) Personality predictors of creative achievement and lawbreaking behavior. Curr Psychol 40(8):3629– 3638. https://doi.org/10.1007/s12144-019-00306-w 9. Akpur U (2020) Critical, reflective, creative thinking and their reflections on academic achievement. Think Skills Creat 37:1–8. https://doi.org/10.1016/j.tsc.2020.100683 10. Gube M, Lajoie S (2020) Adaptive expertise and creative thinking: a synthetic review and implications for practice. Think Skills Creat 35:1–14. https://doi.org/10.1016/j.tsc.2020. 100630 11. Mumford MD, McIntosh T (2017) Creative thinking processes: the past and the future. J Creat Behav 51(4):317–322. https://doi.org/10.1002/jocb.197 12. Xie H, Beaty R, Jahanikia S, Geniesse C, Sonalkar N, Saggar M (2021) Spontaneous and deliberate modes of creativity: multitask eigen-connectivity analysis captures latent cognitive modes during creative thinking. Neuroimage 243:1–11. https://doi.org/10.1016/j.neuroimage. 2021.118531 13. Kaplan Sayi A, Akgul S (2021) Psychometric properties of guiding creative thinking competency scale. Think Skills Creat 40:1–11. https://doi.org/10.1016/j.tsc.2021.100811 14. Redifer JL, Bae CL, Zhao Q (2021) Self-efficacy and performance feedback: impacts on cognitive load during creative thinking. Learn Instr 71:1–11. https://doi.org/10.1016/j.learninstruc. 2020.101395 15. Parra González E, Segura Robles A, Romero García C (2020) Análisis del pensamiento creativo y niveles de activación del alumno tras una experiencia de gamificación. Educar 56(2):475–489. https://doi.org/10.5565/rev/educar.1104 16. Suganda E, Latifah S, Irwandani SPM, Rahmayanti H, Ichsan IZ, Mehadi Rahman M (2021) STEAM and Environment on students’ creative-thinking skills: a meta-analysis study. J. Phys. Conf. Ser.1796(1):1–10. https://doi.org/10.1088/1742-6596/1796/1/012101

222

O. Gutierrez-Aguilar et al.

17. Suzan Ç (2018) Creative and critical reflective thinking (C & Crt) skill during connectivist learning process learning spaces: change agents at the university. Revista CIDUI 1:1–13 18. Romo M, Sanchez Ruiz MJ, Alfonso Benlliure V (2017) Creatividad y personalidad a través de dominios: una revisión crítica. Anuario de Psicologia 47(2):57–69. https://doi.org/10.1016/ j.anpsic.2017.04.003 19. Santamaría H, Sánchez R (2012) Creatividad y rasgos de personalidad en estudiantes universitarios: estudio transversal de asociación. Revista Colombiana de Psiquiatría 41(2):284–298. https://doi.org/10.1016/s0034-7450(14)60005-6 20. Salvador BM-M, Flor M-M, Juárez L (2020) Internet y redes sociales en la construcción de la reputación digital de la organización del sector turístico. Espacios 41(14) 21. Marques A (2016) Creación de reputación en la red: una visión de las estrategias de los usuarios en las redes sociales. Opción 32(8):105–119 22. Barbery-Montoya D, Candell-Dávila E (2021) Acciones del marketing para la promoción y creación de marca de artistas ecuatorianos. Acad (Asunción) 43–58 23. Pérez M (2012) Redes sociales, mecanismos generadores de reputación organizacional para las PyMEs. Universidad Empresa 14(22):131–149 24. Szwajca D (2017) The role of social media in corporate reputation management-the results of the polish enterprises. Found Manage 9:161–174. https://doi.org/10.1515/fman-2017-0013 25. Castro-Solano A, Casullo MM (2001) Rasgos de personalidad, bienestar psicológico y rendimiento académico en adolescentes argentinos. Interdisciplinaria 18(1):65–85 26. Hosen M, Ogbeibu S, Giridharan B, Cham T-H, Lim WM, Paul J (2021) Individual motivation and social media influence on student knowledge sharing and learning performance: evidence from an emerging economy. Comput Educ 104262 27. Hair Jr JF, Hult, GTM, Ringle C, Sarstedt M (2016) A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications 28. Bagozzi RP, Yi Y (1988) On the evaluation of structural equation models. J Acad Mark Sci 16(1):74–94 29. Hair JF, Sarstedt M, Pieper TM, Ringle CM (2012) The use of partial least squares structural equation modeling in strategic management research: a review of past practices and recommendations for future applications. Long Range Plan 45(5–6):320–340 30. Dijkstra TK, Henseler J (2015) Consistent partial least squares path modeling. MIS Q 39(2) 31. Fornell C, Larcker DF (1981) Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 18(1):39–50 32. Henseler J, Ringle CM, Sarstedt M (2015) A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 43(1):115–135. https://doi. org/10.1007/s11747-014-0403-8 33. Franke G, Sarstedt M (2019) Heuristics versus statistics in discriminant validity testing: a comparison of four procedures. Internet Res 29 34. Henseler J (2017) Partial least squares path modeling. In: Advanced methods for modeling markets. Springer, Cham, pp 361–381

Machine Learning for the Strengthening of Ecuadorian Sign Language Communication Jeneffer Barberán-Moreira , Miguel Mantuano-Casual , Willian Ocampo-Pazos , Angel Sabando-García , and Fausto Orozco-Iguasnia

Abstract Currently, communication between students with hearing disabilities and other actors that make up an educational establishment, has a gap despite the existence of technological resources. The objective of the study is to provide a comprehensive vision on machine learning to strengthen the communication of Ecuadorian sign languages within a career. This study presents an investigation how machine learning through the Predicate Learning algorithm can be applied to relate a word introduced in a chat, send the corresponding pictogram in Ecuadorian sign language instead of the word, this way bridges the gap in communication between listeners and people with educational disabilities. The results show that machine learning significantly influences Ecuadorian Sign Language communication. Keywords Machine learning · Predicate learning · Hearing disability · Ecuadorian sign language

J. Barberán-Moreira (B) · M. Mantuano-Casual · W. Ocampo-Pazos · A. Sabando-García · F. Orozco-Iguasnia Pontificia Universidad Católica del Ecuador Sede Santo Domingo, Santo Domingo, CP 230203, Santo Domingo de los Tsáchilas, Ecuador e-mail: [email protected] M. Mantuano-Casual e-mail: [email protected] W. Ocampo-Pazos e-mail: [email protected] A. Sabando-García e-mail: [email protected] F. Orozco-Iguasnia e-mail: [email protected] J. Barberán-Moreira · M. Mantuano-Casual Instituto Superior Tecnológico Tsa’chila, Santo Domingo, CP 230109, Santo Domingo de los Tsáchilas, Ecuador © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_20

223

224

J. Barberán-Moreira et al.

1 Introduction 1.1 Background Today, disability is understood “as that condition under which certain people have some physical, mental, intellectual or sensory deficiencies that make it impossible to interact with various routine activities” [9], this condition cannot be alien to new technologies [10]. Worldwide, new projects have been developed such as -Háblalowhich shows the main market niches for people with hearing disabilities, improving their quality of life with applications of this type, since it does not require an internet connection [1]. At the University of Cauca (Colombia), Cano, Muñoz, Collazos & Bustos [3], made a mobile application for learning literacy with FitzGeralt for children with hearing disabilities. Among their results, they managed to create an inclusive educational tool, through obtaining concepts and sentence structure, as well as adapting the level of learning of the game based on the child’s captured attributes. On the other hand, Zamora, Salamanca & Cañon [10], carried out an inclusive application prototype in order to teach the Colombian dactylological alphabet through mobile devices, allowing a person with hearing disabilities to dynamically interact the representation of oral language with dactylological language. Also, Mendoza, Salazar, Del Carmen & Herrera [7], at the Technological University of the Huasteca Hidalguense, they developed an alternative mobile application that improves the communication of people with hearing and speech disabilities. Among their results, they obtained the learning of the letters of the alphabet through images, as well as expressions and phrases through games and playful activities. In addition, the possibility for a hearing impaired person to write a sentence and obtain the audio through the application. Currently, Information and Communication Technologies (ICT), do not have as a central goal to offer accessibility to people with disabilities, however, there are tools focused on improving communication that gradually include more components, such as apps and projects that strengthen social integration [6]. After the on-site visit, it was possible to see teachers with little knowledge of Ecuadorian sign language, which makes communication difficult in the subjects they teach. In addition, the interpreter has basic notions about technical terminology in the aforementioned career, creating a gap in the learning process of students with hearing disabilities. Finally, the entire educational community of the career does not achieve fluid communication, because there is an Ecuadorian sign language interpreter for all subjects, consequently, they cannot transmit their messages in an understandable and timely manner. After the analysis based on the problem raised, it was possible to demonstrate a limited communication between students with hearing disabilities and other actors that make up the educational community of the Gastronomy career of the Instituto Superior Tecnológico Calazacón.

Machine Learning for the Strengthening of Ecuadorian Sign Language …

225

This research focused on the development of a mobile application based on machine learning and its adaptation to the pictogram technique, to strengthen communication between students with hearing disabilities and other actors that make up the academic community of the Instituto Superior Tecnológico Calazacón through a survey and interview to determine the scope of the investigation.

2 Methodology The methodology used was the mixed one, which contains the qualitative approach through data collection through the interview conducted with the Ecuadorian sign language interpreter and the teachers who teach classes to students with hearing disabilities within the Higher Technology career. in Gastronomy. The quantitative approach is reflected in the tabulation of the data collected in the survey of the educational community. The survey allowed the collection of data through strategic questions that determine the scope of the investigation. This technique included a structured questionnaire addressed to the sample of 163 (160 listeners and 3 people with hearing disabilities) who belong to the Higher Technology in Gastronomy career at the Calazacón Higher Technological Institute. The interview is a means of interpersonal communication between the researcher, teachers and interpreter in order to have answers to the questions raised about the proposed problem. It consisted of a bank of open questions, which served to obtain data that supported the development of the mobile application. Statistical analysis was used to process all the data collected through the survey and transform them into numerical data, reflected in graphs and Chi square cross analysis to validate the hypothesis, for which the SPSS tool was used, which allows visualizing the correlation between the indicators.

2.1 Communication Needs of Students with Hearing Disabilities Before carrying out the surveys and interviews, the validation of the information collection instruments was carried out by experts in the area of research, statistics and Ecuadorian sign language. The interview was applied to the Ecuadorian sign language interpreter, who provided substantial information for the knowledge of the communication process. The interviews were applied to teachers who teach classes to students with hearing disabilities, it was considered to do several interviews because it is important to have the point of view and experience of the teaching staff.

226

J. Barberán-Moreira et al.

The educational community of the Gastronomy career of the Instituto Superior Tecnológico Calazacón, having a confidence level of 97.5% and an acceptable limit of sampling error of 5%, a sample of 163 people was obtained, where 160 are listeners and the remaining 3 are students with hearing disability.

2.2 Pictographic System for Students with Hearing Disabilities The SPC system (Pictographic Symbols for Communication) is an aid that requires technical and external resources to carry out the communicative act, it allows intuitive communication through pictographic symbols, it does not have syntactic or grammatical rules. It is used by people who cannot speak, read, write or show difficulty in doing so. From Aymerich’s point of view [2], pictograms are signs or drawings, it is a type of writing that uses graphic resources to symbolize a phrase, sentence or paragraph, explain a fact or any other form of written communication. In addition, they are perceptible, simple and permanent when it comes to understanding the idea that you want to make known. In the process of agreeing on signs, the interpreter and the student with hearing disabilities participate, who identify the terms within the Ecuadorian sign language dictionary “Gabriel Román”, if the sign does not exist, it is created based on synonyms or characteristics of the term. new, as shown in Fig. 1. Through the process of agreeing on signs between the interpreter and the hearingimpaired student, signs were created for terms that did not exist in the Ecuadorian sign language dictionary “Gabriel Román” and were reflected in GIF animations, the terms created are in relation to the technical subjects taught in the semester that are found by people with hearing disabilities. The developed pictograms are: Ecuadorian cuisine, bar and restaurant techniques, high-volume cuisine, the alphabet, general terms used in the classroom.

2.3 Development of the Mobile Application For the development of the mobile application, the Java programming language was used, which is object-oriented, allowing the continuity and updating of the project without having to start from scratch, this is because the objects keep the code organized and easy to modify when necessary. necessary. The aforementioned agrees with Durán et al. [4], who emphasize that Java is a simple, powerful and adapted language for programming applications on the network. Similarly Garrido [5], mentions that Java is very flexible because it facilitates the software life cycle, from analysis, design, implementation to maintenance.

Machine Learning for the Strengthening of Ecuadorian Sign Language …

227

Fig. 1 Diagram of the process of signing signs

Oracle XE was installed first, which is the database engine, then the Payara Server Community (open source application server derived from GlassFish Server Open Source Edition) was configured. The database connection to the Payara server required the Oracle XE driver which is ojdbc8-12.2.0.1. The persistence framework that was used is the JPA that allowed working with persistent entities connected to a database. The JPA configuration file is persistence.xml, where the persistence unit (PU) is inclusivePU of type JTA, that is to say that the PU is in charge of managing the transactions on the server side, isolating most of the programmer from opening or closing the transactions. transactions, for this the Pool JDBC connection was linked with InclusivaDS. Based on the technical histories, the software architecture defined the ModelView-Controller architectural pattern. • Model: this is where the system data is stored, the Oracle XE database was used. • View: allows to interpret the data for the user. • Controller: Its purpose is to receive the data from the model and present it in view for the user.

228

J. Barberán-Moreira et al.

Table 1 Parameterization Model

View

Controller

Categorias.java

index.xhtml

UserLoginController.java

Chat.java

login.xhtml

ContactosController.java

Contactos.java

cambiarclave.xhtml

UsuarioController.java

Destacados.java

contacto.xhtml

CategoriasController.java

Paises.java

mensajes.xhtml

ChatController.java

Pictogramas.java

contactos-edit.xhtml

PictogramasController.java

Roles.java

contactos-form.xhtml

DestacadosController.java

Usuarios.java

registro.xhtml

Agente.java

categorias.xhtml chat.xhtml destacados.xhtml micuenta.xhtml categorias-edit.xhtml categorias-form.xhtml chat-move.xhtml pictogramas-edit.xhtml pictogramas-form.xhtml pictogramas-list.xhtml claves.xhtml chat.xhtml chat-audio.xhtml chat-recognition.xhtml Note: Model, view and controllers obtained from Netbeans

For the functionality of the login, registration, contacts, chat, categories, pictograms, machine learning, the parameterization was created where the models, view and controllers of the different compendiums are observed in the source code of the mobile app and it is shown in Table 1.

2.4 Application with Machine Learning The Predicate Learning algorithm is a neural architecture that describes or creates an object from semantic features, these features are compared and contrasted, then meaningful relationships are built between the created objects. The algorithm places the object that represents the image in a predicate and the relative position of each feature in another, in this way at the end it will have the image and the features that represent it. After the creation of several images with the predicate algorithm, they

Machine Learning for the Strengthening of Ecuadorian Sign Language …

229

can be compared and identified between them, for which the final predicates of the images are compared and the similarities between them are obtained. In Fig. 2, the sendGlobal public method follows the logged in user and destination as parameters, if there is any inconvenience in the method, an IOException is made, it performs an if to know if the globalMessage variable is different from Null, if it is equal to Null it goes through else and warns with a message that there must be text before sending, if it finds text it makes a query if there is an asterisk (*), a character that activates machine learning. If you enter through the condition, an agent type object is created that defines the words that are being written in the chat, it has a spellWord method that separates what is written with the pattern (*). A list of string-type images is created that searches the pictograms for words related to the pattern created by the agent. A while was created that relates the keywords with the pictograms available in the DB, until all of them have been found. This data is sent in gll format which is related to gallery. In order not to overload the server, the globalMessage variable is set to Null. In Fig. 2, the generic agent.java class uses the ArrayList, Arrays and List packages, an array was created where some general union words were determined, an empty constructor was created to instantiate the class and not be sending parameters. In addition, the spellWord method was created, which is of type List String (it returns a list of String), it creates a list of words of type arrayList. If the text string is not null and has a size greater than zero, it separates it with the word.split function using a for with the condition that when it finds a space, it enters the statement; this is important to separate machine learning from the rest of the words and not to saturate the server,

Fig. 2 Predicate Learning Algorithm

230

J. Barberán-Moreira et al.

Fig. 3 Pictogram in Ecuadorian Sign Language

a predicate line was created (within the context of machine learning it is called the attribute). In Fig. 3, it is shown how a word introduced in a chat is related and the corresponding pictogram in Ecuadorian sign language is sent instead of the word, in this way the predicate learning algorithm works.

3 Results 3.1 Validation of the Proposal The survey was applied in two stages (pre and post test) to 160 listeners of the Gastronomy career of the Instituto Superior Tecnológico Calazacón. In Table 2, you can see the pre and post test of 160 listeners. With the data collected, it was observed that a higher percentage of people began to use messaging applications that allowed them to save GIFs or emoticons related to sign language. In addition, the possibility of being able to communicate through sign language in real time increased. Similarly, the possibility of using applications that suggest predictions was extended. On the other hand, a higher percentage of users have Android as their operating system, being the most common software among them. Finally, through the use of the application, listeners can communicate with people with hearing disabilities. Also, the survey was applied in two stages (pre and post test) to the 3 students with hearing disabilities of the Gastronomy career of the Instituto Superior Tecnológico Calazacón. It should be noted that each question was adapted in Ecuadorian sign language in video format and is detailed below. Table 3 shows the pre and post test for the 3 students with hearing disabilities. With the data collected, it was observed that students with hearing disabilities use messaging applications that allow them to save GIFs or emoticons related to sign language. In addition, there is an increase in the use of applications related to LSEC (Ecuadorian sign language communication). On the other hand, the number of daily use applications in the personal, educational and work spheres was determined.

¿How likely is it that you will be able to communicate in real time through sign language with a hearing person?

¿How often do you use apps that suggest predictions of previously used words or pictograms (emoticons or GIFs)?

2

3

Frequently

17.50

Almost always

10.63

25.63

33.13

30.00

Sometimes

23.13

2

25.63

18.13

1

Sometimes

Frequently

0

Pre test

¿How often do you Almost always use messaging apps 10.63 that save emoticons or GIFs related to sign language and automatically suggest them again?

1

Scale and %

Questions

N°

25.00

Seldom

9.38

3

17.50

Seldom

Table 2. Results of the pre and post test to the hearing educational community

16.88

Rarely

8.75

4

28.13

Rarely

58.13

Almost always

0.00

0

87.50

Almost always

Post test

40.63

Frequently

0.00

1

11.88

Frequently

1.25

Sometimes

0.00

2

0.63

Sometimes

0.00

Seldom

0.00

3

0.00

Seldom

(continued)

0.00

Rarely

100.00

4

0.00

Rarely

Machine Learning for the Strengthening of Ecuadorian Sign Language … 231

¿What operating Android system do you have 88.75 on your mobile?

¿What is the 0 probability of being 23.75 able to convey an idea to a hearing person if a sign language interpreter is not found and/or the use of an inclusive application is not available?

7

2 20.00

27.50

0.63

1

KaiOS

10.00

31.25

4–6

iOS

13.75

6

1–3

9.41

22.58

1.25

5

Games

Utilities

0

¿How many apps do you currently have on your phone?

4

Scale and %

Questions

¿What types of Social networks applications do you 41.13 currently have on your digital device(s) (computers, tablets, phones, etc.)? Check all that apply

N°

Table 2. (continued)

10.00

3

0.63

Windows Phone

19.38

7–9

9.95

Sports

18.75

4

34.38

10 or more

16.94

Educational

0.00

0

20.11

Social networks

0.00

1

15.24

Utilities

0.00

2

13.27

Games

0.00

3

17.61

Sports

(continued)

100.00

4

33.70

Educational

232 J. Barberán-Moreira et al.

Questions

¿How many applications on your cell phone allow you to understand and use all its features?

¿Do you think that messaging applications on your smartphone help people with hearing disabilities to communicate through sign language?

N°

8

9

Table 2. (continued)

Much

25.00

11.25

34.38

Too

1–3

45.63

Scale and %

0

24.38

Enough

12.50

4–6

33.13

Little bit

4.38

7–9

6.25

Any

3.13

10 or more

83.13

Too

0.00

0

16.88

Much

100.00

1–3

0.00

Enough

0.00

4–6

0.00

Little bit

0.00

7–9

0.00

Any

0.00

10 or more

Machine Learning for the Strengthening of Ecuadorian Sign Language … 233

How likely is 0 it that you will 100.00 be able to communicate in real time through sign language with a hearing person?

2 0.00

0.00

0.00

Sometimes

1

0.00

0.00

2

Frequently

Almost always

How often do you use messaging apps that save emoticons or GIFs related to sign language and automatically suggest them again?

1

Pre test

Questions

N°

0.00

3

100.00

Seldom

0.00

4

0.00

Rarely

Table 3 Results of the pre and post test for the hearing-impaired educational community

0.00

0

0.00

Almost always

Post test

0.00

1

100.00

Frequently

0.00

2

0.00

Sometimes

100,00

3

0.00

Seldom

(continued)

0.00

4

0.00

Rarely

234 J. Barberán-Moreira et al.

What types of Social apps do you networks currently have 33.33 on your digital device(s) (computers, tablets, phones, etc.)?

How many 0 apps do you 0.00 currently have on your phone?

5

4–6 100.00

0.00

33.33

33.33

1–3

Games

0.00%

Sometimes

Utilities

0.00%

0.00%

4

Frequently

Almost always

How often do you use apps that suggest predictions of previously used words or pictograms (emoticons or GIFs)?

3

Pre test

Questions

N°

Table 3 (continued)

0.00

7–9

0.00

Sports

100.00%

Seldom

0.00

10 or more

0.00

Educational

0.00

Rarely

0.00

0

25.00

Social networks

0.00

Almost always

Post test

0.00

1–3

25.00

Utilities

100.00

Frequently

0.00

4–6

25.00

Games

0.00

Sometimes

100.00

7–9

0.00

Sports

0.00

Seldom

(continued)

0.00

10 or more

25.00

Educational

0.00

Rarely

Machine Learning for the Strengthening of Ecuadorian Sign Language … 235

0.00

0.00

4–6

0.00

0.00

1–3

How many applications on your cell phone allow you to understand and use all its features?

8

2

1

0.00

0.00

100.00

What is the 0 probability of 100.00 being able to convey an idea to a hearing person if a sign language interpreter is not found and/ or the use of an inclusive application is not available?

7

KaiOS

iOS

0

¿What Android operating system do you 100.00 have on your mobile?

6

Pre test

Questions

N°

Table 3 (continued)

0.00

7–9

0.00

3

0.00

Windows Phone

0.00

10 or more

0.00

4

0.00

0

0.00

0

100.00

Android

Post test

100.00

1–3

0.00

1

0.00

iOS

0.00

4–6

0.00

2

0.00

KaiOS

0.00

7–9

66.67

3

0.00

Windows Phone

(continued)

0.00

10 or more

33.33

4

236 J. Barberán-Moreira et al.

Do you think Too that messaging 0,00 applications on your smartphone help people with hearing disabilities to communicate through sign language?

9

Pre test

Questions

N°

Table 3 (continued) Enough 0.00

Much

0.00

100.00

Little bit 0.00

Any 66.67

Too

Post test 33.33

Much 0.00

Enough 0.00

Little bit 0.00

Any

Machine Learning for the Strengthening of Ecuadorian Sign Language … 237

238

J. Barberán-Moreira et al.

Also, it was found that all students have Android. In the same way, a greater use of the application by people with hearing disabilities was determined to be able to communicate with a listener without the intervention of the interpreter. Finally, it was concluded that, with the absolute use of the application, the number of applications increased satisfactorily, where users with hearing disabilities achieve accessibility to all its functions.

3.2 Validation of Hypotheses Once the exploratory analysis was carried out, a Chi square cross analysis was carried out for the indicators of the study variables of the mobile application with machine learning (Table 4). This bivariate analysis allows to visualize the correlation between the indicators, in which it was observed: frequency of use of applications, probability of communicating in real time, frequency of predictions of pictograms, probability of transmitting an idea, the number of applications and messaging applications. in real time they help people with disabilities (p < 0.05). Therefore, the alternative hypothesis (H1) was validated: the mobile application with machine learning significantly influences the communication of Ecuadorian sign language, in the Gastronomy career of the Instituto Superior Tecnológico Calazacón. Table 4 Cross analysis of the indicators depending on the application Questions

X2

gl P

How often do you use messaging apps that save emoticons or GIFs related 183,213 1 to sign language and automatically suggest them again?

0,000

How likely is it that you will be able to communicate in real time through 249,224 1 sign language with a deaf person?

0,000

How often do you use apps that suggest predictions of previously used words or pictograms (emoticons or GIFs)?

154,703 1

0,000

What is the probability of being able to convey an idea to a 219,195 1 hearing-impaired student if the sign language interpreter and/or the use of an educational application are not available?

0,000

How many applications on your cell phone allow people with hearing disabilities to have full access to their features??

4,371 1

0,037

176,340 1

0,000

¿Do you think that messaging applications on your smartphone help people with hearing impairments to communicate using sign language?

Note: Data collected from the community of the Instituto Superior Tecnológico Calazacón; X2 = Chi squared, gl = degrees of freedom, p = significance level.

Machine Learning for the Strengthening of Ecuadorian Sign Language …

239

4 Discussion According to the information obtained from the survey to the entire educational community of the Gastronomy career of the ISTC, in addition to an interview with the interpreter and the teachers who teach classes to students with HD, the communication needs of students with HD were known. hearing impairment to achieve better interaction with peers, teachers and authorities. Those surveyed considered that the development of a messaging application using text, voice and pictograms would improve the communication of people with hearing disabilities with other actors in the educational community. The aforementioned is aligned with what was stated by Mendoza et al. [7], who consider that mobile devices are the most used access channel today for educational purposes, for which they developed a tool that promotes inclusion and facilitates interaction between listeners and people with hearing disabilities. The “SignIES” application was developed with the functionalities that respond to the needs of all the actors of the educational community, unsupervised machine learning was used, through the Predicate Learning algorithm for voice recognition and the sending of GIFs according to the match of characters stored in the application database. The algorithm places the object that represents the image in a predicate and in another the relative position of each character, in this way at the end it will have the image and the characters that represent it. The above aligns with Murphy [8], who mentions that unsupervised learning performs a discovery without any input and the training data is unclassified. The Predicate Learning algorithm allows to describe or create an object from semantic characteristics, they are compared, contrasted and finally significant relationships are built between the created objects. For the development of the mobile application, the Java programming language was used, which is object-oriented, allowing the continuity and updating of the project without having to start from scratch, this is because the objects keep the code organized and easy to modify when necessary. necessary. The aforementioned agrees with Duran et al. [4], who emphasize that Java is a simple, powerful and adapted language for programming web applications. Similarly Garrido [5] mentions that Java is very flexible because it facilitates the software life cycle, from analysis, design, implementation to maintenance.

5 Conclusion In conclusion, it was possible to know the communicative needs of students with hearing disabilities through surveys and interviews, in addition to the activity diagram of a regular class. In the process of communicative needs in the classroom, inconveniences were noted at the time of developing it, since students with hearing disabilities

240

J. Barberán-Moreira et al.

are unable to receive 100% of the information; There are technical terms or phrases that are not found in the Ecuadorian sign language dictionary “Gabriel Román”. Finally, the implementation of the mobile application “SignIES” was well received by the entire educational community, allowing students with hearing disabilities to strengthen communication with others and vice versa. Among its advantages is its adaptable and user-friendly interface, within its modules it has a section of pictograms that are categorized collaboratively, allowing new resources to be uploaded and made available to all application users. Also, the purpose of the Predicate Learning algorithm is to send the GIF associated with the word placed with the special character (*) after performing a character match search. To conclude, it was demonstrated with the data collected in the pre and post test that the application with machine learning significantly influences the communication of Ecuadorian sign language, in the Gastronomy career of the Instituto Superior Tecnológico Calazacón.

References 1. Asteroid Technologies (2020). Hablalo. Obtenido de https://hablaloapp.com/index.html_ 2. Aymerich M (2009) Simbolos, pictogramas y siluetas. Index Book 3. Cano S, Muñoz J, Collazos C, Bustos V (2015) Aplicación móvil para el aprendizaje de la lectoescritura con FitzGerald para Niños con Discapacidad Auditiva. Revista brasileira de informática na educaçâ0 1. https://doi.org/10.5753/cbie.wcbie.2015.240 4. Duran, F., Gutierrez, F., & Pimentel, E.: Programación orientada a objetos con Java. Ediciones Paraninfo (2007). 5. Garrido P (2015) Comenzando a programar con JAVA. Universidad Miguel Hernández de Elche 6. Luna M (2013) Revista digital universitaria. Obtenido de. http://www.revista.unam.mx/vol.14/ num12/art53/ 7. Mendoza L, Salazar H, Del Carmen Y, Herrera S (2018) Aplicaciones móviles alternativas para mejorar la comunicación de personas con discapacidades auditivas y del habla. Tecnologías de la Información ECORFAN 5(15):29. Obtenido de. https://docplayer.es/109927433-Issn-vol umen-5-numero-15-abril-junio-ecorfan.html 8. Murphy K (2012) Machine learning: a probabilistic perspective.Obtenido de, Cambridge. https://ebookcentral.puce.elogim.com 9. Sosa L (2012) Reflexiones sobre la Discapacidad. Dialógica de la Inclusión y Exclusión en las prácticas. Ágora 58 10. Zamora L, Salamanca O, Cañon V (2013): Manos Que Hablan. Prototipo de Aplicación en Android Para el Aprendizaje del Alfabeto Dactilológico Para Colombia. Researchgate, 1. Obtenido de https://www.researchgate.net/publication/275581354_Manos_Que_Hablan_Pro totipo_de_Aplicacion_en_Android_Para_el_Aprendizaje_del_Alfabeto_Dactilologico_Para_ Colombia

Organizational Models and Information Systems

Towards the Evaluation and Continuous Evolution of Open Government Data Portals: A Framework Proposal Castelar Costa Junior, Flavia Cristina Bernardini, Guido Silva Vaz, José Viterbo, Vanessa Nunes, and Claudia Cappelli

Abstract Citizens are gaining broad access to public data sources, made available in Open Government Data Portals (OGDPs). While the number of OGDPs grows over the years, researchers identify recurrent problems in these portals regarding data delivery. We conducted preliminary studies for identifying recurrent issues in OGDPs and we identified that data governance frameworks may be applied for tackling these issues. However, data governance may be difficult to be applied for improving an entire data portal considering all issues at once. So, in this work, we propose a preliminary conceptual framework for analyzing and evolving an OGDP considering (i) the prioritization of items to be evolved in OGDPs; (ii) recurrent issues in OGDPs extracted from literature; (iii) good practices for constructing other similar OGDPs and the analysis of situation; (iv) identification of gaps and (v) improvement proposal based on data governance. We applied our framework for proposing new activities for managing bidding data in an OGDP of the city of Niterói, Brazil. Our findings show how the framework can be used to evaluating and continually evolving OGDPs.

C. C. Junior · F. C. Bernardini · J. Viterbo · V. Nunes (B) Institute of Computing, Fluminense Federal University (UFF), Niterói, RJ, Brazil e-mail: [email protected] C. C. Junior e-mail: [email protected] F. C. Bernardini e-mail: [email protected] J. Viterbo e-mail: [email protected] G. S. Vaz Institute of Science and Technology, UFF, Rio das Ostras, RJ, Brazil e-mail: [email protected] C. Cappelli State University of Rio de Janeiro – UERJ, Rio de Janeiro, RJ, Brazil e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_21

243

244

C. C. Junior et al.

Keywords Open Government Data · Data Governance · Business Process Management · Enterprise Architecture

1 Introduction Governments need to turn their processes, services, technologies, people, and all data manipulated within, into more transparent. In this way, we have seen the fast growth of transparency through Open Government Data Portals (OGDPs) through governmental websites in many countries [1] considering all administrative levels (federal, state, and city), including Brazil. The objective is to promote social participation through the empowerment of citizens, giving them access to public data that are of public relevance. In this context, the act of turning public data open means more than publishing it in an organized way. There are necessary actions to be established, like management and governance processes. It is possible to observe this in some critical domains, such as bidding, contracts and procurement data, as they are directly related to Public Economy and Finance [2]. In Brazil, there is even a specific federal law for legislating government transparency, which states that all data regarding to bidding, contracts, and procurement must be available at all government levels [3]. So, governments have been constantly trying to make data from their bidding processes available to guarantee that citizens have awareness of the rendering of government accounts. OGDPs may be of different types and may have specific issues depending on the type we are tackling [4]. OGDPs that attends the aforementioned Brazilian transparency law lack many important features to guarantee citizen access, understandability and use of information, as pointed out by Machado et al. [5]. These features are related to providing search tools, data files in various formats, data in machinereadable formats, a guarantee of authenticity and integrity, updated data, presence of communication channels and so on. OGDPs managers can adopt several practices inspired by data management and governance good practices, as pointed out by Janssen, Charalabidis, and Zuiderwijk [6]. However, is not a trivial task to use data governance frameworks, as they tackle many and diverse issues in data management. This paper focuses on proposing a preliminary framework for evolving data delivery on a specific OGDP considering a systematic way of (i) prioritizing items to be evolved in the portal; (ii) identifying data governance frameworks best practices for the problems to be tackled; (iii) assessing good and bad practices of providing data in similar OGDPs; (iv) analyzing typical problems in OGDPs from an academic literature perspective; and (v) assessing rules established by legal systems for delivering data to the public. More specifically, our preliminary framework aims to organize the different sources of information (items (ii) to (v)) for iteratively analyzing and evolving OGDPs by public administrators. To this end, we followed the Design Science Research (DSR) paradigm for constructing our framework, which was applied for analyzing and evolving bidding data delivery in an OGDP of the city

Towards the Evaluation and Continuous Evolution of Open Government …

245

of Niterói, Brazil. The goal when applying our framework was to search for problems in delivering bidding data and suggest data governance practices and activities for improving data availability of bidding data. Specifically, we used the following sources of information to apply the preliminary framework in our study: (i) knowledge areas from the DAMA DMBOK data governance framework; (ii) analysis of eight Transparency OGDPs from Brazilian cities focusing on bidding data; and (iii) two previous works conducted by this research group, which indicated the most recurrent OGDP issues and how to tackle them according to different data governance frameworks. This study in the OGDP of Niterói is qualitative, as it aims to discover and refine only the analyzed OGDP through data collection, description and observations. This paper is organized as follows: Sect. 2 presents the material used as theoretical background and our proposed preliminary framework that formalizes the construction and use of a reference model. Section 3 presents a discussion on the use of our framework regarding the analyzis of the three sources of information considered in our framework and presents a discussion on the current situation of the Transparency OGDP of Niterói when delivering bidding data, a gap analysis that took into consideration the recurrent issues raised and their relationship with the knowledge areas of DMBOK and some improvements suggestions for providing bidding data in the OGDP. Section 4 concludes this paper and points out the next steps and future work.

2 Material and Methods The Design Science Research (DSR) paradigm is a qualitative research approach where conducting the design process simultaneously generates knowledge about the method used to design an artifact and the design of the artifact itself [7]. In this research, the artifact is the preliminary framework. So, Sect. 2.1 describes the theoretical reference adopted by this research and who may be benefited from this proposal. Section 2.2 presents the preliminary framework, including the research cycle aiming at organizing and evolving sources of information and diagnosis, the process of building and evolving a reference model and its use to analyze and evolve an OGDP.

2.1 Theoretical Reference Open Data and Open Data Portals: Open Data can be defined as data that can be used freely, reused, and redistributed by anyone subject [8]. Open data initiatives aim to open all non-personal and non-commercial data, especially (but not exclusively) all data collected and processed by government organizations. Open Data Portals are a collection of datasets, which may belong to governments, non-governmental organizations, public institutions, and others that operates in the public sector. Open

246

C. C. Junior et al.

data portals are managed by authorized users who oversee the submission of resources and the completion of metadata fields [9]. They also aim to promote dialogue between society and government and to reason about the most appropriate use of data for a better society. Open Government Data Portals (OGDP): OGDPs are understood as the effort, that can be conducted by governments in all administrative levels, toward the publication and dissemination of public data and information on the Web, allowing for their use and integration. The idea for OGDPs is to publish data in a platform-independent format and made it available to the public, without restrictions that prevent their reuse. Data Governance (DG): It is related to efforts toward establishing structural support, consistency, and balance through data management activities. It helps companies to manage their knowledge of information and answer questions such as (i) What do we need to know about this data?; (ii) What do we need to do with this data?; and (iii) What are the policies and rules that govern this data?. Data governance practices provide a comprehensive approach to managing the creation, improvement, and use of data and information helping the company regarding the efficiency in data management [10]. According to the Data Governance Institute (DGI - https://datago vernance.com/), DG is a decision-making process and responsibility for data-related processes, based on policies, standards, and restrictions. The focus of DG’s activities can start with data privacy and go up to their quality, passing through different areas of the data life cycle. To own a structured and efficient DG, organizations need to define their data management needs, as well as the objectives to be achieved, and then, delimit the scope of action of the DG, regardless of the focus and scope outlined by the organization. There are some commonly DG frameworks such as the Data Management Body of Knowledge Framework (DMBOK) [11], The DGI Data Governance Framework [12], KBF [13], the Notre Dame University framework (NDUF) [14], among others. DMBOK [11] is the data governance framework used to perform our study when applying our framework. It comprises a body of Data Management knowledge developed in 2009 by over 120 professionals worldwide. DAMA (Data Management Association) is an international non-profit organization that sponsors and facilitates the development of the DMBOK. It is also dedicated to promoting concepts and best practices of Data Management and Data Governance. DMBOK provides Data Management’s general overview, through patterns definition, terminologies, and best practices, but without technical details and specific methods. Each knowledge area of the DMBOK defines a context diagram that represents the inputs, outputs, participants, suppliers, and consumers to guarantee its development and management.

Towards the Evaluation and Continuous Evolution of Open Government …

247

2.2 Our Preliminary Framework Our preliminary framework organizes sources of useful information regarding problems and best practices used to feed a reference model whose focus is on the environmental characteristics and requirements defined for an OGDP. The environmental characteristics include the legal aspects that they need to attend to, and the requirements can be defined by both (i) the issues and barriers typically pointed out by literature and (ii) good practices to turn data available by other OGDPs. The framework also presents a mechanism to consume this reference model to: (i) model the current situation of providing data in a specific OGDP; (ii) analyze its data structure and processes; and (iii) use it to model a future situation where weaknesses and threats are solved. The framework is presented in Fig. 1. The four sources of information (on the left of the figure) were elected due to the following premises: • Academic literature is important to consider. So, a (systematic) literature review can be carried out to continuously collect problems and best practices about recurrent and important issues regarding OGDPs; • Data governance frameworks output good practices to be implemented for better managing open data portals in general and, therefore, they must be also useful for the public sector; • There are OGDPs from other (Brazilian) cities that present good practices for data management and governance. The aim is to evaluate other OGDPs to generate a comparison between them to assess common problems, and best practices and to identify standards. The selection of OGDPs may be performed using national and international rankings; and • Legal Orders are mandatory and need to be accounted as requirements that must be addressed by OGDPs, such as the Brazilian Transparency Law [3].

Fig. 1 Proposed Framework for the analyses of Brazilian OGDPs

248

C. C. Junior et al.

On the right size in Fig. 1, we can observe the activities to be executed by the user of the framework. This user must firstly prioritize the items to be improved in the OGDP (for instance, bidding type of data is an item for us) according to public administrators and users of the OGDP. After, the user must model the AS-IS processes of turning data available regarding on specific items of a OGDP. Considering the information sources of the framework, the user must conduct an OGDP Gap Analysis considering (i) the main issues on delivering data in the items of the OGDP under scrutiny; and (ii) good and bad practices of other good OGDPs based on some selection criteria when delivering data in the same selected items. Finally, the user must present TO-BE models for improving data delivery in the OGDP of interest considering data governance best practices for the specific issues. The main idea is iteratively executing these steps for tackling issues of delivering data of specific items previously prioritized. It is worth mentioning that in our study, we considered only biding data due to its importance and it was also reported by the OGDP managers as an important type of data, mandatory by a Brazilian legal order and widely used by the portal users. However, other types of data should be chosen to be the focus of data delivery evolution. The non-exhaustive sources of information must track all best practices to justify the development and evolution of a Reference Model for Data and Information Management and Governance. The aim is to evolve the Reference Model for the entire life cycle of manipulation, management and governance of an OGDP. In this work, we executed only one first step for evolving our reference model: data specification and practices required by an administrative unit responsible for an OGDP for improving it.

3 Applying Our Framework We applied our framework on bidding data publicized by one OGDP of the city of Niterói, presented next. The analyzed OGDP was constructed to the specific Brazilian transparency law. So, in what follows, the Legal Orders of the framework is not discussed in the Reference Model development.

3.1 Development – Constructing Sources of Information Literature Mapping On OGDP Problems: We previously performed in our research group [15] a literature mapping to identify the main drawbacks identified in government open data portals. We analyzed publications published from 2011 to 2016. Among them, we found surveys analyzing open data portals or inspected existing problems and papers presenting how open data portals should be managed and how they should work. We collected the following recurrent problems found in in this research: (i) Difficulty in accessing datasets – it can be represented by the lack of

Towards the Evaluation and Continuous Evolution of Open Government …

249

available descriptions, lack of tags and categories, and the big number of datasets; (ii) Lack of data source traceability – lack of data life cycle registration, responsibility, and authorship; (iii) Data format issues – lack of data standard, machine-readable mechanisms, linked download and data in CSV, TXT, XML, and RDF Format; (iv) Poor metadata – lack of update frequency, update time, and last update. Reference Data Governance Frameworks: Reis, Viterbo, and Bernardini [15] also explored data governance frameworks analyzing how they are aligned with many OGDPs issues and how to face them. DMBOK was chosen as it presents the improvements in data governance to be implemented when delivering data from bidding processes as this is the most cited framework in their literature review. Among the ten knowledge areas of the DMBOK wheel, Data Architecture, Data Quality, and Metadata best practices were used. These three knowledge areas are also directly related to the recurrent problems in OGDPs described in [15]. Analysis of Problems and Best Practices in OGDPs: Since there are more than 5,000 cities in Brazil, an evaluation instrument constructed by the Brazilian federal government to evaluate transparency in cities [16] was used. This instrument assigns a score from 0 to 10 for Brazilian cities according to several criteria, including the availability of bidding data. Eight cities that achieved the highest score (10) in 2021 were chosen to evaluate their OGDPs regarding the quality of the provision of data from their bidding processes. They are: C1 – Mesquita (transparencia.mesquita.rj.gov.br), C2 – São Pedro da Aldeia (transparencia.pmspa.rj.gov.br), C3 – Linhares (linhares-es.portaltp.com.br), C4 – Campo Grande (transparencia campogrande.ms.gov.br), C5 – Londrina (londrina.pr.gov.br/ transparencia), C6 – Recife (transparencia.recife.pe.gov.br/), C7 – João Pessoa (transparencia.joaopessoa.pb.gov.br) and C8 – Sobral (transparencia.sobral.ce.gov.br). In this study, we first inspected if the portals presented the same recurrent issues in OGDPs found in the literature mapping. Next, the previous list of problems were extended by detailing them in conjunction with the selected DMBOK knowledge areas to perform our study in a reasonable period. We identified the following recurrent issues (RI) in the absence of mechanisms to provide them: RI1 - Data descriptions; RI2 - Tags and categories; RI3 - Number of datasets; RI4 - Machine readability; RI5 - Linked download; RI6 - Data in CSV, TXT, XML, RDF Format; RI7 - Data in DOC and XLS Format; RI8 - Update frequency; RI9 - Update time; RI10 - Last update; RI11 - Data use License; RI12 - Authority and Responsibility; and RI13 Data Management System. Table 1 presents the results of our analysis of the OGDPs of the eight cities. The ‘e’ indicates the issue ‘e’xists in the OGDP of the city. With this analysis, we could observe common issues and the OGDPs with more issues to tackle. In this way, João Pessoa (C7) was the OGDP with less issues. R3 and R13 are issues present in all of the 8 analyzed OGDPs. So, this comparison helped us to set a prioritization contact list. OGDP managers of João Pessoa (C7) were contacted to understand their best practices for the issues they do not have. We found out that they invested on a bigger team for managing the portal, but they do not have a proper methodology based on data governance for improving data availability.

250

C. C. Junior et al.

Table 1 Evaluation of Bidding Data in the eight evaluated Brazilian OGDPs Recurrent Issues (RI)

City OGDPs C1

C2

C3

C4

C5

C6

C7

C8

RI1

e

-

-

-

-

e

-

-

RI2

e

e

-

e

-

e

-

e

RI3

e

e

e

e

e

e

e

e

RI4

-

e

e

-

e

e

-

e

RI5

-

e

e

-

e

e

-

e

RI6

-

-

e

-

e

e

-

e

RI7

e

-

-

-

-

e

-

e

RI8

e

-

-

e

-

e

-

-

RI9

e

-

-

e

-

e

-

-

RI10

e

e

-

e

e

e

-

e

RI11

-

-

-

e

e

e

-

-

RI12

e

e

-

e

e

e

-

-

RI13

e

e

e

e

e

e

e

e

It is worth observing that all these non-exhaustive three sources are to be used as a foundation to guide the definition of the Reference Model for Data and Information Management and Governance. For now, these recurrent issues were only used to explore the quality of providing bidding data in the analyzed OGDP of Niterói.

3.2 Tasks Executed by User AS-IS Model and OGDP Gap Analysis: The current situation on how bidding data is delivered in the analyzed OGDP (AS-IS) was registered using Business Processes Modeling Notation (BPMN – https://www.bpmn.org/) for modeling work activities. Dumas et al. [17] recommend the following actions: (i) document analysis – there is often available documentation related to existing business processes; (ii) observation – using observation as a discovery method, we directly follow individual cases processes to gain an understanding of how the process works; (iii) interview-based discovery – interviewing domain experts to inquire about how a process is performed; and (iv) workshop – experts execute their work to be analyzed in real-time. So, we chose to analyze the portal itself and to perform interviews with some stakeholders related to the bidding data. Therefore, a questionnaire was created to guide this interview with those responsible for the Transparency Portal. The employees who participated in the interview were: the manager responsible for the Portal team, the IT manager responsible for the portal, and an employee who publishes the bids. The main questions that guided this interview were: (i) How the management of data is

Towards the Evaluation and Continuous Evolution of Open Government …

251

realized in the Transparency Portal?; (ii) Are the main bottlenecks related to technological or procedural issues?; (iii) How is data processed?; (iv) How is data published in the Portal?; and (v) How is data quality management performed on the Portal?. After that, we analyzed manuals describing the team’s work and extra meetings were conducted where the data flow for feeding the OGDP was shown. The understanding of the AS-IS bidding information architecture was possible from the interviews, observations and those materials collected. The data flow considering the base systems where data is collected is done by a system where the bids are registered. We also discovered that an ERP of the local government stores the bidding data for other purposes. The Transparency Portal provides the data from bids available to the citizen collecting data from de ERP manually by a person. This understanding was essential to understand the relationships between processes, people and data inside the team responsible for the Transparency Portal. Therefore, the main issues raised in the transparency portal from this analysis were: (i) bids are duplicated – the portal does not have a unique and formalized data source; it comes from different systems like Official Diary (DO), ERP and manual entries which makes it impossible to attest data veracity; (ii) there is not a process to assess and measure the quality of the way data is made available on the OGDP, such as absence of tags, categories and quantities of datasets, absence of different data format (XML, CSV, TXT, XML, RDF, DOC and XLS) and data incompleteness; (iii) some important metadata are not available, such as update frequency, update time and last update. Next section describes some improvement suggestions and analyzes regarding DMBOK practices. TO-BE Model: To evolve the feeding process of the OGDP into a new future process and a new system architecture to deal with bidding data, we first related the defined problems to the DMBOK knowledge areas as presented in Table 2. Next, practices from the knowledge area suggestion were selected to solve the problems. In what follows, we focus on the Metadata knowledge area to show what activities should be executed according to DMBOK. Managing metadata means planning, implementing, and controlling activities that enable high-quality access to metadata. To collect metadata and provide such a publication, 3 main activities are necessary: (i) Define which metadata is important; (ii) Define a metadata architecture; and (iii) Create queries, reports and analyze metadata. Regarding to Define which metadata is important, it is necessary to define what is important to register regarding metadata for bidding considering what the portal needs to publicize. In Table 2 Niterói Transparency Portal problems related to DMBOK knowledge areas Transparency Portal Problems

DAMA

Bids are duplicated. The portal does not have a unique and formalized data source

Data Architecture

There is no process to assess and measure the quality of the way data is made available on the data portal

Data Quality

Some important metadata is not available

Metadata

252

C. C. Junior et al.

Fig. 2 Inputs, activities to be executed by the OGDP managers and deliverables for metadata management of bidding data in the OGDP

the questionnaire and in the research on other portals, it was observed that Update frequency, Update Time, and Last Update are important records to be published for accountability to citizens. The next activity, to Define a metadata architecture, demands looking at the structural models of the bidding data to specify how to evolve it to support the demands of the Transparency Portal. Lastly, to Create queries, reports, and analyze metadata, is related to guaranteeing metadata control and quality. It is necessary to define ways for the metadata to be checked by systems against its requirements. Figure 2 presents a summary of what is needed to achieve the objective. It is important to observe that the metamodel and the models generated on it must be supported by technology infrastructure, process and people defined and a cooperative relationship to guarantee interoperability among all stakeholders being suppliers, participants, or consumers. It brings DMBOK Data Integration and Interoperability Knowledge area that deals with the need to provide technological data services to automatically orchestrate the flow of data among systems. For example, it demands that the system used to register bids communicate with the ERP and this to the OGDP. It demands that the data management processes of the Niterói municipality departments to change considering a data-oriented perspective, which are some of the challenges this research group intends to discuss further in the next stages of this research work.

Towards the Evaluation and Continuous Evolution of Open Government …

253

4 Conclusions and Future Work Data management and data governance is becoming crucial do deliver data in OGDPs with quality, understanding and usability for citizens. This research proposes a preliminary conceptual framework to organize useful sources of information for building and evolving a reference model to organize semantic, organizational, processual and technological aspects on management and governance of an OGDP. We used our framework for evaluating and proposing new ways of managing bidding data to be better delivered in the Transparency OGDP of Niterói, Brazil. We explored literature on this theme and on-site evaluations of OGDPs to this end. A deeper systematic literature mapping is on course to be continuously evolved by a formal protocol, which will be able to bring light to the discussions and approaches from academia. A third approach on the horizon is related to diving into other Data Governance Frameworks to compare and analyze best practices for constructing our reference model. The use of Enterprise Architecture is also being explored as a methodology for the modeling and continuous evolution of the reference model based on the best practices provided by the TOGAF Framework (https://www.opengroup.org/togaf). We would like to observe that our mechanism used to organize the construction and continuous evolution of a reference model for an OGDP is related to the fact that it is being initiated by a single area of the municipality, which could weaken the initiative. In this way, the framework allows each area of a municipality, independently, to work on some aspect of the reference model, organizing a base reference to track the inclusions of practices in it. Acknowledgements This study was partially financed by the Niterói City Hall. We also would like to thank the anonymous referees which comments helped us for improving this work.

References 1. Tygel A, Auer S, Debattista J, Orlandi F, Campos MLM (2016) Towards cleaning-up open data portals: a metadata reconciliation approach. In: Proceedings of IEEE 10th international conference on semantic computing, pp 71–78, Laguna Hills, CA 2. Giffinger R, Gudrun H (2009) Smart cities ranking: An effective instrument for the positioning of the cities. In: Proceedings of 5th international conference virtual city and territory, Barcelona, Spain 3. Brazilian Transparency Law (in Portuguese). http://www.planalto.gov.br/ccivil_03/leis/lcp/lcp 131.html.. Accessed 19 Oct 2022 4. Sampaio I, Andrade E, Viterbo J, Bernardini F (2022) Assessing the quality of Covid-19 open data portals. In: Proceedings of IFIP EGOV-CeDEM-ePart 5. Machado V, Mantini G, Viterbo J, Bernardini F, Barcellos R (2019) An Instrument for evaluating open data portals: a case study in Brazilian cities. In: Proceedings of 19th annual international conference on digital government research (DG’O 2018), Delft, Netherlands, (2018). 6. Janssen M, Charalabidis Y, Zuiderwijk A (2012) Benefits, adoption barriers and myths of open data and open government. Inf Syst Manag 29(4):258–268

254

C. C. Junior et al.

7. Carstensen A-K, Bernhard J (2019) Design science research – a powerful tool for improving methods in engineering education research. Proc Eur J Eng Educ 44 8. Otto B (2011) A morphology of the organization of data governance. In.: Proceedings of the European conference on information systems (ECIS), Helsinki, Finland 9. Beno M, Figl K, Umbrich J, Polleres A (2017) Open data hopes and fears: determining the barriers of open data. In.: Proceedings of conference on E-democracy and open government (CeDEM) 10. Helbig N, Cresswell A, Burke GB, Luna-Reyes L (2012) The dynamics of opening government data. Center for Technology in Government. University of Albany, New York 11. Mosley M, Brackett MH, Earley S, Henderson D (2017) DAMA guide to the data management body of knowledge, 2nd edn. DAMA International 12. Thomas G (2020) The DGI data governance framework. The Data Governance Institute 13. Khatri V, Brown C (2010) Designing data governance. Comm ACM 53(1):148–152 14. Chapple M (2013) Speaking the same language: building a data governance program for institutional impact. Educause Review, Learntechlib.Org 15. Reis JR, Viterbo J, Bernardini F (2018) A rationale for data governance as an approach to tackle recurrent drawbacks in open data portals. In.: Proceedings 19th annual international conference on digital government research (DG’O 2018), Delft, Netherlands pp 1–9 16. Transparent Brazil Map (in Portuguese). https://mbt.cgu.gov.br/publico/home. Accessed 8 Sept 2022 17. Dumas M, La Rosa M, Mendling J, Reijers HA (2013) Fundamentals of business process management. Springer, Cham

IT Governance in Brazilian Public Healthcare Sector Roberto Luiz Silva dos Santos , Sandro Luís Freire de Castro Silva , Sandra Regina Freitas da Silva Morgado de Gois , Jose Geraldo Pereira Barbosa , and Antonio Augusto Gonçalves

Abstract Healthcare organizations in the public sector are under pressure to transform and use Information Technologies (IT) to be more effective and serve increasing number of patients. IT Governance is a challenging issue in the public health sector and is still in the initial stage in Brazilian organizations. The objective of this research was to describe the relevant mechanisms for the effectiveness of IT Governance in the view of managers in the public healthcare sector through a case study carried out at the Brazilian National Cancer Institute (INCA). The results showed that the Institution implemented relevant actions to the effectiveness of IT Governance, aiming to add value to the cancer treatment provided to the Brazilian population. Keywords IT Governance · Public Healthcare Governance

1 Introduction Organizations are increasingly dependent on ITs to survive and grow. Previously, boards of directors and senior management executives could delegate, ignore or avoid IT decisions. However, such attitudes are impossible, as they have become R. L. S. dos Santos (B) · S. L. F. de Castro Silva · A. A. Gonçalves Brazilian National Cancer Institute, Rio de Janeiro, Brazil e-mail: [email protected] S. L. F. de Castro Silva e-mail: [email protected] A. A. Gonçalves e-mail: [email protected] S. R. F. da Silva Morgado de Gois · J. G. P. Barbosa · A. A. Gonçalves Universidade Estacio de Sá (MADE), Rio de Janeiro, Brazil e-mail: [email protected] J. G. P. Barbosa e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_22

255

256

R. L. S. dos Santos et al.

crucial in companies’ support, sustainability, and growth. IT Governance is an integral part of Corporate Governance. It addresses the definition and implementation of the organization’s process, structures, and relational mechanisms that allow business and IT professionals to perform their responsibilities in supporting business alignment [1]. The use of IT has the potential to be the primary driver of economic wealth in the twenty-first century. IT is fundamental for managing enterprise resources and dealing with customers, and suppliers, besides enabling increasingly global transactions. IT Governance can be defined as an organizational capacity for formulating and implementing IT strategy. Thus, it can be concluded that IT Governance is a mechanism of controlling IT assets based on a framework to achieve the Institution’s vision and mission [2]. IT Governance defines an accountability framework to sustain appropriated conduct in IT utilization. It reflects broader organizational governance principles by focusing on managing and utilizing IT to accomplish organizational goals. Effective IT governance enhances IT utilization and ensures compliance with the regulations and values of the organization [3]. IT Governance provides organizations with mechanisms to achieve business goals, such as allocating IT decision rights and managing IT risks. It also ensures that the responsibility of IT is not only limited to achieving internal IT efficiency through settling better IT processes or by accomplishing compliance issues. The main objective of IT Governance is to establish synergy between business and IT to reach business value through IT investments [1, 2]. Companies that are mature in IT Governance have four objectives: effective search for IT costs; effective use of IT resources; use of IT for business growth and service; and the use of IT to make the business more flexible and to achieve these objectives, the use of IT Governance mechanisms are suggested, which must be implemented in organizations [4]. The growing level of investment in IT and the expected impact on the performance of healthcare organizations require an active governance posture. IT Governance is a challenging area in the health sector and is still in its initial stage in Brazilian organizations [5, 6]. IT Governance is a set of organizational structures ensuring decision-making rights and responsibilities concerning IT assets [3, 4]. IT Governance ensures that IT supports business processes, investments, risks, and opportunities [7]. Two main perspectives can define IT governance effectiveness. The first is regarding the functional aspect of IT and its characteristic of operational areas (hardware, software, infrastructure, availability of IT services, resources, people, financial and economic assets). The second refers to the business, observing the connection between IT results and organizational performance [7]. IT has the power to support existing operations and principally shape new business strategies. In this scenario, IT has become a critical success factor and an opportunity to obtain a competitive advantage. IT offers the means to improve productivity, leveraging technological assets to transform the enterprise and create new products and services. IT moves from a service provider to a strategic partner [8].

IT Governance in Brazilian Public Healthcare Sector

257

Nowadays, healthcare organizations face many challenges in analyzing organizational performance because managers try to match organizational performance and strategic goals. The ultimate goal is to develop and implement IT Governance, integrate business and IT, and achieve better financial results [9]. High costs, unorganized healthcare services, elevated medical error rates, inefficient communication, and the lack of information to support decision-making characterize the present status of the majority of healthcare systems. IT Governance has a strategic role in qualifying health services transformation to improve this situation [10]. Raising the life quality of the population is the principal goal of healthcare systems. The performance of the healthcare services should improve to increase the overall level of population health and reduce inequalities. A significant evolution in health services is driven by demographic changes, including the increase in the aging population, chronic diseases, progress in sciences and technologies, and the recognition of the requirement for significantly improved quality and safety of healthcare services [11]. IT holds the challenge of catalyzing the healthcare industry’s transformation into an evidence-enhanced practice that enables better analytics and decision support for improving health services. Creating personalized treatments that incorporate targeted procedures based on patients’ characteristics will promote wellness and better outcomes through patient-centered and collaborative care [12]. In this scenario, it is essential to examine IT Governance in the public sector to adopt the best IT Governance practices by the Brazilian Federal Public Administration bodies in the healthcare area. This article aims to describe and analyze the relevant mechanisms for the effectiveness of IT Governance through a case study carried out at the Brazilian National Cancer Institute (INCA).

2 Methodology The Brazilian National Cancer Institute was selected as a unit of analysis of the case study due to the perceived effectiveness of the mechanisms of IT Governance in this federal healthcare organization located in the city of Rio de Janeiro and considered a national reference in cancer prevention and treatment. IT Governance is a set of organizational structures assuring decision-making rights and responsibilities about its IT assets. This qualitative study identified the IT governance domains in cancer hospitals affiliated with Brazilian National Cancer Institute (INCA). Semi-structured interviews were used for data collection. Content analysis was applied. During the interview, participants were advised that all information would keep confidential. This qualitative study was conducted among nine heads of INCA departments. For this study, a descriptive qualitative methodological approach was used. It is the most suitable for investigating a specific area and treating the evidence obtained,

258

R. L. S. dos Santos et al.

seeking understanding and meaning through the views and perspectives of the participants involved. In this way, we sought to bring out the answer to the problem question of this research. The following means were used to ensure the validity of the research: development of analysis categories based on a consistent theoretical framework; multiple sources of data extracted from interviews and records of internal activities; a collection of evidence throughout the discussions; and review of the survey report with a key informant. To ensure the reliability of the research, a protocol was used for the case study containing the procedures and general rules that must be followed to avoid deviations; Also, a database was developed for analysis and comparison of the results found in the case study was performed. Content analysis was applied to analyze the data. Themes and subthemes were extracted from the interviews, and then the main articles related to the study objectives were identified.

3 Results The first step was to identify the objectives of IT Governance in the healthcare organization analyzed. The evidence collected has identified the main issues of IT Governance. Based on the data analysis of the evidence collected, the Information Technology Service defined the main objectives aligned with the Strategic Objectives of the Institute (Table 1). The evidence collected was analyzed to identify the objectives of IT Governance in the institute, based on the goals mentioned by Weill (2004). According to the analysis of the evidence collected, the Information Technology Service (STI) defined the purposes below in line with the Strategic Objectives of the Institute. The identified goals were organized into perspectives. Table 1 Table captions should be placed above the tables Perspective

Objectives

Business growth

Consolidate INCA’s IT as an instrument to enable the execution of strategies, address the organization’s needs, and add value to the services provided to the population

IT costs

Improve Budget and Financial Management

IT resources

Effectively provide technological infrastructure resources, systems and information, ensuring the adequate availability of each IT service

Transparency

Promote transparency and publicity of IT initiatives and the application of public resources

Society

Expand and innovate the provision of digital services

IT Governance in Brazilian Public Healthcare Sector

259

3.1 Perspective: Business Growth • Consolidate INCA’s IT as an instrument to enable the execution of strategies, meet the organization’s needs, and add value to the services provided to the population. It was possible to perceive that this objective is well spread among the interviewees. In this regard, one interviewee stated: "IT governance is fundamental to meet the institutional strategic objectives and the needs of each area of the institute."

3.2 Perspective: IT Costs • Improve Budget and Financial Management. • Adopt good budgetary and financial management practices to effectively use the financial resources necessary to meet IT-related goals. • Provide adequate IT budget planning and the effective execution of budget resources. • Promote and coordinate actions integrated with IT managers for better financial execution of contracts. The interviews did not show an adequate search for IT cost reduction. Thus, it can be inferred that there is no obvious concern of managers in the practical search for IT costs. However, in the interviews, the interviewees informed that the Information Technology Service does not have a budget destined exclusively to achieve its objectives. In this regard, one interviewee stated: "The IT budget is the budget of the Institution as a whole, and you submit the projects, and they are approved or not, but you do not have a budget of your own for the area."

3.3 Perspective: IT Resources Provide technological infrastructure resources, systems, and information effectively, ensuring the adequate availability of each IT service. • Ensure adequate and effective technology for INCA’s strategies, projects, and actions. • Improve work processes and management of IT services. The interviewees enhanced the importance of the decision-making process in healthcare services. They emphasized that these decisions can be made by sênior managers or specialists committees to support hospital operational activities.

260

R. L. S. dos Santos et al.

The IT committee can make decisions on software requirements for the INCA hospitals. This process is related to applications used uniformly in the INCA hospitals like HIS (Hospital Information Systems). In this regard, one interviewee stated: The Institution’s areas are invited to Demand from the IT sector their needs with their respective justifications. From there, the list of demands comes out.

Additionally, another interviewee stated: "The priorities of the information technology director plan projects are defined and approved by the budget committee."

In regard with decision making meetings of the budget committee, another interviewee remarked: "IT is always present, and we have meetings in the budget committee every two weeks where business demands are prioritized."

3.4 Perspective: Transparency Promote transparency and publicity of IT initiatives and the application of public resources. • Adopt actions that expand the use of IT to provide reliable, timely, and updated information on the results produced, whether partial or total, from the application of public resources, more clearly and straightforwardly to the citizen, of the products, services generated, or transformed reality. In this regard, one interviewee stated: "IT governance is fundamental to be able to meet the institutional strategic objectives and the needs of each area of the institute."

Another interviewee remarked: "The objective is that IT Governance manages to align the investments destined for information technology solutions to the strategic objectives of the Institution, that is, the IT solutions they need to contribute as much as possible so that the institution achieves its mission of fulfilling the strategic objectives."

3.5 Perspective: Society Expand and innovate the provision of digital services. • Adopt actions that expand and innovate public services offered by digital means (eservices) through innovative processes and technologies to meet society’s needs.

IT Governance in Brazilian Public Healthcare Sector

261

Expand and encourage social participation in the creation and improvement of public services. • Adopt actions that encourage the active participation of society in the creation and improvement of accessible public services, considering regional diversities and needs enabling the direct interaction of citizens with the government to express their needs and evaluate them and suggest improvements. In this regard, one interviewee stated: "Strategic planning is also important. It must be discussed and approved by the IT strategy committee. Information technology solutions must meet the needs of the areas of assistance, teaching, research, and prevention, providing efficiency in the services provided by INCA.".

4 Conclusions Despite the research showing that the six objectives of IT Governance are formalized in the Institution, it was evident throughout the interviewees’ speeches that the main aim of IT Governance is inserted in the perspective of “business growth.“ The goal focuses on the execution of IT strategies, meeting the organization’s needs, and aiming to add value to the services provided to the population. Managers understand the role of IT as a strategic area and not purely operational. Governance mechanisms must be implemented in organizations o achieve the goals of IT Governance. It was observed that the mechanisms most cited as relevant to IT Governance are closely linked to the alignment of IT strategies to the organization’s business needs: IT Strategy Committee and IT Strategic Planning, confirming the initial assumption of the research. It is noteworthy that the Governance / Project Management Methodologies mechanism was the least cited as relevant to the effectiveness of IT Governance. This fact can be explained by the concern of managers with the alignment of IT strategies to the organization’s business needs and because this processing mechanism is more related to IT Management. The results demonstrate that the organization needs to have IT leadership to ensure that the strategic vision of IT is clearly understood by the areas and, mainly, by the Senior Management. It also acts as a bridge between IT and business, interacting with Senior Management, identifying needs, promoting IT projects, and motivating employees. Respondents also highlighted that it is crucial to have IT leadership so that the other mechanisms are implemented and IT Governance is effective. Identifying the responsibility and power of IT users in major IT decisions is a crucial issue in healthcare organizations. To create such commitment, the organization should clearly define the roles of the IT stakeholders through decision-making processes. Decision-making processes clearly show compliance and accountability consistent with accepted organizational structures of the IT governance. There can be several motivators or expected benefits in adopting IT Governance mechanisms. However, the interviews show that it is necessary to have active participation of the IT leadership. The IT leadership should have decision-making power

262

R. L. S. dos Santos et al.

and direct access to all areas of the organization. In addition, IT leadership must be aware that information can improve the health services provided to the population. This professional should also be responsible for making the senior management aware of the strategic role of IT and the importance of creating a formal IT Governance structure and creating committees at the institutional level; by interacting with the other directors of the organization, identifying needs, and promoting IT projects. Acknowledgements The authors would like to thank the Brazilian National Cancer Institute (INCA). The third author would like to thank the support of Brazilian National Council for Scientific and Technological Development (CNPq—Brazil) and Estácio de Sá University.

References 1. De Haes S, Van Grembergen W (2015) Enterprise governance of information technology. Springer, Cham 2. Van Grembergen W (2009) Introduction to the minitrack IT governance and its mechanisms. In: Proceedings of the 35th Hawaii international conference on system sciences. IEEE 3. Weill P, Ross J (2004) IT Governance: how top performers manage IT decision rights for superior results. Harvard Business School Press, Boston 4. Ali S, Green P (2007) IT Governance mechanisms in public sector organisations: an australian context. J Glob Inf Manag 15(4):41–63 5. Betratarbide E, Kelsey T. (2013) eHealth governance in Scotland: a cross-sectoral and crossnational comparison. In: eHealth: legal, ethical and governance challenges. Springer Berlin, Heidelberg 6. Gonçalves AA, Martins CHF, De Castro Silva SLF, Cheng C, Santos RLS, de Oliveira SB (2019) The Governance of information technology and communication in public health organizations: Case study of the national cancer institute. RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao E17:231–243 7. Silva BAM, Moraes GHSM (2011) Influência dos direcionadores do uso da TI na governança de TI. Revista Brasileira de Gestao de Negocios 13(38):41–60 8. Guldentops E, et al. (2003) Board briefing on IT governance, 2 edn. Rolling Meadows, Ilinois – USA.. Information Systems Control Association 9. Borousan E, et al. (2011) Balanced Scorecard; a tool for measuring and modifying IT governance in healthcare organizations. Int J Innov Manage Technol 2(02) , 141 10. Blendon RJ, Schoen C. DesRoches C, Osborn R, K, Zapert B (2003) Common concerns amid diverse systems: health care experiences in five countries. Health Affairs 22(3), pp 106–121, May/June 11. AbuKhousa E, Nabeel Al-Q (2012). Health information technology governance: a perspective on investment decision processes. In: Proceedings of the 23rd Australasian conference on information systems, ACIS 12. Shahi M, Sadoughi F, Ahmadi M (2014) Information technology governance domains in hospitals: a case study in Iran. Global J Health Sci 7(3):200–208. https://doi.org/10.5539/gjhs.v7n 3p200

Virtual Learning Networks in the Learning of a Foreign Language: Remote Education Perspectives Jehovanni Fabricio Velarde-Molina , Stefany Juliana Noa-Copaja , Kevin Mario Laura-De La Cruz , Silvia Milagritos Bazan-Velasquez , Domingo Nicolás Pérez-Yufra, and Giomar Walter Moscoso-Zegarra

Abstract The intention of the research was to evaluate how English students using a virtual learning network perceive the connectivist pedagogical model. In this sense, descriptive research employing the hypothetical-inductive methodology was conducted. For data collection, a Likert-scale based survey measuring connectivism perception was utilized. The majority of the statements were seen positively by the respondents, according to the findings. On the basis of the results, it is claimed that connectivism can increase English acquisition because of its positive perception, particularly in the way teachers and students behave in the model, where the agree scale averages were 55.94% and 52.24%, respectively. Furthermore, it is required to suggest steps for the implementation of innovative technologies by instructors during the lessons, the accurate equipping of establishments, the formation of educators, and the cultivation of study strategies via learning networks. Keywords Connectivism · English as a foreign language · Perceptions · Virtual learning

J. F. Velarde-Molina (B) · K. M. Laura-De La Cruz · D. N. Pérez-Yufra · G. W. Moscoso-Zegarra Escuela de Posgrado Newman, Av. Bolognesi, 987 Tacna, Perú e-mail: [email protected] K. M. Laura-De La Cruz e-mail: [email protected] D. N. Pérez-Yufra e-mail: [email protected] G. W. Moscoso-Zegarra e-mail: [email protected] S. J. Noa-Copaja · S. M. Bazan-Velasquez Universidad Nacional Jorge Basadre Grohmann, Av. Miraflores S/N, Tacna, Perú e-mail: [email protected] S. M. Bazan-Velasquez e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_23

263

264

J. F. Velarde-Molina et al.

1 Introduction As a result of technological advancements, there are new demands placed on educators to adapt their methods of instruction to meet the changing needs of a student body that is increasingly diverse and multicultural [1, 2]. This demand, in turn, necessitates the exploration of new educational theories that can serve as a roadmap for the enhancement of classroom instruction. Furthermore, researchers as Vallejo et al. [3] argue that the impact of technology on the educational process necessitates the prominence of connectivist theory in the teaching of virtually any subject as the primary factor of virtual learning. According to Siemens [4], the foundation of the connectivist learning theory is the individual, because each person’s knowledge is formed by a network that, via interaction with those around, contributes to the expansion of other networks in ways that benefit both the individual and the larger society. This feedback loop of education and experience helps to maintain the most current links. In a similar vein, Siemens [4] contends that the Connectivist model can be defined by eight defining principles, most of which concern interaction, illuminating the theory’s emphasis on knowledge acquisition via the exchange of perspectives among multiple people to produce enriched and up-to-date knowledge nodes. Since learning would be incoherent and without meaning without the ability to discern the relationship between certain facts or concepts with others, this further emphasizes the significance of this skill over the acquisition of knowledge. The significance of learning networks in the implementation of connectivist theory is high [5], since they enable students to develop competencies in a collaborative and knowledge-sharing setting. Learning networks enhance informal learning experiences; yet, they also benefit formal educational environments. Additionally, research conducted by Valencia Ortiz et al. [6] demonstrates that students who participate in online courses are the ones that benefit most from the usage of social media in order to fulfill their educational goals. Therefore, we should consider the potential benefits of using social networks and the Internet as training tools in traditional classrooms. Numerous individuals rely on online education and social networks without even recognizing it, offering a perfect environment for the development and application of connectivist theory. In light of this, it is crucial to understand what students who rely on connectivist educational networks believe, particularly if they believe connectivism to be an effective model for the English acquisition.

Virtual Learning Networks in the Learning of a Foreign Language …

265

Fig. 1 Age of the sample

2 Method 2.1 Design of the Study A non-experimental study was carried because the sample was not altered. To evaluate the research hypothesis, numerical data were collected with the aid of statistics, resulting in a quantitative method [7]. Besides, the investigation was descriptive, as the primary purpose of the investigation was to describe the features of the participants.

2.2 Population and Sample Participants were members of an online WhatsApp learning group for students of the English language. This group comprised 220 individuals. The study sample consists of 140 students of various ages and nationalities. The percentage of pupils questioned by age is depicted in Fig. 1. Students between the ages of 21 and 28 have the highest perceived proportion, at 24.07%, whilst students between the ages of 49 and 56 have the lowest percentage, at 5.56%. It can be gathered from the data that people of mature age, whose average age is roughly 32, are more interested in studying English. Figure 2 depicts the origin of the surveyed pupils. Indian students hold the biggest share with 16.66%, followed by Mexican students with 12.96%. It is evident that the bulk of respondents are from American countries, followed by those from Asia.

2.3 Techniques for Data Analysis Technique The level of perception regarding the connectivist model was measured by means of a questionnaire.

266

J. F. Velarde-Molina et al.

Fig. 2 Nationality of the sample

Instrument A survey employing the Likert scale format was conducted in consideration of the following characteristics and indicators: Dimension 1: Principles of connectivism. Proves a favorable or negative assessment in accordance with Siemens’ connectivism principles. Dimension 2: Teacher’s role. According to the teacher’s position in connectivism, displays either favorable or negative perception. Dimension 3: Student’s role. According to the student’s position in connectivism, displays favorable or negative perception. Dimension 4: Limitations. Possesses a favorable or unfavorable perspective towards the limitations of connectivism’s applicability.

2.4 Measurement Scale Students’ perceptions of the connectivist paradigm were gathered through the use of a Likert-scale questionnaire. The following criteria were taken into consideration: strongly agree, agree, neutral, disagree, and strongly disagree. Due to the diversity of the student population, the survey was conducted in English.

2.5 Instrument Reliability Cronbach’s alpha, an internal consistency measure [8], was applied in a pilot study, and the resulting value of 0.88 indicated that the instrument used to measure this variable was reliable.

Virtual Learning Networks in the Learning of a Foreign Language …

267

3 Results Table 1 displays the students’ perceptions of the Siemens-proposed Connectivism principles. Item 1 received the largest percentage of positive responses, with 79.70% between strongly agree and agree, while item 5 received the lowest percentage of positive responses, with 40.80% between strongly disagree and disagree. It may be extrapolated that English language learners support the use of social networks and forums for language learning; yet, the respondents’ disagreement with the fifth statement demonstrates that virtual English classes are not sufficient for them. Table 2 displays the opinions of respondents on the role of teaching in connectivism. The item with the most positive score was item 6, which received 92.60 percent between strongly agree and agree. The item with the lowest positive score was item 9, which received 79.30 percent between strongly agree and agree. Nonetheless, all items received significant positive percentages, so it can be inferred that respondents have high expectations for the teaching role and its competitiveness in English classes, necessitating the teacher to promote the use of software, games, and web sites that can assist students in developing their independent learning in English. The perceptions of respondents on the role of the student in connectivism are displayed in Table 3. All questions received large favourable percentages, but questions 11 and 13 stand out with 92.60 percent of respondents who strongly agreed or agreed. Thus, the respondents suggest that they are aware of the significant decisions they make in the lessons and in their own English language acquisition, as seen by their favourable dispositions toward the connectivist qualities of group learning, experience sharing, and information processing. Table 1 Dimension 1: Principles of Connectivism Dimension 1: Principles of Connectivism

SD

D

N

A

SA

Total

1. I learn English better when I use forums and social networks

0.00%

3.70%

16.70%

59.30%

20.4%

100%

2. I think that the information provided by the Internet is enough to study English on my own

0.00%

29.60%

20.40%

44.40%

5.60%

100%

3. I think that it is not necessary to memorize in English classes, but to reflect on what is learned

0.00%

13%

20.40%

42.60%

24.10%

100%

4. I feel better when I learn English through technology than from physical books

0.00%

22.20%

33.30%

31.50%

13%

100%

5. I think I can learn English better in virtual classes than in face-to-face classes

9.30%

31.50%

27.80%

24.10%

7.40%

100%

Note:SD = Strongly disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly agree

268

J. F. Velarde-Molina et al.

Table 2 Dimension 2: Teacher’s role Dimension 2: Teacher’s role

SD

6. I think teachers should promote the use of technology and web pages in English classes

0.00% 0.00%

D

N

7.40% 66.70% 25.90% 100%

7. I think teachers should provide virtual resources, applications and games to learn English better

0.00% 1.90%

7.40% 53.70% 37.00% 100%

8. I think teachers should teach what information and websites on the Internet are reliable for learning English

1.90% 4.00%

7.40% 55.60% 31.50% 100%

9. Teachers should promote autonomous learning in English language learners

0.00% 1.90% 18.50% 46.30% 33.00% 100%

10. I think teachers should teach how to 0.00% 1.90% organize and apply information found on the Internet about the English language

A

SA

Total

9.30% 57.40% 31.50% 100%

Note:SD = Strongly disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly agree

Table 3 Dimension 3: Student’s role Dimension 3: Student’s role

SD

D

11. I think English language learners should seek to communicate and share experiences in English with others virtually

0.00% 0.00%

12. I think English learners should be motivated and try to learn the language on their own

0.00% 1.90% 11.10% 46.30% 40.70% 100%

13. I think English language learners 0.00% 0.00% should try to connect the new information they are learning with their prior knowledge 14. I think students should recognize that group learning is better than individual learning in English classes

N

A

SA

Total

7.40% 66.70% 25.90% 100%

7.40% 55.60% 37.00% 100%

0.00% 0.00% 18.50% 51.90% 30.00% 100%

15. I think students should recognize 0.00% 9.30% 14.80% 40.70% 35.20% 100% that collaborative learning is better than individual learning in English classes Note:SD = Strongly disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly agree

Table 4 displays the opinions of respondents on connectivism’s limitations. All of the questions are supported by the respondents, with item 17 receiving the highest level of approval (87.0% between definitely agree and agree). Thus, it can be inferred from the responses that, in their respective countries, the educational system is not focused toward the teaching of connectivism due to infrastructure deficiencies and

Virtual Learning Networks in the Learning of a Foreign Language …

269

Table 4 Dimension 4: Limitations Dimension 4: Limitations

SD

16. I think that not all English teachers are motivated to apply technology in the classroom

0.00% 3.70% 16.70% 57.40% 22.20% 100%

D

N

A

SA

Total

17. I believe that not all students make good use of technology to learn the English language

0.00% 3.70%

9.30% 57.40% 29.60% 100%

18. I think that in my country there is not 0.00% 7.00% 16.70% 40.70% 35.20% 100% enough willingness to equip schools with technology Note:SD = Strongly disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly agree

teachers’ lack of motivation to use emerging technologies, and as a consequence, students do not make effective use of the Internet for learning.

4 Discussion The primary purpose is to ascertain how students view the connectivist model in English language instruction. Although connectivism, unlike other learning theories, has attracted great interest and controversy, there is a lengthy background of research and adjustment that leads to a defining framework explaining how people in the digital world learn in different ways and most effectively [9]. Connectivism was revolutionary when it was introduced in 2004 for the first time in more than a decade, based on complexity theory, connectivity, intricacy, and self-organization notions. Nonetheless, the original concept of Siemens has not been significantly upgraded and enhanced at the theoretical level. In the first dimension, it can be observed that the majority of participants agreed with the use of social media platforms for English language learning. This is supported by the research of Lima and Araujo [7] who declare that students have a favorable perspective of social networks and like the advantages they provide while strengthening the expressive capacity of the target language when they are utilized responsibly and consciously. Nevertheless, the responses reveal the necessity for an equilibrium between physical learning through a manuscript and virtual classrooms of English, believing that the knowledge gained electronically is insufficient and that they must also frequent face-to-face classes because virtual engagement is not as intense as face-to-face learning. This phenomenon is justified by Aguilar and Del Valle [10], who contend that many students need face-to-face interaction with the professor to accomplish greater learning, because studying in virtual modalities takes a great deal of student motivation and self-control. In the second dimension, it is demonstrated that there is a high level of need and aspiration with regard to the responsibilities professors play, given that they must also

270

J. F. Velarde-Molina et al.

motivate the adequate use of digital technology, emphasizing self - directed learning in students while using English language learning-oriented virtual games, software, or web sites. In light of their involvement in virtual media, respondents view the instructor as a vital actor for English language acquisition, as the teacher’s directing role assists in controlling the teaching process. This is demonstrated by Laura and Velarde’s [11] research, which concludes that the majority of students in education sector can accomplish substantial gains in the understanding of English texts with the execution of the JClic software in the classroom, thereby innovating teaching and the learning experience. Recent technological, educational, and pedagogical breakthroughs have altered the processes for teaching and learning. Extensive research has studied the elements that influence foreign language teachers’ technology integration in varied cultural contexts [12]. In a similar vein, research by Rodriguez [13] demonstrates that a variety of teaching materials, including discussion boards, chatrooms, exercises in the online classroom, web sites, podcasts, and others, can have a significant influence on language learning, producing refinement in the development of skills along with cooperation and partnership among classmates. It is possible that the absence of a significant link between simplicity of usage and intention to use in this study is due to the fact that modern English teachers do not perceive online education to be complicated [14, 15]. Evidence from the third dimension suggests that students know they must take an active position in learning English; hence, they seek out interactive virtual environments in which they may practice and perfect their spoken and written English with native speakers. González [16] backs up the aforementioned finding by arguing that students have an optimistic view of using Blogs and Facebook as platforms that facilitate language learning through engagement with the effort of other peers, the sharing of knowledge, and opinions. Similarly, the participants favor group work over individual work as a method for effectively learning English. This makes sense, as group learning can have many advantages in both learning and teaching, including the opportunity for students to share and discuss information, perform peer evaluations, build community, and practice English communication skills [17]. Participants also agreed that they ought to be self-motivated, because this style of motivation makes learners to accomplish their objectives without the necessity of receiving a recompense for their hard work rather than the accomplishment of fully understanding based on the admiration of wisdom [18]; that they must relate the new information they learn in the classroom with their prior experience to acquire the English language independently; and that they should demonstrate their commitment to learning by completing assignments on their own time. According to Laura et al. [19], learners from public schools strengthened their reading comprehension competency by playing a game that incorporated photos, films, phrases, characters, jokes, and the Quizizz tool’s skills to encourage self-reflection on student progress [20, 21]. They were able to learn at their own speed while taking stock of their progress and identifying areas for improvement thanks to the asynchronous and immediate feedback provided by the game reports. Teachers’ educational, imaginative, and subject knowledge is crucial in the technological sectors [22]. Therefore, training initiatives should combine a focus on innovation and pedagogical issues, coaching and

Virtual Learning Networks in the Learning of a Foreign Language …

271

modeling, and skills in the inventive use of technology-mediated English instruction in order to spread a technological vision. Concerning findings in the final dimension put constraints on the use of connectivism in the classroom and the teaching of English. The majority of the participants said that their home country’s government did not prioritize the use of technology in the classroom. As shown in their conclusion, Sierra-Llorente et al. [23] note that schools lack the infrastructure and resources needed for the number of pupils, and that there is no encouragement of workshops or use of ICTs in the classroom to instructors, which could be contributing factors. However, respondents believe both educators and learners do not play the roles that would be expected based on connectivist theory. Teachers lack enthusiasm for using technology in the classroom, and students lack the incentive to make the most of online resources. According to Ovalles [24], many educators are compelled to take their classes online, and when they do so, they often resort to the same strategies used in traditional classrooms because they assume it is the only way to ensure students receive an adequate education. Equally, Garrote et al. [25] draw the conclusion from their study that many adolescents spend considerable time using technology, viewing it as a resource that enables them to relax, while simultaneously demonstrating behaviors very close to dependence, such as distress when they are unable to connect and the need to frequently consult the cellphone, subordinating its use from an academic perspective.

5 Conclusion The results of this study suggest that the majority of respondents hold a favorable view of connectivism and its defining features, lending credence to the hypothesis that using connectivism in English classrooms can improve language acquisition. Based on the fact that survey participants were unsure about the best teaching methodology for learning English, it can be implied that students might be more inclined to participate in classes with a mixed modality, also known as Blended Learning, which, according to Area and Adell [26], is a creative learning method that produces significant improvements in the interaction between teachers and students, dialogue, and learning outcomes. Students, on the other hand, have great expectations of their professors and the ways in which they employ technology in the classroom, viewing it not just as an integral part of the curriculum but also as an opportunity to impart knowledge about where to locate credible sources of information and how to put that knowledge to practical use in the development of independent learners. In addition, participants are conscious of their role in their own education and the significance of motivation and group collaboration in language learning. On the basis of the research, some suggestions can be drawn as a conclusion. It is essential that educators use this model with extreme caution, keeping in mind that the primary objective is not to stimulate students to use technology, but rather to engage them in the teaching–learning process so that they take ownership for their

272

J. F. Velarde-Molina et al.

own education and acquire the necessary competencies. Another essential component is that effort must be taken to address the shortcomings described in this study. A significant proportion of responses indicated that there are limitations in the variety of nations from which respondents hail. Respondents, who are predominantly young, anticipate a transformation in the government, teachers, and students so that they can collaborate for the advancement of their society, particularly in education. It calls for reflective thinking and inspires the various governments to take steps to incorporate technology devices in their classrooms, to train instructors in their use, and to help educate the appropriate use of the social media and the Internet so that students can develop learning strategies and find an opportunity to study implicitly and cohesively via studying networks.

References 1. Vinueza S, Simbaña V (2017) Impacto de las TIC en la Educación Superior en el Ecuador. Revista Publicando 11(1):355–368 2. Tight M (2021) Globalization and internationalization as frameworks for higher education re-search. Res Pap Educ 36(1):52–74 3. Vallejo P, Zambrano G, Vallejo P, Bravo G (2019) Importancia del Conectivismo en la inclusión para mejorar la Calidad Educativa ante la tecnología moderna. Revista Arbitrada Interdisciplinaria Koinonía 4(8):523 4. Siemens G (2005) Connectivism: a learning theory for the digital age. Int J Instruct Technol Distance Learn 2(1):1–9 5. Solórzano F, García A (2016) Fundamentos del aprendizaje en red desde el conectivismo y la teoría de la actividad. Revista Cubana de Educación Superior 35:98–112 6. ValenciaOrtiz R, Garay U, CaberoAlmenara J (2020) The perception from students and teachers about their usage of internet and online social networks, and its relationship with the type of study. Revista de Educación a Distancia 20(62):2–23. https://doi.org/10.6018/RED.411781 7. Moreira GL, Araújo J (2018) El uso de las redes sociales para el desarrollo de la enseñanza de ELE: EL Investigador y el Profesor. Un diálogo entre estos dos sujetos a través de la entrevista con Daniel Cassany y dayane cordeiro. Trabalhos em Linguística Aplicada 57(2):1274–1296. https://doi.org/10.1590/010318138651882364651 8. George D, Mallery P (2003) SPSS for Windows Step by Step: A Simple Guide and Reference, 4th edn. Allyn & Bacon, Boston 9. Corbett F, Spinello E (2020) Connectivism and leadership: harnessing a learning theory for the digital age to redefine leadership in the twenty-first century. Heliyon 6(1):e03250. https://doi. org/10.1016/j.heliyon.2020.e03250 10. Aguilar M, Del Valle M (2016) De lo presencial a lo virtual: Caso Universidad Metropolitana. Opción 32(9):17–31 11. Laura K, Velarde J (2019) La Aplicación de un Software en Comprensión de Textos en inglés para Estudiantes en Perú. Neumann Bus Rev 5(2):108–121. https://doi.org/10.22451/3002.nbr 2019.vol5.2.10042 12. Teo T, Huang F, Hoi C (2018) Explicating the influences that explain intention to use technology among English teachers in China. Interact Learn Environ 26(4):460–475 13. Rodríguez, L.: Diseño de una metodología M-Learning para el aprendizaje del idioma inglés. Master’s thesis. Escuela Superior Politécnica de Chimborazo, Ecuador (2015) 14. Huang F, Teo T, Zhou M (2017) Factors affecting Chinese English as a foreign language teachers’ technology acceptance: a qualitative study. J Educ Comput Res 57(1):83–105. https:/ /doi.org/10.1177/0735633117746168

Virtual Learning Networks in the Learning of a Foreign Language …

273

15. Venkatesh V, Bala H (2008) Technology acceptance model 3 and a research agenda on interventions. Decis Sci 39(2):273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x 16. Gonzáles M (2012) Uso de blogs y redes sociales para el aprendizaje de lenguas extranjeras en un contexto Universitario. Núcleo 24(29):39–57 17. SalasAlvarado J (2016) El aprendizaje colaborativo en la clase de inglés: El caso del curso de Comunicación Oral I de la Sede del Pacífico de la Universidad de Costa Rica. Revista Educación 40(2):1. https://doi.org/10.15517/revedu.v40i2.16285 18. Barrera J, Curasama A, Gonzales A (2014) La Motivación y su relación con el aprendizaje del idioma inglés en los estudiantes de cuarto grado de secundaria de la Institución Educativa Manuel Gonzales Prada de Huaycán, Vitarte [Bachelor’s Thesis]. Universidad Nacional de Educación Guzmán y La Valle, Lima 19. Laura K, Morales K, Clavitea M, Aza P (2021) Aplicación Quizizz y comprensión de textos en inglés con el contenido de la plataforma educativa “Aprendo en Casa.” Revista Innova Educación 3(1):151–159. https://doi.org/10.35622/j.rie.2021.01.007 20. Cruz KMLDL, Gebera OWT, Copaja SJN (2022) Application of gamification in higher education in the teaching of English as a foreign language. In: Mesquita A, Abreu A, Carvalho JV (eds) Perspectives and Trends in Education and Technology: Selected Papers from ICITED 2021. Springer Singapore, Singapore, pp 323–341. https://doi.org/10.1007/978-981-16-50635_27 21. Cruz KMLDL, RoqueCoronel L-M, NoaCopaja SJ, RejasJunes LR (2022) Flipped classroom methodology in English language learning in higher education. In: Rocha Á, Ferrás C, Porras AM, Delgado EJ (eds) Information Technology and Systems: Proceedings of ICITS 2022. Springer International Publishing, Cham, pp 448–459. https://doi.org/10.1007/978-3030-96293-7_37 22. Llorente JGS, Córdoba YAP, Mora BSR (2018) Causas que determinan las dificultades de la incorporación de las TIC en las aulas de clases - causes that determine the difficulties in the onboarding process of ICT in classrooms. Panorama 12(22):31–41. https://doi.org/10.15765/ pnrm.v12i22.1064 23. Yang S, Huang Y (2008) A study of high school English teachers’ behavior, concerns and beliefs in integrating information technology into English instruction. Comput Hum Behav 24(3):1085–1103. https://doi.org/10.1016/j.chb.2007.03.009 24. Ovalles L (2014) Conectivismo, ¿un nuevo paradigma en la educación actual? Fundación de Estudios Superiores Comfanorte FESC 4:72–79 25. Garrote D, Jiménez S, Gómez I (2018) Problemas Derivados del Uso de Internet y el Teléfono Móvil en Estudiantes Universitarios. Formación Universitaria 11(2):99–108. https://doi.org/ 10.4067/S0718-50062018000200099 26. Area M, Adell J (2009) e-Learning: Enseñar y aprender en espacios virtuales. In proceedings Tecnología Educativa. Málaga, Spain

Understanding the Success Factors of Research Software: Interviews with Brazilian Computer Science Academic Researchers Erica Mourão, Daniela Trevisan, and José Viterbo

Abstract Context: Research in various domains, such as engineering, sciences, and other fields, depends on software developed in academia by researchers. However, some software becomes a successful case, and others do not. Objective: The goal of this paper is to identify and understand the factors that determine successful research software, as well as investigate the factors of non-success and challenges involved in research software to become successful. Method: An interview among twenty Brazilian academic research experts was conducted. To survey the interview, we used GQM. An analysis of the collected data was conducted according to the Grounded Theory aiming to identify factors exposed by the experts. Results: We identified sixteen success factors divided into two categories among the answers of the experts, eight non-success factors, and five challenges. Conclusion: Our study indicates that the academies can conduct more research software by being aware of the success factors, preventing the non-success, and overcoming the challenges. Keywords Experimental Software Engineering · Empirical Research · Research Software · Academia · University · Brazilian Academic Researchers

1 Introduction For decades, many scientists have used software to analyze, visualize or simulate data [1]. Scientists are researchers in several domains, such as sciences, computing, E. Mourão (B) · D. Trevisan · J. Viterbo Fluminense Federal University, Institute of Computing, Niteroi, Rio de Janeiro, Brazil e-mail: [email protected] D. Trevisan e-mail: [email protected] J. Viterbo e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_24

275

276

E. Mourão et al.

engineering, humanities, and other fields, which develop, maintain, and or use software to conduct or support their research [2]. Modern research depends on research software that is most often developed by researchers often within academia, faculty, and university [3]. Software engineering researchers have developed several important practices that result in a software overall with higher quality as: documentation, design patterns, management, version control, peer code reviews [4, 5]. However, there are a number of factors that result in the difference between the development of research software and traditional software [2]. In the academy, many software projects are developed and maintained over a longer time period than the academic tenure of any postdoc or faculty. Most software until is developed in an ad hoc manner, without frameworks and tools to reduce the work to be done [6]. Additionally, best practices that provide researchers methods for performing development and maintenance are not widely used [3]. Several studies about factors that determine the success of traditional software projects are performed in the organizations or industry [7–11]. However, the literature investigating is scarce, and there is a gap regard to identifying which characteristics of the research software that determine it to be a success. Therefore, the goal of this paper is to understand what are the factors that determine successful research software in the context of the Brazilian academic researcher. Additionally, we investigate the factors that lead to a non-success and identify the challenges to an academic research software achieving success. We conducted a case study where we used semi-structured interviews to investigate the research questions [12]. The interviewees are twenty expert academic researchers that have experience in the use and development of research software. They are users of research software, and some of them have the role of conducting the research by guiding developers in the build academic research software. Thus, these experts can explain what factors determine the success of research software, the nonsuccess, and the challenges. We analyze the data collected according to Grounded Theory to identify the factors in the answers of the experts. The paper is organized as follows. In Sect. 2, we provide related works. In Sect. 3, we present the method, research questions, study design, data collect, and analyze. In Sect. 4, we present the results. In Sect. 5, we discuss the findings. In Sect. 6, we discuss the threats to validity and limitations. Finally, in Sect. 7, the final remarks and future works are presented.

2 Related Works The search for related works was performed using Systematic Literature Mapping (SLM) [13] to categorize and summarize studies related to software engineering models, practices, principles, and processes that contribute to research software. The period of search for studies was unrestricted. The hybrid search was used, which consisted of using DatabaseSearch in Scopus to select the set of relevant initial

Understanding the Success Factors of Research Software ...

277

articles followed by parallel Backward and Forward Snowballing [14]. We selected the articles that were most similar to our problem. Katz et al. [3] discuss that software is recognized as an essential part of the research. The study presents organizational structures models for research software development at universities that include the use of software engineering practices and processes. As result, they present three different models for research software development: one traditional Research Software Engineering (RSE) group in the UK at the University of Manchester, and units in two US universities. Heaton and Carver [4] performed a systematic literature review examining claims about how scientists develop software. The study contains 33 different claims about 12 software engineering practices. As result, the majority of the claims indicated that software engineering practices are useful for scientific software development. Elvira et al. [5] performed a systematic mapping study on the use of software engineering practices for scientific application development and their impact on software quality. The authors describe a catalog of SE practices used in scientific software development and discuss the quality attributes of interest to the practices. The results indicate that scientific software developers are focusing on practices that improve implementation productivity and that maintainability is important. de Souza [15] reports the results of an interview study examining how research software engineers subjectively define software sustainability. The interviews were conducted with nineteen research software engineers from five institutions: three UK university and UK Government-funded research facility. The authors used thematic analysis of the data to reveals two interacting dimensions: intrinsic, which relates to internal qualities of software, and extrinsic, concerning cultural and organizational factors. Therefore, we did not find studies related to factors adopted by academics on a research software project that lead to success in the context of Brazil.

3 Method In this section, we present the methodology that we used in this paper. We conducted an empirical research using case study methodology [12]. Our research method is based on the GQM [16] paradigm to define our goals, research questions, and metrics. Our research is exploratory and the data collection method is semi-structured interviews. Finally, the qualitative research method was Grounded Theory [17], which provides a set of procedures for constructing theoretical explanations from collected data. According to GQM paradigm [16], our study aims to analyze research software with the purpose of characterizing with respect to the factors of success, non-success, and challenges regarding the development and adoption of this software from the point of view of Brazilian academics researchers in the context of the software use and development in academia.

278

E. Mourão et al.

In this study, we used the following Research Questions (RQs): RQ1. What are the factors that make an academic initiative a success? We formulated this research question with goal to understand the characteristics or factors that lead research software to become a success. RQ2. What are the factors that make an academic initiative a non-success? We formulated this research question with goal to investigate the characteristics of a non-success research software. RQ3. What are the challenges to make an academic initiative a success? We formulated the research question with goal to identify research challenges to determine software success. To answer those research questions, we planned and conducted an exploratory study using a semi-structured interview based on a questionnaire. We invite academic researchers who are subject-matter experts with in-depth knowledge of academic software areas to participate in our interviews. The metric was based on subjective assessments of the answers to the open questions for collecting participants’ opinions.

3.1 Study Design A semi-structured interview was planned with the aim of investigating the success, non-success, and challenges in research software development in the academy by academics. The interview script was characterized by open-ended questions of a questionnaire, with the aim to keep respondents free to answer questions inviting a broad range of answers. We used a questionnaire to support the participants to answers our questions. We developed a questionnaire in Google Forms1 to accompany the interview, which consisted of the following sections: (i) the objective of the study and the consent form, to participants read and accept the explanatory statement about goals, confidentiality and data privacy of the study; (ii) professional profile; (iii) background in research software; (iv) open-ended questions related to research software. Our samples’ composition from a target population included Brazilian researchers professors who are users and or work in software research development. We defined the following acceptance criteria for selecting interviewees to contact: experience in research software, domain and knowledge in their academics project, and convenience. We selected researchers from our affiliation and collaboration network, comprising contextual diversity. Based on these criteria, we invited the participants by e-mail.

1

https://docs.google.com/forms/.

Understanding the Success Factors of Research Software ...

279

3.2 Data Collection We conducted interviews from July to October 2021, and each interview took 60 min. The period of four months was established to select researchers whom we inserted in this context. The interviews were conducted online via video conferencing using the Google Meet2 tool, allowing the participation of interviewees in their geographical locality. Before the interview started, the participants were requested to read the explanatory statement of the research and fill out online the interview consent form of the questionnaire in Google Forms. In accordance with the consent form, all information provided by the participants was kept confidential, assuring that their names and email addresses would not be recorded with their interview and questionnaire responses. We did not offer any financial rewards to the participants. We started the interview with questions about experience, time, and frequency of use of research software. Then, we asked questions for participants to identify factors that make successful software, factors that make a non-success software, and challenges that make a successful software. Due to the Covid-19 pandemic and social distancing, some researchers responded to invitation email stating that they have not had time available for an interview. So we decided to send the questionnaire of the interview for the researcher to answer the questions.

3.3 Data Analysis The data collected were analyzed qualitatively using procedures of the Grounded Theory [17], widely used to generate the theory from the data, which is composed of three main procedures: open coding, axial coding and coding selective. First, we read the transcriptions of all videos and collect the answers of the questionnaire. After data collection, we followed Grounded Theory procedures to analyze the data. The Atlas.ti3 is a qualitative data analysis software used by researchers. We used it in version 9.1 to support the coding steps. We performed open coding to associate codes with quotations of respondents. After that, we related the codes through axial coding. In this procedure, the codes were merged an grouped into more abstract categories. Then, we used selective coding to identify core categories that best explain factors of success, non-success, and the challenges in the research software. Finally, we analyzed the data to address the RQs. The dataset is available at https://doi.org/ 10.5281/zenodo.7267956

2 3

https://meet.google.com/. https://atlasti.com/.

280

E. Mourão et al.

4 Results In this section, we present the participants’ demographics, factors that make an academic initiative a success (RQ1), factors that make it a non-success (RQ2), and the challenges to make an academic initiative a success (RQ3).

4.1 Respondents’ Demographics The professional role of the all expert participants is high education professors and researchers Doctors in Computing Science. We select twenty participants from eight higher education institutions in the different States of Brazil, such as Rio de Janeiro (RJ), Paraná (PR), Mato Grosso (MT), and Brasília (DF). In total, 95% of participants work in public higher education institutions, comprising 85% in federal universities and 10% in state universities, and 5% (one participant) works in a private institution classified as a university. We anonymize the names of institutions to ensure the privacy of participants. Regarding genre, 65% are male participants (M), and 35% are female (F) of our collaboration network, thus showing the representation of both genders. In total, 85% of the participants have 10 or more years of professional experience with research software. The 10% of the participants have 5 or more years of experience. Only 5% of the participants have less than 5 years of experience. Thus, it seems that the participants have considerable experience with research software. Regarding the frequency of use, 50% of them use frequently, 40% use always, and 10% sometimes use research software. Hence, note that the majority of experts frequently followed by always use research software. Regarding the research subject-matter area, the experts have in-depth knowledge in specific academic software areas: 55% in Software Engineering, 15% in Artificial Intelligence, 15% in Computing Systems, 10% in Algorithms and Optimization, and 5% in Visual Computing. In addition, the participants work on various research topics within the research area of Computing Science. Thus, the participants comprise contextual diversity in the software domain and knowledge area. Finally, they are users and workers with development of research software.

4.2 What are the Factors that Make an Academic Initiative a Success (RQ1)? To answer this question, the procedures of open coding and axial coding of the Grounded Theory were used, with the objective of identifying success factors of

Understanding the Success Factors of Research Software ...

281

Table 1 Factors of Software Usage and Disclosure Factor Comment Be widely used within the academy.

Be widely used outside academia.

Have comprehensive user documentation on different media. Have clear and consistent documentation for using the software. Be widely cited in scientific articles. Have articles published about the software.

Be publicized in social networks, events and media. Be recognized as a reference for the research area.

“Wide membership by the academic community.” (P08), “Membership of the entire academic community ...” (P06), “Community usage ...” (P05), and “Community adoption ...” (P09). “Adoption outside the academy ...” (P03), “... use by industry.” (P04) and “Widely used.”’(P11). “... have extensive documentation including articles, videos and books for your wide use.” (P01) and “... well documented” (P06). “Have good documentation.” (P14) and “Have good support and documentation.” (P19). “Success ... for research are citations.” (P02). “Good dissemination of results.” (P03) and “Success is the impact of publication on research ...” (P02). “User engagement.” (P11) and “Be a very active academic community in the development of modules ...” (P01). “Students working on research on the topic and either using the software or promoting proposals for evolution ...” (P18) and “Community recognition.” (P10).

research software (RQ1). During the open and axial coding activities, it was possible to perceive, through the analysis of the generated codes and the identified relationships, two fundamental categories for this research, also denominated dimensions: (1) Software Usage and Disclosure, and (2) Software Quality. The factors are subcategories of these categories. The factors and some comments of participants4 related to Software Usage and Disclosure category are shown in Table 1 and Software Quality category are shown in Table 2. In the Software Quality category, after axial coding, we identified a high similarity of codes with the quality characteristics of the product quality model defined in ISO/IEC 25010 [18] and then we used in the factors.

4

All answers of participants were translated from Portuguese to English by the authors.

282

E. Mourão et al.

Table 2 Factors of Software Quality Factor Comment Functional Adequacy. Performance. Compatibility. Usability. Reliability Security. Maintainability.

Portability.

“The solution must be open source ...” (P01) and “Be freely accessible, interface with clear/well-defined processes, data visualization.” (P09) “robustness, open source and free.” (P07) and “Collaboration, ease of use and open source.” (P05) “... be integrated with other technologies...” (P14) “Ease of use.”(P08), “User-friendly interface.” (P01), “Utility ...” (P12) and “Free access availability ...” (P17) “Maturity, precision, robustness.” (P09) and “... application maturity ...” (P12) “Reliable, Stable.” (P11) “Constant evolution in the tool.” (P18), “Extensible, reusable and modifiable ...”(P20), “Be a well tested product.” (P06), and “Be open source, have a very active community in the development of modules.” (P01) “Be integrated with other technologies.” (P14)

4.3 What are the Factors that Make an Academic Initiative a Non-success (RQ2)? To answer this question, we used the open coding and axial coding of the Grounded Theory with the objective of identifying non-success factors of research software (RQ2). We identify factors that determine the non-success of the research software and the comment mentioned by participants, as shown in Table 3.

Table 3 Factors of Non-Success Factor Not having ease of use. No documentation on its use.

Comment

“... difficulty in use and perception of value.” (P08) “Lack of adequate documentation ...” (P08) and “... lack of support and documentation.” (P19) No scientific disclosure about the software. “... lack of communication and proper advertising strategy ...” (P08) Not having quality. “Not having ... quality of functions.” (P05) Not having open source. “Not be open source.” (P01) and (P05) Not being free. “Not be free.” (P09) and (P19) No maintenance and continuous evolution. “The initiatives were created ... and use of the initiative, but was not maintained afterwards.” (P16) No adoption by researchers/ professors. “... sometimes research is not the professor’s area of research to continue the research.” P(20)

Understanding the Success Factors of Research Software ...

283

4.4 What Are the Challenges to Make an Academic Initiative a Success (RQ3)? To answer this question, we used the open coding and axial coding of the Grounded Theory to identify in the answer of the participants the main challenges to make an academic initiative a success (RQ3) as shown in Table 4. Table 4 Challenges to Achieve the Success Factor Have funding.

Have maintenance.

Have engagement.

Have disclosure.

Have quality.

Comment “Lack of funding and incentives for research, lack of alignment with real issues ...” (P03) and “... have resources so that the initial investment made is maintained.” (P16) “... in the case of tools as a result of research, lack of staff to keep the tool updated.” (P03) and “... need to create the software and the number of people dedicated to creating it.” (P05) “Engage those who conceived the solution to engage the community to use and evolve this solution ...” (P01) and “Community engagement in adoption and maintenance.” (P17) “One challenge is to publicize the initiative.” (P15), “... sponsorship, interest of some group, marketing.” (P14), and “... dissemination and popularization ...” (P17) “... the solution must be open source, and the community must make videos and materials available to facilitate the use of the solution ...” (P01) and “... control of the evolution of the initiative ...” (P20)

5 Discussion In this section, we discuss the implications of our study findings in RQs. They suggest helping to fill the gaps between the topics studied in academia and practices applied in software research. Implication of Finding in RQ1. The Brazilian academy and universities could provide training about software usage and disclosure, best practices of marketing, for the students and beginners in the research career with the advisor closely monitoring the research. They could structure the main findings and maintain a guide of best prac-

284

E. Mourão et al.

tices for the software be used, maintained such as be free, open source, engaged by the development team and users, has a good dissemination, has quality and software engineering characteristics, organizational interest in software, and mainly funding. Implication of Finding in RQ2. The academy should increase researchers’ awareness about the importance of producing and maintaining software with characteristics of quality, software engineering, and software development process, as good documentation and active support of developers community, version control, requirements management, and testing. On the other hand, scientific dissemination is an efficient means to propagate knowledge and must act with greater intensity. In addition, the findings reinforced the importance of developing open source and free software, contributing to open science and software adoption. Implication of Finding in RQ3. In general, expert software researchers have a high level of academic research knowledge. Therefore, the academy should integrate senior researchers with novice researchers to improve the success of academic initiatives and provide further discussions on the issues in the academy. Additionally, the university could have a project office to choose which research has potential and pay a team to turn it into a product. Ideally, the project office would be self-sufficient, being financed by the projects it manages to successfully carry out without needing external funding.

6 Threats to Validity Although we aimed at reducing the threats to validity of our study methodology, some decisions may have affected the results [12]. We present the threats: Regarding External Validity, we invited researchers in the Computing area with knowledge of research software from our affiliation and collaboration network to the interviews. Due to the small sample, consisting of twenty participants, our results may not be generalized to all other computing fields. Regarding Internal Validity, a threat is that to contextualize the participants, the first questions in the interview questionnaire were about the successful software Weka5 , developed by academics and initially for academics. Thus, to mitigate bias in the interpretation of findings, other author was consulted in case of any doubt during the coding answers. Regarding Construct Validity, a threat is that possible we missed the name of other experts and were excluded from the interview. In addition, some participants responded to the online questionnaire that guided the interview rather than responding during the video conference. Regarding Conclusion Validity, a threat is that we did not adopt statistical tests during the data analysis qualitative.

5

https://www.cs.waikato.ac.nz/ml/weka/.

Understanding the Success Factors of Research Software ...

285

7 Conclusions and Future Works The overall goal of this paper was to advance the research in the area by identifying factors that lead to the success of research software from the perspective of Brazilian academic researchers. First, we defined the goal, research questions, and metrics using the empirical study approach GQM. After that, we conducted interviews with expert researchers in the academy. Finally, we used Grounded Theory to identify the factors that determine success, non-success, and challenges to achieving success in the context of academic research software. The results of this work will provide insights we continue addressing in future research. Due to the nature of this exploratory study and the identification of such success factors are a complex problem, an empirical strategy of type survey is suggested in the future because it would be performed to validate the results obtained from this study. Such a survey would help examine the importance of the factors obtained. Our research provided information of interest to start research about the adoption of academic research software and identify the level of importance of the success factors, non-success, and challenges in the context of Brazilian academics in empirical software engineering research. Acknowledgement This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001 and Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ). The authors thank CAPES, FAPERJ and those who kindly contributed to this research.

References 1. Hannay JE, MacLeod C, Singer J, Langtangen HP, Pfahl D, Wilson G (2009) How do scientists develop and use scientific software? In: ICSE Workshop on Software Engineering for Computational Science and Engineering, pp. 1–8 2. Eisty NU, Carver JC (2022) Developers perception of peer code review in research software development. Empir Softw Eng 27(1):1–26 3. Katz DS, McHenry K, Reinking C, Haines R (2019) Research software development & management in universities: case studies from Manchester’s RSDS group, Illinois’ , and Notre Dame’s CRC. In: 14th International Workshop SE4Science, pp. 17–24 4. Heaton D, Carver JC (2015) Claims about the use of software engineering practices in science: a systematic literature review. Inf Softw Technol 67:207–219 5. Arvanitou EM, Ampatzoglou A, Chatzigeorgiou A, Carver JC (2021) Software engineering practices for scientific software development: a systematic mapping study. J Syst Softw 172:110848 6. Carver JC, Gesing S, Katz DS, Ram K, Weber N (2018) Conceptualization of a US research software sustainability institute (URSSI). Comput Sci Eng 20(3):4–9 7. Chow T, Cao DB (2008) A survey study of critical success factors in agile software projects. J Syst Softw 81(6):961–971 8. Dikert K, Paasivaara M, Lassenius C (2016) Challenges and success factors for large-scale agile transformations: a systematic literature review. J Syst Softw 119:87–108

286

E. Mourão et al.

9. Nasir MHN, Sahibuddin S (2011) Critical success factors for software projects: a comparative study. Sci Res Essays 6(10):2174–2186 10. de Souza Bermejo PH, Zambalde AL, Tonelli AO, Souza SA, Zuppo LA, Rosa PL (2014) Agile principles and achievement of success in software development: a quantitative study in Brazilian organizations. Proc Technol 16:718–727 11. Lindsjørn Y, Sjøberg DI, Dingsøyr T, Bergersen GR, Dybå T (2016) Teamwork quality and project success in software development: a survey of agile development teams. J Syst Softw 122:274–286 12. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in Software Engineering. Springer Science & Business Media, Heidelberg. https://doi.org/10. 1007/978-3-642-29044-2 13. Petersen K, Vakkalanka S, Kuzniarz L (2015) Guidelines for conducting systematic mapping studies in software engineering: an update. Inf Softw Technol 64:1–18 14. Mourão E, Pimentel JF, Murta L, Kalinowski M, Mendes E, Wohlin C (2020) On the performance of hybrid search strategies for systematic literature reviews in software engineering. Inf Softw Technol 123:106294 15. de Souza MR, Haines R, Vigo M, Jay C (2019) What makes research software sustainable? An interview study with research software engineers. In:12th International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 135–138 16. Basili VR, Caldiera G, Rombach HD (1994) Goal, question metric paradigm. In: Encyclopedia of Software Engineering, vol. 1 17. Corbin J, Strauss A (2014) Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, Sage 18. Estdale J, Georgiadou E (2018) Applying the ISO/IEC 25010 quality models to software product. In: Larrucea X, Santamaria I, O’Connor RV, Messnarz R (eds) EuroSPI 2018, vol 896. CCIS. Springer, Cham, pp 492–503. https://doi.org/10.1007/978-3-319-97925-0_42

Software Systems, Architectures, Applications and Tools

Server-side Adaptive Federated Learning over Wireless Mesh Network Felix Freitag, Lu Wei, Chun-Hung Liu, Mennan Selimi, and Luís Veiga

Abstract In federated learning, distributed nodes train a local machine learning model and exchange it through a central aggregator. In real environments, these training nodes are heterogeneous in computing capacity and bandwidth, thus their specific characteristics influence the performance of the federated learning process. We propose for such situations the design of a federated learning server that is able to adapt dynamically to the heterogeneity of the training nodes. In experiments with real devices deployed in a wireless mesh network, we observed that the designed adaptive federated learning server successfully exploited the idle times of the fast nodes by assigning them larger training workloads, which led to a higher global model performance without increasing the training time. Keywords edge computing · federated learning · machine learning

F. Freitag (B) Universitat Politècnica de Catalunya, BarcelonaTech, Spain e-mail: [email protected] L. Wei Texas Tech University, Lubbock, TX, USA e-mail: [email protected] C.-H. Liu Mississippi State University, Starkville, MS, USA e-mail: [email protected] M. Selimi Max van der Stoel Institute, South East European University, Tetovo, North Macedonia e-mail: [email protected] L. Veiga INESC-ID Lisboa, Instituto Superior Técnico, University of Lisbon, Lisbon, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_25

289

290

F. Freitag et al.

1 Introduction Federated learning is a recent paradigm for collaboratively training machine learning models with distributed nodes [1]. Different from centralized training in which all the training data are managed by a single node, in federated learning each node has its own training data to train a local model. The subsequent aggregation of the local models in a central aggregator leads to a new global model. One important advantage of federated learning is that the local training data does not need to leave the node where it is used for training, which preserves the privacy of the data. Federated learning uses a distributed computing infrastructure, and therefore heterogeneity can be expected in several forms: 1) The quantity and quality of local data at each node may vary. For instance, local data may be acquired by sensors at or nearby a node, but the local circumstances of each node can lead to a different number of training samples. 2) The computing capacity of the nodes can be different either due to the proper hardware of the nodes or by concurrent executions of other applications at the node, which reduce the available computing resources dedicated to the federated learning process. 3) For exchanging the trained model, some nodes may face limited network bandwidth either due to permanent or dynamic network conditions. In this paper, we address the federated learning process in a heterogeneous environment with the goal to develop an adaptive design of the federated learning server for exploiting this heterogeneity. Specifically, we propose the federated learning server to determine at each round the workload capacity of each node for model training in a determined time slot. Applying our design, each client receives from the server an individual training instruction that includes the number of samples in each training round. As a consequence, the server is able to exploit the computing capacity of the faster nodes in the training process. In federated learning, this leads to higherperforming global models during the training process compared to the non-adaptive design. We experiment with the proposed adaptive federated learning server design with distributed computing devices consisting of mini-PCs or Single-Board Computer (SBC). This scenario was chosen as being representative of realistic user environments, where computing devices run as home servers to manage several services while at the same time each server is connected over a network with other servers to collectively perform federated learning. The heterogeneity of the nodes is induced by the different hardware of each of these home servers. In addition, the heterogeneity comes from the fact that in real-world usage these devices will not be dedicated exclusively to the federated learning application but concurrently run other useroriented services, which results in each training node having a potentially variable computing capacity, thus making our scenario with heterogeneity realistic. The main contributions of the paper are: 1. We develop the design of an adaptive federated learning server being able to exploit the clients’ heterogeneity that leads to an increase in the performance of the global model.

Server-side Adaptive Federated Learning over Wireless Mesh ...

291

2. We provide the evaluation of the proposed design by experimenting with its implementation on real distributed low-capacity devices connected to a wireless mesh network.

2 Background and Related Works Federated learning has raised the interest of the research community as a technique for model training that does not require the sharing of a node’s local data. An important area for federated learning is the application to wireless communications such as 5G, where the edge nodes generate valuable data for applications but at the same time, these data must be held private [2, 3]. Federated learning can be considered a distributed computing process requiring computing and communication resources. This translates federated learning into having local computing capacities at the nodes which perform the training, as well as to have the communication capability that allows to transmit the machine learning models between the training nodes and the central aggregator. While powerful edge nodes such as that of the 5G system do have such computing and communication capacities to support federated learning, other edge nodes like SBC and tiny embedded Internet of Things (IoT) devices may have computation and communication challenges [4, 5]. The heterogeneity of the communication capacity in federated learning scenarios was addressed by Jiang et al. in [6]. Their work on the BACombo system considers scenarios of training nodes represented by mobile phones or embedded devices, where the network connectivity of each node is different. The solution proposed in the BACombo federated learning system is to leverage the network between the training nodes for pulling segments of the model updates between clients. The process is supported by a worker selection step at each client that monitors the bandwidth to each of the other workers. It can be observed that in BACombo the heterogeneity is addressed on the client side, i.e., the training nodes are given an extended decision capacity to influence the federated learning process. Differently, in the design that we propose, the decision on the training parameters of each client is made at the server. Federated learning in edge environments was proposed in several works, as surveyed in [7]. Specific types of edge devices are investigated for instance in the Flower framework, where Android phones, Raspberry Pi, and NVIDIA Jetson were used [8]. The work on Flower proposes a framework that first addresses the hardware heterogeneity of the clients by providing client-specific software implementations. For instance, the federated learning client for Android phones consists of a Java implementation applying a specific TensorFlow Lite Model Personalization support for Android Studio. The federated learning client for the Raspberry Pi and NVIDIA Jetson is implemented in Python. Secondly, in order to address the different hardware capacities of the clients, the Flower architecture proposes a strategy component as part of the federated learning server that assigns to each client a fixed cutoff time for

292

F. Freitag et al.

sending back the model based on previous offline observations. The value chosen for this cutoff time results in a trade-off between the training time and the accuracy of the model. Differently, in our design, we propose to apply an adaptive training configuration, where the training parameters for each client are determined by the server at each training round and are based on the online client performance observed in previous rounds. In the work of Wang et al. [9], an adaptive federated learning approach is proposed which focuses on the frequency of performing global aggregations under resource constraints. The problem addressed is the optimization of resource consumption of federated learning, both related to computation and communication. The work proposes a control algorithm to determine in real-time after how many local training epochs the model data is sent back to the aggregator node, targeting to minimize a loss function under resource budget constraints. Similar to our approach, in their work the decisions are taken on the server side and are calculated along the training process. The difference is that we propose the server to exploit the faster client’s computing capacity and reduce its idle time. Another work of Wang et al. [10] proposes a communication-efficient compressed federated adaptive gradient optimization framework, FedCAMS, which largely reduces the communication overhead and addresses the adaptivity issue in federated optimization methods. Differently, FedCAMS is evaluated locally and not in a wireless mesh network. The evaluation is performed by simulations and some experiments in real nodes consisting of 3 Raspberry Pi and 2 laptop computers. In FedMax [11], a highly-efficient federated learning framework is presented. The heterogeneity of the worker nodes is related to the context of the IoT. Two measures are suggested, which are relaxed worker synchronization for tolerating dropouts of sporadic workers, and similarity-based worker selection, which aims to select a subset of the most efficient workers. FedMax is similar to our approach as it measures the client performance at the server and allows the server to make decisions for each worker, such as assigning a different amount of training load to these workers. However, FedMax is evaluated in the Google Cloud Platform, where the heterogeneity is configured by the number of CPUs per VM having homogeneous communication. In our work, we use a real edge environment with different SBC nodes interconnected over a wireless mesh network with links of different communication capacities. In our own previous work [4], we demonstrated for the real deployment of federated learning in a wireless mesh network how the heterogeneity of the clients’ computing capacity and that of the communication to the server affects the federated learning process. We showed how slower clients, either due to bandwidth or computing limitations, delay the federated learning process. Therefore, in the current paper, we present as a continuation of the situation observed in our earlier work a federated learning server being able to adapt the training load to the capacity of each client.

Server-side Adaptive Federated Learning over Wireless Mesh ...

293

3 Design of Adaptive Federated Learning 3.1 Server-side Adaptive Federated Learning We consider a scenario where a server conducts federated learning rounds with a set of heterogeneous clients. This heterogeneity is induced by different bandwidths, different client hardware, and/or different computing capacity usage at each client node. Federated learning uses a star topology, where a central node, i.e., the federated learning server, orchestrates the training with the registered clients (Fig. 1). The role of the federated learning server at the end of a training round is to merge the local models received from the clients into a new global model. For the next round, this new model is sent out to the clients, along with training parameter values. We propose to add the capacity of adaptation to the federated learning server. Thus, the server determines for each round the learning parameters for each client. Therefore, the interaction between the server and clients not only includes the sending of the new global model to the clients, but also the individual learning parameters. In order to orchestrate adaptive federated learning, the server takes measurements at each training round. A key metric that the server calculates is the workload capacity, which is a client-specific metric. The metric is defined as the number of samples by epochs trained divided by the time that passed between sending the global model to the client and receiving the local model back. The metric contains the time spent for the communication of the model, from the client to the server and back, and the time of the proper model training at the client.

Fig. 1 Overview of the Architecture of server-side adaptive federated learning.

294

F. Freitag et al.

3.2 Federated Learning Implementation The federated learning network we use for the experimentation is implemented in Python language. The system is composed of two major components which are the code for the client and the server. In our implementation, the server sends both the model parameters and the learning parameters, which relate to how the training has to be done at the clients. These learning parameters are the learning rate, number of local training epochs, and batch size. It is specific to the adaptive server design that the server makes a distinction between the different client nodes, i.e., each client can receive a different value of the learning parameters. These data are sent between servers and clients in JSON format over HTTP POST messages. Both the server and the clients implement a REST API. For both the federated learning server and the client code, we create Docker images in order to instantiate them with Docker containers on the different devices used in the experimentation. The source code of the federated learning network is available on Github1 . Additional information on the code design can be found in [12].

4 Experimental Evaluation of the Adaptive Federated Learning Network 4.1 Experimental Environment The objective of the experimentation is to observe the effect of the adaptive server design in federated learning running in a real edge environment. For conducting the experimentation we use SBC connected to the GuifiSants network2 , a wireless city mesh network. It is a subset of the larger Guifi.net community network3 . This is the same experimental environment we already used in our previous work [4]. In order to experiment with hardware heterogeneity, we use three types of devices for the clients in the federated learning network. Two of them are SBC, specifically the Minix mini-PC NEO Z83-4 with Intel Atom x5-Z8350 processor and 4GB DDR3 RAM4 and the PC Engine APU2 with an AMD Embedded G series GX-412TC processor and 4 GB DDR3 RAM5 . The third type of hardware used for running a federated learning client was a laptop with an i5 processor, hosting also the federated learning server. Figure 2 illustrates the deployment of the three different clients.

1

https://github.com/eyp/federated-learning-network. http://sants.guifi.net/. 3 guifi.net: Commons Telecommunication Network Open Free Neutralhttp://guifi.net/. 4 https://minix.com.hk/products/neo-z83-4-pro. 5 https://pcengines.ch/apu2e4.htm. 2

Server-side Adaptive Federated Learning over Wireless Mesh ...

295

Fig. 2 Testbed nodes within GuifiSants used for the experimentation.

With regards to the communication heterogeneity, it can be observed in Fig. 2 that the two SBC clients have a rather low capacity link with the server, different to the third client that runs on the same machine as the server. The federated learning task to be executed in the experiments is to train a 6-layer Convolutional Neural Network (CNN) model with the Chest_X_ray dataset6 . The CNN model has around 420, 000 parameters.

4.2 Server-side Adaptive Federated Learning with Heterogeneous Clients In this section, server-side adaptive federated learning is studied experimentally. In this experiment, we use three types of hardware to host three federated learning clients. Specifically, we use a VM with two cores running on a host with an i5 processor, the Minix device and the APU2 device, which in terms of computational capacity results in having a fast, medium, and slow client. We perform 50 training rounds. Experiment 1: Baseline. In this experiment, we measure different metrics when the clients train with 6 training samples each round along a total of 50 training rounds (Fig. 3). Slight variations in the fastest client along the 50 rounds are due to the fact that this client was running on a non-dedicated machine. 6

Chest X-Ray Images. https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia.

296

F. Freitag et al.

Experiment 2: Adaptive server. In this experiment during the first 10 rounds, the server applies its default behavior (without being adaptive). Then, from rounds 11 to 50 the server applies the adaptive behavior. The experimental setting for training was having a minimum number of 6 training samples (which the server can increase due to the adaptive behavior) and inference after training was done with 200 test samples at all clients. Since the number of training samples used by each client varies with the adaptive behavior, the federated averaging algorithm at the server was extended to apply weighted averaging. Figure 4 shows the obtained results when the adaptive server is used. In Fig. 4 it can be seen how the training time of the fast clients after 10 rounds increases to that of the slow clients. This is due to the fact that the server increased the number of training samples for the fast clients according to each client’s workload capacity. Specifically, the client running on the Minix mini-PC is trained with about one hundred samples, the client on the APU2 devices with up to around 800 training samples, and the fastest client on the i5 laptop is trained with the original 6 samples. Comparing the accuracy achieved by the adaptive federated learning server in Fig. 4c with the accuracy of the baseline training (Fig. 3c), it is observed that with the adaptive server

Fig. 3 Baseline. Behavior of three clients.(a) Training time. (b) Server-measured client rythm. (c) Test accuracy. (d) Test loss.

Server-side Adaptive Federated Learning over Wireless Mesh ...

297

Fig. 4 Adpative server. Behavior of three clients.(a) Training time. (b) Server-measured client rythm. (c) Test accuracy. (d) Test loss.

it is significantly higher. This is expected since the number of images used for training in the adaptive server configuration is higher. However, since the reference time for adjusting the number of training images for the fast clients is the training time of the slowest client, the overall time for training the model in both baseline and adaptive federated learning is similar.

5 Conclusions This paper presented a server-side adaptive federated learning design. The experimentation with the developed implementation was conducted in a real environment with SBC and wireless network connectivity. It was shown with the adaptive design the server successfully exploited the idle time of the faster clients by assigning them higher training workloads. The better-performing local models of the faster clients improved the overall global model performance by the model aggregation in the federated learning server without increasing the training time.

298

F. Freitag et al.

The results may have interesting applicability in other resource-constrained edge scenarios. Specifically, we aim to extend our results to embedded IoT devices, where the studied design could address critical computing and communication resource constraints. Another line of work is to combine the server-side adaptation with that on the client side. Acknowledgment This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871582 - NGIatlantic.eu and was partially supported by the Spanish Government under contracts PID2019-106774RB-C21, PCI2019-111851-2 (LeadingEdge CHIST-ERA), PCI2019-111850-2 (DiPET CHIST-ERA), and by national funds through FCT, Fundação para a Ciência e a Tecnologia, Portugal, under project UIDB/50021/2020. The work of C.-H. Liu was supported in part by the U.S. National Science Foundation (NSF) under Award CNS-2006453 and in part by Mississippi State University under Grant ORED 253551-060702. The work of L. Wei is supported in part by the U.S. National Science Foundation (#2006612 and #2150486).

References 1. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. 10(2):1–9 2. Niknam S, Dhillon HS, Reed JH (2020) Federated learning for wireless communications: motivation, opportunities, and challenges. IEEE Commun Mag 58(6):46–51 3. Ibraimi L, Selimi M, Freitag F (2021) Bepoch: Improving federated learning performance in resource-constrained computing devices. In: IEEE Global Communications Conference (GLOBECOM) 4. Freitag F, Vilchez P, Wei L, Liu CH, Selimi M (2022) Performance evaluation of federated learning over wireless mesh networks with low-capacity devices. In: Rocha Á, Ferrás C, Méndez Porras A, Jimenez Delgado E (eds) Information Technology and Systems. Springer International Publishing, Cham, pp 635–645 5. Llisterri Giménez N, Monfort Grau M, Pueyo Centelles R, Freitag F (2022) On-device training of machine learning models on microcontrollers with federated learning. Electronics 11(4):573 6. Jiang J, Hu L, Hu C, Liu J, Wang Z (2020) Bacombo-bandwidth-aware decentralized federated learning. Electronics 9(3):440 7. Abreha HG, Hayajneh M, Serhani MA (2022) Federated learning in edge computing: a systematic survey. Sensors 22(2):450 8. Mathur A, et al (2021) On-device federated learning with flower 9. Wang S et al (2019) Adaptive federated learning in resource constrained edge computing systems. IEEE J Sel Areas Commun 37(6):1205–1221 10. Wang Y, Lin L, Chen J (2022) Communication-efficient adaptive federated learning 11. Xu H, Li J, Xiong H, Lu H (2020) Fedmax: enabling a highly-efficient federated learning framework. In: 2020 IEEE 13th International Conference on Cloud Computing (CLOUD), pp. 426–434 12. Parareda EY (2021) Federated learning network: Training distributed machine learning models with the federated learning paradigm

Mobile Development with Xamarin: Brief Literature, Visualizations and Important Issues Renato M. Toasa , Paúl Francisco Baldeón Egas , Henry Recalde , and Miguel Gaibor Saltos

Abstract Mobile applications and devices play a very important role in everyone’s life, due to their easy mobility, small size and interaction with people. But at the time of developing these applications it is a problem to identify the right platform that allows an adequate development. In this context, this paper proposes a literature review of the Xamarin mobile development platform, identifying the current state of development, the visualizations presented by the platform and certain important considerations. It can be stated that there is a great need for research on everything involved in mobile development, so that developers and users can benefit from the use of multiplatform tools. As a result, it is identified that Xamarin is a very robust platform for mobile development, and is one of the most used and popular worldwide. Keywords Mobile · Cross-platform · Xamarin · Xamarin.Forms

1 Introduction Over the years, the development of mobile applications has grown by leaps and bounds, because most humans spend more time connected to their electronic devices such as phones, tablets, computers or televisions, and they need applications to meet their needs of all kinds, from educational, work, to games. As a result, the landscape

R. M. Toasa (B) · P. F. B. Egas · H. Recalde · M. G. Saltos Universidad Tecnológica Israel, Quito, Ecuador e-mail: [email protected] P. F. B. Egas e-mail: [email protected] H. Recalde e-mail: [email protected] M. G. Saltos e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_26

299

300

R. M. Toasa et al.

has become seriously complicated, which brings many challenges to the mobile application development process [1]. To maximize its presence in the market, a mobile application must run on as many devices as possible. One solution consists of developing native applications on each of the existing platforms, the predominant platforms worldwide being Android and iOS [2]. The native mobile development approach remains a predominant and stable way to develop applications for a particular platform [3], however, developers have put great interest in multiplatform development, which are characterized by being created under a single programming language that facilitates its export and therefore its visualization in any type of device regardless of its operating system [4]. Since the applications are developed with the same language, only minimal changes are necessary for their complete adaptation to any device and platform. Cross-platform development, unlike native development, focuses on code reuse. The construction of mobile web applications is an example that represents this approach [5], the limitations derived from running within a web browser, has motivated mobile application developers to turn their attention to other types of cross-platform applications that deliver results closer to native solutions. In this sense, there are different mobile platforms for multiplatform development, among the most widely used and that are trending worldwide are [6]: • Xamarin: It is an open source and free framework developed by Microsoft, use the c# and XAML programing languages. • Flutter: It is a framework developed by Google and uses the Dart programming language. • Inonic: Allows you to create hybrid applications, which are native apps, which hide in their code, a web page. • React Native: Platform for creating hybrid applications, based on JavaScript and a set of ReactJS components. Each of these platforms described has a unique structure for creating their projects and visually organizing and presenting their elements. In this sense, this work intends to start by developing the analysis of the Xamarin Platform, and in future research the analysis of the rest of the platforms. Microsoft offers for the development of mobile applications, the Xamarin platform, which is an extraordinarily fast platform for the development of Android and iOS applications, with capabilities to access all the features of the devices, to write controls and compile as if they were 100% native, thus allowing the reuse of code [7]. Xamarin proposes a cross-platform development approach in which the complete coding of the business logic is shared. However, interfaces must be programmed independently for each of the target platform [5], the integrated development environment (IDE) that Xamarin works with is Visual Studio, the programming languages are C# and XAML [8] and the framework that allows code sharing is Xamarin Forms. For the above reasons, this paper aims to analyze the process of building a mobile application, the types of projects that can be generated, the visual aspects and the organization of files in the Xamarin platform. The remainder of the document is organized as follows: Sect. 2 presents a brief literature review of the project, Sect. 3

Mobile Development with Xamarin: Brief Literature, Visualizations …

301

presents the development of the proposal, the discussion is shown in Sect. 4 and finally the conclusions in Sect. 5.

2 Background In recent years, the market for mobile devices, especially smartphones, has shown remarkable growth worldwide. Particularly in Latin America, the platforms that have grown the most are Android and IOS. This has generated the appearance of different development platforms and one of the most used by developers is Xamarin. In the literature, several works have been identified that use these platforms to carry out novel applications that solve different problems, these present an important field of academic and scientific research that needs to be supported through works related to the subject of study, in this sense, this section is of bibliographic - documentary type: “Documentary research is a process based on the search, recovery, analysis, criticism and interpretation of secondary data; that is, data obtained and recorded by other researchers in documentary sources: printed, audiovisual or electronic” [9]. In this case, research was conducted in high impact scientific databases such as: Scopus, ScienceDirect, IEEE, Google Scholar. To start the literature review process, a key question was posed, for the purposes of this research that allows identifying works related to the topic of study, this question is: What is known and how it has been explained about the development of mobile applications with Xamarin? Another important factor that was taken into account is the year of publication of the work, which should not be older than 5 years, in this sense works published in these databases since 2018 were reviewed. And a third search criterion applied in the literature review was the language in which a paper is published, in this case we chose to use papers published in English. Initially in [10], a very complete historical review of Xamarin and Xamarin Forms is made, where relevant information about the architecture of their platforms such as Android, iOS and Windows Phone is described, the programming languages used are C# for the controllers and XAML for the visual elements. It also describes the design pattern used Model, View, View-Model (MVVM), the Data Binding library that allows the connection of the objects of the design pattern elements. On the other hand, at [11] the authors mention that: Xamarin Forms allows reusing almost all code, including the user interface, across Android and iOS devices, plus it provides access to advanced features such as screen recording or touch input logging, which are used some applications to collect additional data. In [12] Xamarin is claimed to provide mobile development environments on Windows or Mac. The two main options for Xamarin development environments are Xamarin Studio on Mac or Windows, or Visual Studio on Windows with the Xamarin plugin for Windows. A Mac device is always required for iOS app compilation, even if Visual Studio is used as the development environment. There are also comparisons of Xamarin, in [13], a comparative study is made between traditional Xamarin and Xamarin Forms, the former refers mainly to the

302

R. M. Toasa et al.

development of applications using the classic APIS of each platform, the latter is a set of tools created mainly with the aim of streamlining cross-platform development, focusing mainly on maximizing the amount of shared code between platforms. When developing mobile APS with Visual Studio and Xamarin, different templates are presented such as: Floating, Tabbed and Blank control, each template described has its own specific visualization and behavior, besides Xamarin forms provides a template that contains two necessary components for any application: an “Application” component that is responsible for initializing the application, and a “Page” component that represents a single screen in the application [14]. The literature described in previous paragraphs validates the importance of mobile application development using the Xamarin Forms platform with Visual Studio. Everything found gives way to generate new knowledge when managing the development of cross-platform mobile applications.

3 Proposal Development To start the development of mobile applications with Xamarin Forms, it is necessary to consider certain aspects that based on the experience of the authors have been identified over time, these are detailed below:

3.1 Configuration by Creating a Project When naming a new project with Xamarin Forms, it is recommended that you do not use blanks. “”, or special characters such as “_,-,.”, problems were identified at the time of compiling the project since the compiler did not recognize these elements and an error was generated. It was necessary to recompile the solution so that the project would run correctly. Another important aspect is the location of the project, by default Xamarin creates the path: “C:\Users\UserName\source\repos”. With 4 levels (Folders) of depth from the root (C), if you are going to change the path you must keep the same depth, because when adding a new level error may occur when compiling the project, because the path is very long and certain libraries are not recognized. Finally, Xamarin in Visual Studio shows the option to create the solution and the project in the same path of the project, this is necessary to mark it to avoid problems of location and access to the files at the time of compiling the project.

Mobile Development with Xamarin: Brief Literature, Visualizations …

303

3.2 Templates for Mobile Development Xamarin offers three templates for development (see Fig. 1): Flyout, Tabbed, Blank. • Flyout: A floating page generally displays a list of items Fig. 2 (a). The location of the list of items is identical on each platform, and selecting one of the items will navigate to the corresponding detail page, Fig. 2 (b) shows the generic project structure where coding can be done. • Tabbed: Consists of a list of tabs and a larger detail area, with each tab loading content into the detail area, see Fig. 3(a), Fig. 3(b) shows the generic project structure where coding can be performed. • Blank: It only consists of an empty page with text labels “Labels” as shown in Fig. 4(a). The architecture is very simple and consists of an App.xaml.cs file which is the main class of the project configuration, it also has a MainPage file, the view with xaml format, and the controller of that view with xaml.cs Fig. 4(b). The 3 templates offer an adequate structure for the development of mobile applications, however, according to the experience of the authors, it is recommended to use the blank template, because you have control of all the elements and methods to be used in the development of the application. Fig. 1 Xamarin templates

(a) Fig. 2 Flyout template

(b)

304

R. M. Toasa et al.

(a)

(b)

Fig. 3 Tabbed template

(a)

(b)

Fig. 4 Blank template

3.3 Project Structure A solution created with Xamarin Forms contains 3 projects (see Fig. 5). The first project is the Shared Code Fig. 6 (a), which is where all the programming that will be shared by the two operating systems (Android - iOS) is done, the second the Android project Fig. 6(b) and the third Fig. 6(c), these last two contain the structure and files of a native project, so, if it is required to use functionalities specific to the operating system, it is necessary to program in each one. Fig. 5 Xamarin solution

Mobile Development with Xamarin: Brief Literature, Visualizations …

305

(a)

(b)

(c)

Fig. 6 Projects on Xamarin. a General Project, b Android Project, c iOS Project

3.4 Mobile Project Execution Once the project is created, it is necessary to configure the device where the project will be executed, in this sense there are two execution options: • Virtual machine: Virtual emulators can be created with each operating system, however, for iOS a MAC computer is required, while for Android the emulator can be created by configuring the OS version, the architecture and the Android version. Once the emulator is created, the project can be executed, see Fig. 7. • Real device: To enable running on a real device, it may vary depending on the Android version. In general, it is necessary to enable the developer mode of the device and USB debugging. Once this is done it is necessary to connect the device and Visual Studio will automatically recognize it.

Fig. 7 Virtual machine/ Android

306

R. M. Toasa et al.

The above mentioned are important considerations that authors have identified over time, this at the time of starting the development of mobile applications with Xamarin. They are simple aspects but very effective at the time of execution. In reference to the visualization, it is important to clearly define the requirements of the application to choose a suitable template that fits the visual needs of the user.

4 Discussion The identified literature shows that there are several works regarding the development of mobile applications with Xamarin, but with a very technical approach, without considering the small details that have been identified in this work and that are of great support to start with a successful development, which if not considered in early stages of the project, will cause inconvenience and loss of time in an advanced stage of development. Very few explicit discussions were identified about initial configurations in visual studio with xamarin for mobile application development, many of the articles that were analyzed seem to assume that initial configurations are created by default and bugs will be resolved automatically [15, 16], however, based on the experience of the authors, this is not the case, since the appropriate configuration of the projects can guarantee an adequate consumption of computational resources and the success or failure of the project. It also identifies a lack of systematic approaches to mobile application development; each article proposes its own platform and operating system without considering the basic configurations, as in the book [17], which is presented as a guide in the development of cross-platform mobile applications with xamarin. To conclude this small analysis, it is important to mention that if a project has not been configured correctly in the initial stages, it will be a problem in the production stages, since trying to solve a detail that seems simple, in programming it can involve many aspects that can cause the total failure of the system. Everything described in the paper is purely from the experience of the authors and has been of great help when developing mobile applications.

5 Conclusions Currently, identifying a suitable platform for mobile development has become a critical process since certain fundamental aspects must be considered when configuring the tool. For this reason, the need for this review of the current state of the art in mobile development platforms was born. The results of this literature review show very positive aspects about the development of mobile applications with Xamarin, since it was identified that it is a very powerful platform for development and it is still a wide field and still free to explore

Mobile Development with Xamarin: Brief Literature, Visualizations …

307

and investigate and it will depend a lot on the new research that will be developed worldwide to find the right tools for quality mobile development. This work allows to identify the current state of mobile development with Xamarin, and serves as a starting point to propose the use of different mobile development platforms, allowing to create a stable, adequate and timely application, where customers and developers have great benefits.

References 1. Ahmad A, Li K, Feng C, Asim SM, Yousif A, Ge S (2018) An empirical study of investigating mobile applications development challenges. IEEE Access 6:17711–17728. https://doi.org/10. 1109/ACCESS.2018.2818724 2. Garg S, Baliyan N (2021) Comparative analysis of Android and iOS from security viewpoint. Comput Sci Rev 40:100372. https://doi.org/10.1016/J.COSREV.2021.100372 3. Amatya S, Kurti A (2014) Cross-platform mobile development: challenges and opportunities. Adv Intell Syst Comput 231:219–229. https://doi.org/10.1007/978-3-319-01466-1_21/ COVER/ 4. Leocadio J, Veiga A, Barbosa M, Albertini B, Lohmann L, Saraiva A (2018) A multi-platform mobile application to collect citizen science data for Bignoniaceae Phenological research. Biodivers Inf Sci Stand 2:e25582. https://doi.org/10.3897/BISS.2.25582 5. Delía, LLN (2017) Desarrollo de Aplicaciones Móviles Multiplataforma 6. Singh M (2021) Comparative analysis of hybrid mobile app development frameworks Int J Soft Comput Eng (IJSCE) 2231–2307. https://doi.org/10.35940/ijsce.F3518.0710621 7. Vishal K, Kushwaha AS (January 2019) Mobile application development research based on Xamarin platform. In: Proceedings - 4th International Conference on Computing Sciences, ICCS 2018, pp 115–118. https://doi.org/10.1109/ICCS.2018.00027 8. Visual Studio: IDE y Editor de código para desarrolladores de software y Teams. https://visual studio.microsoft.com/es/. Accessed 8 Jul 2022 9. de los M, Monroy Mejía Á, Sanchez Llanes NN (2019) Metodología de la Investigación, vol 2 (2018). Accessed 09 Jul 2022. https://elibro.net/es/lc/learningbyhelping/titulos/172512%0A, http://petroquimex.com/PDF/SepOct17/Desarrolla-IMP-Metodologia.pdf 10. Esteban Blanco A (2017) Desarrollo multiplataforma IOS vs Android con Xamarin - Archivo Digital UPM. Accessed 28 Jul 2022. https://oa.upm.es/49303/ 11. Andrews K, Zimoch M, Reichert M, Tallon M, Frick U, Pryss R (2018) A smart mobile assessment tool for collecting data in large-scale educational studies. Procedia Comput Sci 134:67–74. https://doi.org/10.1016/J.PROCS.2018.07.145 12. Hermes D (2015) Mobile development using Xamarin. In: Xamarin Mobile Application Development, pp 1–8. https://doi.org/10.1007/978-1-4842-0214-2_1. 13. Versluis G (2017) Xamarin.Forms vs. Traditional Xamarin, Xamarin. Forms Essentials, pp 49–59. https://doi.org/10.1007/978-1-4842-3240-8_4 14. Scholarworks@gvsu S, Radi AA (2016) Evaluation of Xamarin forms for MultiPlatform mobile application. Accessed 28 Jul 2022. https://scholarworks.gvsu.edu/cistechlib 15. del Sole A (2017) Configuring the mac development machine. Beginning Visual Studio for Mac, pp 3–13. https://doi.org/10.1007/978-1-4842-3033-6_1 16. del Sole A (2018) Computer vision on mobile apps with Xamarin. In: Microsoft Computer Vision APIs Distilled, pp 43–67. https://doi.org/10.1007/978-1-4842-3342-9_4 17. Bennett J (2018) Xamarin in Action: creating native cross-platform mobile apps - Jim Bennett Google Libros. Accessed 31 Jul 2022. https://books.google.com.ec/books?hl=es&lr=&id=8Tc zEAAAQBAJ&oi=fnd&pg=PT16&dq=XAMARIN+&ots=U9gDj_FKdf&sig=l9HBTQmXNgGbpwI5ZSm1Zh0TcE&redir_esc=y#v=onepage&q=XAMARIN&f=false

AI-enchanced Crowdsourcing as an Element of Information Systems Development Vaidas Giedrimas, Jurgita Lieponiene, Birute Ragalyte, Henrikas Sinickas, Jurgita Paulaviciene, Audrius Valinskas, Edgaras Sriubas, and Alma Paukstiene Abstract The Information Systems Development process consists of a few elements, which may look contradictory at first sight. On one hand, we reached huge progress in automated software development. It is possible to make a priori correct large-scale distributed software using formal methods. On other hand, the enthusiasts of crowdsourcing and the participants of open source projects emphasize the importance of the human factor. The authors of this paper believe that in computer science the composition of crowdsourcing and automated software development is possible as in other sciences where two or more former competitive theories eventually complement each other. Moreover, it leads to synergy. The conceptual framework of the methodology for crowdsourcing-based software development incorporating artificial intelligence elements is discussed in this paper. Keywords Automated software development · Machine learning · Crowdsourcing · Software components

1 Introduction As pointed in [32], in the context of enterprise system development, business processes, information systems (IS), and software systems should be tightly integrated. Requirements are mapped from business level to lower level systems, and lower level systems are constrained by rules governing processes in higher level systems. Software engineering (SE) is key part of IS engineering process [35]. From the point of view of budget planning, roughly two-thirds of it will be spent on software and IS services [30]. As presented in [13], there are at least 60 different software development methodologies. However, in most of them custom designs and manual coding are intrinsically expensive and error prone [13]. In order to improve this, we need either to involve V. Giedrimas (B) · J. Lieponiene · B. Ragalyte · H. Sinickas · J. Paulaviciene · A. Valinskas · E. Sriubas · A. Paukstiene Panevezys University of Applied Sciences, Panevezys, Lithuania e-mail: [email protected] URL: http://www.panko.lt © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_27

309

310

V. Giedrimas et al.

more people to each stage of software development (e.g. to use crowdsourcing) or to use the algorithms of artificial intelligence - AI (“less humans - less errors”). The crowdsourcing [9] is a problem-solving and task realization model that is being increasingly used. Although the concept of crowdsourcing has no clear boundaries, there exist many successful applications in various areas such as graphical design, text processing, routine work, etc. This collective problem-solving methodology is such popular that its activities are even classified into 5 classes [9]: – crowdcasting (for selection of the best possible work implementation); – crowdcollaboration (for the development of the best ideas via massive online brainstorming sessions); – crowdcontent (for the collective data gathering/improvement from distributed human sources. E.g. the Streetview of the remote mountain town made by its citizens taking the photos with their smart phones in the streets); – crowdfunding (to collect funding for some project); – crowdopinion (for pre-check of user opinions about a particular issue, e.g. planned to market product). Crowdsourcing is widely used in the industries such as Mass media [1], Smart city management [20] and others. Recently crowdsourcing is used even in very specific areas, such as research questions formulation for the scientific research [5, 21]. The goal of this paper is to present state-of-the-art use of Crowdsourcing in SE and to discuss possible extension point by the AI. The rest of the paper is organized as follows: Sect. 2 surveys existing applications of crowdsourcing on Software engineering, Sect. 3 presents current approaches to use artificial intelligence for software engineering, Sect. 4 presents our AICrowd methodology for software developement, which expands the element of crowdsourcing by AI. Section 5 introduces related work in this area, finally, the conclusions are made, and open questions are discussed.

2 Crowdsourcing for Software Engineering The relations of crowdsourcing and software engineering are on the spot of researchers starting from early 2000 s Our analysis is based on the source [18], which covers 200+ papers published up to April, 2015 s from major Software engineering journals and conference proceedings (ICSE, ASE, ISSC, IEEE Software etc.) and few other sources published after 2015 s [2, 3, 27, 31]. The following application areas of crowdsourcing for SE are indicated: – Requirement engineering. The crowdsourcing is used to reduce the cost of reliance on experts to approach stakeholders; to recommend stakeholders via social network; for requirements elicitation and acquisition; to process requirements documentation by extracting requirements from large natural language text (or audio/video) source. It is worth to distinguish different prerequisites for the members of the crowd. For some tasks as for requirements extraction/documentation

AI-enchanced Crowdsourcing as an Element of ISD

311

preparation or for ideas for public available software there are no special qualification needed (e.g. [5]). The involvement of the crowd increases the coverage and decreases the cost of manual requirements extraction [18]. For another tasks, as requirements elicitation or expert assessment, specific knowledge and skills are preferred as an prerequisites for crowd-member. – Design. The use of crowdsource in design could be divided to 2 different groups of cases: Architecture design and User experience (UX) design [2, 16, 18]. Architecture design is very low covered in the literature. Usually crowd is used to get opinion about already made design with the goal to improve its quality. In less frequent cases the crowdcasting activities are used to get the best architecture based on given specification. For the design of UI general purpose or graphical design crowd platforms are used. In parallel there are such crowdsourciong tools as Apparition, and CodeOn which allow to “borrow others’ design ideas and to improve the design quality” in terms of ergonomics and UX. – Development, coding. Most know applications are: program optimization/ synthesis, integrated development environment enhancement, and crowd-oriented programming environments specifically designed for cooperative development. Collaborative programming environments for scientific software as CodeOcean [7], and WikiSPSM [10] have been in use for a decade. In the same time commercial crowd-based IDEs as Collabode, CIDRE, TouchDevelop, and Code Hunt were developed [18]. The last two (TouchDevelop and CodeHunt) evolved to learn-tocode platforms such as Microsoft MakeCode. Many of other platforms and tools currently are not supported and/or not used anymore. In contrast, new cloud-based IDEs were created: Koding, CodeAnywhere, AWS Cloud9, GoormIDE etc. It is observed that all new platforms emphasize the possibility to code using browser only. Collaborative aspect is expressed as the possibility to do pair-programming or work in teams within the organization. However it is possible to extend virtual organization by crowd members and turn these tools to crowdsourcing tools. For some crowds IDE is not a tool, but an object. For example the goal of SeaHawk crowd project is to enhance classic IDE using public knowledge [24]. – Testing and verification is widely covered in crowdsourcing platforms. It is possible to perform performance testing, usability testing, UI testing, QoE testing as well as separate test cases generation. Compared with traditional software testing, crowdsourced software testing has the advantage of recruiting, not only professional testers, but also end users to support the testing tasks [18]. For most cases general-purpose crowdsourcing platforms are used, however there are few software-testing-specific ones: CrowdBlaze, iTest, Caiipa [18]. These platforms are focused for mobile applications and provide many automation features. Crowdsourcing is used also for software verification. The main problem in this process is to attract sufficient amount of highly skilled crowd-members. For example the lack of contracted workers is key problem in VeriWeb platform. There are partial solutions, like Crowd Sourced Formal Verification program, which uses the gamefication to simplify the problem and present it in the form acceptable for nonskilled workers. Similar approach for simplification is used in CrowdMine platform.

312

V. Giedrimas et al.

It simulates execution traces as images, which are processed by the crowd in order to find some pattern mismatch. – Maintenance and other issues: software adaptation, documentation, localization, security and privacy analysis, end user support. Most of the issues can be called as “classic” as they do not require any specific crowd platform. Software evolution and maintenance are second most popular areas (after software testing) of crowdsourcing. Software localization is very important for multilingual applications, however the organization of crowd work have no differences from e.g. simply text translation. The use of crowdsourcing for specific (not related to software life-cycle) areas is also observed. It is important that crowdsourcing is used not only for software development life cycle, but for improving SE process itself. There are approaches for elaborating and development of new software methodologies, research frameworks and so on. Predictive Models in Software Engineering (PROMISE), Mining of Software Repositories (MSR), and Search Based Software Engineering (SBSE) events are good examples of such crowd-based efforts. Some crowdsourcing platforms have strict orientation to particular class of software: websites, scripts, mobile apps etc. For example AppStori, TestFlight, Mob4hire, Testin are oriented to mobile applications development (Table 1). We found that the least covered topics related to crowdsourced SE are: – (design) Architecture design – (coding) compiler optimization and software synthesis;

3 Artificial Intelligence for Software Engineering AI and SE are mentioned together very often. In most cases the sources provide information how SE can help for artificial intelligence development. We are focused

Table 1 SE lifecycle support by comercial crowdsourcing platforms Lifecycle phase Commercial platforms Academic platforms Requirements Engineering

Amazon Mechanical Turk

Design

DesignCrowd, CrowdSpring, TopCoder, 99designs Bountify, TopCoder, CodeOcean GetACoder uTest, Passbrains, 99Tests, CrowdBlaze, iTest, Caiipa, TestBirds, TestBats,Pay4Bugs, VeriWeb, CrowdMine CrowdTesters Ce.WooYun, BugCrowd

Development, coding Testing and Verification

Maintenance, Evolution etc.

CrowdREquire, StakeSource, StakeRare Apparition, CodeOn

AI-enchanced Crowdsourcing as an Element of ISD

313

on opposite application - how SE can benefit for the usage of AI. First paper on this point of view is published in early 1990 s [19], however growing interest is observable only in past decade [4, 14]. There are few significant AI applications software engineering. We provide the examples according to each stage of life cycle: – Requirements engineering There exist semi-automatic tools for requirement analysis phase, e.g. there are an attempts to use Natural Language Processing (NLP) and Machine Learning (ML) for requirements extraction from given textual descriptions, UML diagrams, or voice records. Such tools can produce class diagrams from the user requirement document (as R-Tool, NL-OOPS, CMBUILDER) or produce requirements models based on NLP and documents that are already stored in the internal knowledge base (LOLITA) [4]. Also there are recommendation systems for requirements-related activities such as definition, quality assurance, reuse, and release planning (INTELLIREQ, TLBO). These systems are based on various optimization techniques as Swarm Intelligence, Ant Colony Optimization (ACO), Genetic Algorithms etc. Fuzzy Logic and Case-Based Reasoning were used for case aggregation and evaluation [4]. Generally AI for RE is used as a remedy for requirements extraction, clasterization or refinement. However most approaches still involves human as an expert or as an operator. – Design Knowledge-Based Systems (KBSs) are widely used for software design, especially for database design [4]. Other existing application areas are: UI design; AI-based discover, compose, and development of services in SOA architecture; refinement of design architectural models into object-oriented models. – Development, coding. In Coding phase, there are automated development tools such as Github Copilot [11] and AlphaCode [29]. The same GitHub software repository is used for the training of the algorithms for both tools, however the tools are strict different by their behavior. As its manufacturer DeepMind presents [29], AlphaCode system can solve software problems that require a combination of logic, critical thinking and the ability to understand natural language. For brandnew problems (e.g. during some hackatons) AlphaCode generates a set of possible solutions in Python/C++ programming languages. It is assumed, that each solution will be assessed by the humans. Github Copilot does not generate complete solutions, but provide line-by-line, or block-by-block suggestions. Some sources warns on possible copyright violations [29], stating that Copilot directly reuses already existing in GitHub code. However in contrast, Copilot takes into account even the programming style of human companion [11]. Programmers constantly are reporting that the suggested lines and parts of the algorithms heavy depends on its own written/accepted lines. We can name GitHub Copilot as a life-long-learning system. There exist also other tools as ontology-enabled agent for code generation in Autonomous Software Code Generation (ASCG) approach [4]. Knowledgebased development evolved for decades and created even separate SE branch – Search-Based Software Engineering. – Testing and verification One of successful examples of AI-based testing is Knowledge-Based Interactive Test Script (KITTS), created by Bell Laboratories [4]. Analytics-Driven Testing (ADT) try predict software failures and to measure

314

V. Giedrimas et al.

mean time between failures. Other tools have been used in testing distributed systems and GUIs. They are based on ACO, ANN, Info Fuzzy Network (IFN) and other AI methods [33]. – Maintenance and other issues. The chat bots are one of the most visible part of AI in software maintenance process. Though there are also other less-visible applications such as: Text Mining (the categorization of product reviews, distinguishing real reviews, false negative/positive reviews, brand-specific reviews or a reviews that express no opinion), Support Vector Machines for fake reviews detection (libLinear, libSVM, SMO, RF, and J48 tools), Intelligent search methods (to locate parts of the software to be fixed/optimized) [4]. As is concluded in [4], most of the AI applications are small-scale applied to a formal problem within SE instead of industry. No paper was found that addressed the nonfunctional aspects of SE.

4 The AICrowd Methodology As we state in the previous sections, elements of crowdsourcing as well as the elements of artificial intelligence are used in SE, however the scale and the results are very different for each stage of software life cycle. We made conceptual framework of AICrowd methodology for software development, wich incorporates crowdsourcing and AI together. There are 3 main ways for the collaboration of an elements x and y (Fig. 1): 1. Elements x and y seek their goals in parallel without any communication. The results of both elements are passed to next stage. This way of collaboration we can mark A(x, y). Note that A(x, y) = A(y, x). 2. Elements run in clear sequence: first, x is used and its results are processed by y. The results of y are passed to the next stage. This we mark B(x, y). 3. Elements x and y are cooperating, given data from the previous stage is processed by x, then by y, then, again, by x and after (not defined in the begging) number of iterations the result is pased to next stage. As x and y cooperates, the order is not very important until separate notification. In AICrowd methodology we offer to perform integration of crowdsourcing and elements of AI into each stage of software lifecycle. Particular ways of Crowd+AI collaboration depends on the particular stage: – Requirements engineering As mentioned before, non-functional requirements is still a problem for AI part in SE. The most likely that at this stage the model B(Cr owd, H uman) will be used. It means that first we get the ideas/opinions from the crowd and then they are evaluated by single expert. Besides of this we see and extension B(C(Cr owd, AI ), H uman). As an opinions from the crowd could be biased, the AI element could be used for its filtering. After that filtered opinions

AI-enchanced Crowdsourcing as an Element of ISD

315

Fig. 1 AICrowd interaction models

–

–

–

–

are provided for an Expert for assessment and decision making. Also we see here good possibilities to use Crowd-Machine interaction for Machine learning, as the AI element will get new and new data from the crowd. Design This stage is covered much more by AI applications in comparison to crowdsourcing. We suggest to use B(AI, Cr owd) model, as it allows to use KBS. Knowledge about design patterns, architectures can provide general ideas for design. This ideas latter on could be assessed by the crowd members. For the design of UX we see C(AI, Cr owd) collaboration way more feasible. Development, coding. The success of GitHub Copilot and similar tools shows that here model C(Cr owd, AI ) works very well. Copilot uses Github repository which could be considered as indirect result of crowdsourcing. However, Copilot take as an argument the personal programming style of its "master", so we could describe this model as C(A(Cr owd, H uman), AI ). Testing and verification Both crowdsourcing and AI are used separately for this stage in very intensive way. However, it is observed that it is a lack for high-skilled personnel inside the crowd. We suggest to use here C(Cr owd, AI ) collaboration model. For example we could teach a Machine on the data provided by gamification part of the Crowd Serial Formal Verification project. CloudMine approach can be enhanced by AI, as pattern recognition is classic problem in artificial intelligence domain. Maintenance and other issues. All collaboration models are possible in this stage, depending on the particular problem. For software evaluation the model B(Cr owd, AI ) is recommended. For software localisation - C(Cr owd, AI ). In this case the input of native speakers is more than valuable. However such systems as DeepL shows huge possibilities of the use AI in text translation domain. AI-element could be used in A(Cr owd, AI ) model as well. For the search of particular code lines related with reported bug we offer to use B(AI, Cr owd) model or A( AI, Cr owd) model.

316

V. Giedrimas et al.

5 Related Work The set of related papers by its similarity to our approach can be divided to two groups. First group of the papers is focused on some elements only [8, 12, 17, 26, 28]. Second group of the papers [6, 15, 22, 23, 25, 34] presents the similar ideas to ours, however the details of proposed software methodology are slightly different. The focus of [8] is the crowd-based capability building. They argue, that the engagement of external actors and, consequently, its knowledge and skills, for the exploration of vanguard projects would rapidly improve their capabilities and even come up with scalable crowdsourcing services. Interesting enough that in the paper [8] the co-use of artificial intelligence and digital platforms as facilitators for project exploitation is mentioned as well. However, Füller’s approach do not provide any details how the fusion of crowdsourcing and AI could be achieved. Although the title of [8] includes “as-a-service”, the content of the paper discuss the issues from the management point of view, not from the perspective of software methodologies. In contrast to it, the paper [28] is completely dedicated to service-oriented architecture (SOA) and its relations to crowdsoursing. The papers of the Second group describes various combinations of Crowdsource and artificial intelligence, However none of them do not covers the fusion of “Crowdsourcing+AI+SE”.

6 Conclusions and Future Work We presented the software development methodology which involves machines (AI part) in the process as well as humans (crowdsoursing part). 3 different collaboration models between these parts are offered, and its feasibility for each stage of lifecycle is discussed. Particular model for each stage will be selected depending on the particular software. AICrowd methodology covers many of possible its compositions. However we see that most popular is the model C(Cr owd, AI ). This led us to conclusion that it is required to incorporate AI-based agents in current crowdsourcing platforms in order to increase the quality of results. Our future research includes but are not limited to: – Technical details of AI-based agent in crowdsourcing platform. – Particular model of collaboration of for non-functional aspects of SE. – Compiler optimization and software synthesis using AICrowd methodology.

References 1. Abhari K, Davidson EJ (2016) Creative co-production: the adaption of an open innovation model in creative industries, pp 119–130. Springer, Cham 2. Aghayi E, LaToza TD, Surendra P, Abolghasemi S (2021) Crowdsourced behavior-driven development. J Syst Softw 171:110840

AI-enchanced Crowdsourcing as an Element of ISD

317

3. Alyahya S (2020) Crowdsourced software testing: a systematic literature review. Inf Softw Technol 127:106363 4. Batarseh FA, Mohod R, Kumar A, Bui J (2020) The application of artificial intelligence in software engineering: a review challenging conventional wisdom. In: Batarseh FA, Yang R (eds) Data Democracy, pp 179–232. Academic Press 5. Beck S, Brasseur TM, Poetz M, Sauermann H (2022) Crowdsourcing research questions in science. Res Policy 51(4):104491 6. Abhigna BS, Soni N, Dixit S (2018) Crowdsourcing - a step towards advanced machine learning. Procedia Comput Sci 132:632–642, International Conference on Computational Intelligence and Data Science 7. CodeOcean I (2021) Computational research PLatform. https://codeocean.com/. Accessed 14 May 2022 8. Füller J, Hutter K, Kröger N (2021) Crowdsourcing as a service - from pilot projects to sustainable innovation routines. Int J Project Manag 39(2):183–195, managing Open and User Innovation by Projects 9. Garrigos-Simon FJ, service SO (2015) Advances in Crowdsourcing. Springer, Cham 10. Giedrimas V, Sakalauskas L, Neimantas M, Žilinskas K, Barauskas N, Valciukas R (2016) Wiki-based stochastic programming and statistical modeling system for the cloud. Int J Adv Comput Sci Appl 7(3):218–223 11. GitHub I (2022) GitHub copilot - Your AI pair programmer. https://github.com/features/ copilot. Accessed 1 July 2022 12. Guo S, Chen R, Li H, Gao J, Liu Y (2018) Crowdsourced web application testing under realtime constraints. Int J Softw Eng Knowl Eng 28(06):751–779 13. Jones C (2018) Software Methodologies: A Quantitative Guide. Auerbach Publications, Boca Raton (2018) 14. Kalech M, Abreu R, Last M (2021) Artificial Intelligence Methods for Software Engineering. World Scientific, Singapore (2021) 15. Kamar E, Hacker S, Horvitz E (2012) Combining human and machine intelligence in largescale crowdsourcing. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2012, vol 1. pp 467–474. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2012) 16. LaToza TD, Ben Towne W, van der Hoek A, Herbsleb JD (2013) Crowd development. In: 2013 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), pp 85–88. https://doi.org/10.1109/CHASE.2013.6614737 17. Machado L, Prikladnicki R, Meneguzzi F, de Souza CRB, Carmel E (2016) Task allocation for crowdsourcing using Ai planning 18. Mao K, Capra L, Harman M, Jia Y (2017) A survey of the use of crowdsourcing in software engineering. J Syst Softw 126:57–84 19. Molnár B, Frigó J (1991) Application of AI in software and information engineering. Eng Appl Artif Intell 4(6):439–443 20. Neto MC, Rego JS (2019) Urban intelligence for sustainability. In: Ramos I, Quaresma R, Silva P, Oliveira T (eds) Information Systems for Industry 4.0, pp 139–159. LNISO, Springer, Cham (2019) 21. Nevolin I (2017) Crowdsourcing opportunities for research information systems. Procedia Comput Sci 106:19–24, 13th International Conference on Current Research Information Systems, CRIS2016, Communicating and Measuring Research Responsibly: Profiling, Metrics, Impact,Interoperability 22. O’Leary DE (2014) Embedding AI and crowdsourcing in the big data lake. IEEE Intell Syst 29(5):70–73. https://doi.org/10.1109/MIS.2014.82 23. Orkin JD (2013) Collective artificial intelligence: simulated role-playing from crowdsourced data. Ph.D. thesis, Massachusetts Institute of Technology 24. Ponzanelli L, Bacchelli A, Lanza M (2013) Seahawk: stack overflow in the ide. In: 2013 35th International Conference on Software Engineering (ICSE), pp 1295–1298

318

V. Giedrimas et al.

25. Roy D, Orkin J (2013) Collective artificial intelligence: simulated role-playing from crowdsourced data 26. Saremi RL, Yang Y (2015) Dynamic simulation of software workers and task completion. In: Proceedings of the Second International Workshop on CrowdSourcing in Software Engineering, CSI-SE 2015, pp 17–23. IEEE Press 27. Sarı A, Tosun A, Alptekin GI (2019) A systematic literature review on crowdsourcing in software engineering. J Syst Softw 153:200–219 28. Schall D (2012) Service-Oriented Crowdsourcing: Architecture. Protocols and Algorithms. Springer, New York 29. Sparkes M (2022) Deepmind has made software-writing AI that rivals an average human coder. New Sci 253(3373):10 30. Stair RM, Reynolds GW (2018) Fundamentals of information systems 31. Stol KJ, LaToza T, Bird C (2017) Crowdsourcing for software engineering. IEEE Softw 34:30– 36. https://doi.org/10.1109/MS.2017.52 32. Szel¸agowski M, Lupeikiene A (2020) Business process management systems: evolution and development trends. Informatica 31(3):579–595 33. Tahvili S, Hatvani L (July 2022) Artificial Intelligence Methods for Optimization of the Software Testing Process With Practical Examples and Exercises. Elsevier 34. Tunio M, Luo H, Wang C, Zhao F, Shao W, Pathan Z (2018) Crowdsourcing software development: task assignment using pddl artificial intelligence planning. J Inf Proc Syst 14:129–139 35. Wallace P (2021) Introduction to Information Systems, 4th Edition. Pearson, London

User’s Appreciation About Software Prototyping: A Case Study Pedro-Saul Delgado, Wilmer Bermudez , Brian Pando , and Ronald Ibarra

Abstract One of the problems in the software development projects is the ambiguity of the requirements or the multiple interpretations of each user. Likewise, it is known that the cost of correcting an error is less if it is detected in the early phases such as analysis and design compared to detecting in later phases such as construction and maintenance. Software prototyping is an alternative solution to this problem, helping to minimize changes and rework in the final phases of the software process and thus avoiding costs and dissatisfaction. Despite this, prototyping is not widely used in the analysis, specification, and validation of requirements activities. This work evaluates the user’s appreciation regarding the practice of working with prototypes. A questionnaire was used to assess motivation, user experience, and learning ability. The results show a positive perception of the users who participated in this case study. Keywords prototyping · mockups · Software Requirements · Software Engineering

1 Introduction Software development is an activity that is increasingly in demand around the world [1]. However, projects are still not delivered on time ([2, 3]), in several cases due to problems in the interpretation of requirements [4, 5]. Requirements engineering is a vital phase in the development of a software project since it provides a solution to the need of users [6]. Software engineers obtain and analyze requirements with the help of stakeholders [7]. One of the frequent problems is finding flaws in the requirements analysis because the requirements collection was not very clear [8]. P.-S. Delgado (B) · W. Bermudez · B. Pando · R. Ibarra Grupo de Investigación en Ingenieria de Software, Universidad Nacional Agraria de La Selva, Tingo María, Perú e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_28

319

320

P.-S. Delgado et al.

Current software engineering methods suggest, among other good practices [9], the use of the prototyping technique for the requirements process [10]. However, many companies overlook this practice, neglecting constant feedback from the endusers and early prototyping for mature and productive analysis aimed at avoiding untimely change [11]. It is known that the cost to repair an error in the analysis and design phase is 5 to 10 times less than in the construction and 200 times less than in the maintenance phase [8]. Software prototyping is a choice for solving the problem of making many changes and unnecessary work in late phases since it involves high costs and dissatisfaction of users and team members [12]. Prototyping is a technique that helps to collect and validate requirements, based on tools that facilitate the construction process of theses prototypes [13]. They are an important way to communicate and analyze the requirements, considering aspects of usability and ergonomics [14, 15]. In this way, helping software engineers minimize the effort and make an efficient requirements collection, generating timely delivery of the software, thus, being more accurate in project estimations [16]. The reviewed works show that comparisons have been made between techniques to collect requirements, including questionnaires, direct observation, mockups, prototypes, both high and low fidelity, and other textual methods. The work carried out by [17] compares the use of questionnaires against direct observation. [11] shows that the use of prototypes was more convenient than other textual methods. Abrahamsson and Melin [6] has determined that using high-fidelity prototypes have a better interpretation by users compared to low-fidelity ones. The papers presented also show that using prototyping techniques improves the satisfaction of users [18] and students [19] in academic settings. Finally, in the spirit of improving the design of prototypes, in [20] the proposal of an expert system to determine how easy to use a prototype is presented, giving suggestions for improvement. Some factors about prototyping is the time required for the elaboration of lowfidelity prototypes, beside thinking that the client doesn’t want to participate [21]. Prototyping tools have had an impact on the market since 2003 [21]. The tools focused on prototyping have been extended, there are proposals for tools that allow code to be generated from prototypes [22, 23], as well as proposals to integrate prototyping into continuous delivery [24], even to validate part of the requirements automatically [25, 26]. Despite this, very little is known about the choices and insights about this practice and tools [21]. This work presents the degree of appreciation of the end-users about the prototyping practice, considering motivation, user experience, and learning capacity, when they are involved in a software development project. In this way, show that users are also willing to participate in this practice when developing an application in which they will be involved. This work is organized as follows: Sect. 2 shows a research background, Sect. 3 presents the related work, in Sect. 4 the materials and working method. Section 5 shows the results of the survey and the application of the AHP technique. Section 6 presents discussions, finally, conclusion and future works.

User’s Appreciation About Software Prototyping: A Case Study

321

2 Background 2.1 Software Requirements For SWEBOK [16], the software requirements phase comprises elicitation, analysis, specification, and validation of requirements. In each of these activities, there are recommended techniques. Elicitation is the first stage in which the problems that the software must solve are understood. At this stage communication with those involved is important. The techniques used in this stage are interviews, scenarios, prototypes, meetings, observation, user stories, and others. Requirement analysis is the stage where conflicts and ambiguities between requirements are resolved. The specification allows us t o detail the characteristics of the requirements through a rigorous evaluation. Finally, the validation of requirements includes validation and verification. For this, techniques such as the review of requirements documents, prototyping, model validation, the definition of acceptance tests can be used.

2.2 Prototyping It is a technique used in requirements engineering for the requirements elicitation, specification and validation activities that helps the user and the development team to understand and confirm their expectations of the software to be developed, allowing the visualization of the graphical interface of the software in a way very close to how it will be after development. For [16], prototyping is a way of validating the interpretation of the software requirements, establishing the bases of the functionality and behavior of the system according to the established requirements. Prototypes can be low-fidelity or high-fidelity. Software engineers often use both in combination as needed [27]. Low-fidelity prototypes are quick to create and do not represent a difficulty, but do not give a reliable representation of the interface. While high-fidelity prototypes require more effort but are a true representation of the interface. Figure 1 shows an example of how easy it can be to create low-fidelity prototypes using a whiteboard. The Mockups are low and high fidelity drawings, made to capture ideas and transform them into functionalities helping the client to understand what he needs [15]. There are different tools to create mockups. Some examples are Balsamiq, Pencil, LucidChart, among others. Low-fidelity mockups are graphical representations of an initial idea of what the user wants as a graphical interface to some part of the software [15], as used in [28]. However, being unsure of the details, such as the location of controls and content or form of interaction, these representations are made at low resolution with minimal effort, as shown in Fig. 3.

322

P.-S. Delgado et al.

Fig. 1 Colleting of requirements prototyping by freehand

On the other hand, high-fidelity (Hi-Fi) mockups already define the elements with greater clarity, making the graphical interface diagram with realistic sizes and proportions [15], as shown in Fig. 4. Hi-Fi mockups represent what finished software would look like. Some tools allow you to make high-end mockups and produce them in HTML.

2.3 Analysis and Specification of Requirements Using Prototyping It is the process in which the user’s needs are studied to obtain a definition of the software requirements [16]. In this stage, it works with the end-users to determine the services that the software must provide [29]. One of the first steps in the collection of requirements and here basic prototypes can be made for a quick interpretation of user needs. These prototypes are made on the paper, whiteboard, or some software tool as in Fig. 1. The design of prototypes is an initial approximation, based on a sketch or idea, which must then be developed in detail on the product to be obtained [8]. The details that escaped at the time of collection are also gradually being defined, but they will manifest themselves while working with the prototyping. Figure 2 shows a more consolidated prototype after analysis and confirmation to the user.

User’s Appreciation About Software Prototyping: A Case Study

323

Fig. 2 Low Fidelity (Lo-Fi) Mockup Example

Fig. 3 High Fidelity (Hi-Fi) Mockup Example

3 Related Work In [30], the importance of communication in the collection of requirements is presented, where interpersonal skills play an important role in software engineering. They explain 2 techniques widely used in this phase called interviews and brainstorming. The authors suggest that if the interviews are not conducted well, it wastes time for requirements engineering, since the focus of the conversation is lost. They also suggest that brainstorming should be presented in an engaging and interactive way to arouse the interest of the group. Finally, they say that it is necessary to continue researching in reducing communication barriers to reduce errors in the elicitation of requirements.

324

P.-S. Delgado et al.

Fig. 4 Example of prototyping in the design phase

In [31], the authors also mention that communication is an important factor during RQ collection and that this is a challenge for requirements engineers. They recommend using a participatory design to facilitate this communication. They run an experiment to validate the use of the SPIDe technique to elicit RQ. This technique applies (1) story creation, (2) brainstorming, (3) brain-draw, (4) thinking aloud. The technique allows users to feel like co-authors of the software, thus improving stakeholder involvement, since the prototypes made on paper were reviewed and improved by each participant. In [32], they present advantages and disadvantages of the scenario technique for the elicitation of RQ through a review of 63 investigations. According to the authors, this technique is efficient for medium and small projects, but it depends a lot on natural language and is not easy to document. It defines the scenario technique as an initial prototype based on the narrative of possible interactions between clients, users and systems. They consider that a good time to apply this technique is after the first version of the requirements specification. Among his observations, he mentions that many reject this technique when there is a lot of uncertainty in the users. In [33], they suggest that using pencil and paper to represent HCI requirements is faster than using software tools and that the technique does not need a learning curve, therefore any client or user is eligible to participate in this activity. They do a bibliography review and run a case study on public transportation software. In the case study, they make interactions by levels, going from paper to mockup and finally functional prototype during 3 weeks of elicitation. The authors mention that the use of prototyping tools consumes more than twice the time than the use of paper. In [17], a study is carried out aimed at determining the importance of requirements elicitation techniques based on user satisfaction. Two techniques were considered: (1) questionnaire and (2) observation, finding that there is greater user satisfaction in the process of identifying and analyzing requirements when using the questionnaire than when using observation. Furthermore, the study highlights that both techniques are important to incorporate new knowledge about the domain and the needs of the user.

User’s Appreciation About Software Prototyping: A Case Study

325

In [11], a methodology called Mockup DD is presented, as a combination of Model Driven practices with an agile development approach, using mockups to obtain requirements. Their results indicate that the application of the use of prototypes improves the obtaining of the requirements compared to textual methods and facilitates communication between developers and clients. In [6], the differences between working with high-fidelity and low-fidelity prototypes are determined. The research was carried out in a software development company, where the development staff was interviewed to obtain their opinion regarding working with prototypes. It was applied in two software projects, with high-fidelity prototypes and low-fidelity prototypes. Their results show that with the prototyping technique there is better communication with customers, in addition, it is possible to have more them involved during the project and improve the quality of communication, they also indicate that there is a better interpretation with high fidelity prototypes unlike of the low fidelity prototypes. In [18], the use and impact of the prototyping technique on the quality of software companies called PSI (Pakistani Software Industry), is shown. The users of these software companies have been surveyed with a questionnaire about the prototyping technique and the impact on product quality. Results show that it reduces the ratio of failures and errors in the project, besides, the use of this technique produces satisfaction in the users who participate in the elicitation process.

4 Materials and Method 4.1 End-User Evaluation Questionnaire To determine the perception of users regarding the prototyping technique in the requirements engineering process, the instrument of [34] was adapted, obtaining a questionnaire of 20 items to be evaluated using the Likert scale. The questionnaire was validated by consulting five experts with different profiles: two statistical professionals with more than 5 years of teaching experience, two professionals in software engineering with more than 7 years in both teaching and the software industry, and finally, to a psychologist with more than 10 years of experience developing questionnaires to measure user perception. You can see the full questionnaire at https:// forms.gle/7VzTTUAMZ6f9pc5NA. With the validated questionnaire, a pilot survey was carried out with 5 end users of the software to be evaluated. Cronbach’s alpha was α = 0.973 for reliability, which means that the results of the interviewees’ opinion regarding the items considered are correlated; being acceptable and defining the final version of the questionnaire. To determine the user’s appreciation regarding the prototyping technique, three dimensions were considered with their respective indicators, also using the Analytic Hierarchy Process technique (AHP) as described in [35], to find the weight of each indicator and dimension as shown in Fig. 5. In this way, a specific assessment was

326

P.-S. Delgado et al.

Fig. 5 Indicator weights (W) used in this work

made for each dimension and the general variable, using the mean of each item in the questionnaire based on the nominal values of the Likert scale and their respective weights. The software evaluated in the case study has a total of 225 users, of which the questionnaire was applied to a sample of 90 users with 95% reliability. The users were grouped by academic areas, organizing themselves into 8 groups, they participated in the requirements engineering phase of the aforementioned module, and finally, they were given the questionnaire to know their appreciation regarding the prototyping technique.

4.2 Case Study The case study was a project for the development of the syllables elaboration module of subjects in the academic management system of the Universidad Nacional Agraria de la Selva. The module consists of 13 functionalities and the 5 most important ones were evaluated.

User’s Appreciation About Software Prototyping: A Case Study Table 1 Participants groups of Case Study

327

Academic Area

Professors

Ages

G1

Administration

13

[30–64]

G2

Zootechnics

13

[40–72]

G3

Accounting

10

[44–65]

G4

Economy

13

[37–66]

G5

Food industries

12

[39–74]

G6

Environmental

10

[29–67]

G7

Forestry

8

[33–68]

G8

Humanities

11

[30–70]

Total

90

[29–74]

The web version of the academic management system started in 2010 and new functions have been added over time. The application is used by teachers, students, and administrative staff involved in academic management. Throughout this period, the prototyping technique was never used for communication with users, since the processes on which the software is based are being documented in a regulation. Despite this, many users always gave their opinions regarding the functionalities where they were allowed to interact. This research coincided with the need to implement the functionalities of syllabus management by professors through the academic management system. So, the sessions were made only by professors grouped in Table 1. The dev team used mockups as a technique because they considered the most appropriate for the project. So, using the Balsamiq tool, 5 low-fidelity prototypes of the new module functionalities were developed, including navigates between them. The finished prototypes were reviewed by 1 developer and 1 worker in the academic management office before presentation to users to validate technical characteristics and restrictions. Figure 6 shows the prototypes of the syllable automation software. The full prototypes are available for review in https://github.com/brianpando/protot yping. It was coordinated with the offices of each academic area requesting the organization of a workshop on a date where the greatest number of their teachers can be to validate the functionalities of the syllabus module. 8 of the 10 academic areas agreed to participate in these sessions. Which are shown in Table 1. According to the dates planned with each group, the validation session was carried out for those who first were quickly reminded of the manual process that these users did to create their syllabus employing some word processor. Then, the objective of the office in charge of academic management was explained to them about this new module. Finally, they were presented with the prototypes elaborated through a multimedia projector and the details of each functionality were described. Every session takes an average duration of 20 min.

328

P.-S. Delgado et al.

Fig. 6 Prototype created in Balsamiq for the case study Fig. 7 Prototype validation session with one of the case study groups

Users expressed their comments on the prototypes verbally, which were documented to be transferred to the developers. At the end of each session, the participants were asked to fill out the assessment questionnaire regarding the use of the prototyping technique in the software process. Figure 7 shows one of the sessions in the academic area of humanities. For privacy term, the participant faces are shadow.

5 Results 5.1 Motivation about Prototyping Practice The motivation of the users was determined through 4 indicators: attention, relevance, confidence, and satisfaction. Each indicator is evaluated with 2 questions in the questionnaire. Figure 8 shows the percentages obtained according to each indicator,

User’s Appreciation About Software Prototyping: A Case Study

329

distinguishing between sky-blue colors that reveal totally in agreement, light green, in agreement, finally the amber yellow color revealing indifferent. In general, it is seen that in the results related to confidence, relevance, and attention, more than 90% have a positive appreciation before the prototyping technique. More modestly, in the case of satisfaction, 62% have a positive appreciation and 32% are indifferent. To calculate motivation, a frequency table is generated by the sum of the evaluation of the 8 items expressed in Table 1. According to Table 2, the majority of users manifest themselves between the levels: “agree” with 35 users and “strongly agree” with 45 users, that is, 79 out of 90 users have a positive motivation for this good practice. For this work, the motivation represents the degree of motivation that users feel about applying the prototyping technique when developing the software that they themselves will use later. It is observed in Fig. 9 that 89% of the users have a positive appreciation (agree and strongly agree) regarding the prototyping technique in terms of motivation. However, there is 9% with an undecided rating. Table 3 shows the results from the application of AHP to the same data, resulting in motivation with a value of 4.47 on a scale of 1 to 5.

Fig. 8 Percentage of indicators to Calculate Motivation

Table 2 Frequency Table for Motivation

Level

Values

Fi

hi%

Strongly disagree

[20–23]

2

2%

Disagree

[24–27]

0

0%

Undecided

[28–31]

8

9%

Agree

[32–35]

35

39%

Strongly agree

[36–40]

45

50%

90

100%

Total

330

P.-S. Delgado et al.

Fig. 9 The motivation of users regarding the practice of prototyping

Table 3 Indicator scores for Motivation.

Motivation (M) Indicator

Weight (W)

Mean (X)

Attention (a)

0.13

4.39

Relevance (r)

0.28

4.43

Confidence (c)

0.20

4.48

Satisfaction (s)

0.39

4.51

M = WaXa + WrXr + WcXc + WsXs

4.47

5.2 User Experience about Prototyping Practice The user experience was determined through 4 indicators: social interaction, fun, competition, and challenging. Each indicator is evaluated with 2 questions except for the funny indicator that was evaluated with 3 questions, for a total of 9 questions. Figure 10 shows the percentages obtained according to each indicator, differentiated from each other by the levels where the light blue color is completely in agreement, light green, in agreement, and amber yellow is indifferent. Users consider that this good practice promotes social interaction with a 62% rating between 4 and 5, but there is 29% who are indifferent. 95% consider this practice fun. 96% consider this way of working very competent, finally, 91% consider it challenging. To calculate the user experience, a frequency table is generated by the sum of the evaluation of the 9 items expressed in Table 4. As shown in Table 4 and Fig. 11, 82 of 90 users had a positive experience with the prototyping technique, which represents 91% of the users. Likewise, in Fig. 12, 6% is observed with an undecided rating, which is explained by the indicator of social interaction. Table 5 shows the value and weight of each indicator and the User Experience dimension, for which a value of 4.36 was obtained on a scale of 1 to 5.

User’s Appreciation About Software Prototyping: A Case Study

331

Fig. 10 Percentage of indicators to calculate User Experience

Table 4 Frequency Table for User Expirience

Level

Values

fi

hi%

Strongly disagree

[20–24]

1

1%

Disagree

[25–29]

2

2%

Undecided

[30–34]

5

6%

Agree

[35–39]

38

42%

Strongly agree

[40–45]

44

49%

90

100%

Total

Fig. 11 The user experience of the practice of prototyping

5.3 Learning About Prototyping Practice The appreciation of user learning was determined through three indicators: shortterm learning, long-term learning, and application. Each indicator is evaluated with 1 question in the questionnaire, for a total of 3 questions. Short-term and long-term learning means that the user has understood how to participate in a prototyping workshop. The application means that the user plans to apply this knowledge in the future in their personal or work environment.

332

P.-S. Delgado et al.

Fig. 12 Percentage of indicators to Calculate Learning

Table 5 Indicator scores for User Experience

User Experience (UE) Indicator

Weight (W)

Mean (X)

Challenging (ch)

0.23

4.32

Competent (c)

0.25

4.42

Funny (f)

0. 31

4.29

Social Interaction (s)

0. 22

4.41

UE = WchXch + WcXc + WfXf + WsXs

4.36

As can be seen in Fig. 12, users positively value short-term, long-term learning, and its application with more than 45% (agree and strongly agree) for the three indicators. In these results, an undecided assessment of almost 50% of users in the three indicators stands out. To calculate the appreciation for learning, a frequency table is generated by the sum of the evaluation of the 3 items expressed in Table 6. According to the frequency analysis in Table 6 and according to Fig. 13, users manifest themselves between the levels: “agree” with 40 users and “strongly agree” with 11 users, but there are 34 undecided users. Table 6 Frequency Table for Learning

Level

Values

fi

hi%

Strongly disagree

[9–10.1]

3

3%

Disagree

[10.2–11.3]

2

2%

Undecided

[11.4–12.5]

34

38%

Agree

[12.6–13.7]

11

12%

Strongly agree

[13.8–15]

40

44%

90

100%

Total

User’s Appreciation About Software Prototyping: A Case Study

333

Fig. 13 Learning in the practice of prototyping

Table 7 Indicator scores for Learning

Learning (L) Indicator

Weight (W)

Mean (X)

Short-term learning (s)

0. 41

4.42

Long-term larning (r)

0. 14

4.43

Application (a)

0. 45

4.41

M = WsXs + WrXr + WaXa

4.42

However, the experts value the long-term learning criterion with weak weight, compared to the other 2 criteria presented in Table 7. The application of the prototyping technique in other contexts is the criterion with the highest evaluation. Applying AHP, as shown in Table 7, a value of 4.42 was obtained on a scale of 1 to 5 for the learning dimension, which represents a positive assessment.

5.4 End-User Perception About Prototyping Practice When making a general analysis, the appreciation of end- users were determined through the 3 dimensions, motivation, user experience, and learning, totaling 20 questions. These items are rated on a Likert scale from 1 to 5 in ascending order, where 1 is strong disagreement and 5 is in total agreement. To analyze these results, a frequency table is created with the result of the 20 questions. According to Table 8, users express themselves between the levels: “agree” with 29 users and “strongly agree” with 42 users, but there are 16 undecided users. In other words, 71 out of 90 users had a favorable opinion of this good practice. From Fig. 14, it can be seen that 47% of users manifest at a level of complete agreement, 32% of users at a level of agreement, accumulating a total of 79% between values 4 and 5. However, there is 18% with an undecided rating explained by the rating

334 Table 8 Frequency Table for User Perception

P.-S. Delgado et al.

Level

Values

fi

hi%

Strongly disagree

[53–61.4]

2

2%

Disagree

[62.4–70.8]

1

1%

Undecided

[71.8–80.2]

16

18%

Agree

[81.2–89.6]

29

32%

Strongly agree

[90.6–100]

42

47%

90

100%

Total

Table 9 Dimensions scores for User Perception

User Perception (UP) Dimensions

Weight (W)

Mean (X)

Motivation (m)

0. 23

4.45

User Experience (u)

0. 45

4.35

Learning (e)

0. 32

4.42

UP = WmXm + WuXu + WeXe

4.40

of 3 in the learning indicators. Likewise, obtaining the weights of the dimensions, the following operation is performed to determine the value of the investigation variable. According to Table 9, a score of 4.40 was determined for user perception according to the Likert scale. This tells us that this good practice of prototyping is important from the users’ perspective. Additionally, it has been found in the analysis of the data that the younger people are more in agreement with the practice of prototyping concerning to the older ones. However, this does not represent a problem, it simply shows a slight difference in their evaluation but both positive. These results are shown in Fig. 15. Finally, a comparison of the responses was made for each academic area, but no significant differences were found between the responses. i.e., for the case studied, the professional profile of the users is indifferent.

Fig. 14 Assessment of user perception of the practice of prototyping

User’s Appreciation About Software Prototyping: A Case Study

335

Fig. 15 Valuation by ages of the prototyping practice

6 Discussions An advantage of the prototyping technique is the visual communication through these prototypes, closing the idea created in the heads of the users during the understanding of the requirements, reinforcing interpersonal communication skills with this technique [30]. The results of this research show that like [30], the use of prototypes also arouses the interest of the participants, thus helping to reduce communication barriers in the requirements phase. SPIDe [31], is a proposed technique for the requirements process. Its results show after the application of the first meeting recommended by the technique, a second meeting would be held where the validation of the prototypes designed by the same users would be carried out, as was done in this investigation. Thus, each requirement is confirmed, taking advantage of the enthusiasm of the users to feel that they are co-authors of the software. [32] affirms that the scenarios have the advantage of describing the user’s interactions with the software through concrete examples of how each task is carried out. consequently, it allows to validate these requirements effectively. This research shows that users are willing to work on RQs, therefore with the help of prototyping the scenarios could be better interpreted for both users and developers. For [33], the mocks are intended to be a communication object and generate feedback and that the creation time is less than that of the prototypes. The authors mention that a negative point of this technique is that it requires a solid commitment between the development team and the client. The use of prototyping could help to complement this negative aspect since when the number of requirements to be elaborated is very high, the manual task becomes repetitive, generating demotivation in the participants. However, the results of this research show that with the support of an RQ engineer it is possible to quickly assemble the prototypes at runtime with the suggestions of the participants, helping to reduce repetitive tasks while maintaining motivation. In this research, it has been found that the prototyping technique has satisfaction of 79%, which, even when a generalized comparison cannot be made, it is important to point out that satisfaction with the questionnaire technique is superior according to the results of [17].

336

P.-S. Delgado et al.

The results of this research coincide with [11] and [6] in the sense that there are positive results, saying that prototyping facilitates communication between developers and users, in addition to this technique, Users are willing to collaborate in the software process. This research evaluated low-fidelity prototypes, however [6] mentions that having high-fidelity prototypes increases the imagination of end-users approaching the final product and this has the effect that users propose more changes. Although the work [18] focuses on the impact on the quality of the software product, this research is aimed at satisfying the people who participate in the process, it could be said that both results complement each other, one with the perspective of product quality and the other in the appreciation of the user involved. Although the research [19] evaluated two ways of learning programming using mockups, this research is oriented towards the appreciation of the users who participate in this process, it is stated that both types of research coincide with their results in terms of motivation, experience user and learning using the prototyping technique. Finally, despite the comparisons presented in the related works, the authors of these investigations indicate that the techniques evaluated are complementary. i.e., there is no one better than the other but rather depends on the characteristics of the project to be developed. However, everyone agrees that the use of prototypes improves communication with users. Finally, this work presents the appreciation of the perspective of the final users of the system unlike several works reviewed who evaluate developers.

7 Threats to Validity The users who participated in this case study are people who are not constantly involved with software projects, for many of them, it is the first time participating in a software validation session with prototyping. However, the results could vary if they were active users of software projects. In this way, try to generalize the results.

8 Lessons Learned For this case study the following points were achieved: • As we have a large number of people who validate the prototypes, it was considered to group by affinity according to academic department. • To ensure greater attendance at the session, a space for the session was requested within a monthly meeting as an academic department. • The resolution of conflicts of interest during the session must be guided by the development team but decided by the participants of the session.

User’s Appreciation About Software Prototyping: A Case Study

337

• Plan the session in a short time of maximum 1 h and do not extend the estimated time. • Share a light snack between sessions as a stimulus to stay connected to the interest of the session.

9 Conclusions According to the results, 89% of users were motivated and 91% with positive experiences. However, only 56% consider that this way of working will serve them in the future. The authors consider that this last value is low since the users of the case study do not have constant participation in software projects, despite this, the results are favorable. The general results indicate that, for the case studied, 79% of the users agree and strongly agree with the practice of prototyping in the software process, demonstrating that, as well as the methodologies, the end-users also agree with the validation of requirements through prototyping. The user’s appreciation of the prototyping technique is positive in the same way as the impact of the use of this technique on the quality of the software product. Therefore, we can conclude that the prototyping technique is a very important and determining technique in the software process, ensuring the quality of the process and in turn the quality of the software product itself.

10 Future Works The results of this investigation leave pending the study of more cases that demonstrate the need for software prototyping in other contexts. It could be very contributing, promoting research aimed at making comparison experiments of techniques not only in terms of user satisfaction but also in terms of their impact on product quality. Likewise, evidencing the importance of the prototyping technique, it is important to carry out research aimed at incorporating this technique in the training content of academic institutions. This could facilitate the work of Front-end developers, considering that this role is part of the trend in software engineering [36].

References 1. Sambare T (2017) Agility: the need of an hour for software industry. Int J Adv Res Comput Sci 8(9):41–46. https://doi.org/10.26483/ijarcs.v8i9.4886 2. Hussain A, Mkpojiogu EOC (2016) Requirements: towards an understanding on why software projects fail. AIP Conf. Proc. 1761:020046

338

P.-S. Delgado et al.

3. «The Standish Group report 83.9% of IT projects partially or completely fail | Open Door Technology» (2015). [En línea]. https://www.opendoorerp.com/the-standish-group-report-839-of-it-projects-partially-or-completely-fail/. Accedido 22 abr 2020 4. Brito IS, Alburqueque M, Barreiros J, Cruz M (2018) «Industry needs in Requirements Engineering». In: International Conference of the Portuguese Society for Engineering Education (CISPEE 2018) 5. Chari K, Agrawal M (2017) Impact of incorrect and new requirements on waterfall software project outcomes. Empir. Softw. Eng. 23(1):165–185. https://doi.org/10.1007/s10664017-9506-4 6. Abrahamsson L, Melin Wenström P (2018) Prototyping as a Technique for Requirements Engineering in Agile Software Development–A Case Study» Linköpings universitet 7. Suaib M, Iqbal T (2014) Requirement elicitation technique-a review paper. J. Comput. Math. Sci. IJCMS. 3(9):6 8. Boehm BW (1984) Software engineering economics. IEEE Trans. Softw. Eng. SE-10(1):4–21. https://doi.org/10.1109/TSE.1984.5010193 9. Pacheco C, García I, Reyes M (2018) Requirements elicitation techniques: a systematic literature review based on the maturity of the techniques. IET Softw. 12(4):365–378. https://doi. org/10.1049/iet-sen.2017.0144 10. Clerissi D, Leotta M, Reggio G, Ricca F (2016) «Test driven development of web applications: A lightweight approach» In: Proceedings - 2016 10th International Conference on the Quality of Information and Communications Technology, QUATIC 2016, pp. 25–34 (2017) 11. Rivero JM, Grigera J, Rossi G, Luna ER, Montero F, Gaedke M (2014) Mockup-driven development: providing agile support for model-driven web engineering. Inf. Softw. Technol. 56(6):670–687. https://doi.org/10.1016/j.infsof.2014.01.011 12. RodríguezBarajas CT (2017) Impacto de los requerimientos en la calidad de software | Tecnología Investigación y Academia. Tecnol. Investig. Acad. 5(2):161–173 13. Diéguez, M., Sepúlveda, S., Canullan, D.: «Diseño de un Documento para la Elicitación y Especificación de Requerimientos: Caso Práctico», en Cuarto Encuentro de Informática y Gestión (EIG), October, p. 10 (2010) 14. Ahmed M, Adil M, Latif S (2015) Web application prototype: state-of-art survey evaluationational. Nat. Softw. Eng. Conf. NSEC 2015(2016):19–24 15. Porto JP, Gardey A (2019) Definición de mockup - Quées, Significado y Concepto. [En línea]. https://definicion.de/mockup/. Accedido 30 ene 2020 16. Swebok IC (2014) Software Engineering Body of Knowledge, v3 edn. IEEE Computer Society, Los Alamitos 17. Monsalve ES, Werneck VMB, Do Prado Leite JCS (2010) Evolución de un juego educacional de ingeniería de software a través de técnicas de elicitación de requisitos. In: 13th Workshop on Requirements Engineering, WER 2010, pp. 63–74 18. Khan MJ, Khan RA (2019) An investigation of prototyping technique in pakistani software industry. Mehran Univ Res J Eng Technol 38(4):945–960. https://doi.org/10.22581/muet1982. 1904.08 19. Estacio B, Valentim N, Rivero L, Conte T, Prikladnicki R (2015) Evaluating the use of pair programming and coding dojo in teaching mockups development: an empirical study. In: 2015 48th Hawaii International Conference on System Sciences, pp. 5084–5093 (2015) 20. Butt SM, Onn A, Tabassam N, Butt MM (2014) Enhancing the effectiveness of software development and interface evaluation. J Comput Sci Inf Technol. 2(1):75–89 (2014) 21. Rocha T, Hak J-L, Winckler M, Nicolas O (2017) A comparative study of milestones for featuring gui prototyping tolos. J Softw Eng Appl SCIRP 22. Bajammal M, Mazinanian D, Mesbah A (2018) Generating reusable web components from mockups. In: ASE 2018 - Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pp. 601–611 23. Pereira PFF, Rodrigues F, Ferreira C (2019) Code generator from mockups. In: Iberian Conference on Information Systems and Technologies, CISTI, vol. 2019-June

User’s Appreciation About Software Prototyping: A Case Study

339

24. Alperowitz L, Weintraud AM, Kofler SC, Bruegge B (2017) Continuous Prototyping. In: Proceedings-2017 IEEE/ACM 3rd International Workshop on Rapid Continuous Software Engineering RCoSE 2017, pp. 36–42 25. Ligman J, Pistoia M, Tripp O, Thomas G (2016) Improving design validation of mobile application user interface implementation. In: IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft) 26. Clerissi D, Leotta M, Reggio G, Ricca F (2016) A lightweight semi-automated acceptance testdriven development approach for web applications. In: Bozzon A, CudreMaroux P, Pautasso C (eds) Web Engineering. Springer International Publishing, Cham, pp 593–597. https://doi. org/10.1007/978-3-319-38791-8_55 27. Suranto B (2015) Software prototypes: enhancing the quality of requirements engineering process». In: 2nd International Symposium on Technology Management and Emerging Technologies, ISTMET 2015 - Proceeding, pp. 148–153 (2015) 28. Ghazali M, Suaib NM, Sulaiman S, Ibrahim N, Hananto AL, Beran A (2018) Preliminary work on design thinking: addressing challenges using low-fidelity prototyping with rural teenagers. In: Proceeding - 2018 International Conference on ICT for Rural Development: Rural Development through ICT: Concept, Design, and Implication, IC-ICTRuDEv 2018, pp. 158–161 29. Von Wangenheim CG, Savi R, Borgatto AF (2013) SCRUMIA - an educational game for teaching SCRUM in computing courses. J Syst Softw 86(10):2675–2687 30. Dias HF, da Costa J, Pimentel HC (2017) O Papel da Comunicação na Elicitação de Requisitos – Entrevistas e Brainstorming. Revista de Engenharia e Pesquisa Aplicada 2(4):1–13. https://doi. org/10.25286/repa.v2i4.845 31. Rosa JCS, Matos E, Santos FS, Silva GJF (2018) Experimentando o SPIDe aplicado à Elicitação de Requisitos. In: do WER 2018 - Work. em Eng. Requisitos (2018) 32. Tascon A, Domínguez F (2017) Analysis of the usefulness of the scenario technique on requirements elicitation. Rev Antioqueña las Ciencias Comput Ing Softw. 7(1):59–67 (2017) 33. Schimiguel J, de Fabris L, Lopes CS, Peixoto A, Martins CA (2017) Técnicas para requisitos funcionais de Interfaces Homem Computador com recursos em elicitação. Res Soc Dev 6(1):47–63. https://doi.org/10.17648/rsd-v6i1.148 34. Savi R, von Wangenheim CG, Borgatto AF (2011) A model for the evaluation of educational games for teaching software engineering. In: 2011 25th Brazilian Symposium on Software Engineering, pp. 194–203 35. Goepel KD (2018) Implementation of an online software tool for the analytic hierarchy process (AHP-OS). Int J Anal Hierar Process 10(3):469–487. https://doi.org/10.13033/ijahp.v10i3.590 36. Gurcan F, Köse C, Kose C (2017) Analysis of software engineering industry needs and trends: implications for education. Int J Eng Educ. 33(4):1361–1368

Predicting the Number of Pedestrians per Street Section: A Detailed Step-by-step Example Christophe Guyeux

Abstract There are currently software solutions for estimating pedestrian flow by street section, but they are proprietary and expensive. Moreover, they rely mainly on cell phone technology, or even wifi, and are only applicable in areas on land with good network coverage. The goal of this application article is to show that reasonable results can be obtained for free from a limited number of manual pedestrian flow surveys, taking advantage of OpenStreetMaps data and advances in artificial intelligence. We will show concretely such a realization, step by step, in a neighborhood of Paris, France, and we will discuss the advantages and disadvantages of the approach, as well as future developments. Keywords Pedestrian prediction · Geomarketing · OpenStreetMaps

1 Introduction Knowing the flow, both pedestrian and car, for all the road sections of a city, is useful in many ways, especially for geomarketing. For example, the location of a restaurant assumes that there is a significant pedestrian flow at meal times, regardless of the day of the week. For a pharmacy, on the other hand, the pedestrian flow must be sustained during its opening days, but the meal times have no impact here. Proprietary solutions exist, which mainly rely on geolocation data from smartphones. Google, for example, provides information on the time of day when people are in stores, or on road traffic for its “Google maps” solution. Such knowledge is made possible, on the one hand, thanks to the knowledge of the position of smartphones that have accepted to be geolocated, and on the other hand, thanks to the knowledge of the geolocation of wifi terminals following the scanning by Google Cars. Flux vision of Orange, for its part, takes advantage of the distribution of smartphones in its network, and knowing its market share, proceeds by extension to deduce the number of smartphones, then of individuals, in the area of interest. These solutions are expensive, making them impossible to use for small companies or local authorities. They also have a difficult objective to reach, that of the C. Guyeux (B) FEMTO-ST Institute, UMR 6174 CNRS, Université de Bourgogne-Franche-Comté, Besançon, France e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_29

341

342

C. Guyeux

instantaneous flow, when a knowledge of an average flow per hour and per day of the week is often sufficient. As for accuracy, it is often questionable, especially in approaches that use successive approximations. For example, changing the number of smartphones in an area to the number of pedestrians implies fixing the proportion of individuals equipped with smartphones, as well as the proportion of individuals moving on foot, which varies from one area to another and is difficult to know. Finally, these solutions are at the limit of legality, given the exploitation of personal data, often without the users’ knowledge. The objective of this paper is to show that OpenStreetMaps data, artificial intelligence tools and software solutions present in python modules are mature enough to achieve, at low cost, a reasonable estimation of the pedestrian flow by sections of a city, provided that one sticks to an average approach per hour and per day in a week. This article is meant to be pedagogical and practical, detailing step by step the elements to be implemented, up to the choice of the modules to be considered, up to the software bricks. The case considered here is the pedestrian flow, which is easier to estimate than the car flow. The way of constituting the learning base is detailed, and it is applied to a prediction of the pedestrian flow, thanks to a database of 310 manual surveys. A discussion closes this first study, which should be seen more as a proof of concept than as an operational solution. The advantages and disadvantages of the approach are presented, as well as future improvements. The remainder of this applicative article is as follows. In the next section, the way to design the basis of knowledge is presented in details. Section 3 is devoted to the learning process, to the prediction of the pedestrian flow, and to its evaluation. A discussion is outlined in Sect. 4, detailing the pros and cons of the proposal. This article ends by a conclusion section, in which the approach is summarized and intended future work is outlined.

2 Designing the Dataset Let us present the problem again. We assume that we have a list of addresses with the number of pedestrians per day, which was obtained manually in the field. We wish to extract knowledge from this list to predict the number of pedestrians per day at new addresses. For each known address, we will retrieve from OpenStreetMaps all the points of interest (pub, cinema, school...) that can explain this pedestrian flow. These points of interest will constitute the features of a regression problem, for which the pedestrian flow will be the target variable. The purpose of this section is to present how such a learning base can be built in practice. For convenience, we first create a Point object class, which will contain the OpenStreetMaps identifier, its longitude and latitude (WGS84), and a more detailed name. We then define a function which, at a given address, returns its point (i.e., an object of the above class). This can be done with the Nominatim function of the geopy [1] library (geocoders module). Applied to the address “11 rue des boulangers, Paris, France”, this function returns “Node 707655981: (48.8458011, 2.3538782)”.

Predicting the Number of Pedestrians...

343

We then need to define a function that, at two points, returns the distance between these points. The latter can correspond to two realities: either the shortest path of a pedestrian, or the geodesic distance (as the crow flies). In the first case, the Router function of the pyroutelib3 module [4] can be considered, while the geodesic function of the geopy distance module allows to obtain the distance of the same name. It is this last distance that will be used in what follows. We then define a boundingBox function, which receives a point P (latitude, longitude) and a distance h in km, and returns the coordinates of the corners of the spherical square surrounding P and of side 2h . We first need to compute the Earth radius at a given latitude, according to the WGS-84 ellipsoid: / radius =

An 2 + Bn 2 Ad 2 + Bd 2

where: – – – –

An Bn Ad Bd

= MaS 2 ∗ cos(lat), = Mi S 2 ∗ sin(lat), = MaS ∗ cos(lat), = Mi S ∗ sin(lat),

in which lat is the given latitude and MaS (resp. Mi S) is the major (resp. minor) semiaxis of WGS-84 geoidal reference. Then the bounding box of half side h surrounding the point at given coordinates (lat, lon), assuming local approximation of Earth surface as a sphere of radius given by WGS84, is provided by: – – – –

lat Min = lat − h/radius, lat Max = lat + h/radius, lon Min = lon − h/ pradius, lon Max = lon + h/ pradius,

where pradius is the radius of the parallel at given latitude, that is: radius ∗ cos(lat). For the address provided above and a distance of 1 km, the obtained bounding box is for instance equal to: (48.83680090201974, 2.3402019121323954, 48.854801297980266, 2.3675544878676047).

These “bounding boxes” provide the limits to the investigations of the points of interest. We will collect all the points of interest in a square of side 1km centered in P, then we will keep only those which are, for example, at 300 m. of P for a pedestrian. We can now implement a search_amenity function which, given a type (store, tourism...) and a name (bookstore, museum...) of center of interest, as well as a boundingBox, returns the list of all points corresponding to this center of interest. To do this, we need to set up an overpass query (overpy module [3]) on OpenStreetMaps nodes and paths, with type, name and boundingBox:

344

C. Guyeux

def o v e r p a s s _ t o _ c o o r d i n a t e s ( q u e r y ) : api = o v e r p y . O v e r p a s s () r = api . q u e r y ( q u e r y ) r e s u l t s = [ P o i n t ( i d e n t = int ( node . id ) , name = node . tags [ ’ name ’ ] if ’ name ’ in node . tags else ’’, l o n g i t u d e = float ( node . lon ) , l a t i t u d e = float ( node . lat ) ) for node in r . n o d e s ] r e s u l t s + = [ P o i n t ( i d e n t = int ( way . id ) , name = way . tags [ ’ name ’] if ’ name ’ in way . tags else ’ ’ , l o n g i t u d e = f l o a t ( way . c e n t e r _ l o n ) , l a t i t u d e = f l o a t ( way . c e n t e r _ l a t ) ) for way in r . ways ] return results def s e a r c h _ a m e n i t y ( a m e n i t y _ t y p e , amenity , bbox ) : q u e r y = ’’’ [ out : json ][ t i m e o u t : 100 ]; ( node [" ’ ’ ’ + a m e n i t y _ t y p e + ’ ’ ’ "=" ’ ’ ’ + amenity + ’ ’ ’ "] ’ ’ ’ + str ( bbox ) + ’ ’ ’ ; way [" ’ ’ ’ + a m e n i t y _ t y p e + ’ ’ ’ "=" ’ ’ ’ + amenity + ’ ’ ’ "] ’ ’ ’ + str ( bbox ) + ’ ’ ’ ; r e l a t i o n [" ’ ’ ’ + a m e n i t y _ t y p e + ’ ’ ’ "=" ’ ’ ’ + amenity + ’ ’ ’ "] ’ ’ ’ + str ( bbox ) + ’ ’ ’ ; ); out c e n t e r ; ’ ’ ’ return overpass_to_coordinates ( query )

Applying search_amenity to the address previously used as an example, for the amenity type “pub” and a distance of 200m will produce, for example : Epsilon. Node 310363590: (48.8456039, 2.3547969) L’In\’{e}vitable. Node 310363627: (48.8453724, 2.3548312) Baker Street Pub. Node 2298160176: (48.8457908, 2.3539346)

We now create a POI class which, – to a given address P,

Predicting the Number of Pedestrians...

345

– to a list of points of interest (fountain, fast food...), – to a maximum number N of places per center of interest, – to a radius bounding the bbox where the search is performed, will return N values per center of interest, corresponding to the N closest places of this type to P. These values are between 0 and 1, where 1 corresponds to the closest place to P (and 0 the farthest), for the distance chosen above. This class is easily built from search_amenity. For example, if there are two fountains in the bbox, the first one being very close to P and the other one being quite far, and we set N to 5, the 5 values for the explanatory variable fountain will be: 0.9, 0.35, 0, 0, 0. More precisely, if we are interested in the 3 nearest pubs, mailboxes, and bus stops around address A, and for a search box limited to 400 m., we will construct a POI object as follows: poi = POI ( A . name , { ’ a m e n i t y ’ : [ ’ pub ’ , ’ p o s t _ b o x ’ ] , ’ highway ’ : [ ’ bus_stop ’]} , 3, 0.4)

and the obtained features will be equal here to: ’pub0’: 0.9903957263041293, ’pub1’: 0.841447593474445, ’pub2’: 0.8107175585541035, ’post_box0’: 0.8536324093838593, ’post_box1’: 0.6631732523156194, ’post_box2’: 0.6148255899296251, ’bus_stop0’: 0.8450284438921247, ’bus_stop1’: 0.8296554668571713, ’bus_stop2’: 0.6503689376957169.

A graphical representation, in a jupyter notebook, of the location of these features in relation to the address considered can be obtained via the ipyleaflet module [2]. We are then left to define the list of amenities and the number of items to consider, to obtain the basis of knowledge. The chosen list of amenities is detailed in Table 1, and we have selected the 5 closest locations (5 features per point of interest).

3 Predicting the Pedestrian Stream We collected the daily pedestrian flow for 310 addresses in France, and associated them with the feature base as detailed in the previous section. With 5 features for each of the 69 point of interest types, we have 345 features per address, with values between 0 and 1 (no need for standardization). A basic descriptive statistical analysis leads to the following values (in pedestrian per day):

346

C. Guyeux

Table 1 Considered list of points of interest amenity

highway historic office railway shop

tourism mean std min 25% 50% 75% max

arts_center, atm, bank, bar, bicycle_parking, cafe, car_rental, courthouse, drinking_water, fast_food, fountain, ice_cream, marketplace, parking, pharmacy, place_of_worship, post_box, post_office, pub, public_bookcase, recycling, restaurant, school, taxi, toilets, townhall, bus_stop, pedestrian, monument, tower, insurance, estate_agent, telecommunication, subway_entrance, alcohol, antiques, bag, bakery, beauty, books, butcher, clothes, confectionery, convenience, cosmetics, deli, department_store, florist, gift, greengrocer, hairdresser, hospital, houseware, interior_decoration, jewelry, laundry, music, newsagent, optician, pastry, shoes, stationery, supermarket, tobacco, toys, hotel, information, museum, viewpoint

4526.690323 4672.910784 97.000000 1116.000000 3418.500000 6029.250000 37998.000000

For the sake of evaluation, we set a default value: what error would be made by predicting everywhere the average number of pedestrians (i.e. 4526.69). The mean absolute error obtained by using this dummy predictor is equal to 3372.54 pedestrian per day, which is high compared to the average flow. We separate our knowledge base into learning (80% of the data, randomly drawn) and testing sets. We perform the learning, then we measure the prediction error (mean absolute error MAE). The considered learning methods were respectively the extreme gradient boosting XGBoost [7], the random forests [6, 12], lightGBM [10], and support vector machines [8], all with their predefinite hyperparameters as chosen in the Scikit-learn library [12]. Obtained results are listed in Table 2.

Predicting the Number of Pedestrians...

347

Table 2 Mean absolute error of various predictive models MAE Model 1673 XGBoost 2315 Random forest 1912 LightGBM 2459 Support vector machine

This baseline study, which of course should be further investigated in a more systematic paper, led us to prefer XGBoost as a predictive model. This choice is reinforced by the fact that it is currently considered as one of the best predictive algorithm. We only varied the “max depth” hyperparameter (from 2 to 17), because the other hyperparameters had a lesser effect on the score variation. The obtained MAEs are represented in the Fig. 1. As can be seen, the best MAE (1333.22) was obtained for a max depth of 4. This MAE is much better than the one obtained by the naive approach (prediction by the mean), and is low enough for the predictive tool to be useful in practice.

4 Discussion Online services exist, such as Google, which allow for example to access the affluence of a place of interest. But these services are not free, and they are based on continuous geolocalization readings via smartphones. The objective of this study is to show that we can, with easily accessible python modules, easily manage to predict the daily flow by artificial intelligence techniques, provided that we have a reasonable number of field surveys. This is only a first study, not optimized, whose only claim is to be a proof of concept. This work can be improved in a number of ways. First of all, the choice

Fig. 1 Mean absolute error of XGBoost prediction, depending on the max depth

348

C. Guyeux

of the types of points of interest must be done in a thorough way, for example by feature selection techniques. The number of features per type should also be fixed precisely, and there is no reason why this number should be the same for all POI types. Moreover, a selection of the learning model must be really made, while we have chosen XGBoost in a rather quick way, and not at all thorough. Methods such as deep neural networks [5, 11] should be compared too. But the goal of this article was not to make such a comparison, nor to obtain the best possible predictions, but just to prove the feasibility of the approach. Above all, the database here is really too small, and having only 310 manually collected addresses is not enough. Of course, this survey takes time and money. Also it would be interesting to find the best compromise between the quality of the predictions and the number of addresses in the learning database, but this study remains to be conducted. Another interesting study to conduct is to extract knowledge from this learning, to better understand pedestrian habits. It would be relevant to cluster street sections to derive various categories: residential streets, cinema-friendly streets, office-only streets, etc. Then learn from each cluster, and derive the feature importance, to understand in depth what makes a street more suitable for a bakery, etc. This new knowledge will not only be useful for geomarketing studies, but could also be useful for geographic urbanization studies. Finally, we have only discussed pedestrian flow here, whereas vehicle flow (cars, trucks, etc.) is also of practical interest [9]. However, our first studies tend to show that our approach is currently too coarse to be applied, mutatis mutandis, to such flows. There are various reasons for this lack of success. First, pedestrians and vehicles do not always use the same sections. And it is difficult to distinguish between a pedestrian, a cyclist, and a car moving at a walking pace. Finally, the pedestrian route is small compared with that of a car, and is therefore highly dependent on local facilities and characteristics. There must be other reasons for this lack of success, which have not yet been identified. However, given the practical interest of predicting such a flow, efforts will be made in our next work to better understand and model these vehicle paths, in order to obtain a simple yet efficient solution for predicting this flow.

5 Conclusion In this paper, we were interested in showing the feasibility of pedestrian flow prediction, using free and open source tools: OpenStreetMaps data, python modules, artificial intelligence libraries. We focused on the case of prediction by hour and by day of the week, whatever the postal address, the only prerequisite being to have a sufficient number of manual pedestrian flow surveys. The approach has been detailed step by step, until the evaluation of the predictions made, and the discussion of the results obtained. In our future work, we first want to improve the flow prediction scores, notably by making a better model selection, and by evaluating the importance of features.

Predicting the Number of Pedestrians...

349

We also want to take a more serious look at the flow of cars, which is proving to be more difficult to predict. Finally, we would like to make a python library available to all those who might be interested in such predictions.

References 1. 2. 3. 4. 5.

6. 7.

8. 9.

10.

11. 12.

Geopy. https://geopy.readthedocs.io/en/stable/index.html. Accessed 09 Sep 2022 ipyleaflet. https://ipyleaflet.readthedocs.io/en/latest/index.html. Accessed 09 Sep 2022 overpy. https://python-overpy.readthedocs.io/en/latest/. Accessed 09 Sep 2022 pyroutelib3. https://github.com/MKuranowski/pyroutelib3/wiki. Accessed 09 Sep 2022 Aloysius N, Geetha M (2017) A review on deep convolutional neural networks. In: 2017 International Conference on Communication and Signal Processing (ICCSP), pp 0588–0592. IEEE Breiman L (2001) Random forests. Mach. Learn. 45(1):5–32 Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM sigkdd International Conference on Knowledge Discovery and Data Mining, pp 785–794 Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297 Floros G, Van Der Zander B, Leibe B (2013) Openstreetslam: global vehicle localization using openstreetmaps. In: 2013 IEEE International Conference on Robotics and Automation, pp 1054–1059. IEEE Ke G et al (2017) Lightgbm: a highly efficient gradient boosting decision tree. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 3149–3157, Red Hook, NY, USA, Curran Associates Inc LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444 Pedregosa F et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825– 2830

Information Technology Monitoring in Healthcare: A Case Study Vasco Ramos, Carolina Marques, Hugo Peixoto, and José Machado

Abstract The healthcare environment is particularly relevant when discussing information technology infrastructure monitoring since availability and communication are vital for the proper functioning of healthcare units. It is important to be able to easily monitor and observe each unit from a single point of access so that actions can be swiftly taken when there is a problem. This paper proposes a multi-site and multi-organization web and microservices-based information technology infrastructure monitoring solution. In addition to exploring the developed system and its architecture, it presents a case study resulting from the system’s implementation in an organization and holds a discussion about the obtained results to determine whether a multi-platform monitoring system improves information technology availability in the healthcare industry. Keywords IT Infrastructure Monitoring · Microservices · Health Information Systems · Healthcare

1 Introduction The digital transition has been, for many years, a priority in the European Union (EU) in all sectors of society and with a particular focus on healthcare [12]. Hospitals are intricate workplaces where people and information are dispersed, necessitating extensive coordination and communication between the experts employed there [5]. Moreover, Healthcare Information Systems must be considered in terms of a wide range of heterogeneous, distributed, and ubiquitous systems that speak various V. Ramos · C. Marques · H. Peixoto · J. Machado (B) ALGORITMI/LASI, University of Minho, Campus Gualtar, 4710 Braga, Portugal e-mail: [email protected] V. Ramos e-mail: [email protected] C. Marques e-mail: [email protected] H. Peixoto e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_30

351

352

V. Ramos et al.

languages, integrate medical equipments, and are individually tailored by various entities, which in turn were established by various persons with various aims [8]. The healthcare industry is under pressure to improve patient safety, operate more efficiently, reduce medical errors while keeping costs under control and protecting patient privacy. In this spirit, new and more complex information systems and digital standards have arisen. Thus, healthcare is becoming increasingly digital, with the hope of increasing the quality and efficiency of healthcare services and, therefore, improving the lives of the citizens who need them and everyone around them [2–4]. However, it is critical to ensure that technology does not fail when it is most needed [1]. Information Technology (IT) infrastructure monitoring systems are critical in this context because they allow for the analysis of the status of the provided services as well as various metrics related to the servers and infrastructure on which they are hosted. On this topic, an article was published with the main goal of understanding what constitutes a good monitoring solution, and analyzing the monitoring solutions currently available in the optics of a multi-location healthcare environment [6]. According to the findings of this study, current solutions do not provide APIbased IT monitoring management and consumption, a fully integrated multi-site and multi-organization integration schema, or a hybrid data collection mechanism that supports both pull-based and push-based data collection paradigms. Because of these constraints, the article proposed a monitoring system capable of dealing with such environments. In subsequent work, a monitoring solution was developed, implemented, and tested in a real-world environment (Tâmega e Sousa Hospital Center (CHTS)).

2 Previous Work and Updated Proposal As referenced above, this paper is the continuation of work exposed in a previous academic contribution [6]. In the previous work, research was conducted on the context and scope of monitoring systems and infrastructure in healthcare, followed by a requirements analysis and the design of a solution’s architectural proposal. However, some limitations and problems were found in the original architecture, given the more specific infrastructure and security requirements and constraints of healthcare facilities, discovered after requirements and architecture inspection carried out with the collaboration of the healthcare organization CHTS. The issues found in the proposed architecture were mainly related to data and services communication and consistency throughout the entire system. The data storing and alerting procedures being replicated in both the local and off-site servers were the main problem since it could, and most likely would, lead to widespread data and information inconsistencies, which could highly jeopardize the quality and usefulness of the proposed IT infrastructure monitoring solution.

Information Technology Monitoring in Healthcare: A Case Study

353

Therefore, given these new findings and the overall feedback and suggestions from the collaborating healthcare organization’s IT administrators a new solution architecture was designed while the high-level requirements remained the same. Figure 1 depicts the final architectural design, constituted by two different types of servers: the local server, which will be deployed in each healthcare facility and is responsible for all monitoring operations and tasks, and the guardian server, which will be deployed in an off-site location and provides aggregated monitoring information for each and all healthcare facilities. Each local server sends monitoring data to the guardian server through HTTP requests. All the information can be consumed by external clients through the guardian server’s public REST APIs. In this architecture’s implementation, as it is possible to see in Fig. 1, a progressive web application was developed to provide accessible and easy-to-use monitoring management and visualization to the end users.

Fig. 1 Architecture of the proposed monitoring solution

2.1 Guardian Server The guardian server has four main services: authentication service, core service, alert service, and data export service. All these services are virtually aggregated, safeguarded and carefully exposed to outside communication through an API gate-

354

V. Ramos et al.

way, which enables load balancing, caching, SSL encryption [9] and credential validation (authentication and authorization). The authentication service is in charge of user authentication and authorization. It is also in charge of user creation and administration. The API gateway uses this service to authenticate requests depending on the credentials given. • The core service contains all the logic related to the creation and management of organizations, groups and monitoring agents. It is also responsible for the creation and management of the monitoring agents’ health/status level settings. • The alert service is in charge of creating and managing notification channels as well as alert settings. It is also in charge of organizing and carrying out the duties related to the alerts’ occurrences. • The data export service exposes all monitoring data in standard formats, allowing any client to interpret the information. It also offers information on each monitoring agent’s current state and status history. • To support the four services, there are two databases: a time-series database, InfluxDB, for storing and consuming monitoring data, and a document-based database, MongoDB, for storing and managing all the remaining data (users, organizations, groups, monitoring agents, health/status levels, notification channels and alerts). Figure 2 presents a diagram that illustrates the described architecture.

Fig. 2 Guardian server — internal architecture

Information Technology Monitoring in Healthcare: A Case Study

355

2.2 Local Server The local server has two services: a push-based service known as “receiver”, and a pull-based service known as “executor”. These services connect with the guardian server via HTTP requests using access tokens, which are provided to each organization. This architecture with two services exists to accommodate the two aforementioned data collecting paradigms — pull-based and push-based —, as the services’ names indicate. The push-based service, or “receiver”, is in charge of all monitoring agents that are of the “deadman” type, i.e., those that do not require a service to query some target for metrics or information. Rather, are the monitoring services that communicate its status to the “receiver”. The “receiver” then sends the data to the guardian server to be stored, processed, and acted on (if there is a need to notify of any problems, i.e., send alerts). On the other hand, the pull-based service, or “executor”, handles the other types of monitoring agents, that is, the pull-based ones. This service is in charge of contacting the guarding server on a regular basis to update its internal information on which pullbased monitoring agents its organization has. It schedules the pull-based monitoring agent’s execution depending on each monitoring agent repetition configuration, and for each execution reports the gathered data to the guardian server to be stored, analyzed, and acted on, as was the case with the prior service. Figure 3 presents a diagram that illustrates the described architecture.

Fig. 3 Local monitoring server — internal architecture

356

V. Ramos et al.

3 Case Study After the platform has been developed, it was necessary to validate it in a real-world context and environment. Subjective and formal assessment and validation processes were used to demonstrate that the produced technology is capable of adoption and usage in the IT departments of healthcare organizations, as initially planned.

3.1 Methodology In this work, an evaluation was conducted with the collaboration of CHTS to test and validate the system. This evaluation was performed using a questionnaire which was carried out via an anonymous online form with the intent to collect useful information from possible end-users regarding important topics, both from the point of view of modeling and design, as well as validation of results. The questionnaire was divided into two sections. The main goal of the first section was for users to test and experience the platform by completing tasks such as searching, creating, and editing agents; managing, creating, and deleting alerts; and checking the agents’ health using the graphical visualization. The second section, on the other hand, included questions designed to assess user satisfaction with the monitoring system. The questionnaire was treated through organization and data interpretation techniques. The data from the closed questions were treated as being of the nominal type, allowing a statistical representation of the data, followed by the respective analysis and interpretation. On the other hand, the answers to the open questions were treated individually, manually extracting the topics and suggestions considered most important and relevant for the development of the platform, given the context of its execution.

3.2 Results A group of 13 participants, all of them administrators in a healthcare setting, responded to the questionnaire. During the first section, users stated that they were unfamiliar with the structure type and that the learning curve was slightly different when compared to users who had never used this type of tool before because this appeared to new users to be the only way to monitor agents and their metrics. Even though they struggled with the system at first, once they got used to it, the responses to the tasks were usually around the Easy mark, whereas in the beginning, they were Neutral or even Hard. In the second section there were seven statements that evaluated the overall satisfaction with the monitoring solution and its quality in the context of their day-to-day duties. The statements presented to the participants were as follows:

Information Technology Monitoring in Healthcare: A Case Study

357

1. The monitoring agents’ variety is appropriate/sufficient for your organization’s monitoring needs. 2. Every time a monitored service failed you were warned per the configurations you specified. 3. The alerting delivery mechanisms are appropriate/sufficient for your organization’s setup and needs. 4. The notification alerts provided enough context and information to successfully inform you of the problem at hand. 5. The deployment of the local server (i.e. the data collection clients) was simple and straightforward and occurred with zero or minor installation problems. 6. The monitoring platform, as a whole, simplified the monitoring management and consumption in your organization. 7. The monitoring platform, as a whole, fulfilled your organization’s needs better than the previously used monitoring solution. The answers were scored as numbers, being one the highest level of disagreement with each statement and five the highest level of agreement with each statement, with the mean and median values serving as primary indicators. Figure 4 summarizes the overall validation results. As can be seen in Fig. 4, it is possible to understand that the system presented a satisfiable set of functionalities with some shortcomings, namely the diversity of the monitoring agents’ types available. According to the participant’s responses, the system’s current strongest advantages are the deployment simplicity and good alerting capabilities. Finally, given the good scores to the system’s capability of meeting the needs of users and outperforming previous solutions, it can be concluded that the system successfully addresses the problems and necessities that were central

Fig. 4 Distribution of responses to the statements of the questionnaire’s second section

358

V. Ramos et al.

to this paper because of feedback on its capabilities and performance, when compared to other monitoring solutions, was also positive, it is also possible to conclude that the system has a good chance on being adopted by more healthcare facilities which is also a good indicator of its usefulness and quality. The final two questions, which are open-ended, were focused on understanding what functionalities the participants think the system is missing and also how the current functionalities can be improved and expanded. Most of the answers were related to increasing the diversity of the monitoring agents and alerting delivery mechanisms available. The other answers were related to UI-specific changes that could enhance the overall user experience and satisfaction.

4 Discussion The case study aimed to understand how the developed solution was perceived and used by IT administrators in real healthcare environments to validate whether the system successfully addresses the problems, concerns, and requirements discussed in the early stages of this paper. The feedback from the participants was generally very positive, as all of the indicators derived from the questionnaire responses pointed to the system’s success, both in terms of its ability to address the problems and concerns that inspired this paper, as well as the needs and expectations of the IT administrators who tested it, and also in terms of its positive behavior when compared to other solutions already on the market. The questionnaire yielded positive results, with a focus on the system’s ability to overcome the problems and difficulties associated with multi-organization and multi-site IT monitoring, which were at the heart of this work’s exploration topics, as well as the overall participant’s acceptance of the system and willingness to use it in place of the organizations’ previously adopted monitoring solutions. Although the feedback and subsequent evaluation results were positive, some flaws and limitations were discovered, primarily due to the limited number of monitoring agent types and alerting delivery mechanisms. These limitations will be addressed through future work planning discussed in more detail in the next section. These results allow an important conclusion, which is the answer to the guiding research question that originated this project. The question intended to understand if a multi-device/site and multi-organization monitoring solution would help improve the availability and, consequently, quality of the highly heterogeneous IT infrastructures of the healthcare industry. All results indicate a positive answer and, as such, corroborate the initial hypotheses made in [6] as well as the success of this solution’s architectural design and implementation. Furthermore, the case study and architecture designed and validated in this work overcome the limitations found in previously investigated monitoring solutions, namely API-based monitoring management and consumption, with a fully integrated

Information Technology Monitoring in Healthcare: A Case Study

359

multi-site and multi-organization integration schema and a hybrid data collection mechanism.

5 Conclusion This work introduced a novel IT monitoring solution tailored specifically for the healthcare industry. The solution, and respective architecture, that was designed and validated in this work, addresses particular problems and requirements such as full API-based IT monitoring management and consumption, fully integrated multi-site and multi-organization integration schema and a hybrid data collection mechanism that supports both pull-based and push-based data collection paradigms. To accomplish this, a microservices-based architecture was used, in which the solution’s key components are made available from an off-site installation known as the guardian server, and an on-site local server responsible for the organization’s monitoring data collection. The initial discussion with the CHTS’s IT administrators and the subsequent suggestions, which served as a critical instrument of technical and conceptual input, substantially affected the final approach and design. Overall, it can be concluded that it was possible to address and overcome the limitations and problems found, discussed in the first paper, [6], and summarized in this paper’s introduction. It was also possible to understand, from the validation process and results, that a multi-device/site and multi-organization monitoring solution can, indeed, improve the availability and quality of healthcare IT infrastructure and, consequently, their healthcare service’s quality, which was this work’s driving question.

5.1 Future Work Because this effort had to handle several difficulties in order to produce an integrated solution, there was no opportunity to further refine and improve all parts, resulting in implementation prioritization. The resultant solution may and should be utilized to leverage other functions not covered in this study. Taking into account the suggestions regarded interesting but not a priority, as well as the findings and comments from the validation questionnaires, the following features and enhancements were deemed relevant for future work: 1. Extend monitoring agents support. The implemented solution makes available a core set of monitoring agent types. To enable the solution to serve a larger variety of organizations, healthcare or not, it would be interesting to explore and implement additional monitoring agent types, as well as, improve the current agent type model to further simplify the process of adding new agent types to the solution.

360

V. Ramos et al.

2. Extend alerting delivery mechanisms support. Every organization has a different set of communication and technological tool stacks. Thus, although the current solution already provides a large option set of alerting delivery mechanisms, it would be interesting to further expand it and allow even more integrations, such as SMS, PagerDuty, WhatsApp, Telegram, and Signal, among others. 3. Extend data export standards support. The solution’s data export functionality is aimed at simplifying the integration of this system with other external monitoring systems or visualization dashboards. Since integration and interoperability is key in healthcare, it should be interesting to understand how this functionality could be expanded to support additional data metrics standards, namely OpenMetrics and OpenTelemetry, which are the two fastest-growing projects in this area of expertise [11]. 4. Improve real-time alerting. Real-time alerting is an existing functionality of the developed solution. However, its implementation was based on a Socket.IO web server configuration, which, despite allowing quick implementation of this type of mechanism, is not the most reliable, fault-tolerant or efficient option. As such, it should be investigated the possibility to replace the Socket.IO implementation with a combination of a message queue, e.g. RabbitMQ, ZeroMQ, and others, with WebSocket clients [7, 10]. Acknowledgements This work has been supported by “FCT-Fundação para a Ciência e Tecnologia” within the R&D Units Project Scope: UIDB/00319/2020.

References 1. 5 reasons to build a health monitoring system for a hospital. https://cprimestudios.com/blog/ 5-reasons-build-health-monitoring-system-hospital. Accessed 8 Oct 2021 2. Alolayyan MN, Alyahya MS, Alalawin AH, Shoukat A, Nusairat FT (2020) Health information technology and hospital performance the role of health information quality in teaching hospitals. Heliyon 6(10):e05040. https://doi.org/10.1016/j.heliyon.2020.e05040 3. Alotaibi Y, Federico F (2017) The impact of health information technology on patient safety. Saudi Med J 38:1173–1180. https://doi.org/10.15537/smj.2017.12.20631 4. Helms MM, Moore R, Ahmadi M (2008) Information technology (IT) and the healthcare industry. Int J Healthc Inf Syst Inf 3(1):75–92 5. Machado J, Alves V, Abelha A, Neves J (2017) Ambient intelligence via multiagent systems in the medical arena. Eng Intell Syst Electr Eng Commun 15:151–158 6. Marques C, Ramos V, Peixoto H, Machado J (2022) Pervasive monitoring system for services and servers in healthcare environment. Procedia Comput Sci 201:720–725. https://doi.org/10. 1016/j.procs.2022.03.097, the 13th International Conference on Ambient Systems, Networks and Technologies (ANT)/The 5th International Conference on Emerging Data and Industry 4.0 (EDI40) 7. O’Riordan M (July 2022) Message queues: the right way to process and transform realtime messages. Ably Blog. https://ably.com/blog/message-queues-the-right-way. Accessed 19 Aug 2022

Information Technology Monitoring in Healthcare: A Case Study

361

8. Peixoto H, Santos M, Abelha A, Machado J (2012) Intelligence in interoperability with aida. In: Chen L, Felfernig A, Liu J, Ra´s ZW (eds) Foundations of Intelligent Systems. Springer, Berlin, pp 264–273 9. Richardson C (2019) Microservices patterns: with examples in Java. Manning Publications, Shelter Island, New York 10. Tandon R (November 2021) System design: lessons from Netflix’s notification service design. Ravi’s System Design Newsletter. https://ravisystemdesign.substack.com/p/system-designlessons-from-netflixs. Accessed 19 Aug 2022 11. Varshney B (May 2022) OpenMetrics vs OpenTelemetry - a guide on understanding these two specifications | SigNoz. https://signoz.io/blog/openmetrics-vs-opentelemetry. Accessed 19 Aug 2022 12. Øvretveit J, Scott T, Rundall TG, Shortell SM, Brommels M (2007) Improving quality through effective implementation of information technology in healthcare. Int J Q Health Care 19(5):259–266. https://doi.org/10.1093/intqhc/mzm031

Quantum Computing and Lawful Interception Applications Kourtis Michail-Alexandros, Xilouris George, Chochliouros Ioannis, and Kourtis Anastasios

Abstract This paper proposes a secure turnkey framework for lawful interception (LI) solutions addressing the challenges that the new technologies of 5G and quantum computing impose to the work of police authorities on prevention and detection of illegal activities. Current LI technologies need to be extended to comply with the 5G architecture and a fully distributed LI platform is proposed, including virtualized components, dynamically instantiated within the 5G network, in the vicinity of the appropriate 5G network functions, across the whole chain of legal interception. Zero touch management system can hide the network and operational complexity from the operator, while Quantum Computing (QC) and Artificial Intelligence (AI) tools can enable easier extraction of intelligent information from large volumes of intercepted data and IoT devices. Keywords Lawful Interception · Quantum Computing · Network function virtualization · software defined networking · edge cloud computing · 5G

1 Introduction New technology drives changes in all aspects of society, including crime and the criminal justice system. Law enforcement agencies today employ a wide variety of technology to deal with crime and enforce the law in a more effective, efficient, and safe manner. Consequences of law enforcement use of technology are not empirically well understood, however. Scholars are just beginning to explore technology policy decisions, how police uses of technology evolve over time, and how and to what extent technology affects police outcomes. The rapid growth of technological innovation K. Michail-Alexandros (B) · X. George · K. Anastasios National Center for Scientific Research “Demokritos”, Athens, Greece e-mail: [email protected] C. Ioannis Hellenic Telecommunications Organization S.A. (OTE), Research Programs Section, Athens, Greece © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_31

363

364

K. Michail-Alexandros et al.

in policing has outpaced efforts to measure its impacts. This lack of knowledge and rapid evolution of form and function has been seen in LI, as well as predictive algorithms, and eWarrants. For-profit companies consider technology as a solution to a multitude of problems in public safety, yet the adoption of these tools and their uses in practice remain mostly unexamined. The array of motivations for adopting technology, the lack of rigorous research on its effectiveness, and the sheer number of technological products that currently exist suggests what is true for one type of technology is probably not generalizable. Thus, the need for a unified toolset of various of innovative tools that can be easily and seamlessly integrated into Law Enforcement Agencies (LEA) everyday operations and efficiently impact criminal persecution and terrorist activity prevention. Investments in information and communication technologies (ICTs) aimed at increasing judicial efficiency and reducing the cost of administration and management of the judiciary are common all over the world. Research has highlighted the importance of considering the complex impact of digitization of the judiciary on the key judicial values of accessibility, legitimacy, legality, and economy of judicial systems. Judicial reforms led by ICTs can be very effective in addressing these problems. However, positive outcomes require proper design and management of ICT-driven judicial reforms [1]. The proposed framework aims to provide LEAs a complete LI architecture supporting different LI operations and covering a wide spectrum of use cases. The complimentary integration of Quantum will also prepare and train LEAs to adapt to the ever-evolving technology landscape [2] and upgrade their operations to combat advanced terrorist and criminal tactics. Such implementation in a user-friendly and unified interface will also accelerate LEA adoption of LI mechanisms and in parallel with GDPR and EU regulation compliance [3] will also improve evidence collection as-a-whole. It is crucial to develop a holistic management framework for LEAs [4–6] and their LI operations without omitting noteworthy technological innovations of the not-so-distant future. The remainder of the paper is organized as follows. We initially present the technological enablers relevant to LI in the present context. In Sect. 3, the overall proposed system architecture and focus on the management modules is detailed. Section 4 concludes the paper and draws future research lines.

2 Lawful Interception and New Technology Drivers – State of the Art 2.1 5G Implications in LI 5G technology is much more than an evolution of 4G standards. It promises a significantly faster and higher transfer rates through improved mobile broadband connections, shorter response times (latency), ultra-reliable connections and a secure internet of things. Given the deployment scenarios of 5G and the inherent virtualisation and

Quantum Computing and Lawful Interception Applications

365

distribution of its architectural components (i.e. NF), support for network slicing and exploitation of edge computing there are several challenges that need to be addressed when it comes to lawful interception. Moreover, this project will take a closer look to two verticals that will be affected by 5G adoption and will inevitably impose challenges for lawful interception namely IoT and Automotive. Apart from technological improvements, 5G will offer very high security standards. On the other hand, 5G may provide enhanced tools in the hands of LEAs as for example in the more accurate location of the target. As the deployment of 5G will be based on small radius cells (in the order of tens of meters in contrast to 4G where cells are Kilometres wide), this will provide LEAs information about the exact location of the target (within meters). . Virtualisation of physical parts of the network (NFV) The new 3GPP specifications for the 5G Core System is based on a servicebased architecture definition (SBA) that decomposes the 5G system to separate Network Functions (NF). This new specification allows for virtualisation and distribution of the NFs across the Telco coverage area and in addition allows multiple instances of the same NF to run in different locations fulfilling network planning and connectivity service requirements. This model is best served by the Network Function Virtualisation (NFV) framework that provides means for orchestrating and managing the deployment if 5G system. In this context the challenges implied for LI are to meet similar virtualisation and orchestration capabilities for LI functions as well. For example, deployment and configuration of virtual Point of Interception (vPOIs) so that they can be instantiated closer to the corresponding intercepted modules. . Network slicing The availability and accessibility of information through lawful interception can also be impacted by network slicing, a core feature of 5G. It refers to the slicing of the resources of a single Public land mobile network (PLMN)-based mobile radio network into multiple virtual networks (slices). Network slicing builds upon the virtualisation of 5G core system. It allows multiple virtual networks to be created on top of a common shared physical infrastructure each fulfilling different requirements demanded by the vertical service. Each UE can connect to a maximum of 8 slices concurrently. In addition, different slices may event comprise of different instantiations of NFs creating additional challenges for LI to carry out lawful interception. It is evident that a close cooperation between LI Management and Communication Service Provider (CSP) is required in order to collect the required information for the deployment and configuration of LI functions within each Slice instance. In the future, law enforcement will therefore require the cooperation of numerous network providers both at home and abroad. Whereas many will be subject to (national) regulation, there is also the potential of ‘private slices’ held by ‘private third parties’ that may not be subjected to such regulation. Either way, the existence of network slicing leads to potential challenges as information is fragmented and may either not be available or accessible for law enforcement.

366

K. Michail-Alexandros et al.

. Multi-Access Edge Computing (MEC) Multi-access edge computing (MEC) [7] allows 5G operators to provide services and, store and process data, in decentralised computing infrastructures so that information exchanged among UEs will not necessarily be directed to central office infrastructure, where lawful interception is currently taking place. This locality in exchanged data when MEC is used with combination of Local Break Out (LBO) direct communication between users leads to consequences in terms of data interception for law enforcement. Communication content and identifiers no longer have to be directed via central nodes, which means information may not be available or accessible for law enforcement using the legacy LI deployment modes. . e-warrant In the era before 5G, the authorities had to deal with a single network provider to monitor a target. Within a 5G infrastructure, to monitor communications, the cooperation of numerous network providers under different jurisdictions is required, thus making it almost impossible to be granted access and coordinate everyone under strict deadlines. The proposed framework proposes a platform wherein the warrant can be published to the specific network providers within the platform needed to actively monitor the target, thus reducing time constraints, and delays for the interception implementation to initiate. The proposed framework will benefit from the defined eWarrant interface imposed by ETSI TR 103 690 [8]. Upon receiving the published warrant, the network providers will be held responsible to uphold and provide a complete copy of the technical information. For a secure handover of the interception data, Content of Communication (CC) and Intercept-related information (IRI). The proposed framework proposes an architecture that will tackle each of the above challenges. First and foremost, it proposes a LI platform that will follow virtualisation principles, to this end it will offer virtualised counterparts of LI functions that traditionally are being deployed at Cloud Service Provider (CSP) environment.

2.2 Quantum Processing in LI The evolution of quantum computers is expected to revolutionize computing in many fields [9]. Their unprecedented power will enable them to break the digital encryption system upon which the modern information and communication infrastructure depends. Quantum computing has the potential to put under risk law enforcement and first responder communications, if it is exploited by criminal and terrorist organizations in the case their communications’ encryption protocols do not migrate to the post-quantum era. There is already a race among nations and corporations attempting to develop quantum computers and many expected commercial applications are unrelated to cryptography. Quantum computers capable of undermining

Quantum Computing and Lawful Interception Applications

367

current cryptography are a few years behind, but they are already introducing risks, and these risks will grow over time., an aspect that Police Authorities regulatory bodies and policy makers should be aware of. Furthermore, quantum computers may become a useful tool in the hands of police authorities, when it comes to the retrieval of intelligent information from E2E encrypted services. Machine learning algorithms that rely on quantum properties have a better performance in analysis of large volumes of data in order to identify correlations among them. Post Quantum Cryptography (PQC) solutions are under development but will need to be improved, standardized, and implemented. This transition will be challenging and time-consuming, potentially stretching out over decades. Moreover, the advent of quantum computers presents retroactive risk, because information being securely communicated today without PQC, which may have been captured and stored but never decrypted, may be revealed once quantum computers are created. The period during which cryptography is expected to lag quantum computing developments presents a vulnerability that prudence requires us to address today. At the same time, another quantum-based communications technology, Quantum Key Distribution (QKD) has already reached significant maturity and commercial devices are already on the market. QKD achieves totally tamper-proof key exchange over optical links, complementing PQC techniques towards next-generation cryptography.

3 Proposed LI Architecture Powered by 5G and Quantum Technologies 3.1 The proposed framework Reference Architecture The centerpiece of the reference architecture depicted in Fig. 1, exploring a wide set of solutions and technologies for data driven distributed manufacturing spaces. The architecture is split among different layers covering all aspects of data management and processing in the lawful interception ecosystem, namely: i) the Warrant Management System (WMS), ii) the Administration Function (ADMF), iii) the Mediation and Delivery Function (MDF), iv) the Management and Orchestration of 5G (MANO) and v) Quantum Powered Operations (QPO). Three types of LI can be identified: Active, passive and hybrid: Active interception means that the interception solution is an integral part of the network infrastructure. The interception management system is able to directly control the network elements (e.g., the routers, switches) and to filter and retrieve the IRI and CC directly at the network node. The IRI and content are then sent to the interception management system, where they are mediated, and from there to the LEA monitoring centre. Passive interception means that the network elements transmit a copy of all network traffic to the interception management system. The filtering takes place on the copy of the traffic within the management system, the traffic belonging to non-interception

368

K. Michail-Alexandros et al.

Fig. 1 Proposed High Level Architecture

targets is discarded whereas the IRI and communication content of targets is passed on to the LEA monitoring center. The proposed framework will follow the method of Hybrid Interception, which as its name suggests, is a combination of active and passive interception techniques, and it is becoming increasingly common. According to this architecture, the IRI and content from some user terminals and IoT devices are first filtered by the application analysis server and then sent to the interception management system (LIMS), while the IRI and content from other user terminals and IoT devices are sent directly to LIMS for further filtering. LIMS combines the resulting IRI and content from both paths and forwards it to the LEA monitoring center. LIMS will also ensure that the activity of the surveillance system can be monitored and can be securely enabled/ disabled on request, ensuring integrity and correctness of delivered data.

3.2 Warrant Management System The proposed framework warrant management system (WMS) module offers a fully automated workflow. In more detail, it offers integration with electronic HI1 interfaces to automatically handle lawful interception and lawful disclosure requests from any law enforcement agency. The received warrants are presented to the handlers in a user-friendly fashion in a tailored user interface. The module can compare the technical details of the eWarrant to the official warrant to minimize the required time

Quantum Computing and Lawful Interception Applications

369

needed for verification processes. Once an eWarrant is accepted by the handler the intercept functionality is automatically installed on the relevant network elements and the interception is initiated. Moreover, the warrant management system module offers auditing and logging during the warrant lifespan. This module will consist of different functionalities covering the entire process and ensuring data protection and an accessible user interface. This will be the module to support the ETSI compliant interfaces (ETSI TR 103 690), where the eWarrant interface will be the physical and logical interface across which the production measures are requested from a Communication Service Provider (CSP), and the results are delivered from a CSP to a designated location of a LEA, and the Handover Interface 1 (HI1), which is the data interface supporting the receipt of eWarrant requests pursuant to the case. Other security measures provided by MC: i) Support for Radius login authentication, ii) Support of “RSA Secure ID” strong authentication for management nodes, iii) Public Key Infrastructure (PKI) support, iv) Secure shell communication, v) Isolated delivery (Fan-out) of data and content to different agencies, vi) Filters to address the needs of different warrant types, vii) Centralized operational management of call data delivery, viii) Security scans of each subsystem, ix) Encrypted database, x) Logging of all administrative/user activities on the system, xi) Restricted access to system error logs and alarms, xii) Periodic testing of the latest security patches to assure the security of the Operating System.

3.3 Quantum Powered Operations (QPO) One area of research that has attracted considerable interest is the design of machine learning algorithms that inherently rely on quantum properties to accelerate their performance. One key observation that has led to the application of quantum computers to machine learning is their ability to perform fast linear algebra on a state space that grows exponentially with the number of qubits. These quantum accelerated linear-algebra based techniques for machine learning can be considered the first generation of quantum machine learning (QML) algorithms tackling a wide range of applications in both supervised and unsupervised learning, and will be investigated in the frame of the proposed framework including principal component analysis, should support vector machines and k-means clustering. These algorithms often admit exponentially faster solutions compared to their classical counterparts on certain types of quantum data. These new algorithms use parameterized quantum transformations called parameterized quantum circuits (PQCs) or Quantum Neural Networks (QNNs). In analogy with classical deep learning, the parameters of a QNN are then optimized with respect to a cost function via either black-box optimization heuristics, in order to learn a representation of the training data. In the proposed model, quantum machine learning is the development of models, training strategies, and inference schemes built on parameterized quantum circuits of a Quantum computer.

370

K. Michail-Alexandros et al.

The main action points of the Post-Quantum Cryptography (PQC) module of the proposed framework are: i) Define a pathway that facilitates adoption, to minimize the final number of algorithms to be standardized by the EU regulatory bodies and follow through on creating an international standard. In keeping with current EU evaluation criteria, the standard should be designed to avoid market fragmentation, maximize interoperability, and facilitate widespread adoption, ii) Mandate PQC transition for LEAs, local member-state and EU Directorates and authorities should consider a mandate to transition to PQC for government agencies, critical infrastructure, and other organizations. It should assure adequate enforcement and grant few waivers, iii) Coordinate the involved LEAs to drive change and improve awareness, by expanding the representation of the central coordinating body to include staff from many more departments and agencies in the corresponding member state. Task them soon with (1) convening stakeholders across government and the private sector to increase awareness and address the risk from quantum computers, (2) issuing frequently updated guidance on PQC transition and cryptographic agility, and (3) driving broad information technology changes.

3.4 Management and Orchestration of 5G The Management & Orchestration of 5G (MANO) is split into two parts, the Virtualized Service Management module, and the Core & Ran Orchestration. Being a key element of the new 5G Service Based Architecture the proposed framework Network Repository Function (NRF) provides a single record of all network functions (NFs) available in a public land mobile network (PLMN) [10– 14], combined with the profile and the services each NF supports. Moreover, the NRF is responsible to support service discovery functions, while also providing a subscription-based model for other NFs to subscribe and get notified about the registration of new NF instances. The NRF serves as a central repository for Network Functions (NFs) on the network and in the the LI virtualized functions. In practice this means when a new Network Function / Network Element is instantiated, the LI Orchestrator only needs to point it at the NRF, which will tell it about other Network Functions on the network, register the new Network Function and let every other interested Network Function know about the newly deployed service. The NRF will be implemented over HTTPS and JSON to communicate with other components and will provision the seamless operation of LI functions in the LI system and corresponding LEMF. Network Slice Selection A network slice is defined as an end-to-end logical communications network within a mobile network and includes the core network, control plane, user plane functions and 5G access network. Mobile operators intend to use these network slices to house isolated service applications for tenants much like cloud providers sell their compute, storage, and memory resources. The idea is a user will provide their bandwidth,

Quantum Computing and Lawful Interception Applications

371

capacity, and latency requirements for the application they want to run on a network slice. The mobile operator does a requirements analysis based on current network status and load, creates a service level agreement, and allocates the necessary capacity and bandwidth for the slice. Once the slice is created, the tenant runs their application using the network slice and they can be assured the network slice will adhere to the service level agreement. The Network Slice Selection Function (NSSF) picks the appropriate Network Slice Instance (NSI) based on information provided during the UE attach. The NSSF will dynamically switch slices when deemed necessary (intercept target changing geo-location). Network Exposure Function Operators looking at their network as a platform, opening it to application developers. There are new capabilities 5G brings, that all these application developers will want to use, and rely on the operator’s network. Network Exposure Function is one of the capabilities that is natively built into the 5G network, Applications can subscribe to certain changes in the network and command the network to exploit its programmable capabilities providing new innovative services for the end-users. The 5G Network Exposure Function (NEF) facilitates secure, robust, developerfriendly access to exposed network services and capabilities. This access is provided by a set of northbound RESTful (or web-style) APIs from the network domain to both internal (i.e., within the network operator’s trust domain) and external applications. The NEF is an analogous function to the 4G Service Capabilities Exposure Function (SCEF). A combined SCEF + NEF node is necessary to hide the specific network technology from applications and user devices that can move between 4 and 5G. Therefore, flexibility on the southbound network interfaces is also an essential feature, for integration with 4G core endpoints using the Diameter protocol and with 5G core end-points via the service-based interfaces that are specified as part of the new 5G Core Service Based Architecture (SBA). Core and RAN Orchestration The 5G Session Management Function (SMF) is a fundamental element of the 5G Service-Based Architecture (SBA). The SMF is primarily responsible for interacting with the decoupled data plane, creating updating and removing Protocol Data Unit (PDU) sessions and managing session context with the User Plane Function (UPF). To meet the architectural requirements of 5G, the proposed framework must be entirely designed and delivered as a Cloud-Native network function, dynamically deployed and scaled-up on demand in a completely automated manner. This is a particularly complex proposition when it comes to high-availability control components with asynchronous call flows across geo-diverse infrastructures requiring long and short-lived state maintenance for sessions traversing elements that might quiesce without notice. These functions must therefore employ established design patterns for building and deploying massively scalable web applications while adapting to fit the constraints of real-time communications networks. REST is inherently stateless and the 3GPP has defined a Structured and Unstructured Data Storage Functions (UDSF), which can be used by any Network Function to achieve stateless reliability and load distribution. However, a strong background in these design principles

372

K. Michail-Alexandros et al.

will ultimately be required to deliver on a truly Cloud-Native 5G Session Management Function. The Session Management Function (SMF) provides session management (session establishment, modification, release) DHCP functions, IP address allocation & management, and NAS signaling termination (related to the session management). The User Plane Function (UPF) is a fundamental component of a 3GPP 5G core infrastructure system architecture. The UPF represents the data plane evolution of a Control and User Plane Separation (CUPS) strategy, first introduced as an extension to existing Evolved Packet Cores (EPCs) by the 3GPP in their Release 14 specifications. CUPS decouples Packet Gateway (PGW) control and user plane functions, enabling the data forwarding component (PGW-U) to be decentralized. This allows packet processing and traffic aggregation to be performed closer to the network edge, increasing bandwidth efficiencies while reducing network. The PGW’s handling signalling traffic (PGW-C) remain in the core, northbound of the Mobility Management Entity (MME). The User Plane Function (UPF) is tasked with packet routing & forwarding, packet inspection, and Quality of Service (QoS) handling.

4 Conclusions This paper presented a proposed LI system architecture, which aims to evolve and improve the performance of infrastructures, combatting illegal and terrorist activities by leveraging 5G and Quantum Computing capabilities.. The manuscript not only presented the current issues in the related field, but also discussed viable solutions to these problems. Firstly, the presented NFV enabled architecture proposes a converged ecosystem for 5G and QC, which is flexible, programmable and efficient. Regarding, small cell management functions, this manuscript described a mapping paradigm for RRM in 5G New Radio. As future line a wider set of use cases can be exploited in the frame of 6G, along with their corresponding challenges and novelties. Furthermore, the concept of cSON small cells [14] will be investigated in relevance to RRM and how it can enhance 6G in terms of performance and efficiency. Acknowledgements The research work presented in this article was supported by the European Commission under the Horizon 2020 Programme, through funding of the RESPOND-A project (G.A. no. 883371) and SANCUS (“Analysis Software Scheme of Uniform Statistical Sampling, Audit and Defense Processes”) project (no. 952672).

References 1. McGuire MR, Holt TJ (2017) The Routledge Handbook of Technology, Crime and Justice. Taylor & Francis, London

Quantum Computing and Lawful Interception Applications

373

2. Hendrix JA, Taniguchi T, Strom KJ, Aagaard B, Johnson N (2017) Strategic policing philosophy and the acquisition of technology: findings from a nationally representative survey of law enforcement.Policing Soc. 1–17 (2017). https://doi.org/10.1080/10439463.2017.1322966 3. Sanders CB, Hannem S (2012) Policing “the risky”: technology and surveillance in everyday patrol work. Can Rev Sociol 49(4):389–410. https://doi.org/10.1111/j.1755-618X.2012.013 00.x 4. Lum C, Koper CS, Willis J (2017) Understanding the limits of technology’s impact on police effectiveness. Police Q 20(2):135–163. https://doi.org/10.1177/1098611116667279 5. Berg M (1998) Order(s) and disorder(s): of protocols and medical practices. In: Berg M, Mol A (eds) Differences In Medicine: Unraveling Practices, Techniques, and Bodies. Duke University Press, Durham, NC 6. Bovens M, Zouridis S (2002) From street-level to system-level bureaucracies: how information and communication technology is transforming administrative discretion and constitutional control. Public Adm Rev 62(2):174–184 7. ETSI, Mobile-edge computing: introductory technical white paper (2014) 8. Network Function Virtualization (NFV): Use Cases. ETSI GS NFV 001 v1.1.1 (2013–10) 9. Chien SF, Lim HS, Kourtis MA, Ni Q, Zappone A, Zarakovitis CC (2021) Quantum-driven energy-efficiency optimization for next-generation communications systems. Energies 14:4090 10. Twamley P, Muller M, Bok P-B, et al (2018) 5GTANGO: an approach for testing NFV deployments. In: 2018 European Conference on Networks and Communications (EuCNC), IEEE 11. Kourtis, M-A, McGrath et al (2017) T-NOVA: an open-source MANO stack for NFV infrastructures. IEEE Trans Netw Serv Manag 14:586–602 12. Kourtis M-A, et al (2015) Enhancing VNF performance by exploiting SR-IOV and DPDK packet processing acceleration. In: 2015 IEEE Conference on Network Function Virtualization and Software Defined Network (NFV-SDN), IEEE 13. Kourtis M-A, Koumaras H, Xilouris G et al (2017) An NFV-based video quality assessment method over 5G small cell networks IEEE MultiMedia. IEEE 24:68–78 14. Kourtis M-A, Blanco B, Perez-Romero, et al (2019) A cloud-enabled small cell architecture in 5G networks for broadcast/Multicast services. IEEE Trans BroadCast. 65(2):1–11

Log4j Vulnerability Pedro Ferreira, Filipe Caldeira, Pedro Martins, and Maryam Abbasi

Abstract Log4j vulnerability was publicly disclosed on December 10, 2021, and it is considered the most severe security breach of all time. Hundreds of millions of devices were vulnerable to attackers executing malicious code remotely on any target device. For that reason, the name Log4jShell was given to the vulnerability. This article is going to talk about the impact that has on the world. Black-Hat Hackers leveraging log4jShell for personal gain. How federal governments, security researchers, security companies responded, and organizations protect themselves. A practical section that shows how a Log4j vulnerability can be discovered and exploited. Showing a methodology to look for log4jShell on a large scope to then explain how to exploit by manual testing in one application. Keywords Cyber Security · Log4J · Log4JShell · CVE-2021–44,228 · Remote Code Execution

1 Introduction Cybersecurity has been used for information security. With the increase of social engineering attacks [18], the aspect of the human in the security process needs to be advised. While this is considered an additional dimension, the objective is to focus P. Ferreira Polytechnic of Viseu, Viseu, Portugal F. Caldeira · P. Martins (B) CISeD - Research Centre in Digital Services, Polytechnic of Viseu, Viseu, Portugal e-mail: [email protected] F. Caldeira e-mail: [email protected] M. Abbasi Univeristy of Coimbra, Coimbra, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_32

375

376

P. Ferreira et al.

on the person as a potential target. To combat this issue that is part of Cybersecurity, frameworks and models have been created. Frameworks and models are updated daily as a result of new vulnerabilities being found. Vulnerabilities are given a CVE [1]. In 2021, a CVE was given the maximum severity score, also known as CVE-202144228 [6] the Log4j Vulnerability [14]. Where did the vulnerability come from? Almost all complex applications incorporate their logging. Log4j is a renowned logging package for Java, under the Apache Software License, a complete open source license that has been approved by the open source initiative. Logging does have its drawbacks, slowing down an application. To alleviate these concerns, Log4j is intended to be quick and adaptable because logging is rarely an application’s primary priority, Log4j strives to be simple to understand and use. The number of applications that are vulnerable to log4jShell was enormous. In later sections will be talked about some attacks. On the other side, organizations needed to patch, mitigate and protect against log4jShell [5]. For this research, the objective of the practical section is to show how difficult, and time-consuming it is to find a vulnerable log4j application and exploit it. For that reason, the testing will be divided into two parts, the first part is tenting on Tesla bug bounty program [10], and the second one will be solving a CTF [12] made by google. The objective is to see how different it is looking for log4j vulnerability in an enormous scope of assets and possible targets, and the other ongoing manual testing and a deep dive on a single application to exploit log4j. For this testing will be needed the usage of open source tools for different reasons, from the gathering of assets by finding subdomains, checking if subdomains are alive, port scanning, and a parser that will return the input format needed for the scans that will find and exploit log4j. This document is organized into four sections. Section 2, presents the state-of-the-art review. Section 3, discusses Black-Hat attacks. Section 4, shows the practical experiment. Section 5, final results. Finally, in Section 6 conclusions are drawn, followed by the introduction of future work guidelines.

2 State of the Art/Related Work(s) According to Sonatype CTO Brian Fox, Log4j has been downloaded more than 8 million times on Maven Central since the vulnerability was first disclosed. On December 16, 2021, Mave Central had more than 35,000 of the available Java artifacts dependent on code using log4j [19]. Approximately more than 8% of Maven Central packages had at least one version vulnerable. Disclaimer “These numbers do not encompass all Java packages, such as directly distributed binaries”. The number 8% shows how significant the impact is. In an ecosystem, 8% is an absurd number. A Black-Hat Hacker can leverage the Log4JShell to compromise a system. By compromising a system, he can just create chaos or use this access to run malware. Organizations needed to take action immediately to respond. Experts advise organizations to implement additional safety measures. Also, implement corrective measures

Log4j Vulnerability

377

and other immediate corrective measures to protect against any threat related to Log4j [3]. In the practical section, the first part will show how to look for log4j vulnerabilities in a real-world application with a gigantic scope. The other part of the objective is to focus on just one target with log4j. This will be demonstrated by solving the Google capture the flag event for log4j. Capture the Flag (CTF) [12] in computer security is an exercise in which “flags” are secretly hidden in purposefully-vulnerable programs or websites. The main objective is to show how to look for log4j vulnerability in a wide range of assets and how to exploit log4Shell. The tools used in this experiment are all free, public, and open source. Let us talk about the tools used. Let us start with “Subfinder” which is a discovery tool made by project discovery that discovers valid subdomains by using passive online sources. Next, “HTTPX” made by project discovery is a fast HTTP toolkit also by project discovery to allow to run of multiple probers using a retry-able HTTP library. “DNSX” made also by project discovery has multiple functionalities, but the one that will be used is to extract A records. Also, a util tool will be needed to append lines from stdin to a file, but only if they don’t already appear in the file, this one is made by “tomnomnom” and is known as Anew. As a port scanner, we will be using Masscan made by “robertdavidgraham”, and a converter for the result of Masscan made by “laconicwolf” with the name of Masscan-to-CSV. And finally, the scanner for log4j with the name log4j-scan is made by “fullhunt”. All these tools are available on GitHub. The material used in this experiment was the website that hosted the CTF challenge made by google for log4j vulnerability “https://log4j-web.2022.ctfcompet ition.com/” and an account on BugCrowd “https://www.bugcrowd.com/”, which is a crowdsourced security platform that will let us look for log4j vulnerability in a realworld application legally. The rules of the Tesla program can be found here “https://bugcrowd.com/tesla”.

3 Dangers of Log4j - Black-Hat Attacks 1. Microsoft: Iranian hackers still exploiting Log4j bugs against Israel August,26,2022. Black-Hat hackers continue to target susceptible applications with the Log4j vulnerability, as demonstrated by the Iranian threat actor known as “MuddyWater”, who was discovered concentrating his efforts on Israeli firms utilizing the SysAid software [8]. Already known as “MERCURY”, MuddyWater is an espionage organization that has recently been spotted targeting telecommunications firms throughout the Middle East and Asia. It is believed to be run directly by the Iranian Ministry of Intelligence and Security (MOIS). The operations of this hacker group are coherent with Iran’s national interests, so they constantly implicate Israeli entities that are considered enemies of the state. The most recent MuddyWater hacking campaign, which was described in Microsoft research, is the first instance of exploiting SysAid program’s vulnerability to

378

P. Ferreira et al.

break into corporate networks [9]. In the next figure, we can see how to exploit SysAid for initial access. 2. Attack by China-based threat actor disrupted by CrowdStrike. Nation-state actors were attempting to utilize the Log4j vulnerability to execute attacks against targets, according to researchers from Mandiant and Microsoft. CrowdStrike disrupted an attack against a large academic institution by China-based threat actor [2]. The attack was discovered during suspicious activity involving the VMware Horizon Tomcat web server. VMware published guidance on potential Log4j vulnerabilities related to VMware Horizon in December. CrowdStrike researchers declined to provide specific geographic information or other details about the attacked organization. “While we cannot directly state that we are seeing a broader use of this particular vulnerability by espionage actors, its viability as an access method is already proven,” Param Singh, VP of Falcon OverWatch at Crowdstrike told Cybersecurity Dive via email. 3. 235 ransomware incidents against Canadians Between January 1 and November 16, 2021, the CCCS, Canada’s pre-eminent cyber-security group, is aware of at least 235 ransomware occurrences against Canadians. However, it can be tough to determine the exact number because some businesses hide it to prevent additional costs related to negative publicity or penalty for inadequate protection, which has an impact on businesses outside the ransomware assault. The human element is also largely ignored by these statistics and figures. Individually impacted Canadians are not likely to report being victims of such attacks. According to CCCS, ransomware payments have steadied at around $200,000. However, the cost of the recovery process may be substantially higher. The average cost of recovering from a ransomware attack globally is $2.3 million. According to CCCS, the critical infrastructure sector, which includes the energy, health, and industrial industries, was home to more than half of the ransomware victims in the first half of 2021. Most of the attacks are suspected to use the Log4jShell because of the dates of the attacks [17] (Fig. 1).

Fig. 1 Latest MuddyWater attach chain (Microsoft)

Log4j Vulnerability

379

Let us now talk about the results of the attacks. What are untrusted Scanners targeting? The majority of scans target HTTP-related ports, such as 80, 8080, and 8000. One of the very active Russian scanners is also focusing on port 5480. The three most popular ports account for more than 50%, likewise, the top ten ports account for more than 85% of the events. Overall, we observe between 36 and 48 different ports as targets. A port that stands out among the top ten in European Union is 7547. This port is related to the TR-069 vulnerability related to home routers. Although this port is scanned often, Java is not the best solution for home routers, and therefore, Log4j is not likely used in that environment. Probably the scanners are just testing the port in passing. What Payloads are used? All watched Log4Shell events contain a payload within the JNDI lookup string. Payloads of the scanners are either GET (between 91% and 98%) or PUT requests. Note that it is not possible to observe all payloads as they may be encoded in an application-particular format. Anyway, binary rules of conduct that contain a matching string in ASCII would still be detected. What is the objective? Using Log4j vulnerability for Malware Download. The main objective for Black-Hat [11] is simply to infect the host with malware. The java object is just an intermediate step for the primary goal. There are two main bypasses, “Groovy bypass” and “Tomcat bypass”. Both bypasses use different ways to run the code. The Tomcat bypass instantiates a script engine to run JavaScript code, while the Groovy bypass builds on Groovy itself. The script code is encoded in the serialized objects as ASCII. One of the Tomcat samples executes PowerShell code, which likely targets Windows. While most other scripts include a mechanism to determine the local Operation System, e.g., by checking the direction of slashes in a file path, they execute bash commands either way and are thus unlikely to run on Windows. One of the malware acquired between the mid and end of Jan’22, while Log4Shell attacks were taking place via the PowerShell code is itself a PowerShell script and downloads a binary with the name of a known crypto miner [15].

4 Practical Experiment The experimental setup is divided into two parts. In the first part of this experimental setup, we are going to the real world and test against organizations using a Bug Bounty program [10]. The objective is testing on a large scale. The second part is going deep into just one application, in this case, the capture the flag challenge by google about log4j vulnerability. Starting with the Tesla program to go after only Log4j on a large scale [4]. Checking the Tesla program, the scope has various wildcards where that test can be made. In Cybersecurity terms, a wildcard is “*.target.com”. Meaning all subdomains on “X.target.com” are available for testing, for example, dev.target.com or forum.target.com are available for testing. Tesla has various wildcards in its scope. The chosen one for the testing was “*.Tesla.com”. To go against so many applications we need to create a methodology. Each hacker has a perspective, a set of skills, and experiences that shape the methodology he or she follows when approaching a

380

P. Ferreira et al.

target. The next part will show a description of every step with the command used and a detailed explanation of each command. These commands were made in a virtual private server, with the operating system Ubuntu 22.04 Using a Bash shell. Methodology: 1. Get many subdomains as possible for the wildcard “*.tesla.com” using subfinder [16]. Command: subfinder -d tesla.com -all | anew subdomains.txt Command detailed: Flag “-d tesla.com” will receive the wildcard tesla.com meaning all subdomains under the domain tesla.com will be the ones the scan will find. Flag “-all” is used to retrieve subdomains from all sources available on this tool [13]. And finally, “| anew subdomains.txt” is used to add all subdomains to a text file under the name subdomains.txt. 2. Use httpx on all subdomains to return the ones that are alive. Command: cat subdomains.txt | httpx -silent | anew hosts.txt Command Detailed: Flag “-silent” is used not to show unnecessary details, only to return the subdomains alive. Furthermore, “| anew hosts.txt” is used to add all subdomains valid to a text file under hosts.txt. 3. Convert all subdomains to internet protocol addresses using dnsx. Command: cat hosts.txt | dnsx -silent -a -resp-only | anew ipfile.txt Command Detailed: Flag “-silent” is used not to show unnecessary details, then the flag “-a” is to query a “A record”. The flag “-resp-only” is to display DNS response only. And finally, “| anew ipfile.txt” is used to add all internet protocol addresses valid to a text file under the name ipfile.txt. 4. Scan all ports with Masscan on the internet protocol addresses found. Command: masscan -p1-65535 -iL ipfile.txt –max-rate 1800 -oX result.xml Command Detailed: Flag “-p1-65535” is used to scan all ports, then “-iL ipfile.txt” is the input of internet protocol addresses to scan. Flag “–maxrate 1800” is the speed of scan that the value used is 1800 packets/second. The last flag “-oX result.xml” is to output the result in an XML file. 5. Use a parser to return a format Ip:Port, to facilitate and automate the scan input. Command: python3/Masscan-to-CSV/masscanxmlparser.py -f result.xml pw|anew ipport.txt Command Detailed: Flag “-f result.xml” is the input on the parser, then the “-pw” flag is used to return with the format “ip:port” in each line of the text file. And finally “| anew ipport.txt” to write the data on the text file “ipport.txt”. 6. Scan all subdomains alive and internet protocol addresses with ports open with log4j scanner. Command: python3 log4j-scan.py -l hosts.txt –run-all-tests –waf-bypass | anew OutputHostsScan.txt Command: python3 log4j-scan.py -l ipport.txt –run-all-tests –waf-bypass | anew OutputIpPortScan.txt Command Detailed: Flag “-l” is used to input the targets, and the flag “–runalltests” will run all tests available on the scan. The flag “–waf-bypass” will extend

Log4j Vulnerability

381

scans with WAF bypass payloads. And finally, “| anew XScan.txt” will output the result of the scan. The second part is the solving of CTF made by google with the objective is to exploit a Log4J vulnerability hosted on the website “https://log4j2web.2022.ctfcom petition.com/”. The first page shows that there is a chatbot interface. This challenge is a “chatbot” Java application that uses a Flask-based web application that interfaces the chatbot through the command line. Reading the source code, verify that the application is using Python 3.8.10 running a Web Flask server in Java 11.0.15 in the backend with Log4j 2.17.2 and with nsjail process isolation. Now the best thing to do is to see if the Log4j version is vulnerable. Let us use exploit-database to check if the version is vulnerable. However, in this version, JNDI lookups are disabled by default, and Log4Shell is not exploitable. 1) Let us check what commands are available on the chatbot. Going to the source code, in the java backend, we can see the function “doCommand” where we have the commands that can be used. The commands available are “/help” ; “/time” ; “/repeat” ; “/wc”. Not a single one of these commands will call the Log4j. So it is pointless to try injecting anything with the commands. 2) There is an interesting file with the name “log4j2.xml”. so let us read it.

This configures the behavior of Log4j: Stderr is where logs are written. Each line of a log is printed with the command property by system properties lookup $sys:cmd. The message is then held in msg, which is followed by a line break %n. 3) Getting the Flag The command is passed as-is to the process in Java and injected into the pattern layout: ”%dHH:mm:ss.SSS %-5level %logger36 executing $sys:cmd - %msg %n”

382

P. Ferreira et al.

We could read the flag via $env:FLAG Log4j’s environment lookups, which are still available. This might be injected into the cmd command as a first pass. So let us try the Payload $env:FLAG Sadly, the frontend will not be able to grab the FLAG because this would publish it to stderr on the backend returning the log ”the command should start with a/.” Additionally, we could introduce brand-new conversion patterns and start stdoutlogging exceptions. Let us attempt the incorrect pattern using the Payload %invalid arg. The Payload returned ”Error Unrecognized format specifier”. Recursively, Log4j will resolve patterns. Conversion patterns and environment lookups can thus be combined. There will be a deviation as a result. So let us try the Payload %d$env:FLAG. However, the exception will not leak the resolved environment lookup. Returning the log ”WARN Could not instantiate …” So the conditions are a combination of lookups that will trigger an exception, and the exception must leak parts of its arguments. Fortunately, Log4j provides us with many lookup and transformation patterns. If we dig into the Log4j source code, we see that Java search can do things like $java:version. However, if we give it a different key, we get an IllegalArgumentException revealing the argument. So, let us give this one $java:$env:FLAG a try then. Got the flag CTFd95528534d14dc6eb6aeb81c994ce8bd and the challenge is completed. A normal scan would flag this application as secure since the Log4j version is not vulnerable. But manual testing proved it was vulnerable.

5 Results and Analysis The main objective to take from this practical analysis is to show how a company with a significant amount of applications will test for log4j versus a deep dive on just one application and exploit log4j. Starting with the results of testing on the Tesla bug bounty [10] program. Being the first step to gathering all targets possible, this phase is named Reconnaissance, also known as “Recon” in the Cybersecurity world. The number of Applications/ Targets can be analyzed in the Table 1. So during the Reconnaissance, are scanned 461 assets. All hosts and Internal Protocol Addresses with ports open were scanned. We can see the difference between the subdomains found to the subdomains alive. The reason is the tool used to find subdomains is using passive online sources, that sources can be outdated. The scans did not catch any application vulnerable. Because of the rules of the Tesla program, the result of scans can’t be shown. We can assume the organization is protected. Nevertheless, remember that the companies have more information about the assets they own than a freelancer tester, so the testing becomes more accessible when the complete scout of assets is available. For example, an organization can have the CIDR ranges and then go from there to test everything single application in that range to check if there are any vulnerabilities. Now the second part of the practical analysis. On the contrast side, the CTF was to show how to manually test and find a log4jShell on a single application. Started checking that

Log4j Vulnerability Table 1 Tesla Assets

383

SubDomains Found

10,635

SubDomains Alive

276

Unique IP’s

103

Unique IP’s:Port

185

IP’s with Port 443

94

IP’s with Port 80

89

IP’s with Port 4643

1

IP’s with Port 8443

1

a Log4j version is not vulnerable, if we used a scan test probably that would mark this application as secure. There is a real possibility that assets from organizations are not secure to log4jShell, either the scan did not get or simply being an asset that the company forgot. Scans are good and help organizations, but for now, they cannot replace manual testing.

6 Conclusions and Future Work The impact of Log4j Vulnerability was enormous. Jen Easterly [7], director of the Cybersecurity and Infrastructure Security Agency (CISA), told CNBC News that this is “the most serious vulnerability that I’ve seen in my decades-long career. Everyone should assume that they are exposed and vulnerable and to check [that they are not vulnerable].”. The impact was so bad that Quebec shut down almost 4,000 websites, just as a precaution. The Cybersecurity and Infrastructure Security Agency (CISA) issued Emergency Directive 22-02 on Dec. 17, which directed U.S. federal government agencies to mitigate, patch, or remove all applications and services affected by the Log4j exploits. CISA required federal agencies to report on affected applications by Dec. 28. What makes Log4Shell so critical and does not get enough attention is the way Log4Shell can turn exploits into malware. Also, the available public information explains in detail how to exploit it. Hackers attempted to exploit unpatched users, organizations, and governments. But every bad side has a good one, the vulnerability saw wide coverage online as blog posts, lists of vulnerable applications and detection tools were quickly developed. Concluding the impact of log4jShell was simply phenomenal. Due to the logging library’s integration into numerous widely used applications to the easy nature of the exploit and to make matters worse the easy way to turn into malware. These are the three reasons why log4jShell was that bad. However, with the help of the American government, Cybersecurity, Infrastructure Security Agency, Security researchers, and everyone involved, the response was incredible. At the present moment, there are still attacks on going, as we discussed previously. The open source maintainers and users who have already updated their versions of log4j deserve praise and gratitude.

384

P. Ferreira et al.

6.1 Future Work The hunting on the Tesla program could be improved, by trying to gather more assets and make an even larger scope. For example, trying to use more tools for the step of subdomain finding, then brute-force subdomains, and finally permutate all subdomains with a list of common words to find sub-sub-domains (example:portal.dev.tesla.com) on tesla. Since it has not been a long time since the CVE was released to the public. I would like to know if, in a year or more, another exploit is found on log4j. For the moment, looks like we are in a good position with companies protected against this vulnerability with all the material available online. For the future, let us see how many open source programs will have vulnerabilities that get to the scale of log4j vulnerability impact. If the answer is yes, maybe that will be a future article. Acknowledgements “This work is funded by National Funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project Ref UIDB/05583/2020. Furthermore, we would like to thank the Research Centre in Digital Services (CISeD), the Polytechnic of Viseu, for their support.”

References 1. Christey S, Martin RA (May 2007) Vulnerability type distributions in cve. Mitre report 2. CrowdStrike. Overwatch exposes aquatic panda in possession of log4shell exploit tools during hands-on intrusion attempt (2021) 3. Cybertalk. Log4j protecting your organization from dynamics threats. (2021) 4. Everson D, Cheng L, Zhang Z (2022) Log4shell: Redefining the web attack surface 5. A. S. Foundation. Log4j – changes. Computer (2021) 6. https://nvd.nist.gov/vuln/detail/CVE 2021-44228. Cve 2021-44228. nvd nist (2021) 7. https://twitter.com/CNBC/status/1471532296905887749. Cisa director jen easterly. Twitter (2021) 8. https://www.bleepingcomputer.com/news/security/microsoft-iranian-hackersstill-exploitinglog4j-bugs-againstisrael/. Microsoft: Iranian hackers still exploiting log4j bugs against Israel. Bill Toulas (2022) 9. https://www.microsoft.com/security/blog/2022/08/25/mercury-leveraging-log4j2-vulnerabi lities-in-unpatched-systems-to-target-israeliorganizations/. Mercury leveraging log4j 2 vulnerabilities in unpatched systems to target Israeli organizations. Microsoft (2022) 10. Laszka A, Zhao M, Malbari A, Grossklags J (2018) The rules of engagement for bug bounty programs. In: International Conference on Financial Cryptography and Data Security, pp 138– 159. Springer, Cham 11. Mahmood MA, Siponen M, Straub D, Rao HR, Raghu T (2010) Moving toward black hat research in information systems security: an editorial introduction to the special issue. MIS Q 34(3):431–433 12. L. McDaniel, E. Talvi, and B. Hay. Capture the flag as cyber security introduction. In 2016 49th hawaii international conference on system sciences (hicss), pages 5479–5486. IEEE, 2016. 13. Notess GR (2002) The wayback machine: the web’s archive. Online 26(2):59–61 14. N. I. of Standards and Technology. Nvd - cve-2021–44228. Computer (2021)

Log4j Vulnerability

385

15. Pastrana S, Suarez-Tangil G (2019) A first look at the crypto-mining malware ecosystem: A decade of unrestricted wealth. In: Proceedings of the Internet Measurement Conference, pp 73–86 16. Ramadhan RA, Aresta RM, Hariyadi D (2020) Sudomy: information gathering tools for subdomain enumeration and analysis. In: IOP Conference Series: Materials Science and Engineering, vol 771, p 012019. IOP Publishing 17. Rudolph A (2022) What is log4j and why did the government of Canada turn everything off? Computer 18. Salahdine F, Kaabouch N (2019) Social engineering attacks: a survey. Future Internet 11(4):89 19. Wetter J, Ringland N (2021) Understanding the impact of apache log4j vulnerability Computer

Integrating Security and Privacy Mechanisms with Fast Health Interoperability Resources (FHIR), a Scoping Review João Pavão , Rute Bastardo , and Nelson Pacheco Rocha

Abstract The scoping review reported by this article aimed to analyse how security and privacy mechanism are being integrated with Fast Health Interoperability Resources (FHIR). An electronic search was conducted, and 37 studies were included in the review. The results show that 19 studies (i.e., more than half of the included studies) reported on the use of blockchain technology to (i) assure secure data sharing, (ii) provide secure Personal Health Records, (iii) support authentication and auditing mechanisms, (iv) support smart legal contracts, and (v) monitor the access to clinical data. The remainder 18 articles reported on the implementation of security and privacy mechanisms related to (i) data security at transmission, (ii) data security at storage, (iii) access control; (iv) data anonymization, and (v) management of informed consents. Keywords Scoping review · Fast Health Interoperability Resources · FHIR · Security and privacy mechanisms

1 Introduction The protection of individuals regarding the processing of personal data and on the movement of such data is guaranteed by law (e.g., EU Directive 2016/679). Therefore, robust security and privacy mechanisms are required to protect clinical data, J. Pavão INESC-TEC, Science and Technology School, University of Trás-os-Montes and Alto Douro, Vila Real, Portugal e-mail: [email protected] R. Bastardo UNIDCOM, Science and Technology School, University of Trás-os-Montes and Alto Douro, Vila Real, Portugal N. P. Rocha (B) IEETA, Department of Medical Sciences, University of Aveiro, Aveiro, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_33

387

388

J. Pavão et al.

according to a set of principles such as openness, transparency, inviolability of the data being gathered, and individual participation (i.e., individuals should have the opportunity to verify the validity of their data). In 2013, Health Level Seven International (HL7) proposed the Fast Healthcare Interoperability Resources (FHIR) as the next-generation healthcare interoperability standard. FHIR presents improvements when compared to existing interoperability protocols, by allowing easy-to-understand specifications and healthcare data exchange at the level of discrete data elements. Moreover, it recommends the use of standardized protocols for data transfer such as Extensible Markup Language (XML) or Java Script Object Notation (JSON). This article reports on a scoping review aiming to analyse how security and privacy mechanism are being integrated with FHIR. After this first section with the introduction, the rest of the paper is organized as follows: (i) the methods that were applied; (ii) the respective results; and (iii) discussion and conclusion.

2 Methods This scoping review was informed by the following research questions: . RQ1 - what type of security and privacy mechanisms are being integrated with FHIR? . RQ2 - what is the maturity level of the applications being developed? To answer these research questions a review protocol was prepared to define (i) search strategies, (ii) inclusion and exclusion criteria, and (iii) synthesis and report of the results, which is briefly described below. Boolean queries were prepared to include all the articles that have in their titles, abstract or keywords the expressions ‘Fast Healthcare Interoperability Resources’ or ‘FHIR’. The resources considered to be searched were two general databases, Scopus, and Web of Science, and one specific medical database, PubMed. As inclusion criteria, the authors aimed to include peer-reviewed studies that reported on evidence of the integration of security and privacy mechanisms with FHIR. Exclusion criteria were defined to exclude references that: . . . . .

Did not have abstracts or authors’ identification. Were written in other languages than English. Their full texts were not available. Reported on reviews or surveys. Were books, tutorials, editorials, special issues announcements, extended abstracts, posters, panels, transcripts of lectures, workshops, and demonstration materials.

Integrating Security and Privacy Mechanisms with Fast Health …

389

. Reported on studies already covered by other references (i.e., when two references reported on the same study in different venues, such as scientific journal and conference, the less mature one was excluded). . Reported on studies that are not relevant for the objective of this scoping review. Concerning the screening procedures, the selection of the studies to be included in this scoping review was performed according to the following steps: . First step, the authors removed the duplicates, the references without abstract or authors, references not written in English, and references reporting on reviews or surveys. . Second step, the authors assessed all titles and abstracts for relevance and those references clearly not meeting the eligibility criteria were removed. . Third step, the authors assessed the full text of the remaining references against the outlined eligibility criteria to achieve the list of the references to be included in this scoping review. Finally, in terms of synthesis and report of the results, the included studies were analysed considering their: . Demographic characteristics: (i) number of studies published in scientific journals and conference proceedings, (ii) distribution of the studies according to respective publication years, and (iii) distribution of the studies by geographical areas, according to the affiliation of the first author. . Characteristics of technological developments: (i) purposes of the security and privacy mechanisms being integrated with FHIR, (ii) the publicly available development tools that were used, and (iii) the maturity level of the security and privacy mechanisms being reported. To characterize the maturity level the following development stages were distinguished: (i) requirements and design, the study included the requirements’ elicitation, and a general overview of the application architecture or some of the respective components; (ii) technical testing, the study included results of a performance evaluation of the application or some of its components (e.g., the performance of a specific algorithm); and (iii) pilot testing, the study included a real-world evaluation by end users in their daily context during a certain period.

3 Results 3.1 Selection of the Studies The search of the studies to be included in this scoping review was conducted in April 2022. A total of 1343 references was retrieved from the initial search: (i) 599 references from Scopus; (ii) 394 references from Web of Science; and (iii) 350 references from PubMed.

390

J. Pavão et al.

The initial step of the screening phase yielded 594 references by removing duplicates, references without abstracts or authors, references not written in English, and references reporting on reviews or surveys. Based on titles and abstracts (i.e., the second screening step), 516 references were removed since they did not meet the eligibility criteria. Finally, in the third screening step the full texts of the remaining 78 references were screened and 41 were excluded. Specifically, six articles were excluded because they report on studies also reported by more recent articles that were considered for inclusion and the remainder 35 articles were excluded because they were not relevant for the specific objective of this review. Therefore, 37 studies were considered eligible for this study [1–37].

3.2 Demographics of the Included Studies In terms of publication types, one study [2] was published as a book chapter, 15 studies [1, 6, 13, 14, 17, 20, 22–24, 26, 29, 31, 32, 35, 37] were published in scientific journals, being the remainder 21 studies [3–5, 7–12, 15, 16, 18, 19, 21, 25, 27, 28, 30, 33, 34, 36] published in conference proceedings. The included studies were published between 2016 (i.e., one study [1]) and 2022 (i.e., three studies [35–37]). Fourteen studies [18–31] were published in 2020, the year with the most publications. In terms of geographical distribution, United States was the country with the highest contribution (i.e., 10 articles [2–4, 6, 12–14, 25, 27, 31]). In turn, European countries contributed with 14 articles (i.e., Germany, two articles [21, 22], Greece, three articles [19, 30, 36], Hungary, three articles [5, 32, 35], Portugal, three articles [15, 24, 29], Belgium, one article [16], Ireland, one article [33], and Netherlands, one article [17]), and Asian countries contributed with 10 articles (i.e., Korea, four articles [7, 8, 10, 20], Taiwan, three articles [23, 28, 37], Bangladesh, one article [18], Jordan, one article [1], and Vietnam, one article [34]). Finally, Australia contributed with two articles [9, 26] and Brazil contributed with one article [11]. Moreover, six studies involved multinational research teams: Korea and United States [20]; Netherlands and United Kingdom [17]; Portugal, Austria, Slovenia, and Italy [15]; Taiwan, Hong Kong, India, Thailand, and Philippines [23]; Taiwan, India, Thailand, and Philippines [37]; and United States and Switzerland [31].

3.3 Characteristics of the Technological Developments Nineteen studies [6–12, 14, 17, 18, 20, 23, 26, 28, 29, 31, 33, 34, 37] used blockchain technology with different purposes: . To assure secure data sharing for collaborative clinical decision-making, four studies [6, 8, 10, 18].

Integrating Security and Privacy Mechanisms with Fast Health …

391

. To provide secure Personal Health Records (PHR), ten studies [7, 9, 11, 14, 17, 23, 28, 31, 33, 34]. . To support authentication and auditing mechanisms, two studies [29, 37]. . To support smart legal contracts, two studies [12, 26]. . To implement alert systems aiming to allow patients to monitor the access to their clinical data, one study [20]. The remainder 18 articles [1–5, 13, 15, 16, 19, 21, 22, 24, 25, 27, 30, 32, 35, 36] were classified considering the following purposes: . Data security at transmission - two articles [16, 19] reported on the use of encryption techniques to guarantee secure messaging mechanisms for data exchange. . Data security at storage - three articles [15, 32, 35] reported on structured policy enforcement approaches that include different strategies (e.g., identification of security challenges or categorization of the required policies) to improve data security. . Access control - eleven articles [1–5, 13, 21, 24, 25, 27, 30] reported on access control mechanisms. . Data anonymization - one article [36] presented a mobile library to either anonymize or pseudonymize personal data using FHIR resources. . Informed consents - one article [22] proposed an application to manage informed consents. Looking specifically to the articles related to access control mechanisms, different solutions were identified: . Article [1] presented the integration of permission specifications in XML schemas modelling FHIR resources. . Article [4] described the incorporation of Role-Based Access Control (RBAC) into FHIR. . In addition to RBAC, articles [3, 5] described the use of Attribute Based Access Control (ABAC) to allow granular authorization decisions. . Article [13] presented the use of RBAC and Mandatory Access Control (MAC) to define permissions based on role and the sensitivity level of the services. . Article [25] described a Multi-Level Security (MLS) using Lattice-Based Access Control (LBAC). . Article [2] described a global security policy and enforcement mechanism for cloud services with RBAC, MAC and Discretionary Access Control (DAC). . Article [21] presented security measures based on patients’ smartphones and public cloud infrastructures to obtain patients reported outcomes. . Article [27] described a model for attaching trust profile authorization data to FHIR resources. . Article [24] presented a parser to allow the definition of access control policies and rules using natural languages.

392

J. Pavão et al.

. Article [30] described temporary mechanisms to secure access to Electronic Health Records (EHR) during emergency situations, through a record indexing method. Moreover, considering the public available development tools, (i) four articles [5–7, 25] reported on the use of HAPI server and (ii) five articles [2, 4, 13, 14, 27] reported on the use of the HAPI server and SMART on FHIR. Finally, concerning the maturity level of the proposed security and privacy mechanisms, (i) 22 articles [1–5, 8–10, 12, 13, 15, 18, 19, 21, 22, 25–28, 30, 31, 33] reported on the requirements and design of the proposed solutions, and (ii) the remainder 15 articles [6, 7, 11, 14, 16, 17, 20, 23, 24, 29, 32, 34–37] reported on the technical testing of the proposed solutions or some of their components. None of the included studies implemented a pilot testing.

4 Discussion and Conclusion The oldest study [1] included in this scoping review was published in 2016. In turn, 14 studies [18–31] were published in 2020. This recent interest in the integration of privacy and security mechanisms with FHIR is a natural consequence of the fact that the first FHIR proposal, the FHIR Draft Standard for Trial Use 1, was published in 2013. After this first publication, the Draft Standard for Trial Use 2 was published in 2015, the Standard for Trial Use 3 was published in 2017, and the Release 4, the first version with normative content, was published in 2019. Significantly, it was in 2019, with Release 4, that the XML and JSON became normative, as well as the Representational State Transfer (REST) or the RESTful Application Programme Interface (API). In terms of the first research question (i.e., what type of security and privacy mechanisms are being integrated with FHIR?), (i) nineteen studies (i.e., more than half of the included studies) were focused on the use of blockchains technology (i.e., to assure secure data sharing, provide secure Personal Health Records, support authentication and auditing mechanisms, support smart legal contracts, and monitor the access to clinical data), (ii) three articles proposed structured policy enforcement approaches, (iii) two articles reported on the use of encryption techniques to guarantee secure messaging mechanisms to support data exchange, (iv) one article described an application to manage informed consents, (v) one article presented a mobile library to either anonymize or pseudonymize personal data, and (vi) eleven articles proposed access control mechanisms. Moreover, in terms of the use of public available development tools, the use of HAPI server was reported by nine articles [2–7, 13, 14, 25, 27], and the use of SMART on FHIR was reported by five articles [2, 4, 13, 14, 27]. In terms of the maturity level of the solutions being proposed (i.e., the second research question), the included studies were focused on requirements’ elicitation,

Integrating Security and Privacy Mechanisms with Fast Health …

393

the design of the applications’ architecture or a performance evaluation of the applications or some of their components: (i) 22 studies were classified as requirements and design; and (ii) 15 studies were classified as technical testing. None of the included studies implemented a pilot testing. These results might be explained by the fact that the FHIR is a relatively recent standard. A limitation of this scoping review is related to the fact that the search procedure only considered references indexed by scientific databases. This strategy has the drawback of excluding potential interesting industrial studies that were not published in indexed journals or conferences. Acknowledgements This work was supported by Programa Operacional Competitividade e Internacionalização (COMPETE 2020), Portugal 2020 and Lisboa 2020 of the Fundo Europeu de Desenvolvimento Regional (FEDER)/European Regional Development Fund (ERDF), under project project ACTIVAS - Ambientes Construídos para uma Vida Ativa, Segura e Saudável, POCI-01-0247-FEDER-046101.

References 1. Altamimi A (2016) SecFHIR: a security specification model for fast healthcare interoperability resources. Int J Adv Comput Sci Appl 7(6):350–355 2. Baihan MS, Demurjian SA (2017) An access control framework for secure and interoperable cloud computing applied to the healthcare domain. In: Research Advances in Cloud Computing, pp 393–429. Springer, Singapore 3. Mukherjee S, Ray I, Ray I, Shirazi H, Ong T, Kahn MG (2017) Attribute based access control for healthcare resources. In: Proceedings of the 2nd ACM Workshop on Attribute-Based Access Control, pp 29–40. ACM, New York 4. Sánchez YKR, Demurjian SA, Baihan MS (2017) Achieving RBAC on RESTful APIs for mobile apps using FHIR. In: 5th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud), pp 139–144. IEEE, Piscataway, New Jersey 5. Jánki ZR, Szabó Z, Bilicki V, Fidrich M (2017) Authorization solution for full stack FHIR HAPI access. In: 30th Neumann Colloquium (NC), pp 000121–000124. I IEEE, Piscataway, New Jersey 6. Zhang P et al (2018) FHIRChain: applying blockchain to securely and scalably share clinical data. Comput Struct Biotechnol J 16:267–278 7. Kim JW, Lee AR, Kim MG, Kim IK, Lee EJ (2019) Patient-centric medication history recording system using blockchain. In: International Conference on Bioinformatics and Biomedicine (BIBM), pp 1513–1517. IEEE, Piscataway, New Jersey 8. Lee AR, Kim MG, Kim IK (2019) SHAREChain: Healthcare data sharing framework using Blockchain-registry and FHIR. In: International Conference on Bioinformatics and Biomedicine (BIBM), pp 1087–1090. IEEE, Piscataway, New Jersey 9. Rupasinghe T, Burstein F, Rudolph C, Strange S (2019) Towards a blockchain based fall prediction model for aged care. In: Proceedings of the Australasian Computer Science Week Multiconference, pp 1–10. ACM, New York 10. Kim MG, Lee AR, Kwon HJ, Kim JW, Kim IK (2018) Sharing medical questionnaries based on blockchain. In: International Conference on Bioinformatics and Biomedicine (BIBM), pp 2767–2769. IEEE, Piscataway, New Jersey (2018). 11. Garcia V, Roehrs A, d Costa CA, Righo R, Mayer AH (2019) B4HEALTH–an architecture model for personal health records with HL7 FHIR and Hyperledger Fabric. In: 18th International Conference on WWW/Internet 2019, pp 19–26. IADIS, Lisbon

394

J. Pavão et al.

12. Li M, Xia L, Seneviratne O (2019) Leveraging standards based ontological concepts in distributed ledgers: a healthcare smart contract example. In International Conference on Decentralized Applications and Infrastructures, pp. 152–157. IEEE, Piscataway, New Jersey 13. Sánchez YKR et al (2019) A service-based RBAC & MAC approach incorporated into the FHIR standard. Digital Commun. Netw. 5(4):214–225 14. Hylock RH, Zeng X (2019) A blockchain framework for patient-centered health records and exchange (HealthChain): evaluation and proof-of-concept study. J Med Internet Res 21(8):e13592 15. Aranha H et al (2019) Securing mobile e-health environments by design: a holistic architectural approach. In: International Conference on Wireless and Mobile Computing, Networking and Communications, pp 1–6. IEEE, Piscataway, New Jersey 16. Beni EH et al (2019) DataBlinder: a distributed data protection middleware supporting search and computation on encrypted data. In: 20th International Middleware Conference Industrial Track, pp 50–57. IEEE, Piscataway, New Jersey 17. Hawig D et al (2019) Designing a distributed ledger technology system for interoperable and general data protection regulation–compliant health data exchange: a use case in blood glucose data. J Med Internet Res 21(6):e13665 18. Al Asad N, et al (2020) Permission-based blockchain with proof of authority for secured healthcare data sharing. In: 2nd International Conference on Advanced Information and Communication Technology (ICAICT), pp 35–40. IEEE, Piscataway, New Jersey 19. Kiourtis A, Mavrogiorgou A, Menesidou SA, Gouvas P, Kyriazis D (2020) A secure protocol for managing and sharing personal healthcare data. In: Integrated Citizen Centered Digital Health and Social Care, pp 92–96. IOS Press, Amsterdam 20. Sung M, Park S, Jung S, Lee E, Lee J, Park YR (2020) Developing a mobile app for monitoring medical record changes using blockchain: development and usability study. J Med Internet Res 22(8):e19657 21. Ploner N, Prokosch HU (2020) Integrating a secure and generic mobile app for patient reported outcome acquisition into an EHR infrastructure based on FHIR resources. In: Digital Personalized Health and Medicine, pp 991–995. IOS Press, Amsterdam 22. Bild R et al (2020) Towards a comprehensive and interoperable representation of consent-based data usage permissions in the German medical informatics initiative. BMC Med Inform Decis Mak 20(1):1–9 23. Lee HA et al (2020) An architecture and management platform for blockchain-based personal health record exchange: development and usability study. J Med Internet Res 22(6):e16748 24. Rosa M, Barraca JP, Zuquete A, Rocha NP (2020) A parser to support the definition of access control policies and rules using natural languages. J Med Syst 44(2):1–12 25. Demurjian SA, Agresta T, Sanzi E, DeStefano J (2020) Alternative approaches for supporting lattice-based access control (LBAC) in the fast healthcare interoperability resources (FHIR) standard. In: WEBIST, pp 93–104. Scitepress, Setúbal 26. Milosevic Z (2020) Enacting policies in digital health: a case for smart legal contracts and distributed ledgers? Knowl Eng Rev 35:1–22 27. Sanzi E, Demurjian SA (2020) Trust profile based trust negotiation for the fhir standard. In: DATA, pp 242–251. Scitepress, Setúbal 28. Kung HH, Cheng YF, Lee HA, Hsu CY (2019) Personal health record in FHIR format based on blockchain architecture. In: International Conference on Frontier Computing, pp 1776–1788. Springer, Singapore 29. Rosa M, Barraca JP, Rocha NP (2020) Blockchain structures to guarantee logging integrity of a digital platform to support community-dwelling older adults. Clust Comput 23(3):1887–1898 30. Kiourtis A et al (2020) Health record index: Secure access of cloud-stored healthcare data. Stud Health Technol Inf 272:221–224 31. Dubovitskaya A, Baig F, Xu Z, Shukla R et al (2020) ACTION-EHR: patient-centric blockchain-based electronic health record data management for cancer care. J Med Internet Res 22(8):e13598

Integrating Security and Privacy Mechanisms with Fast Health …

395

32. Szabó Z, Bilicki V (2021) Access control of EHR records in a heterogeneous cloud infrastructure. Acta Cybernet 25(2):485–516 33. Alamri B, Javed IT, Margaria T (2021) A GDPR-compliant framework for IoT-based personal health records using blockchain. In: 11th IFIP International Conference on New Technologies, Mobility and Security, pp 1–5. IEEE, Piscataway, New Jersey 34. Do Hoang H, Nhut TC, Quyen PDT, Duy PT, Pham VH (2021) A Blockchain-based secured and privacy-preserved personal healthcare record exchange system. In: International Conference on Machine Learning and Applied Network Technologies (ICMLANT), pp 1–5. IEEE, Piscataway, New Jersey 35. Szabó Z (2022) Evaluation of a policy enforcement solution in telemedicine with offline use cases. Pollack Periodica 17(1):12–17 36. Dimopoulou S, Symvoulidis C, Koutsoukos K, Kiourtis A, Mavrogiorgou A, Kyriazis D (2022) Mobile anonymization and pseudonymization of structured health data for research. In: Seventh International Conference on Mobile and Secure Services (MobiSecServ), pp 1–6. IEEE, Piscataway, New Jersey 37. Lee HA et al (2022) Design of a vaccine passport validation system using blockchain-based architecture: development study. JMIR Public Health Surveill 8(4):e32411

An Approach for Visualizing SVG Sprites Systems Nikolay Buhtiyarov

and Daniela Borissova

Abstract The contemporary design of Internet applications relies on a variety of icons. This requires special attention to the format of the images used as icons. It is important that these images are not only of good resolution but also of sufficiently high quality. These basic requirements are met by SVG format and images could be organized in sprites with different numbers of icons. Due to the huge amount of used icons, it is of high importance not only to organize the icons into SVG sprites but also to be properly visualized. In this regard, the current article deals with the problem of automation of the process of visualizing the SVG sprites. A three-stage approach based on JavaScript algorithms able to organize and visualize multiple SVG sprites with different numbers of icons is proposed. The conducted tests approve the applicability of the proposed approach for visualizing SVG sprites with different numbers of items. Keywords SVG · Sprite system · Algorithms · JavaScript

1 Introduction The modern design of various web applications requires collaboration between all teams involved in the development of the particular product. The concept of modular programming could be used to provide the needed foundation to design an application rapidly thus enables teams to not only work concurrently, but also makes testing, deployment, and maintenance easier for both development companies and the business [1, 2]. On the other hand, modular development allows individual modules to be reused for other applications, saving valuable development time. Time to market N. Buhtiyarov · D. Borissova (B) Institute of Information and Communication Technologies at the Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria e-mail: [email protected] N. Buhtiyarov e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_34

397

398

N. Buhtiyarov and D. Borissova

plays an important role in product development, and design teams need to streamline the process without compromising the user experience. The user interface (UI) is a key component of any interactive software application and it is important for software developers to be focused on UI design using the proper tools [3]. Many model-based UI development environments have been proposed that allow semi-automated UI source code generation by using models with different levels of abstraction [4–6]. In some cases, web applications need to contain different information according to the particular problem, and UI is generated on the fly [7]. While designing the UI, it is good to follow a consistent design that is also intuitive design. Visual consistency relies on similar elements such as fonts, sizes, icons, etc. being perceived in the same way. Therefore, typography, colors, spacing, grid, size, and positions must be defined uniformly. The availability of consistent visual elements will contribute to a better user experience [8]. The main aspect of UI design in the modern internet space are icons. They appear on every website and application on the Internet, from navigation menus to social icons and indicators. Icons as carriers of information are an important feature of UI [9]. That is why some authors try to automate process for icon designs assisting in UI development by developing a new tool [10]. Their popularity shows a constant interest from the developers of various applications. As a result, every developer is faced with the dilemma of choosing which icon set to use. On the other hand, the massive use of mobile devices and the advent of high pixel density displays require sophisticated visual assets to render images in the appropriate format [11]. It is worth to mention also the flexibility of JavaScript [12] and its advantage to create variety of web-based modular applications due to the compatibility with variety of browsers [13]. Until recently, developers had to rely only on image formats to meet the requirements of the developing applications. The used images as icons result in low quality and resolution. This problem is solved today by the two solutions available: icon fonts or SVG icons (scalable vector graphics). Both icon formats have proponents who argue the merits of one over the other [14]. Icon fonts that contain symbols and glyphs instead of letters or numbers were introduced in 2012. A comparison between icon fonts and SVG with respect to size, accessibility, performance, scalability, deployment, and cross-browser compatibility can be found in [15]. Recently, there is widespread acceptance of SVG icon format due to its better performance, higher standards of accessibility, high rendering quality, unparalleled flexibility, and complete customization. The SVG format also supports dynamic changes, can be used to create interactive documents, and animations can be performed using declarative animation functions or by using a script [16]. It is worth to be mention also the fact, that De-coupling the SVG visualization specification from SVG rendering, and obtaining a dynamic VDOM, adds flexibility and opportunity for visualization system research [17]. Last but not least, it should be mentioned the fact that using SVG leads to a reduction in the number of HTTP requests to replace images [18]. User interface design work falls to developers, visually oriented interface designers, or subject matter experts. Therefore, each tool will be of great importance in speeding up the application development process. Such a tool that contributes for speed up website loading is the sprite. Image sprite represents a collection of

An Approach for Visualizing SVG Sprites Systems

399

individual images all participating in the formation of a single image. Picture Sprite compiles multiple icons into a single file that is used by designers to improve website performance. In this regard, the current article deals with the problem of visualizing sets of sprites composed of multiple icons. The essence of the proposed approach is the possibility developer to be informed about the overall number of icons, the number of icons that compose each sprite, and the ID of each icon. The rest of the article is organized as follows: Sect. 2 provides a description of the problem, Sect. 3 describes the essence of the proposed approach for previewing all icons from SVG sprites, Sect. 4 contains some results and discussion and the conclusions are drawn in Sect. 5.

2 Problem Description SVG images can be previewed with most graphic software, even with web browsers. When many SVG files are combined into SVG sprite, their initial state is to be invisible, and only a single icon can be displayed, by referring to it by ID. This great advantage is also SVG sprites’ biggest disadvantage because it leads to an inability to preview the collection of icons. From the development point of view, this is a huge drawback. The inability to preview the icons from the sprite in most cases leads to icon repetition. In reality, the sprites we are working with contain about 30% repeated icons. One way to preview the icons is to create an HTML file where to hardcode every icon from the sprite. But this solution will be obsolete soon after its implementation because requires additional work every time new icons are added or removed. The only other solution by the time available on the internet was provided by a highly reputable website for front-end web development “CSS Tricks” [19]. JavaScript code that selects all “symbol” tags from the SVG sprite, and then will add each icon to separate SVG tag to the DOM. The same as the HTML example but automated with JavaScript. At first glance much better solution, but it is the same effort because, in order to work, the developer needs every time to copy the content of the sprite, paste it into the HTML file, and then run the script. In our case study, there are 12 sprites with 600 + icons combined. Although this was not the solution we were looking for, it was helpful and gave a starting point for the concept of what needs to be done, to create easy to use dynamic preview system.

3 Description of the Proposed Approach for Previewing all Icons from SVG Sprites The proposed approach for previewing all icons from SVG sprites can be realized in three main stages as shown in Fig. 1:

400

N. Buhtiyarov and D. Borissova

Fig. 1 Main stages of the proposed approach

The steps could be realized by adequately coding a single HTML file containing the HTML, CSS, and JS codes. This file needs to be located in the directory that contains the SVG sprites. CSS code will not be discussed because it only affects the design.

3.1 Template Creation Following the scheme in Fig. 1, in the first stage, it is needed to create an empty container to store the content of the SVG sprites. For the goal, it is needed to create 2 templates: one of them represents the section that will contain the icons of each sprite, while the second one will be used to represent the container of a single icon. The corresponding codes for these templates are as follows:

A wrapper container that contains the general information such as title, informative texts and input field that is used for dynamic resizing of the icons can be added

An Approach for Visualizing SVG Sprites Systems

401

for additional functionality and information to the user. The HTML templates are mandatory to be able to preview the sprites. They will be populated dynamically with data provided by JavaScript in order for the icons to be visualized.

3.2 Requesting the Content of SVG Sprites The second stage requires to be performed a requesting the content of SVG sprites. This activity could be realized via two JavaScript functions. The aim of the first function is to send requests to the SVG sprites files and adds their content to the DOM tree. The second JavaScript function is needed to render the icons. The code of these functions is listed below:

402

N. Buhtiyarov and D. Borissova

Fig. 2 Algorithm for requesting the content of SVG sprites

The used algorithm behind these functions is realized by the following 6 steps as shown in Fig. 2: Knowing the names of the sprite files provided from the array, the “addSvgSpritedToDOM” function will send a request to each file, extracting its text content and adding it to the DOM. The same as manually copy/paste the content, but automated! At this stage, the icons are not visible. The other code observes when all the sprites are finally added to the DOM and then calls the “renderIcons” function. This approach guarantees the reliability of the application, no matter the number of sprites and icons ape provided, it will execute properly every time.

3.3 Rendering the Icons The third stage is intended to render the icons. This activity can be implemented by another JavaScript function:

An Approach for Visualizing SVG Sprites Systems

403

The used algorithm behind the function “renderIcons” is realized by the following 8 steps as shown in Fig. 3: At this stage, the icons can be visualized within the web browser environment. The results obtained using the described above algorithms that are appropriately implemented as JavaScript functions are presented in the next section.

404

N. Buhtiyarov and D. Borissova

Fig. 3 Algorithm for rendering the icons

4 Results and Discussion To send the HTTP requests for SVG sprite icons preview, a simple web server is needed to serve the HTML file. The server used for this research is “Live Server”, which can be installed as NPM package [20]. Considering the fact developers already work on a WEB app, the chances to have installed Node.js and NPM already are almost 100%, the only thing needs to be done is to execute the command “npx” liveserver”. To simplify this process even further, a batch script file with this starting command is added. The described approach has been tested in practice for visualizing a set of 12 sprites with more than 250 icons. The final result we are looking for about the described problem is presented in detail in the repository GitHub [21]. Let’s see what is going on, why this is the only working solution, you can find on the internet, and how with a single click of the mouse you can get such beautiful collection of icons. The corresponding result is shown in Fig. 4.

An Approach for Visualizing SVG Sprites Systems

405

Fig. 4 SVG sprites visualizing: (1) block of single sprite; (2) menu for changing the size of icons within sprites

It could be seen that each SVG sprite is visualized with following attributes: name of the sprite and number of the icons within each sprite. For example, the sprite “buttons.svg” contains only four icons unlike to “main.svg” sprite that contains 72 icons. The default size of all icons is 32 px. Some icons might have different viewport, or this size is too small to see all needed details. For this reason, an additional menu (see position #2 in Fig. 4) is included. This menu allows to change the icons dimensions within interval from 16 px to 1260 px. Thanks to this functionality, it is possible to see in detail each icon included within a given sprite. In the end, to summarize all the effort, only one developer needs to do the initial set-up process by adding the names of all sprites to the array in the HTML file. A double click on the “Show icons.cmd” file will preview all icons. This approach is 100% dynamic and when we add new icons to the sprites, they will be automatically displayed, the same as when icons are removed.

5 Conclusions In the modern design of Internet applications, there is an increasing tendency to use a large variety of icons. This requires paying particular attention to the format of the images used as icons. Along with this, it is essential these images are not only of good resolution but also of sufficiently high quality and be able to load quickly. These basic requirements are completely satisfied by the SVG format which is the preferred choice for most UI developers. Of particular importance are SVG sprites composed of many icons. In the current article an approach to visualize such SVG sprites is proposed. This approach is realized following three main stages concerning: 1) template creation; 2) requesting

406

N. Buhtiyarov and D. Borissova

the content of the sprites and 3) rendering the icons. The first stage is realized by using HTML, while the second and third rely on JavaScript. The described approach contributes to the automation of the process of icon previewing. The proposed solution can easily be updated by adding/removing icons or new sprites. The conducted tests show the applicability of the proposed approach to visualizing given sets of sprite systems. Thanks to well-designed software modules for SVG sprites visualization, it is possible to easily extend them as well as reuse them. These circumstances are a prerequisite for achieving good resilience, maintenance, and adaptation. Acknowledgements This work is supported by the Bulgarian National Science Fund by the project “Mathematical models, methods and algorithms for solving hard optimization problems to achieve high security in communications and better economic sustainability”, KP-06-N52/7/19-11-2021.

References 1. Seydnejad S (2016) Modular Programming with JavaScript. Packt Publishing, Birmingham 2. Mustakerov IC, Borissova DI (2013) Data structures and algorithms of intelligent web-based system for modular design. https://doi.org/10.5281/zenodo.1086885 3. Ruiz J, Asensio ES, Snoeck M (2020) Learning UI functional design principles through simulation with feedback. IEEE Trans Learn Technol 13:833–846. https://doi.org/10.1109/TLT.2020. 3028596 4. Tran TAV, Vanderdonckt J, Kolp M, Wautelet Y (2010) Generating user interface for information applications from task, domain and user models with DB-USE. In: 1st International workshop on user interface extensible markup language UsiXML’2010, pp 183–194 5. Ruiz J, Serral E, Snoeck M (2019) Evaluating user interface generation approaches: modelbased versus model-driven development. Softw Syst Model 18:2753–2776. https://doi.org/10. 1007/s10270-018-0698-x 6. Kennard R, Leaney J (2010) Towards a general purpose architecture for UI generation. J Syst Softw 83(10):1896–1906 7. Dimitrova Z, Borissova D, Dimitrov V (2021) Design of web application with dynamic generation of forms for group decision-making. In: Saeed K, Dvorsky J (eds) Computer information systems and industrial management. CISIM 2021. Lecture notes in computer science, vol 12883, pp 112–123. https://doi.org/10.1007/978-3-030-84340-3_9. 8. Hinderks A, Mayo FJD, Thomaschewski J, Escalona MJ (2022) Approaches to manage the user experience process in agile software development: a systematic literature review. Inf Softw Technol 150:106957. https://doi.org/10.1016/j.infsof.2022.106957 9. Black A (2017) Icons as Carriers of Information. Information Design: Research and Practice, pp 315–329 10. Feng S, Ma S, Yu J, Chen C, Zhou T, Zhen Y (2021) Auto-Icon: an automated code generation tool for icon designs assisting in UI development. In: 26th International Conference on Intelligent User Interfaces (IUI’21), April 2021, pp 14–17. https://doi.org/10.1145/3397481. 3450671. 11. Cryer J (2015) Using grunt with Images, SVG, and Icons. In: Pro Grunt.js. Apress, Berkeley, CA, pp 85–100. https://doi.org/10.1007/978-1-4842-0013-1_5. 12. Kantor I (2022) The Modern JavaScript Tutorial. https://javascript.info/ 13. Mustakerov I, Borissova D (2013) Data structures and algorithms of intelligent Web-based system for modular design. Int Schol Sci Res Innov 7(7):876–881. https://doi.org/10.5281/zen odo.1086885

An Approach for Visualizing SVG Sprites Systems

407

14. Colton: An Optimized Icon System – SVG Sprites, (2021). https://medium.com/the-crazycoder/an-optimized-icon-system-svg-sprites-5067e006a770. Accessed 11 July 2022 15. Nikhil: Icon Fonts vs SVG – Clash of the Icons, (2019). https://www.lambdatest.com/blog/its2019-lets-end-the-debate-on-icon-fonts-vs-svg-icons/. Accessed 07 July 2022 16. Scalable Vector Graphics (SVG) 2. https://www.w3.org/TR/2016/CR-SVG2-20160915/singlepage.html. Accessed 01 July 2022 17. Schwab M, et al (2022) Scalable scalable vector graphics: automatic translation of interactive SVGs to a multithread VDOM for fast rendering. IEEE Trans Visual Comput Graph 28(9):3219–3234. https://doi.org/10.1109/TVCG.2021.3059294. 18. Shivakumar SK (2020) Modern Web Performance Optimization: Methods, Tools, and Patterns to Speed Up Digital Platforms. Apress, New York 19. Coyier CH (2019) A snippet to see all SVGs in a sprite. https://css-tricks.com/a-snippet-tosee-all-svgs-in-a-sprite/. Accessed 16 June 2022 20. An introduction to the Npm package manager. https://nodejs.dev/en/learn/an-introduction-tothe-npm-package-manager/. Accessed 12 July 2022 21. Buhtiyarov N Icons display. https://github.com/nikolaybuhtiyarov/icons-display. Accessed 02 Sep 2022

A Software System to Support Student Engagement in Academic Courses Darina Dicheva, Keith Irwin, and Christo Dichev

Abstract One of the most challenging problems faced by educators today is how to increase students’ engagement in academic courses, especially in STEM (Science, Technology, Engineering, and Mathematics) disciplines. Motivation is a major driver of engagement in learning, particularly in optional (not required) learning activities where learners have to plan and coordinate their learning process without instructors’ intervention. Lack of motivation and inability to engage learners are among the top and most frequently cited barriers for learners’ engagement in such activities. Among the various approaches that have been proposed to improve students’ motivation, gamification has garnered significant attention from the educational community. While research has shown that well-designed gamification may improve student motivation, appropriate software to support educational gamification is generally missing. This paper describes a powerful customizable software platform that can be used for gamifying academic courses. It can support both instructors who seek to gamify their courses as well as researchers in educational gamification. Keywords IT in Education · Educational Gamification · Student Engagement

1 Introduction In recent years, gamification – the use of game design elements in non-game contexts [1] – has seen rapid adoption. The rising interest in it is reinforced by behavioral studies, which reveal that a core set of intrinsic motivators exists in all of us: the D. Dicheva (B) · K. Irwin · C. Dichev Winston-Salem State University, Winston Salem, NC 27110, USA e-mail: [email protected] K. Irwin e-mail: [email protected] C. Dichev e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_35

409

410

D. Dicheva et al.

desire to improve, to achieve, to direct our own lives and to connect with others. These motivators can be stimulated by the right experience. Several studies have been published arguing that gamification can be successfully implemented in educational contexts to improve student motivation [2]. However, a main obstacle that instructors face is the lack of educational software that can be used to support course gamification. This causes a significant burden on instructors who want to apply gamification to their courses. From another side, some studies have reported non-significant, mixed or contradictory results of applying educational gamification [2]. Improving our understanding regarding the sources of such conflicting results entails the need of a platform that can facilitate the creation of gamified learning activities and support experimental studies in varying contexts. To address these two problems, we have implemented a course gamification platform [3] that supports the use of popular game design principles and mechanisms in the organization of academic courses. In this paper, after a short introduction of educational gamification, we summarize the system functionality and exemplify its use for gamifying one particular course. The theoretical foundation of the work and more details are given in [3].

2 Educational Gamification and Its Challenges Gamification in learning uses elements drawn from game design to make learning activities more motivating and engaging and to improve learning outcomes. While the interest in applying gamification in education is growing, given its potential to enhance and sustain students’ motivation [4], research on how to support gamified learning does not follow this trend. This is a consequence of the fact that gamification as a multidisciplinary research domain integrates elements of information technology, human–computer interactions, human motivation, and task design thus requiring knowledge from multiple disciplines. The use of games in an educational setting is not a new idea. However, the concept and implementation of gamification in the educational sector has been introduced recently [5]. Several studies revealed various barriers, concerns, and support needs regarding the gamification of learning. The gamification design and development process require special skills that go beyond the knowledge of the average instructor. On the one hand, designing and developing a gamification solution is different from developing a game [6]. Entertainment is not the aim of gamification; its main goal is to drive desired user behavior [7]. This involves accounting for a variety of user behaviors, motivations, and requirements of different stakeholders such as learners and instructors. Therefore, there are high entry barriers to design adequate tools to support educational gamification [8]. Educators with little or no training in information technologies are very likely to create “bad gamification” with little or no effect on learning. Furthermore, the gamified learning tools involved might not be mature enough to support true gamification that is able to provide meaningful gamification experiences. The lack of customization is another criticism for applying gamification

A Software System to Support Student Engagement in Academic Courses

411

in education [9]. Most current gamified learning tools serve average students and are hardly engaging for high-performing or left-behind students. In customizable gamification platform the gamification experience can be adapted to various learners’ and instructors’ preferences. The currently available support for educational gamification is limited. There are some general gamification platforms, such as Badgeville, Hoopla, Bunchball and PugPharm, but their typical approach is packing selected gamification techniques in ‘one size fits all’ systems, responding to the needs of enterprises with varying organizational structures. Education involves activities different from those in the corporate world, which entails the need for dedicated platforms. There are several gamification platforms targeting education such as ClassCraft, Rezzly, and ClassDojo, but they are designed with K-12 education in mind with a focus on class management and reward systems. At the university level, there are limited number of gamification platforms offered such as Kahoot and Gradecraft [10]. Kahoot, however, is basically a platform for creating simple gamified quizzes. Gradecraft, on the other hand, is focused on grading and student choice of learning paths through a course (which is not always practical for STEM courses). To support gamification, educators often use LMS, such as Moodle, Blackboard, or Canvas, which provide some gamification elements. However, the gamification features offered by most LMS are limited to basic elements such as points, badges, leaderboards, and levels, and their behavior is hard-coded. Finally, game mechanics have been inserted in some previously developed online learning systems, mainly by converting grades to scores, introducing points for some activities supported by the system, and using the points for leaderboard ranking and/ or rewarding specific badges. Such superficial ‘gamification’, often called ‘pointification’, is solely based on extrinsic enablers and empirical studies show that it does not lead to improved motivation of students [11]. The lack of proper general support for gamifying learning in higher education motivated us to design and develop a holistic course gamification platform, OneUp [3], which is aimed at both facilitating the gamification of academic courses and fostering experimental research on gamifying learning.

3 The Course Gamification Platform 3.1 Platform Design To design a course gamification platform that meets learners, teachers, and researchers needs, we had to come up with a general specification of how an academic course which uses the platform should be organized and structured with regard to its content and learning activities and how gamification elements should be linked to the

412

D. Dicheva et al.

underlying course structure. There were three major requirements for the platform design: 1. Content independence: to be both course independent and learning activities independent. 2. High configurability: to allow instructors to choose not only what game elements they want to use in the gamified learning activities but also how these elements work. 3. Data analytics-driven GUI: to provide an intuitive, data analytics-based view of students’ and class performance. Meeting the first requirement suggested that specific learning content should not be built within the system. Instead, like in LMS, the instructor should enter the course content and/or activities which they want to gamify. Meeting the second requirement suggested that the provided gamification support should be configurable, the gamification elements should not have a predefined behavior, and the system should be opened for adding new gamification features. The third requirement concerned the provision of rich visual representation of the results of data-driven analyses of the student learning activities that could be used to provide learners and teachers with timely feedback for optimizing their experience. These requirements suggested the implementation of a component-based architecture utilizing the MVC (Model-View-Controller) architectural design pattern. In addition, to achieve a highly configurable gamification platform, which could be specialized to reflect specific customer (instructor) requirements, it needed to be modeled after the concept of a software product line [12]. Thus, the game elements needed to be designed as loosely coupled components in the system which work independently. The software was implemented in Python using Django, a Model-View-Controller (MVC) Python Web Framework and a PostgreSQL database. The interface was written in HTML5 and JavaScript, allowing the platform to be used also from mobile devices.

3.2 Platform Description The major blocks of the platform are an authoring tool, a gamification tool, a configuration tool, and a learner modeling/learning analytics tool. The authoring tool enables the instructor to create challenges and activities in their course shell. Challenges consist of problems, which are automatically graded by the system, and can be warm-up challenges for self-learning and self-assessment or serious challenges for graded course tests or quizzes. The problems can be static or dynamic. Static problems are such, for which the correct solution is specified at the time of entering the problem in the system. These include multiple choice questions, multiple answer questions, true/false questions, fill-in-the-gap questions, and matching questions. Dynamic problems are intended primarily for STEM-related courses. They are short

A Software System to Support Student Engagement in Academic Courses

413

computer programs which use a random seed to generate a unique instance of a particular question and then grade the correctness of the answer submitted to that question. This allows variants of the same problems to be used by different students on a test, or by the same student to practice. The automatic grading of program questions is essential for implementing the ‘immediate feedback’ game design principle. Somewhat in between the static and dynamic problems are the Parson’s problems. These are a type of code completion problems in which the learner must place given mixed up code lines/blocks in a correct order. By enabeling the platform to dynamically generate problem instances, it can make available a sufficient pool of exercises of a particular type for students to practice. In addition, the platform supports activities that can be any course related activities which are not automatically graded by the system. For example, these can be labs, assignments, student presentations in class, participation in a course-related event, etc., for which the instructor assigns points. These points are entered in the system and could be used in course gamification. The gamification tool consists of gamification rules and a rule engine, which is the heart of the gamification platform. Gamification rules are what link the learning activities defined in the system to the game design elements. For example, a rule can specify the conditions upon which a badge is awarded, or course bucks are earned. In fact, rules combine the learning activities with the game design elements in a coherent gamified course. The rules are in the form of production rules: I f _satis f ied(action, condition) then_o f f er (incenti ve), where action denotes any measurable process performed by a learner and incentive denotes any award supported by the system. For example, IF a student completes 8 challenges from a single topic then award them a badge.

The instructors specify the rules in an appropriate interface. By defining rules with different conditions and game elements that are awarded upon these conditions, instructors can induce different forms of enjoyable experiences, such as, an experience of curiosity, surprise and novelty or experience of choice/autonomy as illustrated below: One of the next five consecutive days is lucky: if you solve three problems in the lucky day you earn 3 course bucks.

The configuration tool supports two kinds of configuration: one related to the course structure and another to the gamification features to be used in the particular course. The course configuration includes specifying the course topical structure, the learning objectives (skills) targeted in the course, and the milestones and activities planned for the course (with their corresponding points), but none of these is required. The gamification related configuration includes choosing the game elements to be used in the course along with specifying gaming rules for them. The system currently supports the following game elements: XP points (based on challenge, skill, and activity points), goals, levels, progress bar, badges, various leaderboards, skill board,

414

D. Dicheva et al.

Fig. 1 Screenshots of OneUp student interface: two variants of course homepages

virtual currency (VC), avatars, random surprises, duels, and callouts. Figure 1 shows screenshots from two different student interfaces displaying some of the elements used in a gamified course. The learner modeling/learning analytics tool utilizes learning-related information that can be derived from the data collected in the system. This data include: the number and category of awards acquired, and VC obtained, high/low/mean challenge points, activity points, skill points, practice sessions’ starting times and length, number of warm-up challenges attempted and the corresponding success rate, problems avoidance, and problems abandonment. All these shed light on student learning and skill progress, targeted skills, persistency, problem solving habits, and meaningful use of VC. When the derived information is related to learners’ goals and the progress toward these goals is tracked, meaningful feedback loops can be created that can sustain desired behavior. The learning analytics tool supports a comprehensive learning dashboard that allows students to see aggregated data on their performance. It also provides analytical support for the instructors enabling them to view the performance of each student and the class performance across various metrics. Figure 2 shows screenshots displaying a student’s Learner Dashboard and an instructor’s Student Summary page. In the next section we provide an example of how the system can be used for gamifying an academic course.

4 An Example of an Academic Course Gamification The creation of a new course in OneUp includes a sequence of steps none of which is required: specifying the course topics, targeted skills, and milestones; specifying course activities; entering warm-up challenges for student practice and serious challenges for course assessment (if desired); and selecting game elements to be used in the course along with creating gaming rules for them. This example describes

A Software System to Support Student Engagement in Academic Courses

415

Fig. 2 A student’s dashboard (left) and an instructor’s student summary page (right)

the gamification of a sophomore-level computer science course taught by one of the authors. The instructor specified the course topics and for each of them entered warm-up challenges in the authoring interface. She also created two categories of (not automatically graded) activities - Labs and Homework. Next, the instructor configured the gamification features to be used in the course. The selected features included: avatars, badges, XP leaderboard, virtual currency (VC), and duels. These selections were set in the instructor’s Gamification Configuration page. While nothing more had to be done for the use of avatars and duels in the course, the use of badges and VC required specifying rules governing the issuing of the awards. The distinguishing feature of OneUp is that instructors can link learning activities to the selected game elements in the way they want. They do this by specifying the conditions under which the awards will be given to students. If virtual currency is used, two sets of rules must be specified: earning rules and spending rules. When students satisfy the condition of an earning rule, they receive the specified course bucks. When a student accumulates some amount of VC in their VC account, they can spend it in the course shop. Table 1 shows some of the badges and associated rules specified by the instructor and Tables 2 and 3 - some of the created VC earning rules and spending rules. Table 1 Sample badges used in the course Badge

Description Take for the first time warm-up challenge with a score >= 80%. Get the highest score in the class for Assignment 1. Complete at least one warm-up challenges a day for 5 consecutive days.

416 Table 2 Examples of earning rules

D. Dicheva et al.

Rule Description

VC

Taking a new warmup challenge with a score >= 80%

1

Attending 5 consecutive classes

1

Complete 5 or more warm-ups on a topic with a score >= 3 80%

Table 3 Examples of spending rules

Rule Description

VC

Limit

Get one day homework deadline extension

10

2

Drop the lowest lab score

6

3

Get a surprise gift from the instructor

20

1

The instructor has experimented with different game rules and selections of different gamification elements. The results of the conducted studies, described elsewhere, confirmed the positive impact of gamification on student engagement.

5 Conclusion As motivation influences students learning behavior, it is a critical factor for students’ success [1]. However, fostering motivation reliably remains an elusive task [4]. Hence, selecting effective strategies to engage and motivate students remains a challenge for the educational community. As part of the efforts for finding a way to foster motivation, gamification has emerged as a potential strategy to boost students’ motivation toward learning activities by employing design principals inspired by games [1, 2]. While it is gaining popularity in education, available sources providing practical guidance on how to gamify learning are scarce and fragmented. In addition, there is no adequate software support that instructors can use to gamify their courses and researchers – to explore the impact of gamification in educational settings. The contribution of the presented here work is towards filling this gap by creating a course gamification platform that can be used for both gamifying academic courses and promoting empirical research. Being highly configurable, it can provide support to both instructors and researchers. The gamification platform described here has been and continues to be used in several courses across different universities and countries and has proven to increase student motivation to participate more actively in the gamified courses. Acknowledgements This material is based upon work funded by NSF Project #1821189. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the NSF’s views. The authors thank to all research assistants in the Intelligent Information Systems Lab at WSSU who contributed to the implementation of the OneUp platform.

A Software System to Support Student Engagement in Academic Courses

417

References 1. Deterding S, Dixon D, Khaled R, Nacke L (2011) From Game Design Elements to Gamefulness: Defining “Gamification”. MindTrek, Tampere, Finland, pp 9–15 2. Dicheva D, Dichev C, Agre G, Angelova G (2015) Gamification in education: a systematic mapping study. J Educ Technol Soc 18(3):75–88 3. Dicheva D, Irwin K, Dichev C (2018) OneUp: supporting practical and experimental gamification of learning. Int J Serious Games 5(3):5–21 4. Dichev C, Dicheva D (2022) Work-in-progress: what motivators matter when gamifying a learning activity. In: 2022 IEEE global engineering education conference (EDUCON). IEEE 5. Kalogiannakis M, Papadakis S, Zourmpakis A (2021) Gamification in science education. A systematic review of the literature. Educ Sci 11(1):22 6. Huotari K, Hamari J (2017) A definition for gamification: anchoring gamification in the service marketing literature. Electron Mark 27:21–31 7. Hamari J, Koivisto J, Sarsa H (2014) Does gamification work? - literature review of empirical studies on gamification. In: Proceedings of the 47th Hawaii international conference on system sciences 8. Liu D, Santhanam R, Webster J (2017) Toward meaningful engagement: a framework for design and research of gamified information systems. MIS Q. 41(4) 9. Buckley P, Doyle E (2017) Individualizing gamification: an investigation of the impact of learning styles and personality traits on the efficacy of gamification using a prediction market. Comput Educ 106:43–55 10. Aguilar SJ, Holman C, Fishman BJ (2015) Game-inspired design: empirical evidence in support of gameful learning environments. Games Cult 1–27 11. Ryan R, Deci E (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25:54–67 12. Sommerville I (2016) Software engineering, 10th edn. Pearson

Virtual Reality and the Cognitive Screening of Older Adults: A Review Rute Bastardo , João Pavão , Bruno Gago , and Nelson Pacheco Rocha

Abstract This paper presents a literature review aiming to analyse the state of the art of virtual reality applications to detect age related cognitive impairments of community-dwelling older adults by identifying (i) the paradigms for interaction being used, (ii) the cognitive domains being assessed, and (iii) how the proposed applications were evaluated. A systematic electronic search was performed using Scopus, Web of Science and PubMed databases and 23 studies were included in the review. The paradigm most used for interaction was simulation of daily tasks, while memory, executive functions, and attention were the cognitive domains most referred. For the evaluation of the proposed applications, five articles reported on diagnostic accuracy studies and their results state that applications supported on virtual reality can discriminate between normal cognition and nonnormal cognition. However, these diagnostic accuracy studies not only constitute a small percentage of the included studies, but also, in general, were conducted with a small number of participants. Keywords Virtual reality · Cognitive Screening · Older adults · Systematic review

R. Bastardo Science and Technology School, UNIDCOM, University ofTrás-Os-Montes and Alto Douro, Vila Real, Portugal e-mail: [email protected] J. Pavão Science and Technology School, INESC-TEC, University ofTrás-Os-Montes and Alto Douro, Vila Real, Portugal B. Gago Department of Medical Sciences, IBIMED, University of Aveiro, Aveiro, Portugal e-mail: [email protected] N. P. Rocha (B) Department of Medical Sciences, IEETA, University of Aveiro, Aveiro, Portugal © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_36

419

420

R. Bastardo et al.

1 Introduction Considering the importance of the early detection of mild cognitive impairment, multiple technologies are being considered to develop new applications able to monitor the individuals in their residential environments [1, 2]. For instance, passive measures extracted from daily activities and interactions can be used for a continuous monitoring of sleep changes, verbal fluency, social interactions, gait, or fine motor control [3–8]. Since virtual reality is being used to implement applications to detect cognitive impairments, this review aimed to analyse the state of the art of applications supported on virtual reality to detect age related cognitive impairments of community-dwelling older adults.

2 Methods The review protocol defined (i) research questions; (ii) search strategies; (iii) inclusion and exclusion criteria; (iv) screening procedures; and (v) synthesis and reporting. Considering the research objective, the following research questions were developed: •RQ1 - what are the paradigms for interaction of virtual reality applications that might be used as a screening tool for age related cognitive impairment of older adults in their residential environments? •RQ2 - what cognitive domains are being assessed by virtual reality applications? •RQ3 – how the proposed virtual reality applications were evaluated? Three electronic databases were used for literature search (i.e., Scopus, Web of Science, and PubMed). Queries were prepared to retrieve articles that have their titles, abstracts or keywords conform with the following Boolean expression: (‘cognitive screening’ OR ‘cognitive test’ OR ‘memory screening’ OR ‘memory test’ OR ‘attention screening’ OR ‘attention test’) AND (‘computer’ OR ‘game’ OR ‘gaming’ OR ‘virtual’ OR ‘online’ OR ‘internet’ OR ‘mobile’ OR ‘app’ OR ‘digital’). References’ inclusion criteria were: (i) full articles in English published in peerreviewed conferences or journals; (ii) reporting on any independent virtual reality solution; (iii) that might be used as a generic self-administrated screening tool for age related cognitive impairment of community-dwelling older adults. References’ exclusion criteria were: (i) lack of abstract or authors’ identification; (ii) unavailability of full text; (iii) reported on reviews or surveys; and (iv) books, tutorials, editorials, special issues announcements, extended abstracts, posters, panels, transcripts of lectures, workshops, and demonstration materials.

Virtual Reality and the Cognitive Screening of Older Adults: A Review

421

Study selection for inclusion in this review followed three steps: (i) removal of duplicates, references without abstract or authors, not written in English, and of review or survey type; (ii) title and abstract were assessed for relevance and references clearly not meeting the inclusion and exclusion criteria were removed; and (iii) the full text of the remaining references was assessed against inclusion criteria to achieve the list of references included in this review. Included studies were analysed in terms of their demographic characteristics: (i) number of studies published in scientific journals and conference proceedings; and (ii) distribution of the studies according to their publication years. Moreover, syntheses were prepared to systematize: (i) paradigms for interaction being used; (ii) cognitive domains being assessed; and (iii) how the proposed applications were evaluated. For the evaluation methods, included studies were classified as: (i) usability evaluation studies; (ii) feasibility studies (i.e., evaluation of the practicality of the proposed applications); (iii) validity and reliability studies (i.e., evaluation of the accuracy and consistence of the results of the proposed applications); and (iv) diagnostic accuracy studies (i.e., evaluation of how well the proposed applications discriminate between healthy and cognitive impaired individuals).

3 Results 3.1 Study Selection The database search of the studies to be assessed in this review was conducted in the first week of May 2022. A total of 8557 references was retrieved from the initial search: (i) 3733 from Scopus; (ii) 1244 from Web of Science; and (iii) 3580 from PubMed. The initial step of the screening phase yielded 4481 references by removing duplicates (n = 3452), reviews or surveys (n = 311), absence of abstract or authors (n = 171), articles not written in English (n = 141) and one retraction. Based on title and abstract, 4443 references were removed since they did not meet the eligibility criteria. Finally, full texts of the remaining 38 references were screening and 15 were excluded since they reported studies not relevant for the specific objective of this review. Therefore, 23 studies were considered eligible [9–31]. In terms of publication types, 16 studies [10, 12–16, 21–24, 26–31] were published in scientific journals, and seven [9, 11, 17–20, 25] published in conference proceedings. The 23 studies were published between 2008 (one study [9]) and 2022 (three studies [29–31]). Fourteen studies [18–31] were published during the last five years.

422

R. Bastardo et al.

3.2 Paradigms for Interaction and Cognitive Functions As depicted in Table 1, the most referred paradigm for interaction was simulation of daily tasks. Some studies used specific tasks such as meal preparation [24], driving [9, 11, 12, 17] and shopping in a virtual supermarket [13, 22, 23, 27–29], while others [16, 19, 21, 30] considered a set of diverse daily tasks (e.g., watch television news, listen to a message on the answering machine, separate food for a meal, find a particular object in the living room, and recall the date of the medical consultation [16]). Moreover, the following paradigms for interaction were also identified: (i) realization of diverse abstract tasks [10, 31] (e.g., throw objects into portals according to their shape and color or memorize light patterns and report when the patterns are repeated [31]); (ii) start to goal spatial navigation tasks [15, 26]; (iii) manipulation of tangible objects using augmented reality [14]; (iv) cave automatic virtual environments [18, 25]; and (v) interaction with an intelligent virtual agent [20]. The included studies assessed 13 cognitive functions (Table 2). Two studies [15, 18] did not specify the cognitive functions being assessed. In the remaining studies, memory was the most referred cognitive functions and was assessed by 14 studies [9, 10, 12, 13, 16, 19, 21, 23, 24, 26–28, 30, 31], while executive functions were assessed by ten studies [13, 14, 16, 19, 21–23, 28, 29, 31] and attention was assessed by seven studies [10, 13, 14, 23, 28, 30, 31]. In turn, multiple paradigms for interaction were used to assess these functions. For instance, in terms of memory assessment, simulation of daily tasks was used by 11 studies [9, 12, 13, Table 1 Paradigms for interaction Paradigm for interaction

Number of articles Articles

Daily Tasks Set of diverse daily tasks 4

[16, 19, 21, 30]

Driving

4

[9, 11, 12, 17]

Meal preparation

1

[24]

Virtual supermarket

6

[13, 22, 23, 27–29]

Set of diverse abstract tasks

2

[10, 31]

Start to goal spatial navigation tasks

2

[15, 26]

Augmented reality for the manipulation of tangible objects

1

[14]

Cave automatic virtual environment

2

[18, 25]

Interaction with an intelligent virtual agent

1

[20]

Virtual Reality and the Cognitive Screening of Older Adults: A Review

423

16, 19, 21, 23, 24, 27, 28, 30] (four of them using the virtual supermarket metaphor [13, 23, 27, 28]), while the execution of diverse abstract tasks was used by two studies [10, 31] and start to goal spatial navigation tasks was used by one study [26].

3.3 Evaluation of the Proposed Applications For evaluation of the proposed applications, two studies [18, 31] present the concepts but did not report on the respective evaluation. Two studies [15, 25] report on the test of the user interaction involving one participant each. The evaluations performed by the remainder 19 studies were classified in the following categories: (i) usability evaluation studies [14, 19, 29]; (ii) feasibility studies [9, 16, 21, 24, 26, 30]; (iii) validity and reliability studies [11, 17, 20, 22, 27]; and (iv) diagnostic accuracy studies [10, 12, 13, 23, 28]. For usability studies, participants varied from five [14] to 24 [29] healthy adults, and the evaluation instruments were the System Usability Scale [14, 19, 29], the In-Game Experience Questionnaire [14], the Simulated Sickness Questionnaire [19], the User Satisfaction Evaluation Questionnaire [19], and the Igroup Presence Questionnaire [19]. The characteristics of the feasibility studies are described in Table 3, validity and reliability studies in Table 4, and diagnostic accuracy studies in Table 5. The outcomes of the validity and reliability studies were the comparison with validated instruments, and for the diagnostic accuracy studies the discrimination between normal and nonnormal cognition. Concerning the diagnostic accuracy studies, sensitivity values (i.e., ability to identify an individual with cognitive impairment) varied from 74% [23] to 87% [28], while specificity values (i.e., ability to identify an individual without cognitive impairment) varied from 73% [28] to 85% [23].

Table 2 Cognitive functions and paradigms for interaction Cognitive function Attention

Number of Articles Articles Daily tasks

4

[13, 23, 28, 30]

Set of diverse abstract tasks

2

[10, 31]

Augmented reality 1 for the manipulation of tangible objects

[14]

(continued)

424

R. Bastardo et al.

Table 2 (continued) Cognitive function Executive functions

Number of Articles Articles Daily tasks

8

[13, 16, 19, 21–23, 28, 29]

Set of diverse abstract tasks

1

[31]

Augmented reality 1 for the manipulation of objects

[14]

Inhibition of cognitive interference

Daily tasks

2

[11, 17]

Language

Daily tasks

3

[19, 28, 30]

Augmented reality 1 for the manipulation of tangible objects

[14]

Interaction with an intelligent virtual agent

1

[20]

Learning

Daily tasks

3

[21, 28, 30]

Memory

Daily tasks

11

[9, 12, 13, 16, 19, 21, 23, 24, 27, 28, 30]

Set of diverse abstract tasks

2

[10, 31]

Start to goal spatial navigation tasks

1

[26]

Orientation

Daily tasks

1

[19]

Perception

Augmented reality 1 for the manipulation of objects

[14]

Perceptual-motor Daily tasks function

2

[21, 30]

Spatial navigation

Daily tasks

4

[13, 22, 23, 29]

Set of diverse abstract tasks

1

[31]

Social cognition

Daily tasks

1

[30]

Time orientation

Daily tasks

1

[28]

Visual and spatial Augmented reality 1 processing for the manipulation of objects

[14]

Cave automatic 1 virtual environments

[25]

Virtual Reality and the Cognitive Screening of Older Adults: A Review

425

Table 3 Feasibility studies Article

Outcomes

Instruments

Participants

[9]

Comparison between the performance of young adults and older adults

Episodic Memory Classical Test, Trail Making Test A and B, and Wechsler Adult Intelligent Scale

113 students and 57 healthy older adults

[16]

Construct validity

Wisconsin Card Sorting Test, the Rey-Osterrieth Complex Figure Test, Wechsler Memory Scale and Dysexecutive Questionnaire

Six healthy adults

[21]

Comparison with Montreal Cognitive Assessment Montreal Cognitive Assessment

60 older adults (37 healthy controls and 23 cognitive impaired)

[24]

Compares game performance according to age groups

Wechsler Memory Scale and California Verbal Learning Test

41 students, 52 healthy older adults, and seven older adults with neuro-degenerative diagnosis

[26]

Analysis of performance

-

46 healthy adults

[30]

Compares performance according to age groups

-

99 healthy older adults

Table 4 Validity and reliability studies Article

Instruments

Participants

[11]

Pencil-and-paper Stroop and ANAM Stroop

20 students

[17]

Pencil-and-paper Stroop and ANAM Stroop

66 students

[20]

Clinical diagnosis

78 older adults (17 healthy controls and 61 cognitive impaired)

[22]

Montreal Cognitive Assessment

76 older adults (43 with subjective cognitive decline and 33 cognitive impaired)

[27]

California Verbal Learning Test

44 healthy adults

426

R. Bastardo et al.

Table 5 Diagnostic accuracy studies Article Instruments

Participants

[10]

Expert neuropsychologists’ assessment

405 older adults (172 controls and 233 cognitive impaired)

[12]

Cognitive Difficulties Scale, Mill Hill Test, Trail-Making Test and Wechsler Adult Intelligence Scale

51 older adults (21 healthy controls, 15 cognitive impaired and 15 with dementia)

[13]

Mini-Mental State Examination, Ray Auditory Verbal 55 older adults (21 Learning Test, Verbal Fluency Test, Rey-Osterrieth Complex healthy controls and 34 Figure Test, Rivermead Behavioral Memory Test, Test of cognitive impaired) Everyday Attention, Trail Making Test

[23]

Mini-Mental State Examination

87 older adults (52 healthy controls and 35 cognitive impaired)

[28]

Mini-Mental State Examination and Montreal Cognitive Assessment

122 older adults (60 healthy controls and 62 cognitive impaired)

4 Discussion and Conclusion This review identified 23 studies related to applications supported on virtual reality used as screening tools for age related cognitive impairment of community-dwelling older adults. Five studies [10, 12, 13, 23, 28] provided evidence about the capacity of the proposed applications to discriminate between normal and nonnormal cognition. Although the first study was published in 2008, most studies were published in the last five years, which shows a growing interest in the topic. Concerning the paradigms for interaction of virtual reality applications that might be used as a screening tool for age related cognitive impairment of older adults in their residential environments (i.e., first research question), the simulation of daily tasks was the paradigm for interaction most used. Shopping in a virtual supermarket was considered by five studies [13, 22, 23, 27–29], while four studies [9, 11, 12, 17] proposed driving exercises and one study considered the tasks related to preparation of a meal [24]. Four studies [16, 19, 21, 30] proposed the realization of various types of daily tasks. The remainder paradigms for interaction include: (i) realization of diverse abstract tasks [10, 31]; (ii) start to goal spatial navigation tasks [15, 26]; (iii) augmented reality for the manipulation of tangible objects [14]: (iv) cave automatic virtual environment [18, 25]; and (v) interaction with an intelligent virtual agent [20].

Virtual Reality and the Cognitive Screening of Older Adults: A Review

427

In terms of cognitive domains being assessed by virtual reality applications (i.e., second research question), memory, executive functions and attention where the most referred cognitive functions and were evaluated using different paradigms for interaction: (i) memory, simulation of daily tasks [9, 12, 13, 16, 19, 21, 23, 24, 27, 28, 30], realization of diverse abstract tasks [10, 31] and start to goal spatial navigation tasks [26]; (ii) executive functions, simulation of daily tasks [13, 16, 19, 21–23, 28, 29], realization of diverse abstract tasks [31] and augmented reality for the manipulation of tangible objects [14]; and (iii) attention, simulation of daily tasks [13, 23, 28, 30], realization of diverse abstract tasks [10, 31] and augmented reality for the manipulation of tangible objects [14]. In addition to memory, executive functions, and attention, other ten cognitive domains were identified: (i) inhibition of cognitive interference [11, 17]; (ii) language [14, 19, 20, 28, 30]; (iii) learning [21, 28, 30]; (iv) orientation [19]; (v) perception [14]; (vi) perceptual-motor function [21, 30]; (vii) spatial navigation [13, 22, 23, 29, 31]; (viii) social cognition [30]; (ix) time orientation [28]; and (x) visual and spatial processing [14, 25]. Surprisingly, two studies [15, 18] did not specify the cognitive functions being considered. Finally, concerning how the proposed virtual reality applications were evaluated (i.e., third research question), two studies [18, 31] did not report on evaluation of the proposed applications and two other studies [15, 25] reported on the test of the user interaction involving one participant each. The remainder 19 articles reported on usability evaluation studies, feasibility studies, validity and reliability studies, and diagnostic accuracy studies. Considering the three studies reporting on usability evaluation, all used the System Usability Scale [14, 19, 29], which is a well-known usability scale [32]. Moreover, the following usability evaluation instruments were also identified: (i) In-Game Experience Questionnaire [14]; (ii) Simulated Sickness Questionnaire [19]; (iii) User Satisfaction Evaluation Questionnaire [19]; and (iv) Igroup Presence Questionnaire [19]. In the three studies, participants were healthy adults and the number varied between five [14] and 24 [29]. Six articles [9, 16, 21, 24, 26, 30] reported on feasibility studies. The objectives of these feasibility studies included: (i) construct validity [16]; (ii) comparison according to age groups [9, 24, 30]; (iii) comparison with a validated cognitive instrument [21]; and (iv) performance analysis [26]. All studies included healthy older adults, and the number of participants varied between six [16] and 113 [9]. Moreover, two studies [21, 24] included cognitive impaired older adults, and another two studies included students [9, 24]. Five articles [11, 17, 20, 22, 27] reported on validity and reliability studies. These studies compared the results of the proposed applications with validated cognitive screening instruments used in clinical practice (e.g., Montreal Cognitive Assessment, California Verbal Learning Test or Stroop Test). The number of participants varied from 20 [11] and 78 [22] and were students [11, 16], healthy older adults [20, 22, 27], and cognitive impaired older adults [20, 22].

428

R. Bastardo et al.

Five articles [10, 12, 13, 23, 28] reported on diagnostic accuracy studies. The discrimination between healthy controls and cognitive impaired was compared with the results of expert neuropsychologists’ assessment [10] or different validated cognitive instruments (e.g., Mini-Mental State Examination or Montreal Cognitive Assessment). All the studies included healthy older adults and cognitive impaired older adults. The number of participants varied from 405 (i.e., the study reported by [10] included 172 healthy older adults and 233 cognitive impaired older adults) to 55 (i.e., the study reported by [13] included 21 healthy older adults and 34 cognitive impaired older adults). The results of these diagnostic accuracy studies show that virtual reality applications can discriminate between normal cognition and nonnormal cognition. From the 23 studies assessed in this review, only five reported diagnostic accuracy studies, a small percentage, given the importance of this category, and in general, they were conducted with a relatively small number of participants. Therefore, further clinical research is required to collect robust scientific evidence to facilitate the translation to clinical practice of virtual reality applications to detect cognitive impairments of community-dwelling older adults. Acknowledgements This work was supported by Programa Operacional Competitividade e Internacionalização (COMPETE 2020), Portugal 2020 and Lisboa 2020 of the Fundo Europeu de Desenvolvimento Regional (FEDER) / European Regional Development Fund (ERDF), under project project SH4ALL – Smart Health for ALL, POCI-01-0247-FEDER-046115.

References 1. Sabbagh, M., Boada, M., Borson, et al.: Early Detection of Mild Cognitive Impairment (MCI) in an At-Home Setting. The Journal of Prevention of Alzheimer’s Disease 7, 171–178 (2020). 2. Bastardo, R., Pavão, J., Martins, A.I., Silva, A.G., Rocha, N.P.: Cognitive Screening Instruments for Community-Dwelling Older Adults: A Mapping Review. In: International Conference on Digital Science, pp. 533–544. , Switzerland (2021). 3. Ibáñez V, Silva J, Cauli O (2018) A survey on sleep assessment methods. PeerJ 6:e4849 4. Tröger J., Linz N., König A., et al.: Telephone-based dementia screening I: automated semantic verbal fluency assessment. In: 12th EAI International Conference on Pervasive Computing Technologies for Healthcare, pp. 59–66. ACM, New York, NY, Unired States (2018). 5. Kourtis L, Regele O, Wright J, Jones G (2019) Digital biomarkers for Alzheimer’s disease: the mobile/wearable devices opportunity. NPJ digital medicine 2(1):1–9 6. Marengo, D., Settanni, M.: Mining facebook data for personality prediction: An overview. Digital Phenotyping and Mobile Sensing. In: Baumeister, H., Montag C. (eds) Digital Phenotyping and Mobile Sensing. Studies in Neuroscience, Psychology and Behavioral Economics, pp. 109–124. Springer, Cham, Switzerland (2019). 7. Grande, G., Triolo, F., Nuara, et al.: Measuring gait speed to better identify prodromal dementia. Experimental Gerontology 124, 110625 (2019). 8. Dagum P (2018) Digital biomarkers of cognitive function. NPJ digital medicine 1:1–3

Virtual Reality and the Cognitive Screening of Older Adults: A Review

429

9. Plancher, G., Nicolas, S., Piolino, P.: Virtual reality as a tool for assessing episodic memory. In: Proceedings of the 2008 ACM symposium on Virtual reality software and technology, pp. 179–182. ACM (2008). 10. Wright DW, Nevárez H, Kilgo P, LaPlaca M et al (2011) A novel technology to screen for cognitive impairment in the elderly. Am J Alzheimers Dis Other Demen 26(6):484–491 11. Parsons, T. D., Courtney, C. G., Arizmendi, B., Dawson, M.: Virtual reality stroop task for neurocognitive assessment. In: Medicine Meets Virtual Reality 18, pp. 433–439. IOS Press (2011). 12. Plancher G, Tirard A, Gyselinck V, Nicolas S, Piolino P (2012) Using virtual reality to characterize episodic memory profiles in amnestic mild cognitive impairment and Alzheimer’s disease: influence of active and passive encoding. Neuropsychologia 50(5):592–602 13. Zygouris S, Giakoumis D, Votis K et al (2015) Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J Alzheimers Dis 44(4):1333–1347 14. Boletsis C, McCallum S (2016) Augmented reality cubes for cognitive gaming: preliminary usability and game experience testing. Int J Serious Games 3(1):3–18 15. Doucet G, Gulli RA, Martinez-Trujillo JC (2016) Cross-species 3D virtual reality toolbox for visual and cognitive experiments. J Neurosci Methods 266:84–93 16. Oliveira, C.R., Lopes Filho, B.J.P., et al.:. Development and feasibility of a virtual reality task for the cognitive assessment of older adults: the ECO-VR. The Spanish journal of psychology, 19, 94–110 (2016). 17. Parsons, T. D., Russo, N., Schermerhorn, P.: Avatar Administered Neuropsychological Testing (AVANT): Stroop Interference Task. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting , 61(1), pp. 2047–2051. Sage Publications (2017). 18. Sobota, B., Guzan, M.: Possibilities of utilization chaos for the cognitive tests using virtual reality technologies. In: 2018 16th International Conference on Emerging eLearning Technologies and Applications (ICETA), pp. 499–504. IEEE (2018). 19. Dulau, E., Botha-Ravyse, C. R., Luimula, M., Markopoulos, P., Markopoulos, E., & Tarkkanen, K.: A virtual reality game for cognitive impairment screening in the elderly: a user perspective. In: 2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pp. 403–410. IEEE (2019). 20. Mirheidari, B., Blackburn, D., O’Malley, R., Walker, T., Venneri, A., Reuber, M., Christensen, H.: Computational cognitive assessment: Investigating the use of an intelligent virtual agent for the detection of early signs of dementia. In: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2732–2736. IEEE (2019). 21. Chua, S. I. L., Tan, N. C., Wong, W. T., Allen Jr, J. C., Quah, J. H. M., Malhotra, R., Østbye, T.: Virtual reality for screening of cognitive function in older persons: comparative study. Journal of medical Internet research,21(8), e14821 (2019). 22. Paulo R, Albuquerque PB (2019) Detecting memory performance validity with DETECTS: A computerized performance validity test. Appl Neuropsychol Adult 26(1):48–57 23. Eraslan Boz, H., Limoncu, H., Zygouris, S., Tsolaki, et al.: A new tool to assess amnestic mild cognitive impairment in Turkish older adults: virtual supermarket (VSM). Aging, Neuropsychology, and Cognition 27(5), 639–653 (2020). 24. Barnett MD, Childers LG, Parsons TD (2021) A Virtual Kitchen Protocol to Measure Everyday Memory Functioning for Meal Preparation. Brain Sci 11(5):571 25. Sobota, B., Kaduk, M., Sivý, M., & Mattová, M. Cognitive testing environment for people with disabilities based on virtual reality and EEG devices. In: 19th International Conference on Emerging eLearning Technologies and Applications (ICETA), pp. 364–368. IEEE (2021). 26. Rodríguez, M. F., Ramirez Butavand, D., Cifuentes, M. V., Bekinschtein, P., Ballarini, F., García Bauza, C.: A virtual reality platform for memory evaluation: Assessing effects of spatial strategies. Behavior Research Methods, 1–13 (2021). 27. Weitzner DS, Calamia M, Parsons TD (2021) Test-retest reliability and practice effects of the virtual environment grocery store (VEGS). J Clin Exp Neuropsychol 43(6):547–557

430

R. Bastardo et al.

28. Yan M, Yin H, Meng Q, Wang S et al (2021) A Virtual Supermarket Program for the Screening of Mild Cognitive Impairment in Older Adults: Diagnostic Accuracy Study. JMIR serious games 9(4):e30919 29. Zygouris S, Segkouli S, Triantafylidis A, Giakoumis D, Tsolaki M, Votis K, Tzovaras D (2022) Usability of the Virtual Supermarket Test for Older Adults with and without Cognitive Impairment. Journal of Alzheimer’s Disease Reports. 6(1):229–234 30. Tan, N.C., Lim, J.E., Allen Jr, J.C., Wong, et.al.: Age-Related Performance in Using a Fully Immersive and Automated Virtual Reality System to Assess Cognitive Function. Frontiers in Psychology, 2022(13), 847590 (2022). 31. Brugada-Ramentol V, Bozorgzadeh A, Jalali H (2022) Enhance VR: A Multisensory Approach to Cognitive Training and Monitoring. Frontiers in Digital Health 4:916052 32. Martins AI, Rosa AF, Queirós A, Silva A, Rocha NP (2015) European Portuguese validation of the system usability scale (SUS). Procedia computer science 67:293–300

Machine Learning Applications that Might Support Home-Based Screening of Parkinson’s Disease: A Scoping Review Catarina Abreu, Gonçalo Costa, João Tiago Oliveira, Vlada Hanchar, and Nelson Pacheco Rocha

Abstract This scoping review aimed to analyse the state of the art of Machine Learning (ML) techniques that might support home-based screening of Parkinson’s disease. An electronic search was conducted, and 33 studies were included. The results show an interest in analysing different data modalities (i.e., voice and speech, body movements, handwriting, and questionnaires) with a variety of ML techniques. The most referred ML techniques were Support Vector Machine, Neural Networks, Random Forests, K-Nearest Neighbor, and Decision Tree. The results point that different data modalities and ML techniques might allowed the implementation of applications to support home-based screening of Parkinson’s disease. However, randomized clinical trials are required to translate current knowledge to clinical practice. Keywords Parkinson’s disease · Machine Learning · Diagnostic · Scoping review

C. Abreu · G. Costa · J. T. Oliveira · V. Hanchar Department of Medical Sciences, University of Aveiro, Aveiro, Portugal e-mail: [email protected] J. T. Oliveira e-mail: [email protected] V. Hanchar e-mail: [email protected] N. P. Rocha (B) IEETA, Department of Medical Sciences, University of Aveiro, Aveiro, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_37

431

432

C. Abreu et al.

1 Introduction Neurological disorders are the leading cause of disability worldwide and Parkinson’s disease incidence is rising more quickly than that of other conditions, with a global prevalence of more than six million individuals. Risk factors include age, male gender, and environmental factors [1]. Parkinson’s disease main pathological signature are the neural inclusions in the form of Lewy bodies and neurites, with cell loss in the substantia nigra, known as the movement control centre, and other brain regions [2]. In addition to its hallmark motor dysfunctions such as slowness of movement (bradykinesia), resting tremor, rigidity and changes in gait, Parkinson’s disease is linked to a wide range of non-motor symptoms that significantly increase the burden of disease [3]. Parkinson’s disease diagnosis remains difficult, and research is still being done to characterise the disease’s early phases, since several clinical features can overlap with those of other neurodegenerative diseases. As a result, even when the disease has fully clinically developed, clinical diagnostical accuracy is still below ideal [4]. New findings relating to diagnostic biomarkers have been made by several modalities of neuroimaging, including novel magnetic resonance imaging techniques, positron emission tomography and single-photon emission computerized tomography, which have shown to help in early diagnosis of the disease [5]. Machine Learning (ML) techniques have shown that they can manage vast amounts of clinical datasets and provide insightful guidance. By analysing huge data from patient records, ML-based solutions might improve care quality improve and patients’ safety and have been widely used for categorization and prediction of different pathologies. In recent years, the number of articles that use ML techniques to diagnose Parkinson’s disease has increased. Therefore, the goal of this scoping review is to provide the reader with the state of the art of the different ML techniques that might support home-based screening of Parkinson’s disease.

2 Methods Considered the defined objective, this scoping review was informed by the following research questions: . . . .

What are the data modalities being used? What are the ML techniques being applied? What are the experimental setups being conducted? What are the outcomes being measure?

Machine Learning Applications that Might Support Home-Based …

433

The resources chosen for this scoping review were three electronic databases (i.e., Scopus, Web of Science, and PubMed). Queries were prepared to include all the articles that have in their titles, abstract or keywords the terms ‘Parkinson’ and ‘Diagnostic’ together with at least one of the following expressions: ‘Machine Learning’ or ‘Artificial Intelligence’. The inclusion criteria were i) full English articles, ii) published in peer-reviewed journals, iii) that report on the implementation of ML algorithms able to discriminate between patients with Parkinson (PwP) and healthy controls (HC), iv) which might be used to implement home-based Parkinson’s disease screening tools. In turn, the exclusion criteria were articles that i) did not have abstracts or authors’ identification; ii) their full texts were not available; iii) were published before 2016; iv) reported on reviews or surveys; and v) were books, tutorials, editorials, special issues announcements, extended abstracts, posters, panels, transcripts of lectures, workshops, and demonstration materials; and vi) reported on algorithms to distinguish Parkinson subtypes or different stages of disease progression. After the selection processes, the included articles were analysed in terms of demographic characteristics, data modalities being considered (e.g., voice and speech or handwriting), ML techniques being used, experimental setups being prepared, and the outcomes being measure.

3 Results 3.1 Selection of the Studies As shown in the Preferred Reporting Items for Systematic Reviews and MetaAnalyses (PRISMA) flowchart presented in Fig. 1, a total of 775 studies were originally yielded through database searching, which was performed on the first week of May 2022. After removal of duplicates, reviews or surveys and studies with no author or abstract, 475 studies were screened. Exclusion based the inclusion and exclusion criteria resulted in a total of 33 studies [6–38] that were further analyzed.

3.2 Demographics of the Included Studies More than half of the included studies were published during the last three years (i.e., 2020 [21–27], 2021 [28–34] and beginning of 2022 [35–38]). The remainder 15 articles were published in 2016 [6, 7], 2017 [8, 9], 2018 [10–15] and 2019 [16–20]. In terms of geographical distribution, considering the affiliation of the first author, China contributed with five articles [6, 9, 14, 17, 36], Italy with four articles [10, 12, 13, 34], Turkey with three articles [7, 24, 27], and Egypt with three articles [25, 33, 38]. Moreover, five countries (i.e., Brazil [18, 31], India [11, 15], Malaysia [19,

434

C. Abreu et al.

Fig. 1 PRISMA Flowchart of the incorporated study selection process

21], Pakistan [16, 26], and United States of America [28, 29]) contributed with two articles each, and other eight countries (i.e., Australia [8], Canada [20], Estonia [35], France [30], Germany [32], Luxembourg [23], Poland [22], and Saudi Arabia [37]) contributed with one article each. Looking at the author’s affiliations, most authors are affiliated at technological research and development institutions, such as engineering faculties or computer science departments. A minority of articles (i.e., 11 articles [7, 10, 13, 20, 22, 28–31, 35, 36]) were authored or co-authored by researchers affiliated at medical faculties or healthcare institutions.

3.3 Data Modalities and ML Techniques Table 1 presents the data modalities and ML techniques used by the different studies. Four data modalities were identified: i) questionnaires [11]; ii) voice and speech [6, 7, 9, 14, 17, 19, 20, 23–26, 30, 33, 37, 38]; iii) handwriting [12, 18, 29, 31, 34–36]; and iv) body movements (i.e., lower limbs movements [13, 16, 21, 27, 28],

Machine Learning Applications that Might Support Home-Based …

435

Table 1 Data modalities and ML techniques References

ML techniques

Questionnaires [11]

Support Vector Machine, Logistic Regression, Random Forests, Boosting Decision Tree

Voice and speech [6]

Extreme Learning Machine, and Kernelized Version of Extreme Learning Machine

[7]

Wavelet Kernel Extreme Learning Machine

[9]

K-Nearest Neighbor

[14]

Fuzzy K-Nearest Neighbor

[17]

Neural Networks and Linear Discriminant Analysis

[19]

Support Vector Machine, Random Forests, Decision Tree, Naïve Bayes, and Neural Networks

[20]

Support Vector Machine

[23]

Gaussian Process

[24]

Support Vector Machine, and K-Nearest Neighbor

[25]

Adaptative Neuro-Fuzzy Inference

[26]

Support Vector Machine, Random Forests, and Multilayer Perceptron

[30]

Deep Neural Networks

[33]

Support Vector Machine, Logistic Regression, Random Forests, Decision Tree, K-Nearest Neighbor, Naïve Bayes, Neural Networks, Convolutional Neural Networks, Adaptative Boosting, and Stochastic Gradient Descent

[37]

Support Vector Machine, Random Forests, K-Nearest Neighbor, Naïve Bayes, and Multilayer Perceptron

[38]

Logistic Regression, Random Forests, and Deep Neural Networks

Handwriting [12]

Ensemble: Support Vector Machine, Random Forests, K-Nearest Neighbor, Boosting Decision Tree, and Naïve Bayes

[18]

Recurrent Neural Networks

[29]

Random Forests

[31]

Support Vector Machine, Random Forests, K-Nearest Neighbor, and Convolutional Neural Networks

[34]

Support Vector Machine, Random Forests, and Decision Tree

[35]

Ensemble: Support Vector Machine, Logistic Regression, Random Forests, Decision Tree, K-Nearest Neighbor, and Adaptative Boosting

[36]

Continuous Convolutional Neural Networks

Lower limbs movements [13]

Support Vector Machine, Random Forests, and Naïve Bayes

[16]

Support Vector Machine (continued)

436

C. Abreu et al.

Table 1 (continued) References

ML techniques

[21]

Artificial Neural Networks

[27]

Neighborhood Representation Local Binary Pattern

[28]

Logistic Regression, and Recurrent Neural Network

Upper limbs movements [32]

Support Vector Machine, Boosting Decision Tree, Artificial Neural Networks, and Deep Neural Networks

Hands or fingers movements [8]

Ensemble: Support Vector Machine, Logistic Regression, Random Forests, Decision Tree, K-Nearest Neighbor, Nu-Support Vector Classification, Multilayer Perceptron, and Quadratic Discriminant Analysis

[10]

Support Vector Machine, Logistic Regression, and Naïve Bayes

Combination of features [15]

Decision Tree, and K-Nearest Neighbor

[22]

Support Vector Machine, and Linear Discriminant Analysis

upper limbs movements [32], and hands or fingers movements [8, 10]). Moreover, two articles reported on the use of combined data modalities (i.e., handwriting, and voice and speech [15], and hand tremor, and voice and speech [22]). In turn, 26 ML techniques were identified: Support Vector Machine, Logistic Regression, Random Forests, Decision Tree, Boosting Decision Tree, Naïve Bayes, K-Nearest Neighbor, Fuzzy K-Nearest Neighbor, Neural Networks, Convolutional Neural Networks, Continuous Convolutional Neural Networks, Recurrent Neural Networks, Artificial Neural Networks, Deep Neural Networks, Adaptative Boosting, Adaptative Neuro-Fuzzy Inference, Extreme Learning Machine, Kernelized Version of Extreme Learning Machine, Wavelet Kernel Extreme Learning Machine, Gaussian Process, Linear Discriminant Analysis, Multilayer Perceptron, Neighborhood Representation Local Binary Pattern, Nu-Support Vector Classification, Quadratic Discriminant Analysis, and Stochastic Gradient Descent.

3.4 Experimental Setups Table 2 synthetizes the experimental setups in terms of datasets being used, number of participants and measured outcomes.

Machine Learning Applications that Might Support Home-Based …

437

Table 2 Experimental setups and outcomes are being measure References Source

Participants Accuracy (%)

Questionnaires [11]

Parkinson’s Progression Markers Initiative

474 PwP + 94.6 198 HC

Voice and speech [6]

UCI Machine Learning Repository

23 PwP + 8 HC

96.5

[7]

UCI Machine Learning Repository

23 PwP + 8 HC

96.8

[9]

UCI Machine Learning Repository

23 PwP + 8 HC

98.0

[14]

Oxford Parkinson’s Disease Data and Istanbul Parkinson’s 43 PwP + Disease Data 28 HC

97.9

[17]

UCI Machine Learning Repository

20 PwP + 20 HC

82.1

[19]

Oxford Parkinson’s Disease Data

23 PwP + 8 HC

99.5

[20]

Previous research study

147 PwP + 92.2 48 HC

[23]

UCI Machine Learning Repository

40 PwP + 40 HC

96.9

[24]

UCI Machine Learning Repository

23 PwP + 8 HC

91.6

[25]

UCI Machine Learning Repository

188 PwP + 87.5 64 HC

[26]

PC-GITA Spanish Language Dataset

50 PwP + 50 HC

[30]

Collected data

121 PwP + (*) 100 HC

[33]

Oxford Parkinson’s Disease Data

23 PwP + 8 HC

96.9

[37]

UCI Machine Learning Repository

40 PwP + 40 HC

88.3

[38]

UCI Machine Learning Repository

188 PwP + 92.0 64 HC

99.7

Handwriting [12]

Previous research study

37 PwP + 38 HC

73.4

[18]

São Paulo University Parkinson Data

21 PwP + 14 HC

92.2 (continued)

438

C. Abreu et al.

Table 2 (continued) References Source

Participants Accuracy (%)

[29]

Previous research study

62 PwP + 15 HC

97.6

[31]

Collected data

20 PwP + 20 HC

83.1

[34]

Collected data

68 PwP + 73 HC

90.0

[35]

Previous research study and Collected data

61 PwP + 69 HC

92.2

[36]

Collected data

20 PwP + 20 HC

89.3

Lower limbs movements [13]

Collected data

30 PwP + 30 HC + 30 IH(a)

96.7

[16]

PhysioNet Data

93 PwP + 73 HC

90.3

[21]

PhysioNet Data

93 PwP + 73 HC

97.4

[27]

PhysioNet Data

93 PwP + 73 HC

98.3

[28]

Private Insurance Company Database

8382 PwP + 41,936 HC

(**)

Upper limbs movements [32]

Collected data

260 PwP + 81.9 89 HC + 101 DD(b)

Hands or fingers movements [8]

Collected data

32 PwP + 71 HC

(***)

[10]

Collected data

16 PwP + 12 HC

81.4

Combination of features [15]

Oxford Parkinson’s Disease Data, Istanbul Parkinson’s Disease Data, São Paulo University Parkinson Data and collected data

275 PwP + 92.2 46 HC

[22]

Collected data

61 PwP + 20 HC

85.4 (continued)

Machine Learning Applications that Might Support Home-Based …

439

Table 2 (continued) References Source

Participants Accuracy (%)

Notes: (a) IH – Idiopathic Hyposmia patients (b) DD – Patients with Differential Diagnoses (*) The reported outcome was the equal error rate (EER), which ranging from 32 to 39% (**) The reported outcome was the Area under the ROC Curve (AUC) with a value of 0.87 (***) The reported outcome was the AUC with a value of 0.98

4 Discussion and Conclusion This scoping review identified 33 studies focused on the development of ML algorithms that might support home-based screening of Parkinson’s disease, using different data modalities. Concerning the data modalities being used (i.e., the first research question), voice and speech was the most used data modality, and was considered by 46% of the studies. In turn, 24% of the studies considered body movements, namely lower limbs movements (15%), upper limbs movements (3%), and hands and fingers movements (6%). Moreover, 21% of the studies were focused on handwriting, and 3% of the studies aimed to process data from questionnaires related to Parkinson’s disease. Finally, 6% of the studies considered voice and speech in combination with other data modality (i.e., handwriting or hand tremor). In terms of the second research question (i.e., what ML techniques are being used?), 26 different ML techniques were identified, and 9% of studies assessed ensembles composed by various ML techniques. Support Vector Machine was referred by 52% of the articles, while Neural Networks and variations (i.e., Convolutional Neural Networks, Continuous Convolutional Neural Networks, Recurrent Neural Networks, Artificial Neural Networks, and Deep Neural Networks) were referred by 42% of the articles. Moreover, Random Forests was referred by 39% of the articles, K-Nearest Neighbor or Fuzzy K-Nearest Neighbor were referred by 30% of the articles, and Decision Tree or Boosting Decision Tree were referred by 27% of the articles. All studies trained and evaluated their ML models on PwD and HC and the number of participants varied from 31 to 50,258. Nearly all the data used in the voice and speech studies were taken from UCI Machine Learning Repository or Oxford Parkinson’s Disease Data (the data taken from the same database, was not necessarily the same dataset). In turn, Jeancolas et al. [30] collected their own data, Zahid et al. [26] used a Spanish dataset named PC-GUITA, and Lahmiri and Shmuel [30] used data from a previous research study. Four [8, 10, 13, 32] out of the eight studies focusing on body movements collected their data. The datasets of the remainder studies were obtained from the PhysioNet (i.e., three studies [16, 21, 27]) and from a private insurance company [28].

440

C. Abreu et al.

In the studies where the PwP classification was based on handwriting, the researchers collected their own data [31, 34–36] or used data from previous research studies [9, 29, 35]. In turn, the dataset of one study [18] was the São Paulo University Parkinson Data. The numbers of participants of most datasets may raise some concern as they were relatively small. This is particularly important since ML classifiers require a large amount of data to be properly implemented and interpreted. Some studies also use unbalanced datasets of PwD and HC, which may have a negative impact on the model’s validation. The outcomes measured by most of the studies were accuracy and Area under the ROC Curve (AUC), with one exception [30] that measured the Equal error rate (EER). However, comparison between studies may not be so straightforward, considering there are aspects of the models that can affect the comparison, such as the presence of a feature selection technique, the splitting strategy that was used and the crossvalidation method that was selected. The reported accuracy values varied from 73.4% [12] and 99.5% [19], which implemented the Random Forests ML classifier. However, these high accuracy values should be carefully interpreted. The researchers worked with datasets in which PwD and HC were known from the start, which means that it was possible to optimize the algorithms to obtain the best possible accuracy values. Consequently, randomized clinical trials are required, in which the ML results should be interpreted without knowing the diagnostics using reference standards. This might be difficult to achieve, since most of the authors of the included studies, considering their affiliations, are engineers or computer scientists. In this scoping review we came across some limitations namely the selected databases, the search keywords and the exclusion of all articles not published in scientific journals and not written in English. However, this scoping review followed rigorous methods for the selection and analysis of the included studies, which allows to conclude that the application of ML techniques to different data modalities (i.e., voice and speech, body movements, handwriting, and questionnaires) might allowed the implementation of applications to support home-based screening of Parkinson’s disease. The results of this scoping review also point the need of randomized clinical trials to provide robust evidence to translate ML algorithms to clinical practice. Acknowledgements This review was carried out within the scope of the course unit Clinical Information Management of the Master’s in Clinical Bioinformatics at the University of Aveiro.

References 1. Dorsey ER et al (2018) Global, regional, and national burden of Parkinson’s disease, 1990– 2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet Neurology 17(11):939–953

Machine Learning Applications that Might Support Home-Based …

441

2. Karaman O, Çakın H, Alhudhaif A, Polat K (2021) Robust automated Parkinson disease detection based on voice signals with transfer learning. Expert Syst Appl 178:115013 3. Aarsland D, Batzu L, Halliday GM, Geurtsen GJ et al (2021) Parkinson disease-associated cognitive impairment. Nat Rev Dis Primers 7(1):1–21 4. Tolosa E, Garrido A, Scholz SW, Poewe W (2021) Challenges in the diagnosis of Parkinson’s disease. The Lancet Neurol. 20(5):385–397 5. Poewe W, Seppi K, Tanner CM, Halliday GM et al (2017) Parkinson disease. Nat. Rev. Dis. Primers 3(1):1–21 6. Chen HL, Wang G, Ma C, Cai ZN, Liu WB, Wang SJ (2016) An efficient hybrid kernel extreme learning machine approach for early diagnosis of Parkinson’s disease. Neurocomputing 184:131–144 7. Avci D, Dogantekin A (2016) An expert diagnosis system for parkinson disease based on genetic algorithm-wavelet kernel-extreme learning machine. Parkinson’s Disease 2016:5264743 8. Adams WR (2017) High-accuracy detection of early Parkinson’s Disease using multiple characteristics of finger movement while typing. PLoS ONE 12(11):e0188226 9. Zhang YN (2017) Can a smartphone diagnose Parkinson disease? A deep neural network method and telediagnosis system implementation. Parkinson’s Disease 2017:1–11. https://doi. org/10.1155/2017/6209703 10. Butt AH et al (2018) Objective and automatic classification of Parkinson disease with Leap Motion controller. BioMedical Eng. OnLine 17(1):1–21. https://doi.org/10.1186/s12938-0180600-7 11. Prashanth R, DuttaRoy S (2018) Early detection of Parkinson’s disease through patient questionnaire and predictive modelling. Int J Med Inform 119:75–87 12. Impedovo D, Pirlo G, Vessio G (2018) Dynamic handwriting analysis for supporting earlier Parkinson’s disease diagnosis. Information 9(10):247 13. Rovini E, Maremmani C, Moschetti A, Esposito D, Cavallo F (2018) Comparative motor preclinical assessment in Parkinson’s disease using supervised machine learning approaches. Ann Biomed Eng 46(12):2057–2068 14. Cai Z, Gu J et al (2018) An intelligent Parkinson’s disease diagnostic system based on a chaotic bacterial foraging optimization enhanced fuzzy KNN approach. Comput Math Methods Med 2018:2396952 15. Gupta D et al (2018) Optimized cuttlefish algorithm for diagnosis of Parkinson’s disease. Cogn Syst Res 52:36–48 16. Channa A, Baqai A, Ceylan R (2019) Design and application of a smart diagnostic system for Parkinson’s patients using machine learning. Int J Adv Comput Sci Appl 10(6):563–571 17. Ali L, Zhu C, Zhang Z, Liu Y (2019) Automated detection of Parkinson’s disease based on multiple types of sustained phonations using linear discriminant analysis and genetically optimized neural network. IEEE J. Transl Eng Health Med 7:2000410 18. Ribeiro LCF, Afonso LCS, Papa JP (2019) Bag of Samplings for computer-assisted Parkinson’s disease diagnosis based on recurrent neural networks. Comput Biol Med 115:103477 19. Mostafa SA et al (2019) Examining multiple feature evaluation and classification methods for improving the diagnosis of Parkinson’s disease. Cogn Syst Res 54:90–99. https://doi.org/10. 1016/j.cogsys.2018.12.004 20. Lahmiri S, Shmuel A (2019) Detection of Parkinson’s disease based on voice patterns ranking and optimized support vector machine. Biomed Signal Process Control 49:427–433 21. Veeraragavan S, Gopalai AA, Gouwanda D, Ahmad SA (2020) Parkinson’s disease diagnosis and Severity assessment using ground reaction forces and neural networks. Front Physiol 11:587057 ´ 22. Chronowski M, Kłaczy´nski M, DecCwiek M, Por˛ebska K, Sawczy´nska K (2020) Speech and tremor tester - monitoring of neurodegenerative diseases using smartphone technology. Diagnostyka 21(2):31–39 23. Despotovic V, Skovranek T, Schommer C (2020) Speech based estimation of Parkinson’s disease using gaussian processes and automatic relevance determination. Neurocomputing 401:173–181

442

C. Abreu et al.

24. Yaman O, Ertam F, Tuncer T (2020) Automated Parkinson’s disease recognition based on statistical pooling method using acoustic features. Med Hypotheses 135:10483 25. ElHasnony IM, Barakat SI, Mostafa RR (2020) Optimized ANFIS model using hybrid metaheuristic algorithms for Parkinson’s disease prediction in IoT environment. IEEE Access 8:119252–119270 26. Zahid L, Maqsood M et al (2020) A spectrogram-based deep feature assisted computer-aided diagnostic system for Parkinson’s disease. IEEE Access 8:35482–35495 27. Yurdakul OC, Subathra MSP, George ST (2020) Detection of Parkinson’s disease from gait using neighborhood representation local binary patterns. Biomed Signal Process Control 62:102070 28. Yuan W, BeaulieuJones B, Krolewski R et al (2021) Accelerating diagnosis of Parkinson’s disease through risk prediction. BMC Neurol 21(1):1–12 29. Chandra J, Muthupalaniappan S et al (2021) Screening of Parkinson’s disease using geometric features extracted from spiral drawings. Brain Sci 11(10):1297 30. Jeancolas L, PetrovskaDelacrétaz D et al (2021) X-vectors: new quantitative biomarkers for early Parkinson’s disease detection from speech. Front Neuroinform 15:578369 31. Folador JP, Cecilia M et al (2021) On the use of histograms of oriented gradients for tremor detection from sinusoidal and spiral handwritten drawings of people with Parkinson’s disease. Med Biol Eng Compu 59(1):195–214 32. Varghese J et al (2021) Sensor validation and diagnostic potential of smartwatches in movement disorders. Sensors 21(9):3139. https://doi.org/10.3390/s21093139 33. Mohammed MA et al (2021) A multi-agent feature selection and hybrid classification model for Parkinson’s disease diagnosis. ACM Trans Multimed Comput Commun Appl 17(2s):1–22 34. Parziale, A., Senatore, R., Della Cioppa, A., Marcelli, A.: Cartesian genetic programming for diagnosis of Parkinson disease through handwriting analysis: performance vs. interpretability issues. Artif Intell Med 111:101984 (2021) 35. Valla E, Nõmm S, Medijainen K, Taba P, Toomela A (2022) Tremor-related feature engineering for machine learning based Parkinson’s disease diagnostics. Biomed Signal Process Control 75:5103551 36. Li Z, Yang J, Wang Y, Cai M, Liu X, Lu K (2022) Early diagnosis of Parkinson’s disease using continuous convolution network: handwriting recognition based on off-line hand drawing without template. J Biomed Inform 130:1040385 37. Saeed F, AlSarem M, AlMohaimeed M et al (2022) Enhancing Parkinson’s disease prediction using machine learning and feature selection methods. Comput Mater Continua 71(3):5639– 5658 38. Omar M, Abdelalim AM, Mohamed NH, AbdElaty HM et al (2022) Enhancing Parkinson’s disease diagnosis accuracy through speech signal algorithm modeling. Comput Mater Continua 70(2):2953–2969

Threat Modeling to Design a Decentralized Health Data Management Application Christina Erler, Shanshan Hu, Alexa Danelski, Wilhelm Stork, Ali Sunyaev, and Martin Gersch

Abstract Decentralized applications are increasingly recognized as efficient tools to overcome security challenges in the management of highly sensitive health information. However, research still lacks the understanding of how to address the complex security challenges in the design of health data management applications (HDMAs). In this study, we structurally analyzed security requirements of health data management systems to improve the efficiency of their design. We leveraged threat modeling to inform the design of HDMAs and proposed the system architecture accordingly to facilitate the security of interorganizational health data exchange. Our results contribute to the exaptation of threat modeling to the innovative design of decentralized applications for health data management and to the exploration of benefits of decentralization for healthcare. Keywords Data Security · Data Privacy · Threat Modeling · Self-Sovereign Identity · Decentralized Systems · Blockchain · Health Data Management Application C. Erler (B) Embedded Systems and Sensors Engineering, FZI Research Center for Information Technology, 76131 Karlsruhe, Germany e-mail: [email protected] S. Hu · A. Sunyaev Institute of Applied Informatics and Formal Description Methods, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany e-mail: [email protected] A. Sunyaev e-mail: [email protected] A. Danelski · M. Gersch Department of Information Systems, Free University of Berlin, 14195 Berlin, Germany e-mail: [email protected] M. Gersch e-mail: [email protected] W. Stork Institute for Information Processing Technologies, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_38

443

444

C. Erler et al.

1 Introduction The technical evolution of health data management has become one of the most critical topics in the digitization of the health sector [8]. Broadly speaking, health data management applications (HDMA) are computerized systems designed for the collection, storage and use of data related to the health status of individuals. HDMAs usually interoperate with or even comprise of electronic health records (EHRs), electronic medical records (EMRs) or personal health records (PHRs). Effective and efficient health data management can improve medical delivery and enhance the accuracy of patient care by reducing healthcare expenses and facilitating intersectoral collaboration [7]. However, existing efforts of HDMAs have tended to focus on the place of data generation (e.g., hospitals) [4]. As a result, fragmented patient data and heterogeneous health care services can lead to friction and uncertainties in patients’ daily lives [4]. To overcome these challenges, new solutions towards secure HDMA have been proposed [8]. Among others, decentralized HDMA have received popularity in research because decentralization mainly overcomes reliance on a central point and provides a trustless environment to enhance the system security and data integrity [3, 5]. Moreover, decentralized applications promise to enhance the data sovereignty of individuals and increase efficiency and transparency of collaborations based on inherent characteristics such as tamper resistance and democratization of data [9], thereby centering patients in the realization of the medical and economic value associated with their health data. Table 1 provides a selection of well-known proposed decentralized HDMAs. While most studies indicated the security concerns that should be addressed through the proposed HDMA, few studies have investigated the design of decentralized HDMAs with a fundamental consideration of the underlying security mechanisms. For example, the study by Zhang et al. [19] identified requirements and considered the implication of these for the system design. However, their analysis is limited to the defined list of requirements. Overall, it remains unclear whether and to what extent researchers have relied on systematic approaches to address security concerns [11]. This motivates us to ask the following research question (RQ): How can established security analysis methods inform the design of a HDMA? To answer our RQ, we leveraged threat modeling to inform the design of a trustworthy system architecture as it is used to analyze system vulnerabilities and their consequences structurally [18]. Table 1 Analysing security requirements of decentralized HDMA in extant literature Security Concerns to Analysis of Security Potential Security Decentralized be addressed Requirements Issues in Design HDMA Yes n/a Yes Ancile [2] FHIRChain [19] Yes Yes Yes OmniPHR [12] Yes n/a Yes MeDShare [17] Yes n/a Yes Shae & Tsai [14] Yes n/a Yes n/a n/a Yes MedRec [1] Yes Yes Yes This Study

Threat Modeling to Design a Decentralized Health Data ...

445

2 Methods We applied a three-step qualitative method to investigate the security requirements of a HDMA. First, we identified the security problems in the targeted use case by defining assets to be protected and drawing a high-level Data Flow Diagram (DFD) of the system. Second, we analyzed the potential threats based on the STRIDE-method and defined the countermeasures as requirements for the design of HDMA. Accordingly, we proposed a decentralized health data management application to mitigate the identified threats. Starting with creating a DFD, we visualized the functionalities of each component in the planned system and the interactions between components. Furthermore, we identified assets that are valuable for the system but may also be targeted by attackers. Given the inter-organizational data exchange, we also defined the trust levels of each entity and illustrated the trust boundaries to indicate the border between trustworthy and untrustworthy components [15]. Based on the DFD, we identified potential threats following the STRIDE-method. STRIDE was chosen among many threat modeling methods for several reasons. First, it is currently the most mature threat modeling method, owing to precise guidelines and successful utilization in practical use cases [16]. Second, it provides a deep understanding of how a component’s weakness affects the entire system. Third, it specifically targets security properties such as integrity, availability, and confidentiality, which are vital for the HDMA. In the final step, we proposed appropriate technical elements to implement the countermeasures. To initialize and validate our design, we conducted four online workshops with a group of experts. We started with compiling all the countermeasures on a virtual whiteboard and grouped them into several requirements. Then, we discussed possible technical solutions to fulfill each requirement. Following the analysis of the potential impact of each solution’s limitation on all requirements, suitable technical solutions are selected and a system architecture for the HDMA was proposed.

3 Results 3.1 Data Flow Diagram The essential data flows of our HDMA are illustrated in a DFD (Fig. 1). The system consists of the four components: mobile app, network infrastructure, network connector and the prevailing internal health information systems (HIS) of medical institutions. There are the following four types of users in the system: Patients represent data subjects whose health data is generated and managed in various medical institutions. Data subjects should be able to gain insight into their own overall collected health data and decide who can have access to the data. For this purpose, a mobile app will be made available to them, allowing them to view the

446

C. Erler et al.

health data (e.g., EHRs, PHRs) and their metadata (creator, time of creation, etc.). In this app, patients should be able to define the access policies for individual documents on a granular basis, i.e. who is allowed to access which document under which condition. Recipients of such access rights can be medical institutions involved in the treatment and therapy in the future as well as relatives. Data from patients’ selfassessment can be stored locally on the mobile app and made available to medical institutions, which they pass on to the other roles according to the patient’s wishes. In accordance with GDPR, patients can exercise the right to erasure by requesting the deletion of their health data in the various medical institutions via the mobile app. Healthcare professionals are medical practitioners who work in a medical institution and can create medical data about patients after or during a medical service. As the authors of the medical documents, health professionals can read, edit or delete data. The data is stored in the prevailing internal HIS of medical institutions. In addition to their role as providers, healthcare professionals may also act as consumers of health data from other medical institutions, if they have been granted access rights by patients. The enforcement of the patients’ access policies is done at the network connector.

Fig. 1 Data flow diagram of the health data management application

Medical institutions system administrators are responsible for the operation of the soft- and hardware of all system components (network connector, internal HIS) in their medical institution. All data of patients created at medical institutions are made available to a network infrastructure via standardized interfaces from the existing

Threat Modeling to Design a Decentralized Health Data ...

447

systems. A network connector links the network infrastructure and internal systems, which on the one hand enables communication via the network infrastructure, and on the other hand, contains mapping information between the network and the internal systems. The last type of users are relatives. Relatives can be related persons or persons who are trustworthy from the patient’s point of view. Relatives only participate in the system if a patient has granted them access. Once they have access, relatives can use the mobile app to view the patient’s data that has been made available to them. In addition, patients can also nominate relatives as administrators of their EHRs. With those rights, relatives have the same privileges over the data as patients. Trust Levels: Threats to a system can be made by internal, authorized agents or by external, unauthorized third-party agents with malicious intentions. Due to the fact that internal agents have access to certain system resources protection against them is more elaborate than against external ones and depends on their access, knowledge, privileges, skills, risks, tactics and motivation [6]. The assumptions of the trust level for all four internal roles and the corresponding components are described in the following: TL1: Patients can only see via the mobile app the provided data that has been generated and collected about them in the context of their care (incl. metadata), the patient-related data generated locally in their own mobile app as well as the system information generated for the secure transmission of the data via the network infrastructure (e.g. identifier, key material, credentials). They are not allowed to see (meta-)data from other system users unless they have been granted access by the data subject. Patients may act maliciously to access functionalities, privileges and data to which they do not have access. Due to a frequent lack of IT expertise and vested interests in the platform, patients will mostly behave non-maliciously. TL2: Healthcare professionals have access to all health data of their own patients generated by their own institution under a contractual agreement for the purpose of treatment and legal documentation requirements or provided by other institutions with the permission of the patient. Healthcare professionals do not have access to unshared data from other healthcare professionals as well as to unshared self-assessment data. Like patients, health professionals can act maliciously, but also have little IT expertise. We assume that there are legal and social incentive systems that make the misuse of data by health professionals unlikely. TL3: Relatives only see the patient’s data to which they have been given access by the patient and can only manage the EHRs of patients for whom they have received authorization. With regard to their malicious behavior, motivation and IT skills, similar trust assumptions can be inferred for the relatives as for the patients. TL4: System administrators have access to all IT systems in their own medical institution in order to be able to guarantee their functionality. However, they should not have direct access to the patient’s data. As administrators of the network connector, they have insight into the information about the mapping of patient identities in the network infrastructure. It is assumed that the systems managed by the system administrators follow the protocols and are accordingly trustworthy. In addition to the health data generated in their own institution, the systems also reliably store data provided by other medical institutions. With regard to the network con-

448

C. Erler et al.

nector, it is expected that the data stored in the internal HIS will be made available to the network for data requests. The network connector enforces the access policies created by the patients. The system administrators probably have very high IT skills. But there are contractual and legal framework conditions that make active misuse of the data seem unlikely. Poorly trained system administrators can lead to an accidental threat. TL5: The system components on which the different processes are running outside the medical institutions (e.g. mobile devices, network infrastructure) follow the protocols. Threatening behavior triggered by external identities cannot be ruled out. . Identified assets: We define assets as goods that are worth protecting against abuse within the system. The assets identified for our system are described in Table 2. Technical assumptions: In addition to the roles, components and privileges there are technical assumptions that we have made: TA1: Principles of data minimization should be applied so that no redundant data silos are created; TA2: Fundamental changes to the internal HIS apart from the interfaces to the network connector should not be made; TA3: It cannot be assumed that a user’s mobile device will always be available. Also on the side of the medical institutions, constant accessibility cannot always be expected; TA4: Medical institution which generated the data makes it available for the lifetime of the data. Table 2 The identified assets worthy of protection and their description Component

ID

Mobile App

A1

Asset name

Asset description

Patient-related data from the patient’s point All data collected locally by the patient’s mobile of view device in the context of self-assessment as well as all data provided and received by various health care institutions about the patient (EHR/PHR data) A2 Patient-related data shared with relatives All data shared with the relative by the patient A3 User’s communication information Network internal identifier of mobile device users and network addresses required for the exchange of messages and data via the network infrastructure A4 List of all health data releases Information on existing communication relationships with providing and consuming parties as well as their exchange content and access rights Network InfraA5 Patient-related data The patient-related (health) data and messages to be exchanged via the network infrastructure A6 Identifying communication information Network internal identifier of users and network structure addresses required for the exchange of messages and data Network Connector A7 Access logs List of all access requests, status about the request success, and the associated data and metadata of the requester (e.g. network address, date and time of access) A8 Access control list (ACL) The ACL defines the extent to which individual users and systems have access to the internal objects in the medical institutions (such as services, files, etc.) A9 Mapping information Internal mapping of the identities of different users in the network to those in the internal HIS A10 Patient-related data The (health) data collected in the internal HIS in Internal HIS the context of the internal treatment (EMR data) as well as the data provided by other healthcare professionals

Threat Modeling to Design a Decentralized Health Data ...

449

3.2 Determination of Threats and Derivation of Security Mechanisms Up to this point, the foundations for understanding the system concept and corresponding data flows have been established, now the task is to grasp which security problems can occur within this system. Figure 2 summarizes the potential threats captured during the application of the STRIDE method. Novel threats in decentralized systems like blockchains, such as sybil attacks, selfish mining, 51% attacks or wallet thefts were also taken into account [13]. Based on the identified threats, possible protection mechanisms were derived considering the countermeasures and mitigating tactics of Shostack [15]. These are listed in the right column of Fig. 2. Overall, when setting up the decentralized system, special attention should be paid to verify that a communication partner and system is the desired contact in order to prevent sensitive health data from being disclosed. In the medical context a personal contact with the healthcare professional often precedes a health data storage. This point of contact was seen as a possibility for trustworthy identification and entry point to the system, where corresponding identifiers without personal reference and cryptographic keys can be exchanged in direct contact. Without this direct contact, no connection can be established, which makes it more difficult for potential attackers. The authenticity of the data sent over that communication channel can be guaranteed by using digital signatures. Once this communication channel is established, it should also not be possible for attackers to extract information from network content and traffic, which is why end-to-end encryption and secure network protocols should be used, as well as the obfuscation of the two communication partners and their data exchange frequency by using e.g. mix networks. In addition to data transfer, within the target systems, the confidential data should be encrypted and passwords hashed. To prevent third parties from viewing the data in the event of loss or theft, the applications should be password-protected. Privacy-by-design and privacy-by-default should be applied, to make potential threats based on erroneous user interactions less likely. Appropriate access control mechanisms should be used to grant access only to those actors who have been authorized by the patient. Logging of authentication actions, access permissions, data requests and data retrieval should ensure the repudiation.

3.3 Proposed System Architecture The following section describes the proposed system architecture based on the threat modeling process (Fig. 3). The core of the architecture is the network infrastructure, as this connects the decentralized peers (mobile apps and the internal HIS of medical institutions) and enables communication and data exchange between patients, relatives and/or healthcare professionals. The network infrastructure is a decentralized peer-to-peer communication network supplemented by blockchain technology. The

450

C. Erler et al.

Fig. 2 The identified security threats and their description as well as the security measures to counter them

Threat Modeling to Design a Decentralized Health Data ...

451

network complies with the modern open-source standard DIDComm1 (PM2, PM4, PM7, PM8, PM24) and is based on the open-source implementations of Hyperledger Aries and Indy2 by the Hyperledger Foundation. Connection, communication and data transfer in the network infrastructure take place via the decentralized DIDComm network consisting of Aries Agents3 . Such an agent is a software for storing private cryptographic keys, connections and credentials in a digital wallet, which are necessary for end-to-end connection to other agents and blockchain nodes, as well as for encrypted messaging between edge agents using the DIDComm protocol (PM16-18). We define edge agents as the Aries Mobile Agents integrated into the mobile apps and the Aries Cloud Agents used by the network connector of the medical institutions. So-called Aries Mediator Agents4 enable as a kind of mailbox asynchronous, encrypted communication between edge agents and that mobile agents do not have to be permanently available. For this, the mediator agent is set up by the edge node and has its keys and delivers messages only after decrypting an outer envelope to reveal a forwarding request. To not allow correlations based on agent relationships, each edge agent can use a configuration of mediator agents with multiple transport routes through mix networks that are not fully known to the sender to receive messages over the network (PM19). In addition, to disguise its own network activities, the edge agent can use different paths over the intermediary mediator agents for sending and receiving messages (PM20). The DIDComm network is supplemented by a blockchain of Self-Sovereign Identities (SSI-Blockchain), which enables data subjects to dispose and control their own identity data, thereby serving as an additional trust anchor for the network participants. To enable data subjects to own and control their identity data, the public SSI-Blockchain publishes a decentralized public key infrastructure of cryptographic digital keys and network addresses as uniquely decentralized identifiers (public DIDs) [10]. In the case of our system architecture, the permissioned Indy SSI-Blockchain registers public DIDs of medical institutions and relatives, enabling patients and other medical institutions to establish a new agent-to-agent connection and communicate in a trustworthy manner based on verified digital identities (PM27). In addition, the SSI-blockchain store’s data models (schemas) and credential definitions for issuing and verifying certified and digitally signed identity claims (credentials) via the DIDComm network by the state, medical associations or medical institutions (e.g. a certificate of insurance), as well as a revocation registry for revoking the validity of a claim. DIDComm Messaging provides protocols for issuing and exchanging such digital credentials. Utilizing the verifiable credentials, the authenticity of a person’s identity can be verified both technically and manually when two edge agents establish a new connection on site. To keep the correlation probability of information and identities low, a pairwise peer DID5 is created 1

https://identity.foundation/didcomm-messaging/spec/. https://www.hyperledger.org/use. 3 https://github.com/hyperledger/aries. 4 https://github.com/hyperledger/aries-rfcs/blob/main/concepts/0046-mediators-and-relays/ README.md. 5 https://identity.foundation/peer-did-method-spec/. 2

452

C. Erler et al.

Fig. 3 The proposed system architecture

for each private connection between two agents in the DIDComm Network, which are only known to the two parties involved. In addition, the network connector of the medical institution is composed of an API gateway, a metadata database and a web frontend. The API gateway is the central interface for the connection, communication, access and data management functionalities of medical institutions, which can be accessed via RESTful web services. Medical institutions system administrators can provide medical data from the internal HIS to the network and receive data from other institutions through this API gateway. A metadata database stores metadata relevant to the system functionalities, e.g. the mapping of peer DIDs to User IDs in the internal HIS, a list of the treating healthcare professionals, time-limited access policies or access logs (PM10, PM12, PM13, PM25, PM26). On top of the API gateway is a web frontend that healthcare professionals can interact with to execute business logic, e.g. to establish new connections with patients, provide data from the internal HIS. The mobile app is a password protected SSI-capable application that integrates an Aries mobile agent and enables the intended connection, communication, access and data management functionalities and user interactions through a user interface as well as the associated encrypted storage of confidential data and the authentication logging functionalities (PM3, PM6, PM9, PM11, PM14-16, PM21-23).

3.4 Example of Communication In the following, it will be explained to what extent a data owner (patient) can connect to health professionals of a data provider (medical institution) within the presented architecture and how data provided by the latter to the patient can also be made available to data consumers (e.g. another medical institution or relative). Figure 4 illustrates this process. 1.) Registration and establishment of a new connection: To retrieve health data from a provider as a data owner or create releases for data consumers, a one-time personal registration is required (PM1). To initiate registration, the data owner scans

Threat Modeling to Design a Decentralized Health Data ...

453

with the mobile app an invitation QR code on-site, which is provided by the data provider. This QR code decodes the publicly resolvable DID of the data provider. The endpoint information and public key in that public DID are used to send an encrypted connection request back to the provider to establish a connection and generate and exchange corresponding peer DIDs that are known only between the two actors (e.g. DIDOP of data owner for a connection with the data provider with the DIDPO or DIDOC of data owner for a connection with the data provider with the DIDCO ). After a successful connection, both actors will be in each other’s list of registered contacts in their password-protected digital wallet including the cryptographic keys necessary for further communication and can be displayed in the mobile app or the web application (PM5). As an additional proof of identity, the two actors can exchange legal digital credentials to be able to match and verify these with physical ID documents in personal contact. 2.) Data provision to the data owner by a data provider: The healthcare professional stores the health data to be shared in its internal HIS and shares it via the web frontend with the data owner in his contact list. The API gateway is used to retrieve meta-information about the requested data in the internal HIS. The data owner is then automatically notified in the mobile app with a message via the DIDComm network to the Aries mobile agent that new health data with the corresponding metainformation and the resource ID (e.g. ID1 ) is available for retrieval. The data owner can then access the health data via the mobile app by retrieving the data from the data provider also via the DIDComm network and the network connector and storing a local copy in their mobile app. 3.) Granting data access to data consumers by the data owner: In order to release data, registration between the data owner and data consumer and between the data owner and data provider must already have taken place. Through the mobile app, the patient selects the data available to him from already linked medical institutions for release. Since there is no direct connection between the data provider and the data consumer, the data owner first forwards the public DID (DIDP ) of the data provider and the resource ID (ID1 ) to the data consumer via the established channel of the network infrastructure. In addition, the data owner’s mobile app generates a collision-free authentication token that is passed to the data consumer and data provider, authorizing the data consumer to retrieve the data from the data provider (PM12). Using the public DID, the data consumer can establish a connection with the

Fig. 4 Data sharing process within the network

454

C. Erler et al.

data provider (DIDCP ) of the data connector for a connection with the data provider with the (DIDPC ). By presenting the authentication token, the data consumer proves that he is authorized to retrieve the data from the data owner. After successful verification of access rights in the metadata database, the data provider transmits the health data to the data consumer via a secure connection. Overall, the presented system architecture covers all protection mechanisms derived in Fig. 2. By using a decentralized peer-to-peer network, no central intermediary is needed, preventing possible data tampering or disclosure. The exchange of data and messages is fully encrypted through the DIDComm protocol, which provides confidentiality and integrity. Registration in personal contact forms the basis for confidential network communication. Newly created random IDs (peerDIDs) for each connection make it possible to reveal as little information as possible. This confidential and secure network infrastructure is supplemented by user-friendly, protected applications.

4 Conclusion This study proposed a decentralized architecture for a HDMA by leveraging threat modeling into the system design. Our results reinforce the importance of systematic security requirement analysis to improve the efficiency of the system design phase. By incorporating threat modeling into the contextual specification of system requirements, our study contributes to the exaptation of threat modeling to the innovative design for health information systems. Given the specific requirements in healthcare and their vital implications for the society and individuals, the design of HDMA would be continually improved by researchers and practitioners. This study is limited by several aspects that also provide starting points for future research. First, we have only used one common threat modeling method and may not be able to identify all types of attacks comprehensively. We aimed to account for this by discussing it with experts from different areas. Second, our analysis of threats is based on our security assumptions only. It could be worthwhile for future research to test the validity of these assumptions and verify whether the identified threats need to be addressed in a real system.

Acknowledgments The authors thank all participants of the expert workshops (Dr. Jochen Rill, Dr. Elias Strehle, Dr. Markus Schinle, Philip Andris, Dr. med. Christian Sigler) within the resarch project BloG3 , funded by the German Federal Ministry of Education and Research (BMBF) (16SV8371). Special thanks to Dr. Jochen Rill, Dr. Elias Strehle and Nil Busra Bedir for discussing the design of the system architecture.

Threat Modeling to Design a Decentralized Health Data ...

455

References 1. Azaria A, Ekblaw A, Vieira T, Lippman A (2016) MedRec: Using Blockchain for Medical Data Access and Permission Management. ICBDR 2. Dagher GG, Mohler J, Milojkovic M, Marella PB (2018) Ancile: privacy-preserving framework for access control and interoperability of electronic health records using blockchain technology. Sustain Urban Areas 39:283–297 3. Erler C, Schinle M, Dietrich M (2022) Decision Model to Design a Blockchain-based System for Storing Sensitive Health Data. In: CIS 2022 Research Papers 4. Gersch M (2022) Digitalisierung im Gesundheitswesen. Handbuch Digitalisierung, Vahlen 5. Ghayvat H, Sharma M, Gope P, Sharma PK (2022) SHARIF: solid pod-based secured healthcare information storage and exchange solution in Internet of Things. IEEE Trans Indus Inform. 18(8) (2022) 6. Hunker J, Probst CW (2011) insiders and insider threats - an overview of definitions and mitigation techniques. J Wirel Mob Netw Ubiq Comput Dependable Appl 7. Häyrinen K, Saranto K, Nykänen P (2008) Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 8. Ismail L, Materwala H, Karduck AP, Adem A (2020) Requirements of health data management systems for biomedical care and research: scoping review. J Med Internet Res 9. Kannengießer N, Lins S, Dehling T, Sunyaev A (2020) Trade-offs between distributed ledger technology characteristics. ACM Comput Surv 53:2 10. Liu Y, Lu Q, Paik HY, Xu X, Chen S, Zhu L (2020) Design pattern as a service for blockchainbased self-sovereign identity. IEEE Softw. 37(5) (2020) 11. McGhin T, Choo K, Liu CZ, He D (2019) Blockchain in healthcare applications: Research challenges and opportunities. J Netw Comput Appl 135:62–75 12. Roehrs A, da Costa CA, da Rosa Righi R (2017) OmniPHR: a distributed architecture model to integrate personal health records. J Biomed Informat 13. Saad M, et al (2019) Exploring the Attack Surface of Blockchain: A Systematic Overview. ArXiv 14. Shae Z, Tsai JJ (2017) On the design of a blockchain platform for clinical trial and precision medicine. In: IEEE 37th International Conference on Distributed Computing Systems 15. Shostack A (2014) Threat Modeling: Designing for Security. Wiley Publishing 16. Tuma K, Çalikli G, Scandariato R (2018) Threat analysis of software systems: a systematic literature review. J Syst Softw 144:275–294 17. Xia Q, Sifah EB, Asamoah KO, Gao J, Du X, Guizani M (2017) MeDShare: trust-less medical data sharing among cloud service providers via blockchain. IEEE Access 18. Xiong W, Lagerström R (2019) Threat modeling - a systematic literature review. Comput Secur 19. Zhang P, White J, Schmidt DC, Lenz G, Rosenbloom ST (2018) FHIRChain: applying blockchain to securely and scalably share clinical data. CSBJ 267–278 (2018)

Software Application to Assist the Publishing Sector: A Tool in MS Excel Environment Daniela Borissova , Zornitsa Dimitrova , and Nina Keremedchieva

Abstract The rapid change in technology influences any sector of economics including the media industry. The new circumstances imposed by the coronavirus crisis require to take adequate response to adjust to the new reality. In response to these challenges, the business model print-on-demand seems to be a promising solution, especially in the production of digital/print books. In this context, the current article describes the developed software application to assist the publishing sector realized as a tool in MS Excel environment. Developed application can be used as a decision support tool considering different customer requirements, different circulation, digital publishing, or print on demand. Implementing it in Google Sheets allows other manuscripts to be easily added, as well as to change the printing price under different circulation. The developed application was used in a small publishing company to meet the requirements of various clients. The obtained results are encouraging and demonstrate the applicability of the described application. Keywords Software application · Digital manufacturing · Decision-making · Print on demand · Digital Books

D. Borissova (B) · Z. Dimitrova · N. Keremedchieva Institute of Information and Communication Technologies at the Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria e-mail: [email protected] Z. Dimitrova e-mail: [email protected] N. Keremedchieva e-mail: [email protected] D. Borissova University of Library Studies and Information Technologies, 1784 Sofia, Bulgaria © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_39

457

458

D. Borissova et al.

1 Introduction The entry of new e-media is a prerequisite for expanding and modification of traditional business models in the publishing sector. These trends in book publishing are due not only to the development of information and communication technologies but also to the need for information to reach more readers and on time. On the other hand, the application of green technologies, ecological management of resources, including improvement of logistics systems through the introduction of new technologies is part of the consequences due to the ongoing digitalization process [1]. These prerequisites are the base for achieving a knowledge-based economy that contributes to a sustainable and competitive economy. Digitization dictated by technological changes and new challenges encourage the printing sector to change existing business models, expanding its portfolio [2]. It should also be noted that the development of the e-book market also depends on the adaptability of publishers to implement new business models, taking into account with consumer attitudes [3]. All this leads to the implementation of new business models that would improve customer satisfaction with the way products are offered. For example, a product-service system for printing houses has been proposed as a new business model [4]. It worth to mention, that media and entrepreneurship have strong relevance to each other [5]. Crucial to building success in the media business is entrepreneurship, which necessarily involves innovation in content creation. The role of the media, which plays a vital role in fostering the entrepreneurial spirit by conveying values and images attributed to it, should also not be overlooked. Today, all sectors of the economy are affected by the ongoing digitization and this is most clearly manifested in the business models used [6–8]. Business models are closely related to various technological innovations, but the construction of a specific business model can essentially be separated from the technologies. Some authors treat business models for digitization from a micro-management point of view [9]. The business model in the digitization process is seen as a model based on the printed content. A characteristic feature here is that the publisher re-orientates existing print content to a suitable digital platform, such as an e-book reader, and can resell it in a new format. Regardless of e-book format each manuscript need to be properly organized following a certain page’s format like A4, A5, Letter, etc. In order to be able to convert any manuscript, some business model is to be followed to evaluate the parameters of the digital book [10]. Digital transformation and new technologies in printing, in addition to encouraging innovation in digital printing, also encourage the development of new business models [10]. Digital work in the organization is a digital challenge [11]. In this regard, to cope with such changes, different open platform is proposed. The parameters of such platforms in respect to how open are they is regarding suppliers, customers, complementary service providers, product categories and channels are discussed in [12]. Digital transformation requires of new business models to be able to answer the

Software Application to Assist the Publishing Sector: A Tool in MS …

459

new reality [13–16]. Along with the digitization process, some additional circumstances must be taken into account, such as the impact of the coronavirus crisis on business models [17, 18]. The book in the media industry, needs special attention as it is considered to be the medium with the highest cultural value. The book industry management is facing a considerable contradiction between cultural value and economic activities due to economic constraints [19]. That is why it is need to utilize different business models, sharing platforms or intelligent business decision to be up to date [20]. Now many businesses are embracing the on-demand trend to make a service that has not even been introduced before in the digital age. Everything a customer needs from a service to a product should be a customized and flexible solution. To fulfill all these requirements means to extend on-demand services. These requirements concern also publishing sector. Any approaches that could improve the logistics and customer satisfaction is of great importance. In this regard, using of the proposed generalized approach to support business by group decision-making using of different strategies could be consider as a step in the right direction [21]. Another approach aims to satisfy the needs of users and answer whether the manuscript can be considered a monograph according to the Bulgarian legislation and is based on business logic realized as web application [22]. In order to satisfy different customer requirements, a printing company need to be very flexible. Therefore, an easy and effective solution for small medium or even large companies could be seen in the usage of MS Excel [23]. That is why the current article is focused on development of software tool in MS Excel environment to support some activities in publishing sector. A business model for assessment the parameters of books under different customer requirements considering different circulation or print on demand is described. The rest of the article is structured as follows: Sect. 2 describes the problem in brief, Sect. 3 provide the used expert estimations used in generation of digital books, Sect. 4 provides description of developed software application as a tool to assist publishing sector, Sect. 5 contains results analysis and discussion, and finally a conclusion is given in Sect. 6.

2 Problem Description Due to the dynamics of the ongoing digital economy, the use of various digital tools is imperative. This trend is also valid for publishing companies, which need to be flexible and meet different customer requirements. Developing a web application involves both an out-of-the-box development effort and a domain to install it on. For a small company, which must also be competitive in the market, these are additional activities. All these considerations require to use less expensive solutions and if possible quick decision. Such possible decision is the use of functionalities of MS Excel or Google Sheets spreadsheet. Considering all of these requirements it is need to develop a software application as tool to assist the publishing sector. This

460

D. Borissova et al.

tool should be able to estimate overall parameters of different manuscripts. When evaluating the parameters of digital books, it is necessary to roughly estimate the pages occupied by text, figures, tables and equations. Along with the parameters of the author’s manuscript, the page format for a digital/print book should also be considered.

3 Expert Evaluations of Some Parameters of Production Digital/Print Books In order to make an expert estimate of the number of pages of a digital book, some preliminary assumptions are to be done. These assumptions are related to the font and font size that will be used. It is normal for a smaller page size to be associated with a smaller font size, less line spacing, and smaller margins. Considering the variety of possibilities for publishing in different formats, a variety of expert judgments can be made to reflect a specific print request. Some of the most commonly requested sizes for digital printing are A4, A5 and B5 sized pages. For these three types of dimensions, the expertise of experts from the publishing sector is shown in Table 1. The estimates listed above are made taking into account the following additional parameters per printed page: 1) for page size A4: margin-right equal to 2.0 cm and 2.5 cm for the rest; font type Calibry; font size 13 pt.; line spacing multiplied by 1.4; 2) for page size A5: margin-left equal to 1.6 cm and 1.3 cm for the rest; font type Calibry; font size 11 pt.; line spacing multiplied by 1.2; 3) for page size B5: margin-right equal to 2.0 cm and 2.3 cm for the rest; font type Calibry; font size 12 pt.; and line spacing multiplied by 1.4. All of these assumptions are used to make the following predictions for digital books to be used to generate an estimated total page count: • The number of text-only digital book pages is determined using the following relation Nt xt = N S P P/O AS [10], where the number of symbols per page is denoted by N S P P, while O AS represent the overall number of symbols in manuscript. Table 1 Expert estimations of single page parameters and estimations about required space for images, tables and equations Parameter Type

Paper size

Dimensions of digital book page

A4

A5

B5

Number of characters within a page including spaces

3740

2240

1980

Number of characters within a row

110

70

66

Number of rows per page

34

32

30

Number of images per page

2

2

2

Number of tables per page

2

2

2

Number of equations per row

3

3

3

Software Application to Assist the Publishing Sector: A Tool in MS …

461

The N S P P parameter vary depending on the used dimensions of formats for the digital-book (A4, A5, „ etc.). • The number of additional pages occupied by images Nimg i s determined by the ratio between the number of images of the entire manuscript N F and constant C1 which expresses the expert estimate about the size of the images expresses by number of rows as Nimg = N F/C1 . • The number of additional pages occupied by tables Ntbl are determined by the relation Ntbl = N T /C2 where the number of tables of the entire manuscript is denoted by N T , while constant C2 express the expert estimation of pages number due to the tables. • The number of additional pages occupied by formulas N f are determined by the relation N f = N F ∗ C3 /R P P and the number of formulas of the entire manuscript is denoted by N F, the constant C3 express the expert estimation of the used number of row per to visualize single equation, while the rows per page is denoted by R P P. The total digital book pages can be determined by summing the pages occupied by text (Nt xt ), images (Nimg ), tables (Ntbl ) and formulas (N f ).

4 Software Application as Tool to Assist the Publishing Sector in MS Excel Environment The developed software application as a tool supporting the publishing sector in MS Excel environment is shown in Fig. 1. The application is implemented in three main groups of columns. The first group contains information on each individual manuscript, the second group is related to the price, while the third group shows the calculated parameters of two types of offers for printing.

Fig. 1 Main view of the developed tool in MS Excel environment

462

D. Borissova et al.

Columns from B to E composes the first group and contain information for the manuscripts regarding the number of symbols along with spaces, number of images, number of tables, and number of formulas. The next columns G and H are the columns from the second group and contain information about the printing price for a single page in different quantities. A third group of columns from J to R is designed to display calculations by expert assessment of the total number of pages of digital books and, accordingly, the price for a certain circulation. Columns J, M and P contains formulas for calculation the overall digital book pages. For example, the total digital book pages for manuscript-1 is determined in cell J 4 that contains the following formula:  =

B4 3740



 +

C4 2



 +

D4 2



 +

E4 ∗ 3 34

 (1)

The first part of the relation (1) represents the number of pages occupied by text, determined by using the constant 3740, expressing the number of characters in one page, including spaces. The second part of the relation (1) expresses the number of pages occupied by images accordingly expert’s estimation from Table 1. The third part of (1) expresses pages formed by the presence of tables, while the last part specifies those pages of the book that are formed by the presence of formulas and evaluated by expert evaluation in Table 1. Analogically to cell J 4, the cell contains information for the total page number considering the given parameters for A5 paper size. The corresponding formula written in cell M4 is as follows:         C4 D4 E4 ∗ 3 B4 + + + (2) = 2240 2 2 32 The first part of relation (2) represents the number of text pages, determined by the constant 2240 as an expert estimate for the number of characters per page, including spaces. The second and third parts of (3) represent the number of pages taken by images and tables, while the fourth part expresses the pages taken by formulas. The cell P5 contains formula that determine the total pages considering the given parameters for B5 paper size as follows:  =

B4 1980



 +

C4 2



 +

D4 2



 +

E4 ∗ 3 30

 (3)

The cells J4, M4, and P5 should be formatted as numbers, and more precisely as integer values as they will keep the number of pages. The developed software application is realized to calculate two types of offers for printing a digital books with the already determined parameters: an offer for unsealing in a run of 100 copies and an offer for unsealing in a run of 300 copies. The values of these offers are visualized in cells K 4 and L4 for A4 paper size and respectively in cells K 4 and L4. The formula behind cell K 4 is as follows:

Software Application to Assist the Publishing Sector: A Tool in MS …

 =

463



G4 2

∗ 100 ∗ G4

(4)

The first part of (4) indicates that the book will be printed double-sided. The second constant, equal to 100, indicates the number of circulation issued. The last item of (4) contains information about the price for double-sided printing on one sheet. An analogous formula is written in cell Q4 to determine the cost of printing 100 copies of the digital book in B5 format as = (P4/2) ∗ 100 ∗ G4. Like cell K 4, the content of cell L4 contains a formula for determining the price for a print run of 300 copies of an A4-sized book:  =

G4 2

 ∗ 300 ∗ H 4

(5)

Equation (5) is composed of the information for double-sided printing, a print run of 300 editions and the price for double-sided printing of one sheet for A4 format. The price for printing in A5 format per 100 copies is determined in cell N 4 using the following formula:  =

M4 4

 ∗ 100 ∗ G4

(6)

For printing of 300 copies, the constant 100 in (6) need to be changed to 300 in cell O4. The price for printing in B5 format per 300 copies is determined in cell R4 using the following formula = (P4/2) ∗ 300 ∗ H 4. All these descriptions are valid for manuscript-1, presented in row 4 in the software application (see Fig. 1). For the remaining manuscripts shown in Fig. 1, similar formulas are used to calculate the two offers in both formats.

5 Results Analysis and Discussion The advantage of MS Excel environment usage is the fact that many different charts could be used to visualize the obtained results. The estimated total pages of digital books determined by using the described software application as a tool to assist the publishing sector for 5 different manuscripts and for two types of page size formats are shown in Fig. 2. Manuscript-1 can be transformed into a digital book of 70 pages if the page is of A4 size, into a digital book of 100 pages when using the A5 format, and into a digital book of 110 using B5 format. This manuscript contains 165362 symbols, 30 images, 16 tables, and 37 formulas. Using the proposed expert estimates given in Table 1, it is possible to calculate the approximate number of pages’ digital book. It should

464

D. Borissova et al.

Fig. 2 Estimated total number of pages of 5 digital books in different page formats obtained using the proposed software application as a tool to support the publishing sector

be noted that the estimated page count of digital books should be considered a good approximation of the actual page count. This is due to the expert judgment used for the sizes of the images and tables in the manuscript. To a very good approximation, the pages occupied by formulas can be considered real, since the length of formulas is rarely more than one line. Similarly, the number of digital book pages for the remaining four manuscripts represents a good approximation of the actual number of pages. Knowing the number of pages of digital books ready for printing, it is now possible to determine a specific offer for the price of one issue for a particular circulation. To make it easier for the user to generate a digital book printing quote, two options have been implemented in the developed application. The first option refers to the printing of the digital book in an edition of 100 copies, and the second option refers to the printing in an edition of 300 copies. The comparison between the total number of pages given three digital book sizes ( A4, A5 and B5) and considering two circulation options are shown in Fig. 3. The printing paper is assumed to be white and weighs 80g. The value for the price of two-sided black and white printing of one sheet is equal to e0.09 for the A4 and B5 format, and e0.06 for the A5 page. It is easy to change this price if color printing is required.

Fig. 3 Comparison between the total number of pages in different formats and circulation

Software Application to Assist the Publishing Sector: A Tool in MS …

465

Depending on the purpose of the specific digital book, it is also possible to prepare other offers for a larger circulation. For example, such digital books could be various teaching aids or tests for students or students, as well as special purpose questionnaires for conducting various survey surveys. Any manuscript that will be distributed in electronic or printed form needs to be properly formatted according to the specified page size. Various expert evaluations, such as those implemented in the developed application, can be used to determine the total number of pages. In such away, the developed application can be used as a decision support tool. Implementing it in an MS Excel or Google Sheets spreadsheet allows other manuscripts to be easily added, as well as to change the printing price for a set print run. The use of an appropriate business model creates a prerequisite for combining different autonomous operations in specific tasks. On the other hand, the use of different IT contributes not only to digitalization in general, but also to better economic sustainability.

6 Conclusion The rapid change of technology and the new circumstances imposed by the coronavirus crisis require an adequate response to adapt to the new reality. In this context, the print-on-demand business model appears to be a promising solution, especially in digital/print book production. Based on this business model and taking into account the given specific requirements of the small publishing company, a software application should be developed to support the publishing sector. The described application is implemented as a tool in an MS Excel and Google Sheets environment. This approach is cheap and easy to maintain due to the popularity of this software. The developed application has been used in a small publishing house and the results obtained show its applicability. The main benefit of using such an approach is the very easy changes that need to be made to the parameters of the digital/print book or print price for a different print run determined by different customers. The application can be used as a decision support tool reflecting various customer requirements, incl. circulation, digital publishing or print on demand. Acknowledgements This work is supported by the Bulgarian National Science Fund by the project “Mathematical models, methods and algorithms for solving hard optimization problems to achieve high security in communications and better economic sustainability”, KP-06-N52/7/19-11-2021.

466

D. Borissova et al.

References 1. Dzwigol H, Trushkina N, Kwilinski A (2021) Green logistics as a sustainable development concept of logistics systems in a circular economy. In: Proceedings of 37th International Business Information Management Association (IBIMA), 30–31 May 2021, Cordoba, Spain, pp. 10862–10874 2. Magadan-Diaz M, Rivas-Garcia JI (2021) Coming face to face with innovation and digitisation: the case of book printing in Spain. J Print Media Technol Res 10(1):47–59. https://doi.org/10. 14622/JPMTR-2008 3. Benhamou F (2020) The economics of e-books. In: Bille T, Mignosa A, Towse R (eds) Teaching Cultural Economics. Edward Elgar Publishing, pp 220–224. https://doi.org/10.4337/978178 8970747.00040 4. Salwin M, Kraslawski A, Lipiak J, Golebiewski D, Andrzejewski M (2020) Product-service system business model for printing houses. J Clean Prod 274:122939. https://doi.org/10.1016/ j.jclepro.2020.122939 5. Hang M (2020) Media and entrepreneurship, a revisit with a decade of progress: a bibliometric analysis of media entrepreneurship research between 2005 and 2017. Nordic J Med Manage 1(2):187–207. https://doi.org/10.5278/njmm.2597-0445.4295 6. Remane, G., Schneider, S., Hanelt, A.: Digital business model types: Understanding their mechanisms as recipes to commercialise digital technologies. International Journal of Innovation Management 2240019 (2022), https://dx.doi.org/https://doi.org/10.1142/S13639196224 00199. 7. Caputo A, Pizzi S, Pellegrini MM, Dabic M (2021) Digitalization and business models: Where are we going? A science map of the field. J Bus Res 123:489–501. https://doi.org/10.1016/j. jbusres.2020.09.053 8. AL, Echeberria (2020) The new economy and new business models. In: Echeberria AL (ed) A Digital Framework for Industry 4.0: Managing Strategy. Springer International Publishing, Cham, pp 25–56. https://doi.org/10.1007/978-3-030-60049-5_2 9. Teece DJ (2010) Business models, business strategy and innovation. Long Range Plan 43:172– 194 10. Borissova D, Keremedchieva N (2022) Decision support approach in evaluating the parameters of books for digital manufacturing. In: Rocha Á, Ferrás C, Porras AM, Delgado EJ (eds) Information Technology and Systems: Proceedings of ICITS 2022. Springer International Publishing, Cham, pp 165–174. https://doi.org/10.1007/978-3-030-96293-7_16 11. Oiestad S, Bugge MM (2014) Digitisation of publishing: exploration based on existing business models. Technol Forecast Soc Chang 83:54–65. https://doi.org/10.1016/j.techfore.2013.01.010 12. Davison RM, Ou CXJ (2017) Digital work in a digitally challenged organization. Inf Manage 54(1):129–137. https://doi.org/10.1016/j.im.2016.05.005 13. Broekhuizen TLJ, Emrich O, Gijsenberg MJ, Broekhuis M, Donkers B, Sloot LM (2021) Digital platform openness: Drivers, dimensions and outcomes. J Bus Res 122:902–914. https:/ /doi.org/10.1016/j.jbusres.2019.07.001 14. Lee C-H, Liu C-L, Trappey AJC, Mo JPT, Desouza KC (2021) Understanding digital transformation in advanced manufacturing and engineering: a bibliometric analysis, topic modeling and research trend discovery. Adv Eng Inform 50:101428. https://doi.org/10.1016/j.aei.2021. 101428 15. Curtis SK, Mont O (2020) Sharing economy business models for sustainability. J Clean Prod 266:12519. https://doi.org/10.1016/j.jclepro.2020.121519 16. Schallmo D, Williams ChA, Boardman L (2017) Digital transformation of business models – best practice, enablers, and roadmap. Int J Innov Manag 21(08):1–17 17. Yoshinov R, Chehlarova T, Kotseva M (2021) The e-facilitator as a key player for interactive dissemination of STEAM resources for e-learning via webinar. In: Auer ME, Tsiatsos T (eds) Internet of Things, Infrastructures and Mobile Applications: Proceedings of the 13th IMCL Conference. Springer International Publishing, Cham, pp 675–686. https://doi.org/10.1007/ 978-3-030-49932-7_63

Software Application to Assist the Publishing Sector: A Tool in MS …

467

18. Ritter T, Pedersen CL (2020) Analyzing the impact of the coronavirus crisis on business models. Ind Mark Manage 88:214–224. https://doi.org/10.1016/j.indmarman.2020.05.014 19. Seetharaman P (2020) Business models shifts: impact of Covid-19. Int J Inf Manage 54:102173. https://doi.org/10.1016/j.ijinfomgt.2020.102173 20. Wirtz BW (2020) Business models and value creation in the book market. In: Wirtz BW (ed) Media Management: Strategy, Business Models and Case Studies. Springer International Publishing, Cham, pp 115–124. https://doi.org/10.1007/978-3-030-47913-8_6 21. Jiang F, Zheng X, Fan D, Zhang P, Li S (2021) The sharing economy and business model design: a configurational approach. J Manage Stud 58(4):949–976. https://doi.org/10.1111/ joms.12711 22. Borissova D, Keremedchieva N, Keremedchiev D (2020) Business intelligence approach to support decision making in publishing sector. In: 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO), pp. 1268–1273. https://doi.org/ 10.23919/MIPRO48935.2020.9245424. 23. Borissova D, Korsemov D, Keremedchieva N (2020) Generalized approach to support business group decision-making by using of different strategies. In: Saeed K, Dvorský J (eds) Computer Information Systems and Industrial Management. Springer International Publishing, Cham, pp 122–133. https://doi.org/10.1007/978-3-030-47679-3_11 24. Borissova D, Korsemov D, Mustakerov I (2018) Multi-attribute group decision making considering difference in experts knowledge: an excel application. In: Proceedings of 12th International Management Conference – Management Perspectives in the Digital Era, pp. 387–395

Web Application with Facial and Emotional Recognition for the Control of Human Talent at the “España” Institute of Ambato-Ecuador Fernández Villacrés Gustavo Eduardo , Navas Olmedo Walter Humberto , Reyes Bedoya Donald Eduardo , Pazmay Pazmay Pablo David , and Arcos Naranjo Gissela Alexandra

Abstract This Research arises from the problem observed at the “España” Institute in the city of Ambato-Ecuador, which has to do with the difficulties generated by the manual control of the human talent that works there. The research objective was: “To develop a web application that contains facial and emotion recognition techniques to improve control. “The methodological approach was quantitative, the technique used was the survey, and the associated instrument was the questionnaire. The research population was 50 people. The Research ratified the problem because the processes are manual and with errors, and automation would allow their improvement. The web application development was done in free software and worked with the Laravel environment, working with external routines from Amazon web services related to facial and emotion recognition. The way to perform calculation of the threedimensional information of the face is done through a structured light projection system that shows a set of fringe patterns with the phase shift technique, and the descriptors of the beginning are obtained with the Principal Component Analysis F. V. G. Eduardo (B) · N. O. W. Humberto · R. B. D. Eduardo · P. P. P. David · A. N. G. Alexandra Instituto Superior Tecnológico España, Ambato, Ecuador e-mail: [email protected] Bolívar Y Castillo, Ambato, Ecuador N. O. W. Humberto e-mail: [email protected] R. B. D. Eduardo e-mail: [email protected] P. P. P. David e-mail: [email protected] A. N. G. Alexandra e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Á. Rocha et al. (eds.), Information Technology and Systems, Lecture Notes in Networks and Systems 692, https://doi.org/10.1007/978-3-031-33261-6_40

469

470

F. V. G. Eduardo et al.

method. Finally, the Linear Discriminant Analysis technique makes the classification and identification. It was concluded that artificial intelligence generates new labor control possibilities based on facial and emotional recognition. Keywords Web application · Facial recognition · Emotion recognition · Artificial intelligence · Control · Human talent

1 Introduction Human talent management refers to the set of processes that an organization, through its human resources department, puts in place to attract, recruit, and incorporate new employees and retain those who are already part of the company [1] It can be pointed out that: worldwide, the attendance control of a company’s human talent is part of the management of the department of the same name; however, it is a somewhat tedious process, and it is essential for the proper functioning of the company and therefore to provide the best customer service [2] The non-attendance of an employee often generates disorganization and economic losses in the company. Although they are not companies, educational institutions also require attendance control of their human talent, teaching, and administrative staff [3] Institutions of higher education in Ecuador, within the framework of the development of institutional self-evaluation processes, have detected problems related to academic activity due to the non-attendance of human talent to regular class schedules, which directly affects the quality of the training of those attending, the same problem is extended to administrative staff, although clearly with less impact on the educational service provided to students [4] The “Instituto España” is a private educational institution located in Ambato, specifically in Bolívar and Castillo streets, province of Tungurahua; it can be cataloged as a regular academic center and remote support, with Hispanic jurisdiction. The educational Institution has a manual attendance control system, which is inaccurate and is carried out by inspectors. On many occasions, problems have arisen due to teachers reporting that they have attended classes and have been recorded as “non-attending.“It should also be mentioned that being a manual process, there are delays at the end of the month when issuing a labor attendance report; also, it can be said that: on several occasions, the accounts have calculation errors. On the other hand, it should be noted that since March 2020 and due to the pandemic, all staff has been teleworking, which has further complicated the control of staff attendance; this suggests the need for a process of labor control via the web at home. The problem can be summarized in the following research question:

Web Application with Facial and Emotional Recognition for the Control …

471

How to improve the attendance control of the human talent of the “Instituto España” city of Ambato? To solve this problem, an innovative project has been developed related to access control of human talent based on facial recognition, the general objective of this project: “Develop a web application containing facial recognition and artificial intelligence techniques to improve the attendance control of human talent at the basic education institute “Instituto España” in the city of Ambato.” The following aspects have been reviewed as a theoretical basis for the research work: Artificial Vision Artificial vision covers all industrial and non-industrial applications. The combination of hardware and software provides operating instructions for the equipment to perform its functions according to image capture and processing. Although machine vision applied to industry uses the same algorithms and methods as academic/educational and government/military applications of machine vision, the limitations are different. [5] According to Velez [6], computer vision uses computers to extract information about the physical world from images. This ambitious and complex goal is still in the machine vision system and acts on the representation of reality, providing information about brightness, color, shape, etc. Computer vision or machine vision is related to artificial intelligence, which can not only determine specific parameters of an image but also describe the image. Since everything starts with the basics of image processing is essential, you can process image interpretation or video capture to improve the performance of your algorithm. [7] Areas of Application of Artificial Vision Alvear [7] mentions the following application areas as the most important: . Medicine. - In the medical field, computer vision enables better image classification or visualization, and high-resolution image processing allows specialists to operate remotely without having to travel to a location where surgery can be performed. . Internet of Things with Artificial Vision in Education. - The use of computer vision is a tool that can be implemented to monitor learners and identify moods, discomfort, and even lack of concentration that may be experienced by assessing and recognizing specific physiological behavior patterns. Artificial Intelligence. Since its inception, artificial intelligence has faced the conflict of needing a precise definition. Therefore, it is not surprising that there is still no single target definition today. [8]

472

F. V. G. Eduardo et al.

According to Ponce [9], artificial intelligence is the science and engineering that makes it possible to design and program computers to perform tasks that require intelligence. This will make it possible to replicate human intelligence through machines. Artificial and human intelligence require interfaces and systems to perform their unique functions, but with logical differences. Human senses are different from the perception and learning of operating systems and different from individual, organizational, social, and automatic learning. Artificial intelligence is not a new concept. However, it is happening correctly and should become the most relevant reality of the twenty-first century. [10] Current Artificial Intelligence Applications: According to Rouhiainen [11], these are the most critical applications at present: . Recognition, classification, and labeling of a still image these tools are helpful in many industries. . Improve the performance of algorithmic trading strategies implemented in various forms in the financial industry. . Efficient management of patient data, making medical care outstanding. . Disseminate content on social networks by becoming a marketing tool explicitly used in social networks. . Cyber threat protection for banks and systems to securely send and receive online payments. Phases of Artificial Intelligence: Among the stages of artificial intelligence, we can point out the following: Weak Artificial Intelligence: Automation and Learning: It focuses on automating the process to learn patterns from the data provided easily. With computer vision and language processing. Weak AI can recognize and analyze pattern correlations from data that humans take hundreds of years to understand so that it can replace humans quickly in many activities [10]. Facial Recognition: According to Cabanelas [12], Facial recognition is a technology that has recently been covered by multiple research fields, such as image analysis, feature extraction from digital files, etc. This is because this technological process can simulate the ability of humans to recognize people according to specific patterns in our brains. On the other hand, Knvu [13] defines a facial recognition system as a computercontrolled application that automatically identifies a person in a digital image. This is made possible by analyzing the subject’s facial features extracted from video sources or critical photos, which are compared with a database.

Web Application with Facial and Emotional Recognition for the Control …

473

Facial Recognition Methods: Among the most crucial easy recognition methods are: Holistic Methods: This face recognition method uses multiple samples of the same object whose face is the input image to the system; thanks to mathematical correlation calculations, the comparison characteristics are modeled [14]. Principal Component Analysis: According to Castaño [14], it is a technique used to reduce the size of large data sets to discern the cause of sample variation, especially for low-quality images. Facial Recognition Techniques: Among the main facial recognition techniques are: Eigenfaces: One method is based on a mathematical tool called Principal Component Analysis, which is very useful for developing face recognition and image compression systems. Understanding how this technology works are fundamental and essential to state-of-the-art technology. It is generated in two phases, training and classification. First, through PCA, a faction space, commonly referred to as self-space, is formed using facial images for training. [14] Fisher faces: This facial recognition technology has all the advantages of self-face compared to other available technologies, such as system speed, efficiency, and the ability to process multiple faces concisely. For eigenface to work well, the image must be face-to-face with similar lighting conditions, but Fisher’s face is less sensitive to image illumination and changes in face angle. [14]. Facial Recognition of Emotions: Facial emotion recognition is understood as the process by which facial features are perceived and analyzed to identify the emotional state they denote. It provides additional information for the interpretation of messages and actions of others, playing a central role in regulating social behavior as a component of interpersonal interaction. [15] Assuming the importance of facial expressions of emotions, Ekman emphasizes their recognition, elaborating objective techniques for coding facial actions and considering the functional anatomy of the face, such as the Facial Action Coding System (FACS). The automatic emotion recognition systems that are the least intrusive for users are those based on detecting and analyzing the subject’s facial expression. Facial expression is the movement or change of position of one or more muscles under the skin of the face. Most emotion recognition systems are based on two steps: facial feature extraction and classification. Extraction refers to defining characteristics or attributes that represent an emotion. In contrast, category refers to associating the identified factors with a single feeling in a group, such as joy, anger, surprise, etc. The areas of the face considered most important for emotion detection are the eyes and the lips. [16] Web Applications: According to Norambuena [17], Web application development is one of the most advanced areas of software engineering. This development also includes the emergence of new programming languages, tools, and methods for developing web applications, representing other challenges that software developers must face.

474

F. V. G. Eduardo et al.

Managing software complexity and choosing the right development tools are essential to the change management process of web software development. Human Talent Management: According to Ferrer [18], human talent management shows that leadership and management in an organization are based on effectively managing people’s potential. In other words, it begins with the conformity given to the individual with the ability to develop for the benefit of himself and his environment. On the other hand, in Norambuena [17], human talent creates strategies and achieves results. The organization today is undergoing several renovations. These changes are the result of globalization and have a significant impact on business operations. The main objective of a company’s management is to manage personnel access, respect working hours, and manage overtime, holidays, leaves, and postponements. Internal Control: Talent management also includes managing and recording employees working within the organization. This allows the different areas of the business to function correctly.

1.1 Methods A transversal study has been designed of the problem related to the almost null use of technology 4.0 as an element of control; it was proposed to develop a qualitative, quantitative investigation, the same one that was carried out in the place where the symptoms of the problem could be observed, in this case, the Instituto España of Ambato. Analytical-synthetic methods: Applied to develop a theoretical framework based on the collection of information and its synthesis. This collection was done in the denominated primary and secondary sources. Inductive -Deductive: To induce a general solution from a particular one. The primary technique defined for Research is the survey, both teachers and personnel. The instruments associated with the technique are the respective questionnaires. The population involved in the problem is composed of 50 persons. After the surveys were carried out, the data were tabulated in the Excel program from which the respective frequency tables could be constructed.

Web Application with Facial and Emotional Recognition for the Control …

475

Fig. 1 Results of question 1

Fig. 2 Results of question 2

1.2 Results The following are the results of the Research carried out on teachers and personnel. Question 1. The performance of human talent attendance control reports is? (Fig. 1) Analysis and Interpretation: The vast majority, 76% of those surveyed, stated that the time and attendance control reports of human talent at the Instituto “España” are delayed, 10% of them say they are significantly delayed, while 12% believe that they are fast (Fig. 2). Question 2. Have there been any errors in the human talent attendance control reports?

476

F. V. G. Eduardo et al.

Fig. 3 Results of question 3

Analysis and Interpretation: A slight majority, 45% of the respondents, stated that sometimes there had been errors in the completion of the human talent time and attendance reports; 31% of them said that these errors have rarely appeared and are significantly delayed, while 8% said that mistakes had occurred many times (Fig. 3). Question 3. Is it better to control human talent through an automated process? Analysis and Interpretation: The vast majority, characterized by 84% of those surveyed, affirm that it would be better to control human talent through an automated process; 14% partially indicate that it would be better. Question 4. Should the control of human talent be done automatically and based on facial recognition? (Fig. 4) Analysis and Interpretation: The vast majority, characterized by 92% of respondents, say that it would be better for human talent control to be done automatically and based on facial recognition, with 14% of respondents saying it would be partially advisable. Proposal: The proposal consists of the development of software for the registration of attendance of the human talent of the educational Institution; the novelty is that: the registration will be through facial and emotional recognition, which means that Fig. 4 Results of question 4

Web Application with Facial and Emotional Recognition for the Control …

477

the official will approach the camera and activate facial detection, if it coincides, attendance and mood will be recorded. It must be verified that there is a person in front of the camera for its efficient recognition. Software Development Methodology: The model used for the development of the proposal is the XP (Extreme Programming) methodology since it is an agile project development methodology that presumes an object-oriented approach as the preferred development paradigm and contains a set of rules and practices that, according to Castaño [14] occurs in the following four structural activities: Planning: The objective of this stage is to collect all the necessary information about the problems in the time and attendance department, to determine the requirements, and thus define the user stories and software engineering and plan the project delivery.

1.3 Coding Php routines for face detection (Figs. 5, 6, and 7).

Fig. 5 Facial and emotion recognition algorithm

478

Fig. 6 Technical execution cycle

Fig. 7 Capture of the operations

F. V. G. Eduardo et al.

Web Application with Facial and Emotional Recognition for the Control …

479

Coding. Php routines for face detection