New Perspectives in Software Engineering: Proceedings of the 10th International Conference on Software Process Improvement (CIMPS 2021) (Advances in Intelligent Systems and Computing) 303089908X, 9783030899080

Citation preview

Advances in Intelligent Systems and Computing 1416

Jezreel Mejia · Mirna Muñoz · Álvaro Rocha · Himer Avila-George · Gloria Monica Martínez-Aguilar   Editors

New Perspectives in Software Engineering Proceedings of the 10th International Conference on Software Process Improvement (CIMPS 2021)

Advances in Intelligent Systems and Computing Volume 1416

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Nikhil R. Pal, Indian Statistical Institute, Kolkata, India Rafael Bello Perez, Faculty of Mathematics, Physics and Computing, Universidad Central de Las Villas, Santa Clara, Cuba Emilio S. Corchado, University of Salamanca, Salamanca, Spain Hani Hagras, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK László T. Kóczy, Department of Automation, Széchenyi István University, Gyor, Hungary Vladik Kreinovich, Department of Computer Science, University of Texas at El Paso, El Paso, TX, USA Chin-Teng Lin, Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan Jie Lu, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia Patricia Melin, Graduate Program of Computer Science, Tijuana Institute of Technology, Tijuana, Mexico Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro, Rio de Janeiro, Brazil Ngoc Thanh Nguyen , Faculty of Computer Science and Management, Wrocław University of Technology, Wrocław, Poland Jun Wang, Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing such as: computational intelligence, soft computing including neural networks, fuzzy systems, evolutionary computing and the fusion of these paradigms, social intelligence, ambient intelligence, computational neuroscience, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, Web intelligence and multimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results. Indexed by DBLP, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and Technology Agency (JST). All books published in the series are submitted for consideration in Web of Science.

More information about this series at http://www.springer.com/series/11156

Jezreel Mejia Mirna Muñoz Álvaro Rocha Himer Avila-George Gloria Monica Martínez-Aguilar •

•

•

•

Editors

New Perspectives in Software Engineering Proceedings of the 10th International Conference on Software Process Improvement (CIMPS 2021)

123

Editors Jezreel Mejia Centro de Investigación en Matemáticas A.C. Unidad Zacatecas Zacatecas, Mexico

Mirna Muñoz Centro de Investigación en Matemáticas A.C. Unidad Zacatecas Zacatecas, Mexico

Álvaro Rocha Departamento de Engenharia Informática Universidade de Coimbra Coimbra, Portugal

Himer Avila-George Centro Universitario de los Valles University of Guadalajara Jalisco, Mexico

Gloria Monica Martínez-Aguilar Universidad Tecnológica de Torreón Coahuila, Mexico

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-3-030-89908-0 ISBN 978-3-030-89909-7 (eBook) https://doi.org/10.1007/978-3-030-89909-7 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Introduction

This book contains a selection of papers accepted for presentation and discussion at the 2021 International Conference on Software Process Improvement (CIMPS’21). This conference had the support of the CIMAT A.C. (Mathematics Research Center/Centro de Investigación en Matemáticas); CANIETI (Coahuila-Durango Office); and UTT (Universidad Technological de Torreón Coahuila, México). It took place at UTT as virtual venue, in Torreón, México, from 20 to 22 October 2021. The International Conference on Software Process Improvement (CIMPS) is a global forum for researchers and practitioners that present and discuss the most recent innovations, trends, results, experiences and concerns in the several perspectives of Software Engineering with clear relationship but not limited to software processes, security in information and communication technology and big data field. One of its main aims is to strengthen the drive towards a holistic symbiosis among academy, society, industry, government and business community promoting the creation of networks by disseminating the results of recent research in order to aligning their needs. CIMPS’21 is built on the successes of CIMPS’12, CIMPS’13, CIMPS’14, which took place on Zacatecas, Zac; CIMPS’15 which took place on Mazatlán, Sinaloa; CIMPS’16 which took place on Aguascalientes, Aguascalientes, México; CIMPS’17 which took place again on Zacatecas, Zac, México; CIMPS’18 which took place on Guadalajara, Jalisco, México; CIMPS’19 which took place on León, Guanajuato, México; and the last edition CIMPS’20 which took place on Mazatlán, Sinaloa, México, as virtual venue. The Programme Committee of CIMPS’21 was composed of a multidisciplinary group of experts and those who are intimately concerned with software engineering and information systems and technologies. They have had the responsibility for evaluating, in a ‘blind review’ process, and the papers received for each of the main themes proposed for the conference are organizational models, standards and methodologies, knowledge management, software systems, applications and tools, information and communication technologies and processes in non-software domains (mining, automotive, aerospace, business, health care, manufacturing, etc.) with a demonstrated relationship to software engineering challenges. v

vi

Introduction

CIMPS’21 received contributions from several countries around the world. The articles accepted for presentation and discussion at the conference are published by Springer (this book), and extended versions of best selected papers will be published in relevant journals, including SCI/SSCI and Scopus indexed journals. We acknowledge all those who contributed to the staging of CIMPS’21 (authors, committees and sponsors); their involvement and support are very much appreciated. October 2021

Jezreel Mejia Mirna Muñoz Álvaro Rocha Himer Avila-George Gloria Monica Martinez-Aguilar

Organization

Conference General Chairs Jezreel Mejía Mirna Muñoz

Mathematics Research Center; Research Unit Zacatecas, Mexico Mathematics Research Center, Research Unit Zacatecas, Mexico

The general chairs and co-chair are researchers in computer science at the Research Center in Mathematics, Zacatecas, México. Their research field is software engineering, which focuses on process improvement, multi-model environment, project management, acquisition and outsourcing process, solicitation and supplier agreement development, agile methodologies, metrics, validation and verification and information technology security. They have published several technical papers on acquisition process improvement, project management, TSPi, CMMI, multi-model environment. They have been members of the team that have translated CMMI-DEV v1.2 and v1.3 to Spanish.

General Support CIMPS General Support represents centres, organizations or networks. These members collaborate with different European, Latin America and North America Organizations. The following people have been members of the CIMPS conference since its foundation for the last 10 years. Gonzalo Cuevas Agustín Jose A. Calvo-Manzano Villalón Tomas San Feliu Gilabert Álvaro Rocha Yadira Quiñonez

Polytechnical University of Madrid, Spain Politechnical University of Madrid, Spain Politechnical University of Madrid, Spain Universidade de Lisboa, Portugal Autonomous University of Sinaloa, Mexico vii

viii

Gabriel A. García Mireles Adriana Peña Pérez-Negrón Héctor Orozco-Aguirre

Organization

University of Sonora, Mexico University of Guadalajara, Mexico Autonomous University of México, Mexico

Local Committee CIMPS established a local committee from the Mathematics Research Centre; Research Unit Zacatecas, Mexico, CANIETI Office Coahuila-Durango and the Technological University of Torreon, México. The list below comprises the local committee members. CIMAT UNIT ZACATECAS: Isaac Rodríguez Maldonado Israel Faustino Cruz Einar Jhordany Serna Edgar Bonilla Rivas Elizabeth Villanueva Rosas Carmen Lizarraga Raquel Aguayo Mónica del Carmen Olivarría González

Support, Mexico Support, Mexico Support, Mexico Support, Mexico Support, Mexico Support, Autonomous University of Sinaloa, Mexico Support, Autonomous University of Sinaloa, Mexico Designer, Autonomous University of Sinaloa, Mexico

CANIETI: Hugo Montoya Díaz

President of the CANIETI Office, Mexico

UTT: Raúl Martínez Hernandez Jesus Gerardo Serrano Mejia Rogelio Martinez Mojica Norma Patricia Mota Garcia Gloria Monica Martinez Aguilar

Provost, UTT, Mexico Secretary of Networking, UTT, Mexico Director of Networking, UTT, Mexico Deputy Director of Research and Development, UTT, Mexico UTT, Mexico

Scientific Programme Committee CIMPS established an international committee of selected well-known experts in software engineering who are willing to be mentioned in the programme and to

Organization

ix

review a set of papers each year. The list below comprises the scientific programme committee members. Adriana Peña Pérez-Negrón Alejandro Rodríguez González Alejandra García Hernández Álvaro Rocha Ángel M. García Pedrero Antoni Lluis Mesquida Calafat Antonio de Amescua Seco Baltasar García Perez-Schofield Benjamín Ojeda Magaña Carlos Abraham Carballo Monsivais Carla Pacheco Carlos Alberto Fernández y Fernández Claudio Meneses Villegas Edgar Alan Calvillo Moreno Edgar Oswaldo Díaz Eleazar Aguirre Anaya Fernando Moreira Francisco Jesus Rey Losada Gabriel A. García Mireles Giner Alor Hernández Gloria P. Gasca Hurtado Gonzalo Cuevas Agustín Gonzalo Luzardo Gustavo Illescas Héctor Cardona Reyes Héctor Duran Limón Himer Ávila George Hugo Arnoldo Mitre Hernández Hugo O. Alejandrez-Sánchez Iván García Pacheco Jezreel Mejía Miranda Jorge Luis García Alcaraz

University of Guadalajara, Mexico Politechnical University of Madrid, Spain Autonomous University of Zacatecas, Mexico Universidade de Lisboa, Portugal Politechnical University of Madrid, Spain University of Islas Baleares, Spain University Carlos III of Madrid, Spain University of Vigo, Spain University of Guadalajara, Mexico CIMAT Unit Zacatecas, Mexico Technological University of Mixteca, Oaxaca, Mexico Technological University of Mixteca, Oaxaca, Mexico Catholic University of North, Chile Technological University of Aguascalientes, Mexico INEGI, Mexico National Politechnical Institute, Mexico University of Portucalense, Portugal University of Vigo, Spain University of Sonora, Mexico Technological University of Orizaba, Mexico University of Medellin, Colombia Politechnical University of Madrid, Spain Higher Polytechnic School of Litoral, Ecuador National University of Central Buenos Aires Province, Argentina CIMAT Unit Zacatecas, Mexico University of Guadalajara CUCEA, Mexico University of Guadalajara, Mexico CIMAT Unit Zacatecas, Mexico National Center for Research and Technological Development, CENIDET, Mexico Technological University of Mixteca, Oaxaca, Mexico CIMAT Unit Zacatecas, Mexico Autonomous University of Juárez City, Mexico

x

José Alberto Benítez Andrades Jose A. Calvo-Manzano Villalón José Antonio Cervantes Álvarez José Antonio Orizaga Trejo José E. Guzmán-Mendoza José Guadalupe Arceo Olague José Luis Sánchez Cervantes Juan Francisco Peraza Garzón Juan Manuel Toloza Leopoldo Gómez Barba Lisbeth Rodriguez Mazahua Lohana Lema Moreta Luis Omar Colombo Mendoza Luz Sussy Bayona Ore Magdalena Arcilla Cobián Manuel Mora Manuel Pérez Cota María de León Sigg María del Pilar Salas Zárate Mario Andrés Paredes Valverde Mary Luz Sánchez-Gordón Miguel Ángel De la Torre Gómora Mirna Muñoz Mata Omar S. Gómez Patricia María Henríquez Coronel Perla Velasco-Elizondo Ramiro Goncalves Raúl Aguilar Vera Ricardo Colomo Palacios Roberto Solis Robles Santiago Matalonga Sergio Galván Cruz

Organization

University of Lion, Spain Politechnical University of Madrid, Spain University of Guadalajara, Mexico University of Guadalajara CUCEA, Mexico Polytechnic University of Aguascalientes, Mexico Autonomous University of Zacatecas, Mexico Technological University of Orizaba, Mexico Autonomous University of Sinaloa, Mexico National University of Central Buenos Aires Province, Argentina University of Guadalajara, Mexico Technological University of Orizaba, Mexico University of the Holy Spirit, Ecuador Technological University of Orizaba, Mexico Autonomous University of Peru National Distance Education University, Spain Autonomous University of Aguascalientes, Mexico University of Vigo, Spain Autonomous University of Zacatecas, Mexico Technological University of Orizaba, Mexico University of Murcia, Spain Østfold University College, Norway University of Guadalajara CUCEI, Mexico CIMAT Unit Zacatecas, Mexico Higher Polytechnic School of Chimborazo, Ecuador University Eloy, Alfaro de Manabi, Ecuador Autonomous University of Zacatecas, Mexico University Tras-os Montes, Portugal Autonomous University of Yucatán, Mexico Østfold University College, Norway Autonomous University of Zacatecas, Mexico University of the West, Scotland Autonomous University of Aguascalientes, Mexico

Organization

Sodel Vázquez Reyes Sonia López Ruiz Stewart Santos Arce Sulema Torres Ramos Tomas San Feliu Gilabert Ulises Juárez Martínez Vianca Vega Víctor Saquicela Viviana Y. Rosales Morales Yadira Quiñonez Yasmin Hernández Yilmaz Murat

xi

Autonomous University of Zacatecas, Mexico University of Guadalajara, Mexico University of Guadalajara, Mexico University of Guadalajara, Mexico Politechnical University of Madrid, Spain Technological University of Orizaba, Mexico Catholic University of North Chile, Chile University of Cuenca, Ecuador University of Veracruz, Mexico Autonomous University of Sinaloa, Mexico INEEL, Mexico Çankaya University, Turkey

Contents

Organizational Models, Standards and Methodologies A Proposal of Metrics for Software Requirements Elicitation in the Context of a Small-Sized Software Enterprise . . . . . . . . . . . . . . . C. Pacheco, I. García, J. A. Calvo-Manzano, and M. Reyes

3

CMMI Adoption and Retention Factors: A Systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Francisco Alfaro, Cynthia Silva, and Abraham Dávila

15

Proposal to Integrate Security Practices into the ISO/IEC 29110 Standard to Develop Mobile Apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jezreel Mejía, Mirna Muñoz, Perla Maciel-Gallegos, and Yadira Quiñonez

29

Systematic Mapping of Literature on Applicable Patterns in Conceptual Modelling of Information Systems . . . . . . . . . . . . . . . . . . . . Oscar Carlos Medina, Manuel Pérez Cota, Luis Esteban Damiano, Karen Della Mea, and Marcelo Martin Marciszack A Brief Review of Vertical Fragmentation Methods Considering Multimedia Databases and Content-Based Queries . . . . . . . . . . . . . . . . Aldo Osmar Ortiz-Ballona, Lisbeth Rodríguez-Mazahua, Asdrúbal López-Chau, María Antonieta Abud-Figueroa, Celia Romero-Torres, and Felipe Castro-Medina A Review of Horizontal Fragmentation Methods Considering Multimedia Data and Dynamic Access Patterns . . . . . . . . . . . . . . . . . . . Abraham Castillo-García, Lisbeth Rodríguez-Mazahua, Felipe Castro-Medina, Beatriz A. Olivares-Zepahua, and María A. Abud-Figueroa

41

55

69

xiii

xiv

Contents

The State-of-the-Art of Business Intelligence and Data Mining in the Context of Grid and Utility Computing: A PRISMA Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ernani Damasceno, Ana Azevedo, and Manuel Perez-Cota

83

Knowledge Management Towards Optimisation of Electroconvulsive Therapy with a Little Help from Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jorge Rodas-Osollo, Karla Olmos-Sánchez, Alicia Jiménez-Galina, Marlene Soltero-Romero, and Angélica Pérez-Campos

99

A Literature Review of Hand-Based Interaction in Virtual Environments Through Computer Vision . . . . . . . . . . . . . . . . . . . . . . . 113 Cristian Olvera, Graciela Lara, Arturo Valdivia, and Adriana Peña MLV-Viewer: Universal Decision Support System . . . . . . . . . . . . . . . . . 123 Manuel Pérez Cota, Carlos Manuel Oliveira Alves, and Miguel Ramón González Castro Software Systems, Applications and Tools Towards the Evaluation of Relevant Interaction Styles for Software Developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Adriana Peña Pérez Negrón, Mirna Muñoz, David Bonilla Carranza, and Nora Rangel Software Implementation for Multivariable Control of Dividing Wall Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Reyna R. Díaz-Vásquez, Galo R. Urrea-García, Guadalupe Luna-Solano, and Denis Cantú-Lozano Polyglot Programming with GraalVM Applied to Bioinformatics for DNA Sequence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 José Antonio Romero-Ventura, Ulises Juárez-Martínez, and Adolfo Centeno-Téllez Virtual Assistant as Support for People Visually Impaired . . . . . . . . . . 174 Liliana E. Olguín-Gil, Francisco Vázquez-Guzmán, Eduardo Vázquez-Zayas, Jezreel Mejía, and Itzel Blanco-Cruz A Novel Method Based on Numerical Iterations for Joint-Space Trajectory Generation with a via Point . . . . . . . . . . . . . . . . . . . . . . . . . 189 Yadira Quiñonez, Oscar Zatarain, Carmen Lizarraga, Raquel Aguayo, and Jezreel Mejía

Contents

xv

Main Technological Tools in Virtual Educational Environments According to Different Strategies and Learning Styles to Improve the e-learning Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Raquel Aguayo, Carmen Lizarraga, and Yadira Quiñonez User Research Techniques for User Interface Design of Learning Management Systems: A Decade Review . . . . . . . . . . . . . . . . . . . . . . . . 218 Luis Raúl Méndez-Becerra, Viviana Yarel Rosales-Morales, Giner Alor-Hernández, and Carmen Mezura-Godoy New Quantum Image Manipulation System (NQIMS) . . . . . . . . . . . . . . 233 Rasha F. Aly Classifying the Deceptive and Truthful Responses Through Ocular Measures in an Interview Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Hugo Mitre-Hernandez Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Organizational Models, Standards and Methodologies

A Proposal of Metrics for Software Requirements Elicitation in the Context of a Small-Sized Software Enterprise C. Pacheco1 , I. García1(B) , J. A. Calvo-Manzano2 , and M. Reyes1 1 Division de Estudios de Posgrado, Universidad Tecnologica de la Mixteca,

Carretera a Acatlima, 69000 Oaxaca, Mexico {leninca,ivan,karenina}@mixteco.utm.mx 2 Escuela Tecnica Superior de Ingenieros Informaticos, Universidad Politecnica de Madrid, Boadilla del Monte, 28660 Madrid, Spain [email protected]

Abstract. The requirements elicitation process aims to obtain a preliminary version of the requirements before their specification. However, due the nature of this process, it is very difficult to know if the elicited requirements have quality. Furthermore, there are some metrics that allow requirements engineers to measure some aspects of the elicitation process and its final product (i.e., a preliminary version of the requirements, also called pre-requirements), however there is no way of measuring and controlling the elicited pre-requirements. With the aim of understanding, evaluating, and improving the requirements elicitation process, the existing metrics for this process have been analyzed and, as a consequence, eight metrics using the GQM paradigm are proposed. Additionally, a preliminary validation of these metrics has been performed in the context of a small-sized software enterprise. The obtained results are also presented. Keywords: Requirements engineering · Requirements elicitation process · Metrics

1 Introduction In the context of the software development process, Requirements Engineering (RE) is one of the most important areas that helps software practitioners to decide and define what the software must do [1]. The RE process is frequently described by the following stages: elicitation, analysis, specification, validation and verification, as well as management [2]. Therefore, the final product of the RE process is the Software Requirements Specification (SRS) which establishes the functional and non-functional requirements for the software that will be developed. Nevertheless, a list of pre-requirements must be previously obtained by performing the elicitation and analysis stages. It is worth noting that the requirements elicitation process is not a trivial task and it is one of the main sources of confusion and causes many projects to fail [3, 4]. Thus, the improvement of the software quality depends on the SRS quality and, consequently, the quality of the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 3–14, 2022. https://doi.org/10.1007/978-3-030-89909-7_1

4

C. Pacheco et al.

pre-requirements should improve. According to Bourque and Fairley [5] and Sandhu and Weistroffer [6], the requirements elicitation process is frequently defined by three main activities: (1) the stakeholder identification (i.e., those people who provide the information needed to identify the problem to solve, as well as their needs and constraints); (2) obtaining the list of pre-requirements of each stakeholder by using the elicitation techniques; and (3) the integration, refinement, and organization of the collected information in order to determine the functionalities and limitations of the desired software. However, because human participation is essential for the elicitation process and the large quantity of information that needs to be handled, some problems may arise. For example, the crucial stakeholders do not participate in the elicitation sessions; there are conflicts among stakeholders; the elicited requirements do not reflect the required quality attributes; the time allocated for the elicitation process is not enough; unskilled requirements engineers; the stakeholders are not convinced about the need of a new software; or the lack of good communication and collaboration among stakeholders [7, 8]. With the aim of obtaining better results from the requirements elicitation process, measurement activity may be needed in order to improve the control of this process and provide positive feedback to all stakeholders (e.g., programmers, project management, customers). According to Basili [9], the ideal mechanism for improving a process is through measuring it and its products by establishing measurable objectives supported by the proper models. It is important to bear in mind that a measurement process can promote the maturity of the requirements elicitation process in order to help stakeholders understand what they really want and what they really need, as well as to recognize the constraints and limitations of the software that will be developed [10]. Furthermore, the quality attributes of the pre-requirements (i.e., the indicators for the metrics) are important for determining the quality of the product that will be obtained (i.e., list of prerequirements) as these attributes can be measured by inspection (using the metrics) while the reported metrics can be used to individually evaluate the pre-requirements quality as well as to generally evaluate the full list of pre-requirements. Moreover, the historical data (obtained by the prolonged use of the metrics) can be used for the early identification of potential problems, and to carry out timely corrective actions in order to provide guidance about the decisions that should be taken. Therefore, in this paper, metrics for measuring and controlling the requirements elicitation process are presented to enhance this process by improving the quality to the elicited pre-requirements. The rest of the paper is organized as follows: Sect. 2 examines other approaches which introduce some metrics for the requirements elicitation process. With this aim in mind, Sect. 3 briefly describes the application of the GQM paradigm in order to design specific metrics for understanding and controlling the requirements elicitation process. Section 4 presents the preliminary results obtained in the context of a small-sized software enterprise. Finally, Sect. 5 summarizes the main conclusions of this paper.

2 Related Work Software metrics have been an important research topic in Software Engineering for many years. In this context, the measurement process is an important instrument for

A Proposal of Metrics for Software Requirements Elicitation

5

understanding, evaluating or predicting, managing, and improving software entities such as processes, products, and resources [11]. Therefore, the use of metrics has been highlighted by several authors providing the possible reasons and effects of using metrics that could also be applied to the requirements elicitation process. For example, Palmer and Evans [12] proposed the Advanced Integrated Requirements Engineering System (AIRES) in order to provide automated support for the identification of requirements which are at risk of being ambiguous, redundant, inconsistent, untraceable, or incomplete. To this end, AIRES introduced three types of metrics: counting, ratio of risk, and relative size. Research by Dutoit and Bruegge [13] developed and refined some metrics for obtaining the communication degree among stakeholders. The proposed metrics enabled researchers to collect data on artifacts generated by groupware tools (e.g., e-mail, memos, messages) during requirements elicitation. This study highlighted the use of communication metrics by arguing that they can provide better insights into software processes and products compared with metrics based on code. Moreover, Kaiya and Saeki [14] proposed a set of metrics to measure the degree of domain knowledge in software application development during the requirements elicitation process. This study used an ontology to represent the terms and semantic structure of the problem domain to solve. Thus, by using these definitions, some metrics were proposed in order to evaluate the requirements quality and to identify incompleteness, inconsistency, ambiguity, or errors in the structural characteristics of the ontology and the software requirements. Similarly, Barragáns et al. [15] proposed a metric for determining the level of inconsistency among the stakeholders’ viewpoints (i.e., level of inconsistency in requirements, conflict degree, and the uncertainty of the viewpoints). The designed metric determines the close relationship between disagreement and incompleteness by obtaining the level of inconsistency, meaning that the best model (i.e., model of underspecified states that represents each viewpoint) reflects the combined knowledge of all stakeholders in the elicitation activities. Nakatani et al. [16] highlighted that the requirements volatility indicates the level of maturity of the requirements (i.e., software requirements do not change if they achieve 100% maturity during the elicitation process), and defined a method for evaluating the Requirements Maturation Efficiency (RME) by taking into account stability and accessibility. The RME is obtained by dividing the number of requirements that remain unchanged by the total number of elicited requirements, while the accessibility is calculated as the inverse of the maximum distance of communication between the requirements analyst and the requirements source. Research by Hanakawa and Obana [17] introduced some metrics to evaluate the quality of the elicitation sessions, because these directly influence the quality of the obtained requirements. These metrics were classified in basic (start time and end time of a speaking, stakeholder who is speaking, and the start time and end time of a theme) and advanced (average of a system engineer’s speaking time, average of a stakeholder’s speaking time, the number of times of system engineers’ speakings, the number of times of stakeholders’ speakings, ratio of system engineers’ speaking time to all discussion time, ratio of stakeholders’ speaking time to all discussion time, ratio of the number of

6

C. Pacheco et al.

times of system engineers’ speakings to total number of time of speakings, ratio of the number of times of stakeholders’ speakings to total number of time of speakings, and the maximum number of times of stakeholders’ speakings between a system engineer speaking and a system engineer speaking). Moreover, Zapata et al. [18] performed a study related to distributed software development in order to evaluate the efficiency of an elicitation technique by using the metrics proposed by Lloyd, Rosson, and Arthur [19] with adjustments for the requirements elicitation process. The metric used, called SRS document quality, takes into the account the following features: aspects related to the document’s organization; the percentage of requirements that are identified as an evolution of a basic requirement; average of requirements without defects of precision, ambiguity, consistency, etc. (i.e., defects attributable to deficiencies in the requirements elicitation process); and the percentage of supported requirements which indicates the completeness of the produced document. Finally, research by Byun et al. [20] emphasized the importance of the consistency of requirements as they must be compatible with the proposed objectives and constraints of the software that will be developed. This study proposed six metrics (degree of objective contribution, degree of objective satisfaction, value obtainable for each requirement, degree of constraint conformance, degree of constraint impact, and cost demandable for each requirement) in order to evaluate the requirements considering the objectives and constraints of the software. Additionally, five consistency metrics were designed: a metric to measure the consistency of each requirement that is related to one objective and various constraints, a metric to measure the consistency of each requirement that is related to one objective and one constraint, a metric to measure the consistency of each requirement that is related to various objectives and various constraints, a metric to measure the consistency of each requirement that is related to various objectives and one constraint, and finally a metric to measure the consistency of each requirement using its value and cost. In summary, all of the proposed metrics only focused on the second activity of the requirements elicitation process (i.e., obtaining the list of pre-requirements of each stakeholder by using the elicitation techniques). However, the other two activities (the stakeholder identification, and the integration, refinement, and organization of the collected information) are not covered by the previous studies. Therefore, it is necessary to identify metrics that help requirements engineers to measure the elicitation progress and the quality of the pre-elicited requirements.

3 Eight Metrics for Measuring the Requirements Elicitation Process The Goal-Question-Metric (GQM) paradigm [9, 21] represents a systematic approach to define, adjust, and evaluate a set of objectives based on the objectives of a project or organization as well as by using a measurement process. GQM takes into account three levels for formally defining a measurement program: 1) The conceptual level (Goal) that establishes the measurement goals for an object related to products, processes, or resources. These goals can be defined by using a template for an object [22], 2) The operational level (Question) that generates a set of questions used to specify how the goals

A Proposal of Metrics for Software Requirements Elicitation

7

must be reached, and 3) The quantitative level (Metric) that defines the metrics to quantitatively answer the question(s) established for the previous level [23]. GQM provides a hierarchical structure where the goals, questions, and metrics can have a many-to-many relationship. Moreover, improving RE is a challenge due to its volatility, however, quality assurance can be achieved by developing and applying appropriate (valid) metrics and measurements [24]. With this aim in mind, Tables 1 and 2 show the definition of the measurement goals (the GQM conceptual level) for stakeholder identification, and the integration, refinement, and organization of the collected information. Table 1. Definition of the measurement goal for stakeholder identification. Description Goal

Identify the project stakeholders

Purpose

Improve the stakeholder identification activity using the Volere template [25]

Approach

The identified stakeholders must be relevant and necessary for the project

Perspective

The requirements engineer’s viewpoint

Environment

In the context of a small project (six to twelve months and 50,000–100,000 LOC) that will be developed by a team of fewer than 10 members

Table 2. Definition of the measurement goal for integration, refinement, and organization of the collected information. Description Goal

Integrate, refine, and organize the collected information

Purpose

Analyze if all the obtained pre-requirements by each stakeholder fulfil the quality criteria proposed by IEEE Std. 830-1998 [26], Lauesen [27], and ISO/IEC/IEEE 29148 [28] (i.e., the final list of pre-requirements is complete, each requirement has to be unambiguous, consistent, traceable (to its source or stakeholder) and understandable)

Approach

The final list of pre-requirements must be complete, unambiguous, consistent, traceable, and understandable

Perspective

The requirements engineer’s viewpoint

Environment

In the context of a small project (six to twelve months and 50,000–100,000 LOC) that will be developed by a team of fewer than 10 members

It is worth mentioning that the use of the Volere template [25] was introduced in order to help the requirements engineer to identify the stakeholders (i.e., this template was used because it defines some stakeholder classes that may be present within the software development project). We believe that this strategy provided two main advantages in the context of a small-sized software enterprise: it did not require an advanced level of theoretical knowledge and it provided a good overview of the pre-requirements

8

C. Pacheco et al.

when data was being collected. Tables 3 and 4 provide the basis for the formulation of questions and metrics for each defined goal. Taking into account these goals, the Measurement Construct Template defined by the Practical Software Measurement (PSM) approach [29] was used to link the variables that were measured against our information needs (https://drive.google.com/file/d/1mknDJSpKSQcNS-38R7NmBpAy7wgwY jQK/view?usp=sharing). Within the context of a small-sized software enterprise, these measurement constructs are useful for describing how the relevant activities and products of the requirements elicitation process are quantified and converted into indicators that provide a basis for decision making. Table 3. Metrics proposed for stakeholder identification. Goal

Question

Metrics

Identify the project stakeholders

How many classes of stakeholders were identified by the Volere template?

# of classes of stakeholders that participated in the elicitation process

Were all the software perspectives obtained?

# of pre-requirements proposed by each stakeholder # of elicited pre-requirements

Table 4. Metrics proposed for integration, refinement, and organization of information. Goal

Question

Metrics

Integrate, refine and organize the information

Do the elicited pre-requirements fulfil the quality criteria [26–28]?

# of inconsistent pre-requirements # of ambiguous pre-requirements # of understandable pre-requirements # of traceable pre-requirements Completeness of the list of pre-requirements

The quality of the requirements elicitation process will influence the quality of the software development process because a minimal error at this stage would affect the rest of the life cycle stages. Moreover, an effective requirements definition will allow requirements engineers to remain disciplined during the software development process, provide better support when changing management activities, and obtain better results in testing, thereby decreasing risk, improving the quality, and supporting automatization. Table 5 shows the formal definition of the eight metrics proposed for measuring the requirements elicitation activities 1 and 3, respectively.

A Proposal of Metrics for Software Requirements Elicitation

9

Once the metrics were defined, it was also necessary to define the procedures for collecting them. In order to make the procedures clear and repeatable, the ISO/IEC/IEEE 15939:2017 standard [30] states that this definition should normally describe who is responsible for the measurement, the data source, the frequency of the data collection, and the required tool for supporting this activity. Both requirements elicitation activities 1 and 3 can be divided into tasks, with a template designed for recording measurements. Moreover, a new activity was also proposed for the requirements elicitation process in order to analyze the collected data and provide proper data as well as timely feedback to the requirements engineer about the requirements elicitation process carried out in the project (Activity 4: Post-mortem). The defined procedures required the creation of templates for recording the measurement data, such as the stakeholders’ information and analyzing the information of the elicited pre-requirements. These templates included a sheet of instructions to avoid any errors being made. In addition, with the aim of making the measurement program easier for the requirements engineer of the smallsized software enterprise, Excel spreadsheets were designed for storing and processing the measurement data and recording the historical data. Table 5. Definition of metrics for the requirements elicitation products. Metric

Definition

# of classes of stakeholders that participated in the elicitation process

Classesa of

Unit of measurement Range

Classes of stakeholders identified stakeholders by the Volere template

1–14

# of pre-requirements proposed by each stakeholder

Pre-requirements Individual preproposed by each requirements stakeholder identified according to the Volere template

0 to infinity

# of elicited pre-requirements

Total of pre-requirements elicited in all the elicitation sessions

Total pre-requirements

0 to infinity

# of inconsistent pre-requirements

Pre-requirements in conflict with others

Pre-requirements in conflict

0 to (the total number of pre-requirements – 1)

# of ambiguous pre-requirements

Different Uncertain interpretations of each pre-requirements stakeholder for each pre-requirement on the list

0 to the total number of stakeholders

# of understandable pre-requirements

Pre-requirements understood by all the stakeholders

0 to (the total number of pre-requirements)

Acceptable pre-requirements

(continued)

10

C. Pacheco et al. Table 5. (continued)

Metric

Definition

Unit of measurement Range

# of traceable pre-requirements

Pre-requirements that are associated with a stakeholder

Attributable pre-requirements

0 to the total number of stakeholders

Final version of the list of prerequirements

0 to the total number of stakeholders

Completeness of the All the stakeholders list of must have approved pre-requirements and signed the final version of the pre-requirements list

a The classes of stakeholders define the roles that they can perform with regard to the Volere

template for stakeholder analysis. According to Robertson and Robertson [25], there are 14 classes: interfacing technology, maintenance operator, normal operator, operational support, client, functional beneficiary, interfacing technology, internal consultant, sponsor, customer, interfacing technology, external consultants, negative stakeholders, and core team members.

Finally, in order to validate the feasibility of this approach, a preliminary validation in a small-sized Mexican software enterprise was conducted. The following section provides the details.

4 Preliminary Results An early validation was conducted in a small-sized software enterprise with 20 employees which used Scrum to develop the software projects and the IEEE Std. 830-1998 to obtain the SRS as well as refine the use cases. Previous to the requirements specification, the enterprise used two elicitation techniques in their projects: interviews and focus groups to elicit a pre-requirements list. The project used for this case study was an informative web portal owned by a small corporation for managing all of its internal information and the services focused on customers. This case study used, as a baseline, a pre-requirements list produced by a team who worked in the requirements elicitation process without using the defined measurement program (the control group) and another team who used the measurement program to evaluate its work in the context of this case study (the experimental group). These groups were comprised of three people: a project manager and two requirements engineers. The experimental group members were selected randomly because they all had the same experience and knowledge on the requirements elicitation process and the project’s domain. All of the participants had worked at the enterprise for three years. It is important to mention that no member of the control group had communication with any member of the experimental group, as both groups were comprised of different members, and none of the members of the experimental group knew beforehand about the preparation for this exercise. The experimental group used the Volere template [25] to perform the stakeholder identification activity. Furthermore, the control group did not carry out stakeholder identification because their members considered the client as

A Proposal of Metrics for Software Requirements Elicitation

11

the only stakeholder in the requirements elicitation process. Additionally, the elicited pre-requirements were recorded in the meeting minutes for each session. The data was collected in three ways: review of archival data (pre-requirements list used as baseline), observation of participants, and metrics (including the use of templates to collect the measurements). During the review of the baseline, an expert researcher checked if it was properly carried out and evaluated the quality attributes of each requirement elicited for the control group. The “expert” is a requirements engineer/professor with more than 10 years’ experience, was not part of the experimental group, and was also unaware that his work was part of the validation. The analysis was applied to the quantitative data obtained in the requirements elicitation activities. The expert carried out a completely blind evaluation of the list of pre-requirements obtained by both the control and experimental groups to evaluate their quality, and did not know anything about how they were obtained. For the observation of participants, two requirements elicitation sessions were conducted to get firsthand information concerning the problems of the requirements elicitation activities. Table 6 summarizes the obtained results. Table 6 has two main rows for each group: the first one lists the stakeholders and their efficiency (rows 3 to 7 in Table 6), while the second one specifies the quality attributes that were evaluated from the list of pre-requirements elicited by each project (rows 8 and 9 in Table 6). The appropriateness value was 0.45 for the baseline project, because from the 31 pre-requirements provided by the only stakeholder, a total of just 19 pre-requirements were selected for the final list. This is in contrast to the experimental group, where all 53 proposed pre-requisites appeared in the final list after the metrics were applied. Table 6. Preliminary results from the validation. Control group

Experimental group

Diversity of stakeholders

Appropriateness of stakeholders per classes

Diversity of stakeholders

Appropriateness of stakeholders per classes

Client

0.45

Maintenance operator

0.019

Normal operator

0.340

Client

0.547

Internal consultant 0.056 External consultant 0.038 Coverage of each quality attribute (%) Unambiguous* (0%), Consistent (70%), Traceable (80%), Understandable (45%), Completeness** (100%)

Unambiguous (85%), Consistent (80%), Traceable (100%), Understandable (76%), Completeness** (100%)

31 pre-requirements were elicited

53 pre-requirements were elicited

* This attributed was evaluated taking into account that in the baseline project, only one stakeholder

participated. ** This attributed was evaluated taking into account the full list of pre-requirements.

12

C. Pacheco et al.

Thus, the differences (see last row of Table 6) between the pre-requirements obtained by the baseline project (control group) and those obtained by the new project (experimental group) show an increase in the quality of the elicited pre-requirements as they better fulfill the stakeholders’ expectations and needs. Despite these results, it is important to mention that as a consequence of the training and implementation of the metrics, an additional month was required for the experimental group to perform the requirements elicitation activities than the control group. The experimental group achieved a better coverage of the quality attributes for the elicited pre-requirements, a better diversity of the identified stakeholders, and a higher number of elicited appropriate pre-requirements than the participants of the baseline project. It is clear that these results are promising, however, they are not useful for generalizing the benefits of the metrics. Nevertheless, two measurement programs are currently being conducted to support this approach by providing more quantitative evidence.

5 Conclusions We have proposed eight metrics for the requirements elicitation process which have to be reviewed by all the stakeholders in order to achieve their acceptation (i.e., prerequirements validation). The quality of these pre-requirements must be verified to ensure that they are complete, correct, consistent, unambiguous, verifiable, traceable, and modifiable (i.e., pre-requirements verification). Therefore, as soon the characteristics and constraints of the software are determined, problems and errors will be avoided for the remaining stages of the software process development. Otherwise, the cost of fixing these errors will increase and the project might fail. It is worth mentioning that the presented validation was conducted within only one enterprise who elicited the pre-requirements for only one project, therefore we cannot argue that the obtained results can be replicated for other enterprises and projects. As previously stated, our metrics need to be applied to more enterprises and projects with the aim of analyzing and improving the interpretation of each one. We strongly believe that more specific metrics can be proposed to improve their adaptation to each one of the elicitation techniques and the second activity of the elicitation process (i.e., obtaining the list of pre-requirements of each stakeholder by using the elicitation techniques), which was not included in our research because there are some related metrics for this part.

References 1. Tukur, M., Umar, S., Hassine, J.: Requirement engineering challenges: a systematic mapping study on the academic and the industrial perspective. Arab. J. Sci. Eng. 46, 3723–3748 (2021) 2. Curcio, K., Navarro, T., Malucelli, A., Reinehr, S.: Requirements engineering: a systematic mapping study in agile software development. J. Syst. Softw. 139, 32–50 (2018) 3. Beewolf, R.: Project excellence or failure? Doing is the best kind of learning. IEEE Eng. Manage. Rev. 44(2), 26–32 (2016) 4. Dar, H., Lali, M.I., Ashraf, H., Ramzan, M., Amjad, T., Shahzad, B.: A systematic study on software requirements elicitation techniques and its challenges in mobile application development. IEEE Access 6, 63859–63867 (2018)

A Proposal of Metrics for Software Requirements Elicitation

13

5. Bourque, P., Fairley, R.E.: Guide to the Software Engineering Body of Knowledge (SWEBOK®), Version 3.0. IEEE Computer Society Press, Los Alamitos (2014) 6. Sandhu, R.K., Weistroffer, H.R.: A review of fundamental tasks in requirements elicitation. In: Wrycza, S., Ma´slankowski, J. (eds.) SIGSAND/PLAIS 2018. LNBIP, vol. 333, pp. 31–44. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00060-8_3 7. Ghazi, P., Glinz, M.: Challenges of working with artifacts in requirements engineering and software engineering. Requirements Eng. 22(3), 359–385 (2017). https://doi.org/10.1007/s00 766-017-0272-z 8. Dar, H.S.: Reducing ambiguity in requirements elicitation via gamification. In: 28th International Requirements Engineering Conference (RE 20), pp. 440–444. IEEE Computer Society (2020) 9. Basili, V.R.: Applying the goal/question/metric paradigm in the experience factory. In: Software Quality Assurance and Measurement: A Worldwide Perspective, pp. 21–44. International Thomson, London (1993) 10. Ohashi, K., Katayama, A., Hasegawa, N., Kurihara, H., Yamamoto, R., Doerr, J., Magin, D.P.: Focusing requirements elicitation by using a UX measurement method. In: 26th International Requirements Engineering Conference (RE 18), pp. 347–357. IEEE Computer Society (2018) 11. Tahir, T., Rasool, G., Gencel, C.: A systematic literature review on software measurement programs. Inf. Softw. Technol. 73, 101–121 (2016) 12. Palmer, J.D., Evans, R.P.: Software risk management: requirements-based risk metrics. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 836–841. IEEE Computer Society (1994) 13. Dutoit, A.H., Bruegge, B.: Communication metrics for software development. IEEE Trans. Softw. Eng. 24(8), 615–628 (1998) 14. Kaiya, H., Saeki, M.: Using domain ontology as domain knowledge for requirements elicitation. In: 14th IEEE International Requirements Engineering Conference (RE’06), pp. 189–198. IEEE Computer Society (2006) 15. Barragáns, A.B., et al.: On the interplay between inconsistency and incompleteness in multiperspective requirements specifications. Inf. Softw. Technol. 50(4), 296–321 (2008) 16. Nakatani, T., Tsumaki, T., Tsuda, M., Inoki, M., Hori, S., Katamine, K. Requirements maturation analysis by accessibility and stability. In: 18th IEEE Asia-Pacific Software Engineering Conference (APSEC 18), pp. 357–364. IEEE Computer Society (2011) 17. Hanakawa, N., Obana, M.: A metrics for meeting quality on a software requirement acquisition phase. In: Dieste, O., Jedlitschka, A., Juristo, N. (eds.) PROFES 2012. LNCS, vol. 7343, pp. 260–274. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31063-8_20 18. Zapata, S., Torres, E., Sevilla, G., Abdalla, L., Reus, M.: Effectiveness of traditional software requirement elicitation techniques applied in distributed software development scenarios. In: XXXVIII Conferencia Latinoamericana en Informatica (CLEI 12), pp. 1–7. IEEE Computer Society (2012) 19. Lloyd, W.J., Rosson, M.B., Arthur, J.D.: Effectiveness of elicitation techniques in distributed requirements engineering. In: IEEE Joint International Conference on Requirements Engineering, pp. 311–318. IEEE Computer Society (2002) 20. Byun, J., Rhew, S., Hwang, M., Sugumara, V., Park, S., Park, S.: Metrics for measuring the consistencies of requirements with objectives and constraints. Requirements Eng. 19(1), 89–104 (2014) 21. Basili, V.R., Weiss, D.M.: A methodology for collecting valid software engineering data. IEEE Trans. Softw. Eng. 10, 728–738 (1984) 22. Basili, V.R., Heidrich, J., Lindvall, M., Munch, J., Regardie, M., Trendowicz, A.: GQM+strategies -- aligning business strategies with software measurement. In: First International Symposium on Empirical Software Engineering and Measurement, pp. 488–490. IEEE Computer Society (2007)

14

C. Pacheco et al.

23. Caldiera, V.R.B.G., Rombach, H.D.: Goal question metric paradigm. Encycl. Softw. Eng. 1, 528–532 (1994) 24. Fernández, D.M., et al.: Naming the pain in requirements engineering. Empir. Softw. Eng. 22(5), 2298–2338 (2016). https://doi.org/10.1007/s10664-016-9451-7 25. Robertson, S., Robertson, J.: Mastering the Requirements Process, 2nd ed. Addison-Wesley, New York (2006) 26. Institute of Electrical and Electronics Engineers: IEEE Std. 1233-1998 - Guide for Developing System Requirements Specifications (1998) 27. Lauesen, S.: Software Requirements, Styles and Techniques. Pearson Education, Harlow (2002) 28. International Organization for Standardization: ISO/IEC/IEEE 29148:2011 - Systems and Software Engineering - Life Cycle Processes - Requirements Engineering (2011) 29. McGarry, J., et al.: Practical Software Measurement: Objective Information for Decision Makers. Addison-Wesley, Boston (2002) 30. International Organization for Standardization Standard: ISO/IEC 15939:2007 - Systems and Software Engineering - Measurement Process (2007)

CMMI Adoption and Retention Factors: A Systematic Literature Review Francisco Alfaro1 , Cynthia Silva1 , and Abraham Dávila2(B) 1 Escuela de Graduados, Pontificia Universidad Católica del Perú, Lima, Perú

{alfaro.francisco,c.silvao}@pucp.edu.pe

2 Departamento de Ingeniería, Pontificia Universidad Católica del Perú, Lima, Perú

[email protected]

Abstract. CMMI has increased the productivity and reduced the cost of software development in the software industry. However, there are factors that influence the adoption and retention of CMMI in software organizations, and that need to be studied over time. This article aims to identify factors that influence the adoption and retention of CMMI in the software development organizations. A systematic literature review (SLR) was performed for this study. In the first stage, 2507 articles were obtained from 6 relevant databases and after the SLR process, 40 studies on factors and their possible influence were selected. These factor studies were classified according to a taxonomy based on: organization, people, processes and product. The most studied factors are related to people and organizations, in the CMMI adoption and retention processes, which is consistent with the fact that it is the “people” of the software development “organizations” who manage to carry out the software projects. Studies related to retention factors are still scarce, representing only 10% of the total identified. In addition, the use of alternate terms of factors and the use of “critical success factors” and “success factors” are observed without a clear distinction. Keywords: CMMI · Adoption factor · Critical success factors · SLR

1 Introduction The maturity model, according to [1], has its origin in the works of Nolan in 1973 and Crosby in 1979, based on a framework of management practices. Also, [2] points out, that CMMI is useful to measure different aspects of a process or an organization. Over time, various maturity models have been developed to help organizations improve software quality, most of them are based on Capability Maturity Model Integration (CMMI) and ISO/IEC 15504 [3–6]. In particular, in these three decades, since its inception, the Capability Maturity Model for Software (Software CMM), and later CMMI, have an enormous contribution to the productivity and quality of the software industry worldwide [7]. The adoption of a model, such as the CMMI, reveals a series of factors involved in these initiatives that have been studied in [8], and [9], among others. In these studies, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 15–28, 2022. https://doi.org/10.1007/978-3-030-89909-7_2

16

F. Alfaro et al.

factors are analyzed and organized according to various criteria such as the cases of: (i) Bayona [8], the commitment of senior management is mentioned, objectives aligned with the strategic objectives of the business, training, communication, participation, management of change, knowledge and skills, among others; and (ii) Hameed [9], a conceptual model for IT innovation developed, identifying the different factors that influence innovation, organization, environment, CEO and context of acceptance of use. The aim of this study is to identify the adoption and retention factors of CMMI in software development organizations determining the type of influence. The article is organized as follows: in Sect. 2, the evolution of Software CMM and CMMI is presented; in Sect. 3, related studies are described; in Sect. 4, the definition of systematic literature review is explained; in Sect. 5, the results of this analysis are shown; and, in Sect. 6, the conclusions are presented.

2 CMMI Evolution The foundations of the Capability Maturity Model (CMM) were established by W. Humprey between 1986–1987 [10] and published under the title “Characterizing the Software Process. A Maturity Framework” as a Software Engineering Institute’s technical report [11]. Its publication was part of a research project funded by the Department of Defense (DoD) [7, 10], with data collected from various projects in the DoD and observation of experiences related to processes in an empirical way rather than a theoretical way [12]. The Version 1.0 of the Software CMM was published in 1991 [13, 14] based on experience and previous documents used to identify problems and improving processes. The Software CMM was a model to judge the maturity of an organization’s software process and helps organizations improve the process to increase the maturity [15]. Some characteristics of mature organizations are: software development is performed according to a planned, process that include roles and responsibilities are well defined, schedules and budgets are established using historical data, and product quality is achieved [15]. In 1993, Software CMM Version 1.1 was released [7], and in 1997, it was decided to stop the development of draft of Software CMM Version 2C (draft) in favor to development CMMI. In 2000, CMMI Version 1.02 was published [16, 17]. CMMI integrated the Software CMM v2C, System Engineering Capability Model and Integrated Product Development [16]. CMMI v 1.02 was published for use by development organizations [17]. The CMMI v1.1 was launched in 2002, v1.2 was launched in 2004 and v.1.3 was launched in 2010 [17]. The CMMI v 1.3 includes 3 constellations for development, services and acquisition [18]. The CMMI was based on the need for a software project process management and improvement framework [18], and on the need to identify the key practices that a contractor should perform during software intensive development of systems for [17]: (i) improve the reliability of the software development process; (ii) help the organization improve software development skills; and, (iii) successfully balancing time, budget, and quality requirements.

CMMI Adoption and Retention Factors

17

Finally, in 2018, the CMMI V 2.0 [19] was published, which focuses on improving business performance on operational and business objectives. The CMMI changes in terms of its internal nomenclature, the measurement framework (it uses practice area levels instead of process area capacity levels); reviews the process areas of V1.3 [18] and presents them as a single practice area; and, it defines a maturity model that is easier to understand, maintaining the 5 levels of organizational maturity [19]. It also presents a change in its approach, introducing agility in the organization of the model and showing that the responsibility for the practices lies with the developers. In the next sections, when a sentence refers CMMI, it includes also CMM.

3 Related Work Some secondary studies were identified CMMI adoption factors, but none studies of CMMI retention factors. The identified studies are: • In [20], using a systematic literature review, establishes: (i) a list of success factors, barriers, motivators and demotivators when implementing CMM; (ii) factors grouped in 23 categories based on semantic similarities of the terms used to name them; and (iii) that 14 of the 22 process areas of the People CMM model can be related to 11 of the 23 categories of factors identified. • In [21], based on the identified studies, a catalog of human factors that influence software process improvement (SPI) is elaborated, considering: (i) 15 factors related to people were identified; and (ii) the factor most studied is “senior management commitment” and the second most studied is “role responsibility”. • In the article by [22], based on a mail survey on the influence of six organizational factors and organizational performance, it was established that: (i) small organizations implemented software process improvement (SPI) effectively like big companies; and (ii) small companies as a strength the participation of employees and exploration of new knowledge.

4 Research Method This research was performed using a systematic literature review (SLR) based on [23]. The SLR is on CMMI adoption and retention factors, determining the type of influence and its context. 4.1 Systematic Literature Review According to [23], an SLR has the following stages: (1) planning the review, (2) conducts the review; and finally, (3) document the review. Stage 1 is presented in this section; stage 2, in Sect. 5; and stage 3, throughout the document. The design of the protocol and the execution of the SLR were developed by the first two authors and validated by the third author.

18

F. Alfaro et al.

Identify the Need for Realization. The identification of CMMI adoption and retention factors in software development organizations will be a contribution for those who wish to establish adoption and retention strategies based on the identified Factors. Research Purpose and Questions. For the elaboration of the objective, a scheme taken from the GQM - Goal Question Metrics [24] was used in which it is defined as: (i) objective of the study - capability maturity model integration; (ii) purpose - identify; (iii) focus - adoption and retention factors; and (iv) involved - software development organizations. so, the purpose of the SLR is “to identify in the software development organizations the CMMI adoption and retention factors.“ The research questions (RQ) and motivation are listed in Table 1. The size of the company was not considered, since in a preliminary review of 10 articles, to confirm the research questions and the search string, none of them made reference to the size of the company. The date was not restricted to cover the largest number of studies. In addition, the bayona’s taxonomy [25] was used for categorization and sub-categorization of the selected factors.

Table 1. Research questions and motivation Research question

Motivation

RQ-1 What factors and degree of influence are observed in the adoption when implementing CMMI?

Identify factors when adopting CMMI to improve the quality of your software processes, considering the version of the model

RQ-2 What factors and degree of influence are observed in the retention of CMMI?

Identify factors in retention the practices required by CMMI after their adoption

RQ-3 What factors are categorized according to an established taxonomy (organization, people, process, product)?

Identify how the factors are categorized according to the defined taxonomy

Search String. To define the search string, the Population (P) and Intervention (I) from PICO strategy, established in [23], was used. The P&I components allowed us to cover the largest number of studies and use selection criteria to filter empirical studies. In this study, we have: • ‘P’ with the main term “Capability Maturity Model” and alternate terms to CMM-Sw, CMM and CMMI. • ‘I’ with the main term “adoption or retention factors” and alternative terms success, failure, barriers, resistance, motivators, de-motivators and duration. From the above, the search string was as follows: “(CMMI OR CMM OR “Capability Maturity Model”) AND (factors OR adopting OR success OR failure OR barriers OR resistance OR motivators OR “de-motivators” OR duration)”.

CMMI Adoption and Retention Factors

19

Selection Criteria and Quality Assessment. The inclusion criteria (IC) and exclusion criteria (EC) are presented in Table 2. The search date was left unrestricted to cover all versions of CMMI. It was established to work with an automatic search in the digital databases: Scopus, ACM, ScienceDirect, ProQuest, IEEE Xplore, and Web of Science. The questionnaire used for the quality evaluation was based on [26] and which is presented in Table 3. In addition, for this questionnaire a score taken from [27] was used: Yes = 1, Partly = 0.5 or No = 0. The selection and quality evaluation criteria were applied in 5 stages. In the first stage, the meta-data was extracted from the digital databases. In the second stage, IC.1, IC.2, IC.3, EC.1, EC.2 and EC.4 were applied to the titles. In the third stage, IC.3, IC.4, EC.3 and EC.4 were applied to the abstracts. In the fourth stage, IC.5, EC.5 and EC.6 were applied for a preliminary review of the content. In the fifth stage, the quality assessment was carried out with which the primary studies were determined.

Table 2. Inclusion criteria (IC) and exclusion criteria (EC) Item Criteria IC1

Articles extracted from indexed databases

IC2

Academic articles in scientific journals, specialized conferences

IC3

Primary studies (case studies, experiences, lessons learned, survey, etc.)

IC4

Articles where CMMI has been implemented of any capacity level

IC5

Articles that mention factors of adoption or retention in CMMI

EC1 Duplicate items EC2 Articles whose title is not related to the topic of the RSL EC3 Articles whose abstract is not related to the topic of the RSL EC4 Secondary and tertiary studies, newspaper articles and books EC5 Insufficient presentation of results EC6 Articles that mention factors of adoption or retention, but without considering the CMMI

Table 3. Questionnaire used for the quality evaluation based on [26] #

Question

1

Is the article based on research?

2

Is there a clear statement of the research objectives?

3

Is there an adequate description of the context in which the investigation was carried out?

4

Is the article appropriate to address the objectives of the research? (continued)

20

F. Alfaro et al. Table 3. (continued)

#

Question

5

Was the selection strategy appropriate for the research objectives?

6

Was there a control group with which to compare treatments?

7

Was the data collected in a way that addressed the research topic?

8

Was the data analysis sufficiently rigorous?

9

Has the relationship between the researcher and the participants been an adequate degree?

10 Is there a clear statement of the results? 11 Is the study relevant to research or practice?

Data Extraction and Synthesis. A format to perform data extraction from primary studies was defined based on [26]. The structure of the format included data such as: study identifier, article date, authors, title, source, study objectives, adoption factors indicating whether it was with positive or negative influence, retention factors, benefits obtained, the study’s contribution, among others. The narrative synthesis developed by [28] was used: first, identifying the main concepts that allow answering the research questions; and second, applying the grouping and clustering technique.

5 Analysis of Results This section presents the results of the SLR according to Sect. 4. The steps of the selection process from the data extraction to primary studies selected are presented in Fig. 1. The total number of studies selected from digital databases, in January 2021, were 2507 articles, and after process selection 40 primary studies were selected. The list of selected primary studies is presented in Appendix A and the results of the quality assessment are presented in Appendix B. The answers to the research questions are presented below.

Fig. 1. Primary studies process selection

CMMI Adoption and Retention Factors

21

5.1 RQ-1 What Factors and Degree of Influence are Observed in the Adoption When Implementing CMMI? In the identification of CMMI adoption factors, 30 studies were identified out of a total of 40 selected. Of these, 22 articles studied adoption factors (S18, S16, S01, S02, S05, S07, S15, S09, S11, S12, S13, S38, S37, S36, S35, S34, S33, S27, S29, S30, S31, S22) and 8 articles reported adoption factors in conjunction with retention factors (S14, S25, S28, S24, S26, S23, S20, S21). Appendix C presents the ordered list of factors, according to the descending frequency of studies. In the list they are identified taking into consideration the following: (a) if the analyzed factor coincides or is close to one that is within the taxonomy [25] used then the factor is as defined by Bayona; (b) if the analyzed factor coincides or is close to a previously established one, which is not in Bayona’s taxonomy, then the factor is as it has been established; and (c) if the factor does not coincide with the previous ones, then a new one is established, seeking to be as clear as possible in its description. Table 4 presents a list of those adoption factors that are indicated in at least 20% of the studies. From the first two rows, it can be seen that “Senior management commitment” and “Experienced staff”, 24 and 21 references respectively, are the two factors most studied or reported as influential. Likewise, from the perspective of the type of criticality, they have been reported for: (i) “Senior management commitment” at 50% (12 of 24 references) as a Critical Success Factor (CSF) and 25% (6 of 24 references) as a Success Factor (SF); what could be considered 75% relevance; and (ii) “Experienced staff”, 43% (9 of 21 references) as CSF and 29% (6 of 21 references) as SF; so, 72% relevance can be considered. Both factors are very relevant to take into account in any CMMI adoption strategy. Likewise, Table 4 shows that the factors “Commitment of stakeholders” and “Formal documentation of processes” have had a negative influence on the different implementations. For the factor “Commitment of stakeholders”, 25% (3 of 12 references) consider it CSF and 33% (4 of 12 references) consider it SF, so 58% consider it relevant for the success of the process improvement. For the factor “Formal documentation of processes” 43% (3 of 7 references) consider it as CSF and 29% (2 of 7 references) consider it SF, therefore, 72% consider it relevant. Additionally, using the same reasoning, it can be noted that the next relevant factors are “Tools” and “Staff participation”. In [20], a result is related to the number of factor references, until 2012. According to [20], the two most mentioned factors are “senior management commitment” and “implementation strategy” with 36 and 28 references respectively. The first of them coincides with our study, and the second relates to the following two factors, but with other names. In [21], the most mentioned factors are “senior management commitment” with 11 references and “staff participation” with 4; which is consistent with the results of our study.

S14, S33, S28, S31 S07, S27

S07, S12, S36, S29, S30, S31 S02, S05, S12, S37, S30, S20 S12, S28, S29, S31, S22 S37, S30, S26, S20 S37, S36, S25, S31, S26 S12, S37, S36, S33, S30 S12, S36

Id I+ S02, S07, S14, S37, S36, S35, S33, S27, S28, S29, S26, S20 S02, S14, S37, S33, S25, S27, S28, S29, S30, S31, S26 S35, S07 Tools

S01, S21

Effective communication

S33, S23

S11, S23, S21

S16, S12, S34, S22 S16, S13, S34

S18, S12

S18, S01, S21

S16, S07

Formal methodology for deploying processes Staff training program according to needs Available budget, material, and human resources Formal documentation of processes, reviewed, and approved Leadership

S15, S23, S21

S23

Staff time/resources

Organizational change

S11, S23

S15

Staff participation

S09

S16, S34

S02, S27, S24, S22

S16, S28, S29, S22 S16, S05, S36

Commitment of stakeholders

S38, S23

S18, S16, S12, S37, S36, S34, S33, S22 S14, S28, S22

Experienced staff

S18, S09, S11, S23, S21

S16, S12, S36, S34, S22

Factor Senior management commitment

Id Io S18, S15, S11, S38, S23, S21

Id IS16, S05, S12, S34, S29, S22

6

7

7

7

8

8

9

10

11

11

12

21

Tot 24

S07

23% 20%

S16, S14

23%

S02, S15, S37, S30, S26 S15, S37, S26, S21 S16, S37, S30

S01, S07, S14, S29, S30, S21 S16, S02, S37, S30 S16, S11, S29

S16, S37, S07

Id CSF S18, S16, S02, S07, S15, S14, S11, S37, S29, S26, S21, S29 S16, S02, S14, S11, S37, S29, S30, S26, S21

S18, S16, S01, S07, S21 S16, S11, S21

23%

27%

27%

30%

33%

37%

37%

40%

70%

% 80%

S12, S33, S27

S33

S12, S22

S12, S36

S12, S36, S33

S36

S27, S20, S22

S09, S12, S20, S22, S29 S12, S36, S22

S12, S36, S22

S12, S36, S33, S22

S09, S12, S36, S33, S27, S22

Id SF S12, S36, S33, S27, S20, S22

S13, S34, S28, S31 S18, S23

S34, S23

S34, S23

S25, S31, S23

S05, S28, S31, S23 S24

S05, S28

S28, S31

S18, S38, S35, S34, S23

S18, S34, S25, S28, S31, S23

Id F S05, S38, S35, S34, S28, S23

Table 4. Adoption factors. The first three columns are for study identifiers that have positive influence “Id I+”, negative influence “Id I−” and influence not determined (not declared) “Id Io”. The total column refers to the number of studies that mention this factor and the percentage it represents with respect to the total number of studies. In addition, the last three columns report, as indicated in each study, the level of criticality relevance of the factor as CSF = Critical Success Factor, SF = Success Factors and F = Factor (without declaring criticality).

22 F. Alfaro et al.

CMMI Adoption and Retention Factors

23

5.2 RQ-2 What Factors and Degree of Influence are Observed in the Retention of CMMI? In the identification of CMMI retention factors, 18 studies were identified out of a total of 40 selected. Of these, 8 articles studied retention and adoption factors jointly (S14, S25, S28, S24, S26, S23, S20, S21), another 9 articles studied only retention factors (S08, S19, S17, S03, S04, S06, S32, S39, S40), and one article studied transition factors (S10) between the CMMI maturity levels, which was considered retention. In Appendix D the ordered list of factors is presented according to the descending frequency of studies. The structure and considerations in its elaboration are similar to the one explained in the previous Sect. 5.1 for Appendix C. Table 5 presents a list of those retention factors that are indicated in at least 20% of the studies. Of the first two columns “Id I+”, “Id I−” (Id-factor with positive or negative influence respective), it can be seen that the factors of “Senior management commitment” and “Experienced staff”, 11 and 10 references respectively, have contributed positively to the achievement of results and are the ones that have the greatest study have been presented in the literature. On the other hand, from the perspective of the type of criticality, 28% (3 of 11 references) as CSF and 18% (2 of 11 references) as SF have been reported for “Experienced staff” and for “Senior management commitment”, 40% (4 of 10 references) as CSF and none references as SF; therefore, they can be considered as two very relevant factors to take into account in any CMMI retention strategy. Additionally, using the same reasoning, it can be noted that the next relevant factors are the “Formal methodology for deploying processes” and “Staff time/resources”. The results obtained in this question are consistent with those obtained in [20] in 2012, since they report “senior management commitment” with 16 references as a success factor and 20 as a barrier; and “implementation strategy” with 17 references as a factor and 11 as a barrier. 5.3 RQ-3 What Factors are Categorized According to an Established Taxonomy (Organization, People, Process, Product)? In Table 6, presents a quantified summary of the grouped factors on the Bayona’s taxonomy taken as a reference for this study. All factors have been categorized into Bayona’s taxonomy, because this categorization level is high and include “others” category. At the sub- categorization level, we identify 2 factors in “others” category related to market and 2 factors in the organization category without sub-category. In Appendix E, the complete list of adoption and retention factors grouped by the Bayona’s taxonomy [25] is shown, which takes five main categories such as organization, people, process, product and other. In addition, in the list in Appendix E, the adoption factors are ordered according to the number of studies that mention it and by the importance of its criticality declared as CSF or SF for each primary study of the SLR. In Table 7 show a list of adoption factors with respect to the category defined by Bayona’s taxonomy. As in the previous cases, the cutoff value is those that are 20% higher in the number of articles that study it or that consider it relevant (classified as CSF or SF). In addition, it is noted in both cases that all the factors most studied or those considered relevant are in the Bayona’s taxonomy.

S04, S20,

S14, S28, S28,

S24,

S21, S03, S19,

Tools Staff participation

4 4

4

Organizational change

S23,

5

5

6

7

5

Available budget, material, and human resources Investment and cost of the company

Formal methodology for deploying processes Effective communication

Leadership

S19, S10, S23, S40 S17, S06, S21,

S39

S32,

S04, S14, S32, S28, S04, S32, S28,

S23, S21, S03,

S19, S32

7

22% 22%

22%

28%

28%

28%

33%

39%

39%

56%

10

Experienced staff Staff time/resources

% 61%

Tot 11

Factor Senior management commitment

S10, S23, S21, S40 S06,

S17, S06, S10,

S24

S08, S19,

Id Io S19, S10, S23, S21, S40 S19, S10, S23, S21,

Id IS08, S32,

Id I+ S14, S28, S26, S20, S04, S14, S32, S25, S28, S26, S04, S26, S20, S25, S26,

S14, S21,

S32,

S14,

S21,

S21,

S32,

S26, S21

S26

Id CSF S14, S26, S21, S14, S32, S26, S21

S19, S20,

S17,

S19,

S17, S20,

Id SF S19, S20,

S28, S03, S04, S28,

S10, S24, S23, S40 S04, S06, S32, S28, S04, S28, S23,

S04, S06, S10, S24, S19, S32, S25, S23, S03, S10, S23, S39, S40 S08, S19, S06,

Id F S08, S10, S32, S28, S23, S40 S19, S04, S10, S25, S28, S23,

Table 5. Retention factors. The first three columns are for study identifiers that have positive influence “Id I+”, negative influence “Id I−” and influence not determined (not declared) “Id Io”. The Total column refers to the number of studies that mention this factor and the percentage it represents with respect to the total number of studies. In addition, the last three columns report, as indicated in each study, the level of criticality relevance of the factor as CSF = Critical Success Factor, SF = Success Factors and F = Factor (without declaring criticality).

24 F. Alfaro et al.

CMMI Adoption and Retention Factors

25

5.4 Threats to Validity Threats to the validity of the study are described below. Data Source. The selected digital libraries were 5 indexed databases. Some relevant primary studies may not have been included because they are in databases not considered in this research.

Table 6. Factors classified by category of the Bayona’s taxonomy [25] Category

Within

People

42

Organization

25

Processes

25

Product

2

Others (market)

2

Organization (Not sub-categorized)

2

Selection of Studies. The selection of the studies was based on the professional experience of the first two authors, so there may be a bias in the selection of studies. However, procedures were defined to reduce bias and there was a third author who reviewed the work carried out in some stages.

Table 7. Factors most studied by category of the Bayona’s taxonomy [25] Category

Sub Category

Factor

#

%

Organization

Commitment

Senior management commitment 27

68%

People

Skills

Experienced staff

63%

Organization

Infrastructure

Staff time/resources

14

35%

Organization

Commitment

Commitment of stakeholders

13

33%

25

People

Staff involvement

Staff participation

13

33%

Organization

Organizational culture

Organizational change

12

30%

Organization

Standards and procedures

Tools

12

30%

Organization

Infrastructure

Available budget, material, and human resources

11

28%

(continued)

26

F. Alfaro et al. Table 7. (continued)

Category

Sub Category

Factor

#

%

People

Communication

Effective communication

11

28%

Process

Deployment Process

Formal methodology for deploying processes

11

28%

People

Training

Staff training program according 10 to needs

25%

People

Leadership

Leadership

25%

10

Quality Assessment. In the quality assessment, the studies were scored according to our personal judgment using an assessment model by the first two authors. It was found that 100% of the studies met the minimum score to be selected, therefore, all the studies were included to answer the research questions. Analysis of Results. Of the selected studies, the factors with their influence (positive, negative or neutral) were identified along based on the individual knowledge of the first two authors and reviewed by the third author, who has more professional and research experience. Therefore, there is a risk that some factors may have been omitted.

6 Conclusions Future Work This article presents the results of an SLR on CMMI adoption and retention factors in software development organizations. The identification of adoption and retention factors are relevant to establish strategies that contribute to the success of process improvement projects and the continuity of practices in organizations that have implemented CMMI. The main contribution of this study is offered to the development organizations that are interested in implementing or maintaining the CMMI a set of factors and its influences (positive, negative and neutral) of adoption and retention useful to design their strategy. For the SLR, the term adoption factors have been used considering that it is a neutral term and that it does not imply its type of influence (positive or negative). However, there are other neutral terms that are synonymous and non-neutral terms that are equivalent to “positive factor” as motivators or “negative factor” as barriers that have been included in this study. Likewise, from the knowledge of the software industry and the associated literature, one has on one side the grouped factor models, such as the Bayona’s taxonomy; and on the other, studies on factors that had the greatest impact on process improvement, which are referred to as critical success factors, success factors or simply factors. Additionally, studies on specific factors do not necessarily correspond to the most critical. From RQ1, the most studied and most relevant adoption factors are those of “Senior management commitment”, “Experienced staff”. Coincidentally, from RQ2, the most studied and most relevant retention factors are the same as RQ1. However, studies related to retention factors are still scarce, representing only 10% of the total number identified.

CMMI Adoption and Retention Factors

27

From RQ3, it is found that the most studied and most relevant factors are associated with the People and Organizations categories, in the CMMI adoption processes. This makes perfect sense considering that people are the center of software production and the foundation of a software development organization. The need to search and identify models of retention factors as well as the transition between maturity levels in CMMI emerges from this study. Likewise, it is necessary to identify which set of critical success factors is those that must be considered for a software industry with certain specific characteristics. Acknowledgments. This work has been performed in the context of the Project ProCal-ProSer (Phase 2) and partially supported by the Engineering Department and the Research and Development Group in Software Engineering (GIDIS) from Pontificia Universidad Católica del Perú.

Appendix https://drive.google.com/drive/folders/1Ki65yDFN9LHRIfDCcQP6crn_3iw1i1fP? usp=sharing.

References 1. Kalinowski, T.B.: Analysis of business process maturity and organisational performance relations. Management 20, 87–101 (2016). https://doi.org/10.1515/manment-2015-0052 2. Proença, D.: Methods and techniques for maturity assessment diogo. In: Iberian Conference on Information Systems and Technologies, CISTI, pp. 1–4. AISTI (2016). https://doi.org/10. 1109/CISTI.2016.7521483 3. Monteiro, E.L., Maciel, R.S.P.: Maturity models architecture: a large systematic mapping. iSys - Brazilian J. Inf. Syst. 13, 110–140 (2020). https://doi.org/10.5753/isys.2020.761 4. Saavedra, V., Dávila, A., Melendez, K., Pessôa, M.: Organizational maturity models architectures: a systematic literature review. In: Trends and Applications in Software Engineering, CIMPS 2016, Advances in Intelligent Systems and Computing, pp. 33–46. Springer International Publishing AG 2017, Cham (2017). https://doi.org/10.1007/978-3-319-48523-2 5. García-Mireles, G.A., Moraga, Á., García, F.: Development of maturity models: a systematic literature review. In: IET Seminar Digest, pp. 279–283 (2012). https://doi.org/10.1049/ic. 2012.0036 6. von Wangenheim, C., Hauck, J.C., Salviano, C.F., von Wangenheim, A.: Systematic literature review of software process capability/maturity models. In: International Conference on Software Process Improvement and Capability determination (SPICE), pp. 1–9 (2010) 7. Paulk, M.C.: A History of the capability maturity model for software. Softw. Qual. Profile. 1, 5–19 (2009) 8. Bayona, S., Calvo-Manzano, J.A., San Feliu, T.: Review of critical success factors related to people in software process improvement. In: McCaffery, F., O’Connor, R.V., Messnarz, R. (eds.) EuroSPI 2013. CCIS, vol. 364, pp. 179–189. Springer, Heidelberg (2013). https://doi. org/10.1007/978-3-642-39179-8_16 9. Hameed, M.A., Counsell, S., Swift, S.: A conceptual model for the process of it innovation adoption in organizations. J. Eng. Technol. Manag. 29, 358–390 (2012). https://doi.org/10. 1016/j.jengtecman.2012.03.007

28

F. Alfaro et al.

10. Humphrey, W.S.: Three process perspectives : organizations teams. Syst. Res. 39–72 (2002) 11. Humphrey, W.S., Sweet, W.L.: A method for assessing capability of contractors capability of contractors. Softw. Eng. Inst. 1–46 (1987) 12. Jia, Y., Han, R.: The effective combination of IPD and CMM: IPD-CMM process. In: Proceedings - 2010 2nd WRI World Congress on Software Engineering, WCSE 2010, pp. 193–195. IEEE (2010). https://doi.org/10.1109/WCSE.2010.49 13. SEI: Brief History of CMMI (2009). https://doi.org/10.1055/s-0030-1261874 14. Paulk, M.: Capability maturity model for software. Encycl. Softw. Eng. (1991). https://doi. org/10.1002/0471028959.sof589 15. Bellini, E., Lo Storto, C.: CMM implementation and organizational learning: findings from a case study analysis. In: Portland International Conference on Management of Engineering and Technology, pp. 1256–1271 (2006). https://doi.org/10.1109/PICMET.2006.296694 16. CMMI Product Development Team: CMMI for Systems Engineering/Software Engineering, Version 1.02 (CMMI-SE/SW, V1.02). Software Engineering Institute, Carnegie Mellon University 02, (2000) 17. Chrissis, M.B., Konrad, M., Shrum, S.: CMMI for Development: Guidelines for Process Integration and Product Improvement (2011) 18. SEI: CMMI® for Development, Version 1.3, Pittsburgh (2010). CMU/SEI-2010-TR-033 ESC-TR-2010-033. 19. CMMI-Institute: CMMI Model V2.0 (2018) 20. Matturro, G., Saavedra, J.: Considering people CMM for managing factors that affect software process improvement. IEEE Lat. Am. Trans. 10, 1603–1615 (2012). https://doi.org/10.1109/ TLA.2012.6187605 21. Morales-Aguiar, N., Vega-Zepeda, V.: Factores Humanos y la Mejora de Procesos de Software. Propuesta Inicial de un Catálogo que guíe su Gestión. RISTI - Rev. Ibérica Sist. e Tecnol. Informação. 30–42 (2018). https://doi.org/10.17013/risti.29.30-42 22. Dybå, T.: Factors of Software Process Improvement Success in Small and Large Organizations: An Empirical Study, Popul, English ed., pp. 148–157 (2003) 23. Kitchenham, B., Charters, S.: Guidelines for performing Systematic Literature Reviews in Software Engineering. Tech. report, Ver. 2.3 EBSE Tech. Report. EBSE (2007) 24. Van Solingen, R., Berghout, E.: The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development (1999) 25. Bayona-Oré, S., Calvo-Manzano, J.A., Cuevas, G., San-Feliu, T.: Critical success factors taxonomy for software process deployment. Softw. Qual. J. 22(1), 21–48 (2012). https://doi. org/10.1007/s11219-012-9190-y 26. Dybå, T., Dingsøyr, T.: Empirical studies of agile software development: a systematic review. Inf. Softw. Technol. 50, 833–859 (2008). https://doi.org/10.1016/j.infsof.2008.01.006 27. Rouhani, B.D., Mahrin, M.N., Nikpay, F., Ahmad, R.B., Nikfard, P.: A systematic literature review on enterprise architecture implementation methodologies. Inf. Softw. Technol. 62, 1–20 (2015). https://doi.org/10.1016/j.infsof.2015.01.012 28. Popay, J., et al.: Guidance on the Conduct of Narrative Synthesis in Systematic Reviews A Product from the ESRC Methods Programme Peninsula Medical School, Universities of Exeter and Plymouth, pp. 1–92 (2006)

Proposal to Integrate Security Practices into the ISO/IEC 29110 Standard to Develop Mobile Apps Jezreel Mejía1 , Mirna Muñoz1(B) , Perla Maciel-Gallegos1 , and Yadira Quiñonez2 1 Centro de Investigación en Matemáticas, A.C., Jalisco S/N, Col. Valenciana,

Guanajuato, GJ, México {jmejia,mirna.munoz,perla.maciel}@cimat.mx 2 Universidad Autónoma de Sinaloa, Mazatlán, México [email protected]

Abstract. In recent years, smartphones have accessed to personal information and manage sensitive data, defined as a wide quantity of information encompassing the individual’s personal life. Thus, mobile apps security has become a major area of interest for everyone involved in the development of these. It is important to consider that most of the applications security issues are introduced in the development process. In addition, according to A. Semeney, Founder of Dev Team Space, mobile applications are generally developed by small teams or software development Very Small Entities (VSEs). In this context, there is a need for models, standards or methodologies suitable for VSEs to help in the development of secure mobile applications. In the software development industry, there are few standards suitable for VSEs that integrate security practices. Therefore, the article aims is to present a proposal to integrate security practices into the ISO/IEC 29110 standard for use in the development of secure mobile applications in Very Small Entities. Keywords: ISO/IEC 27001 · ISO/IEC 29110 · CORAS Methodology

1 Introduction Mobile devices have become an essential part of people’s lives. According to data from Newzoo for Statista [1], approximately 3.2 billion people are users of these devices. Smartphones have access to user information, which is put at risk through security holes, provided by the operating system (OS) or by the developers of applications. In the case of the OS, Android has the most vulnerabilities and the most variations of these in its various versions in the market, making it the main victim of exploitation of its vulnerabilities, according to the ESET 2019 report [2]. However, as for the applications, it is well known and affirmed by the author W. Guo [3], that the main security holes are introduced during the development phase, and can be avoided since the initial phases of the project.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 29–40, 2022. https://doi.org/10.1007/978-3-030-89909-7_3

30

J. Mejía et al.

In addition, mobile applications are generally developed by small teams or software development VSEs, which, according to A. Semeney, Founder of Dev Team. Space [4], have 5–9 developing an application for any mobile OS. Examples of these are Swenson He and Blue Whale Apps [5], which were among the Top 10 Web application development companies in the 2020–2021. In this context, there is a need for models, standards or methodologies suitable for VSEs to help in the development of secure mobile applications. In the software development industry, there are few standards suitable for VSEs. The ISO/IEC 29110 standard [6] is being used to certify software development; however, it does not have tasks or activities oriented to good practices of secure software development that can be used in the development of secure mobile applications. Therefore, the article aim article is to describe a proposal to integrate security practices into a standard suitable for VSEs and to enable them to be used in the development of secure mobile apps. The remainder of this paper is organized as follows: Sect. 2 describes briefly the ISO/IEC 29110 standard; Sect. 3 show the proposal describing the performance steps; Sect. 4 presents the security practices integrated into ISO/IEC29110 and finally, Sect. 5 presents a brief conclusion of the paper and indicates the future work.

2 ISO/IEC 29110 Standard The ISO/IEC 29110 is an international standard for Software Engineering that defines Lifecycles Profiles for Very Small Entities (VSEs). This standard was developed to improve the quality of the software product or service. At the same time, they want to improve the organizational performance without excluding the use of diverse methodologies to help in the lifecycle of the software in development. The ISO/IEC 29110 wants to provide a standard that accommodates the needs of a Very Small Entity (VSE). A VSE is defined as an organization, department or project with 5 to 25 people working in it [6–8]. VSEs can be found within large corporations, where they have a meaningful impact, it is crucial for them to be well done and to have an organized process to achieve their goal so as not to end up doing re-working for mistakes made by a bad software engineering [7]. There are “profiles” defined in the ISO/IEC 29110, and they are described as a set of one or more base standards for carrying out a particular function. There are “profiles” defined in the ISO/IEC 29110, and they are described as a set of one or more base standards for carrying out a particular function (See Table 1). The profiles that compose the Generic Profile Group is composed of two processes [6, 8]: 1. The Project Management process (PM): it aims establishing and carrying out the tasks related to a project management in a systematic way, so that the project’s objectives are completed with the expected quality, time and costs. It has 4 activities: project planning, project plan execution, project assessment and control and project closure.

Proposal to Integrate Security Practices

31

Table 1. ISO/IEC 29110 profiles. Profile

Description

Entry

Aimed to the small entities with 6 people developing a project per month.

Basic

The Basic VSE Profile Management and Engineering Guide applies to a Very Small Entity (VSE) (enterprise, organization, department or project up to 25 people) dedicated to software development. The project may fulfil an external or internal contract. The internal contract between the project team and its customer need not be explicit.

Intermediate Aimed to SE developing multiple projects inside the organization context. Advanced

Aimed to SE that want to maintain and grow as an organization of software development independent and competitive.

2. The Software Implementation process (SI): it aims performing in a systematic way, of the activities related to the analysis, design, construction, integration and test, according to the requirements specified of new or modified software products. It has 6 activities: initiation, analysis, design, construction, integration and test and delivery.

3 Selected Framework ISO/IEC 29110 establishes two processes that include Project Management (PM) and Software Implementation (SI) as was described in the previous section. It helps the VSEs to have better control of all phases of the project, and to prevent rework and provide software development activities and tasks in the VSEs. The implementation of this standard has spread both in Mexico and in other countries and its suitability for this type of company has been demonstrated [9]. Therefore, based on the ISO/IEC 29110 standard, a research was carried out to integrate security practices that are most widely implemented for the development of secure software and that can be integrated and used for the development of secure mobile applications. Therefore, in order to select security framework a research question was established: the question was aimed at finding models, standards with a methodology or framework that aids in the development of secure mobile applications. As a result, within the Security frameworks that were identified by the research question are: ISO/IEC 27001 [10, 11] and 27034 [12], Microsoft SDL [13], McGraw’s Seven Touchpoints [14] and the CORAS Methodology [15]. However, they are not oriented to the size of the VSEs that allow them to be easy to implement in their entirety like the ISO/IEC 29110 standard. In addition, 2 frames were identified that can be adapted for use specifically on mobiles: 1) the AWS Mobile SDL [16] which is in a general context of use for mobiles and cloud computing, and 2) Secure Software Development Life Cycle Process (SSDLC) [17], which gives an overview of how this cycle can be followed, but the project is not finished and is still in the process of being created. This leaves us without a real solution of a framework that integrates secure software development practices suitable

32

J. Mejía et al.

for VSEs and that can be applied to the development of secure mobile applications. Each of the selected frameworks is briefly described below. ISO/IEC 27001 The ISO/IEC 27001 [11] is an international standard that is part of the ISO/IEC 27000series, also known as the Information Security Management System (ISMS) Family of Standards, which provides recommendations to establish, implement, operate, monitor, review, maintain and improve an information security managements system [11]. This standard focuses on preserving the integrity, reliability and availability of information and to this end, describes the requirements for an ISMS, these being the establishment, implementation, maintenance and continuous improvement of the ISMS [11]. The ISMS has to be aligned with the organization’s processes and the overall management structure. Therefore, to implement an ISMS, the most important thing is to identify the needs of the organization and based on those needs, to put into action the implementation cycle that the standard provides [1]. The ISO/IEC 27001 standard contains seven implementation stages also known as requirements or clauses. All Clauses can be divided into tasks to be accomplished. ISO/IEC 27034 The ISO/IEC 27034 [12] is an international standard that helps the organizations in integrating security throughout the life cycle of their applications. It offers guidance on information security, using security controls, to those involved in the System Development Life Cycle (or SDLC) of an application system. It addresses all aspects to determine information security requirements to protect information accessed by applications and to prevent unauthorized use and actions in applications [12]. CORAS Methodology CORAS [15] is a model-based methodology for risk assessment that performs a risk analysis using a modeling language with tools specially designed for risk modeling. The initial approach of CORAS was developed within the framework of the CORAS project financed by the European Commission between 2001 and 2003 [15]. It is based on traditional safety analysis techniques, such as HazOp structured brainstorming, Fault Tree Analysis (FTA) and Failure Mode and Effects Analysis (FMEA), combined with system development techniques, such as UML, which is the de facto standard modeling language for information systems [15]. CORAS also takes into account international standards for risk management, such as the Australian/New Zealand standard for risk management, AS/NZS 4360:2004, the ISO/IEC 17799 code of practice for information security management (also now known as ISO/IEC 27002), the ISO/IEC 13335 guidelines for information technology security management and system documentation in the form of a reference model for open distributed processing [15]. CORAS is a risk assessment methodology that works in a defensive manner. It is concerned with protecting what is already there. It provides a modeling language, a method and a tool to carry out the entire risk assessment process.

Proposal to Integrate Security Practices

33

4 Steps to Integrate Security Practices Establishing the ISO/IEC 29110 standard as the base framework to which secure software development practices are integrated and that can be used in mobile application development environments, the following steps were developed to integrate these practices and formalize the proposal from selected security frameworks: 1. Study and analyze the ISO/IEC 29110 standard to identify its nature an explicit structure. The structure of the base framework (ISO/IEC 29110) comprises two processes, each of which is composed of activities that, in turn, are composed of sets of tasks, it is at task level that the processes are described, i.e., the work to be accomplished and the corresponding products. The elements that compose each process are described by the ISO/IEC 29110 as follows: • Name: the process identifier. • Purpose: the general goals and results expected of the effective implementation of the process. • Objectives: the specific goals to ensure the accomplishment of the process purpose. • Activity: a set of cohesive tasks and the first level of process flow decomposition. • Activity Description: the activity description begins with the task summary and is followed by the task descriptions table. The task description doesn’t impose any technique or method to perform it. The selection of the techniques or methods is left to the VSE or project team. • Task: a requirement, a recommended or approved action aimed at contributing to the achievement of one or more processes or objectives and constitutes the second level of decomposition of the process. • Input Product: the product required for execution and the corresponding source (could be another process or an external entity), such as the Customer. • Internal Products: the products generated and consumed by the process. • Output Product: product that is the result of the process and its respective destination. It could be an external entity in another process or project, such as Customer or Organization Management. • Internal Products: the products generated and consumed by the process. The product status may change during the process execution. The source can be another process or an external entity to the project, such as the Customer. • Roles involved: the names and abbreviation of the functions to be performed by project team members. Several roles may be played by a single person and one role may be assumed by several persons. • Incorporation to Project Repository: the list of products to be saved in Project Repository.

34

J. Mejía et al.

2. Establish the explicit element level of the base framework where traceability will be performed. As a result of the analysis of the elements of the processes of the ISO/IEC 29110 standard, the Task element was selected as an element of traceability toward the frameworks that were selected. In addition, the nature of tasks and work products were analyzed, for example: Task element: • Planning tasks: propose or improve this task with security planning aspects. • Requirement task: propose or improve this task with aspects of security requirements. In other words, only propose improvements to the task depending on according to its own nature. Work product (input, internal, output): the starting point of the construction of each work product (input, internal, output) was identified. The possible modification to the work product must be analyzed. In other words, if the modification or improvement is necessary, it should be analyzed where the work product is generated for the first time and the relationship of the associated task or a set of tasks. Hence, the task level is considered the explicit element level of the base framework taking the relation and its relation with the products. 3. Identify and analyze related frameworks of the area of interest. As a result of the research question in the security field described in Sect. 2, the most commonly identified frameworks were the ISO/IEC 27001 and 27034. In addition, the frameworks Microsoft SDL and Seven Touchpoints by McGraw to develop secure mobile app were selected. To complement the identified frameworks, a Risk Methodology was considered because of the importance of the evaluation and mitigation of risk within every framework in the security approach. The CORAS Methodology was chosen because of the availability of the documentation and the easiness in understanding the structural elements for its implementation. 4. Analyze the terminology and concepts that coincide and/or are a priority for the traceability. In the analysis of the terminology of each framework, what stood out was the Lifecycle that each establishes that mostly has the same structure as an average Software Lifecycle, thus making the identification of similarities and differences easier. The similarities were mostly classified by the task description and its related work products in the ISO/IEC 29110 standard. Table 2 shows the traceability. 5. Generate a proposal of security practices towards the ISO/IEC 29110. After an analysis of the base framework and the security frameworks, the security frameworks were integrated into the base framework. The integration of the standard 27001 must be done at clause or requirement level, complementing the already established tasks and products in the base framework. On the other hand, the integration of the ISO 27034 uses the defined Process Application Security Management Process in which the Application Normative Framework is established with the Application Security Controls and the Targeted Level of Trust and integrated as well at purpose and output product level. Also integrated are the Microsoft SDL and the Seven Touchpoints by McGraw, at practice level, and the CORAS Methodology at method level (See Table 2).

Proposal to Integrate Security Practices

35

Table 2. Traceability among selected frameworks. Process

Activity

ISO/IEC 27001

ISO/IEC 27034

Microsoft SDL

Seven touch-points

CORAS

Project Management (PM)

PM.1 Project planning

X

X

X

X

X

PM.2. Project X plan execution

X

PM.3. Project assessment and control

X

PM.4. Project closure Software Implementation (SI)

X

X X

X

X

X

SI.2. Software Requirements Analysis

X

SI.3. Software Architectural and Detailed Design

X

X

SI.4. Software Construction

X

X

SI.5. Software Integration and Tests

X

X

6. Identify common practices to integrate into the base framework and rewrite it. With the ISO/IEC 29110, as the base framework, additional security practices from the ISO/IEC 27001, ISO/IEC 27034, the Microsoft SDL and the Seven Touchpoints by McGraw and the CORAS Methodology to were added. A comparative analysis of the security frameworks was made to identify the common elements that can be integrated into the base framework at the task level. After that, the task was rewritten to add the security practices and the respective output products that are generated by the improvements. 7. Refine the proposed integration, respecting the nature of the elements of the basic framework. Subsequently, from the integration to the base framework, a review must be done to analyze the additions to the task level to ensure that the changes do not change the nature of the task. The next step is to integrate practices for Mobile Security in the form of deployment packages.

36

J. Mejía et al.

8. Improve the proposal through a rigorous analysis of the nature of the explicit elements of both the identified frameworks and the base framework. With the nature of the tasks of the base framework analyzed, security practices that provide specific solutions were considered to improve the proposal. To achieve this improvement, the activities of the first and second steps were taken into consideration to analyze the nature of the explicit element selected from each framework and their own nature, i.e., if the nature of the element of the framework selected refers to planning, these tasks must be integrated into tasks that pertain to planning in the base framework. Following the established consideration, another example would be, If the element of the base framework relates to requirements, tasks related to requirements must be integrated. The same aforementioned analysis was performed for the work products that were linked to the explicit elements selected from the selected frameworks and integrated into the ISO/IEC29110. In others words, if the modification or improvement of the work product is necessary, the associated task or set of tasks should be analyzed, where the work product is generated for the first time. Moreover, the overall nature of the base framework was analyzed. As a result, it was detected that the ISO/IEC 29110 standard does not have an organizational approach. Hence, elements such as practices, phases, touchpoints, among others, that refer to organizational issues from selected frameworks were removed. This, reduced the elements that could be suitable to add to this standard. From this new analysis, a new proposal was created, and practices, input/output products and new involved roles that could be added to each Process of the ISO/IEC 29110 were identified.

5 Security Practices Integrated into ISO/IEC 29110 The security practices integrated to the base framework following the previous steps allowed to improve the ISO/IEC 29110 and can be divided into two overall improvements: 1) the integration of a task-level approach to security, the output products in ISO/IEC 29110, and 2) the deployment packages to integrate security practices at the task level that can be used in the development of secure mobile applications (see Fig. 1). After improving the base framework with the selected security frameworks, the results came as a new proposal from ISO/IEC 29110 part 5 [6]: Management and engineering guide: Generic profile group: Basic profile. In addition, the Deployment Packages describe procedures that can be used directly for the development of secure mobile applications. These are linked from the activities in the improved proposal of the ISO/IEC 29110 Implementation Guide.

Proposal to Integrate Security Practices

37

Fig. 1. General view of the improvements to the ISO/IEC29110.

5.1 Integration a Task-Level The changes done to the task’s levels of the activities in the two processes defined in the Base framework are showed in Figs. 1 into the PM and SI processes. • Project Management Process (PM). Project Planning (PM.1). This activity defines the Project Plan, taking into consideration every deliverable item and the final product that the Client expects. To add a security approach and respect the nature of the tasks, changes were made in PM1.3 task and security practices were added in the form of tasks, e.g., assure the security of the work products, make a list of assets to protect, identify threats, risks and incidents. For the PM.1.5 task, the definition of Resources for both development and security, and the consideration of a list of assets were added. In addition, the PM.1.6 task was complemented with an evaluation of competences of the people in the work team to ensure that the personnel have the skills needed to bring security practices into the entire lifecycle of the project. Output Products included the Record of Competences of personal and a Role Matrix. Finally, in PM.1.9 task, the risk identification must include the information security, the security risks and threats and the response incidents. The generated output product was the Incidents Response of the project. Project Plan Execution (PM.2). In this activity the Project Plan is executed and monitored. A revision meeting with the work team and the customer is conducted. The revision with the work team members in the PM.2.3 task includes a review of status of the risks, which includes the level of risk and likelihood, taken from the security practices, which provide a more detailed status of the risks.

38

J. Mejía et al.

Project Assessment and Control (PM.3). This part of the process evaluates the progress of the Project Plan and compares the actual results with the expected results, to establish actions to correct the problems that could be identified. In PM.3.1 task what can be added to the list of evaluation as regards the security approach is the actual risk against an evaluation of risks. Also, a Final Security Review is proposed to make sure all the security concerns were mitigated, this also can be accomplished with the help of a Security Review from External Auditors foreign to the development team of the Project. As Output Product, an Evaluation of Risks and a Final Security Review Document are generated from the added security practices. • Software Implementation Initiation (SI): Software Requirements Analysis (SI.2). In this activity, the requirements are identified and documented by the work team members, taking into consideration the need established by the client in the Project Plan. Therefore, the security consideration was that the security requirements are considered as part of the Requirement Specification, verifying the testability and correctness of said requirements. Software Architectural and Detailed Design (SI.3). This activity focuses on the Architectural and Detailed Design of the project, which means that from the Requirement Specification the design of the internal and external interfaces is generated and included in the Software Components. The improvements of this activity were located specifically in SI.3.3 task. After the creation of the design, in SI.3.3 a review was introduced. This practice was taken from the Security trends considered in the first draft, then a Security Design Review was generated as an output product. This addition also includes an analysis of architectural risks and their respective assumptions, so that these identified risks can be mitigated and won’t harm the project and the Software Components, and is documented in an Assumptions and possible attacks document. The said assumptions and possible attack document is used in SI.3.4. as an Input Product. 5.2 Deployment Packages for the Mobile Application Development Environment In the ISO/IEC 29110 there are artifacts that help in the implementation of a set of practices named Deployment packages. By implementing a deployment package, VSEs can define a specific approach to reach or demonstrate the scope of the ISO/IEC 29110 Part 5 [6]. For the second part of the improvements to the base framework, regarding to Deployment Packages, the Fig. 1 show the integration of the deployment packages in one activity of the Project Management Process and three in the Software Implementation. However, it does not mean that there must be only four Deployment Packages. For the development of the deployment packages, 2 main aspects were taken into consideration: 1) The activities of Project planning of the Project Management process and Requirements, Design and construction of the Software Implementation process. 2) The considerations of a minimum level of security that a mobile application must have to guarantee its safe use from the point of view of 5 main aspects found in the SLR (see Table 3).

Proposal to Integrate Security Practices

39

Table 3. Top 5 common mobile security. Authentication Authorization Data storage Data access Data transfer

Therefore, deployment packages were developed for Requirements, Design and Build activities, as well as deployment packages for authentication, authorization, data access, data storage, and data transfer. A general view can be found in Fig. 1, which shows the number of deployment packages were introduced.

6 Conclusion and Future Work The contribution of this paper is to present a proposal to integrate Security Practices into the ISO/IEC 29110 Standard to Develop Mobile Apps. According to the proposed steps, first the base framework was selected, considering the area of interest, which is the secure mobile development. Therefore, the ISO/IEC 29110 was chosen because it is oriented to organizations denominated as VSEs, that means that are Very Small Entities, and from the literature it can be associated to the size and type of teams that dedicates to the development of mobile applications; in regards of the security, the security frameworks were identified and integrated to the base framework selected. The First draft of the proposal was defined with the incremental traceability of the security frameworks and the integration of said security practices to the base framework. An analysis to the first draft is done to review the improvements done, doing a check over to the explicit elements modified and corroborate that the said changes does not attempt on the nature of the explicit elements of ISO/IEC29110. After improving the base framework with security considerations, the integration of Deployment packages, defined in the ISO/IEC 29110 as artifacts that helps in the implementation of a set of practices, that complement the integration of the security improvements were developed. Therefore, the integration of security practices into the ISO/IEC 29910 can be divided in two: 1) the integration of a task-level security approach and output products in ISO/IEC 29110, and 2) deployment packages to integrate tasklevel security practices that can be used in the development of secure mobile applications. As future work is to create new formats integrated in the Improvement of the part 5 of the ISO/IEC 29110, to facilitate the implementation of the security practices integrated in the proposal. Also, there are the opportunities of improvement regarding the deployment packages, adding more information and examples to make the implementation easier. Adding to the deployment package, there are also the consideration of introducing explicit procedures to at least the two main Mobile OS (Android® and iOS®) to help in the implementation of good practices.

40

J. Mejía et al.

References 1. Newzoo: Smartphone users 2020. Statista, Statista GmbH (2020). https://www.statista.com/ statistics/330695/number-of-smartphone-users-worldwide/ 2. Giusto Bilic, D.: Semi-annual balance of mobile security 2019. In: WeLiveScurity (2019). https://www.welivesecurity.com/2019/09/05/balance-mobile-security-2019/. Accessed 5 Sept 2019 3. Guo, W.: Management system for secure mobile application development Wei. In: ACM Turing Celebration Conference - China, ACM TURC ‘19, New York (2019) 4. Semeney, A.: DevTeam.Space (2021). https://www.devteam.space/blog/how-many-peopledoes-it-take-to-create-a-successful-mobile-application/ 5. Rose, S.: Top 10 Mobile App Development Companies in 2020–202 (2021). https://medium. com/@scarlett8285/top-10-mobile-app-development-companies-in-2020-d9b4c20f419b 6. ISO/IEC.: ISO/IEC TR 29110-5-1-2 Software engineering — Lifecycle profiles for Very Small Entities (VSEs) — Part 5-1-2: Management and engineering guide: Generic profile group: Basic profile (2011) 7. Robinson, A.D.: Very small entities (VSE); the final systems engineering (SE) frontier. In: Annual IEEE International Systems Conference (SysCon), pp. 1–4 (2018) 8. Laporte, C.Y.: ISO/IEC 29110: Normas y guías de ingeniería de software y sistemas para entidades muy pequeñas. In: Convención Informática. La Habana, Cuba (2016) 9. Asociación Español Normalización y Certificación (AENOR) (2020). https://www.une.org/ normalizacion_documentos/referencia_normas_iso_iec_reg_tecnica.pdf 10. IsecT Ltd.: ISO 27000 Information Security - ISO/IEC 27034. https://www.iso27001security. com/html/27034.html 11. ISO/IEC 27001: Information Technology, Security Techniques, Information Security Management Systems, Requirements (2013) 12. ISO/IEC 27034: Information technology — Security Techniques — Application Security (2011) 13. Microsoft: Security Development Lifecycle. SDL Process Guidance Version 5.2 (2012) 14. McGraw, G.: Software Security: Building Security. Addison-Wesley Professional, Richmond (2006) 15. Lund, M.S., Solhaug, B., Stølen, K.: The CORAS Model-Based Method. SINTEF, Oslo (2006) 16. Amazon: Amazon Web Services (2020). https://aws.amazon.com/es/mobile/mobile-applic ation-development/ 17. Innuy: Secure Software Development Life Cycle Processes (SDLC) (2021). https://www. innuy.com/blog/secure-software-development-life-cycle-processes-sdlc/

Systematic Mapping of Literature on Applicable Patterns in Conceptual Modelling of Information Systems Oscar Carlos Medina1(B) , Manuel Pérez Cota2(B) , Luis Esteban Damiano1(B) , Karen Della Mea1(B) , and Marcelo Martin Marciszack1(B) 1 Universidad Tecnológica Nacional, Facultad Regional Córdoba, Maestro López s/n,

Ciudad Universitaria, Córdoba, Argentina [email protected], {ldamiano,marciszack}@frc.utn.edu.ar, [email protected] 2 Universidad de Vigo, Campus Universitario, s/n, 36310 Vigo, Pontevedra, Spain [email protected]

Abstract. Context: The addition of patterns to the Conceptual Model of Information Systems would allow to optimize the quality level of computer systems. In spite of the fact that there are several studies on the application of patterns in the different phases of software development, since the beginning, these studies have focused on programming rather than analysis and design. Objective: The aim of this study is to carry out a systematic mapping of the field of Software Engineering in an attempt to characterize state-of-the-art patterns that can be applied to Conceptual Modelling, identifying gaps and opportunities for future research. Method: A systematic mapping of existing literature on the topic was performed in order to identify the most important studies on the field. These were then classified and analyzed according to different criteria. We also carried out a subjective assessment highlighting the most important ideas and experiences identified. Results: As a result of systematic mapping, there were found 26 primary studies of interest published between January 1995 and December 2019. In general, these studies are associated with software modelling methodologies which provide empirical evidence of the impact of the use of patterns. Conclusions: In this preliminary study, it was possible to identify the most relevant pattern types that were applied in the Conceptual Modelling of Information Systems. Integrating these types of patterns with existing analysis and design tools is also an important challenge that needs to be addressed in this area. Keywords: Patterns · Conceptual Modelling · Software Engineering · SML · EBSE

1 Introduction In the same way as some information systems base their design on a set of best business practices to solve well-defined issues, such as ERP (Enterprise Resource Planning), it would be desirable to have a set of good practices of analysis and design which could be referenced and used in Conceptual Modelling. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 41–54, 2022. https://doi.org/10.1007/978-3-030-89909-7_4

42

O. C. Medina et al.

The Conceptual Modelling of a information system aims to identify and explain the significant concepts in a problem domain, identifying the attributes and the existing associations between them. Part of the tasks carried out in the Conceptual Modelling are addressed based on the Software Engineering approach [1]. Nowadays, specialists keep on looking for the best way to approach Conceptual Modelling. A possible alternative is to use the concept of patterns in practice recommendations as solutions which are appropriate to the context of business processes, for example, by working with different types of analysis and design patterns that are applicable to a specific problem within a given context which could offer the analyst a clear solution on the given problem. Therefore, we intend to build a Conceptual Model, using the concept of patterns in practice recommendations as solutions, that adapts to the new context by taking advantage of the reusability property of the solution offered by the patterns. Based on this reference framework, the Conceptual Modelling of a new software can reuse typified solutions as patterns. These solutions were typified in previous successful implementations; they solve the same problem, and can be adapted to the context of the model being developed. The patterns not only have the scope of a website template or a list of design suggestions, but are concrete analysis and design solutions that would serve as a guide for the initial construction stage of any system with a proven quality level. In order to be able to develop this proposal, first, it is necessary to know which patterns can be used in the Conceptual modelling of Information Systems. In order to do this, the authors used the research method called Systematic Mapping of Literature (SML). This method is part of the discipline called Evidence-Based Software Engineering (EBSE) that seeks to evaluate and interpret all available empirical evidence related to particular research questions or a phenomenon of interest [2, 3]. An SML is a secondary empirical study which aims to provide a global approach to a topic of interest, the research related to this topic and its published results over a period of time. When there is little evidence on the topic under study or when the topic to be dealt with is too broad, it is recommended to use SML as a starting point, provided it is done with scientific rigor [4]. The SML allows for plotting the evidence of a domain at a high level of granularity. It also enables the identification of groups and evidence gaps that could serve to direct the focus of future systematic reviews, and to detect areas where more primary studies could be conducted. The SML is not only based on empirical evidence. This article details each of the stages of an SML work on patterns that can be applied in the Conceptual Modelling of Information Systems. The SML has three phases: planning, execution and report. The planning phase is detailed in Sect. 2. This section that deals with the planning of the review details the topic foundations, specifies the search strategy used in the study, describes how the data extraction process was designed and explains the synthesis process. The execution and report phase is detailed in Sect. 3. This section explains how the SML was executed and shows the results obtained. Finally, the outcome of the study is analyzed in the conclusions section.

2 Planning The Review The purpose of this study is to determine the state of the art, specifically in the patterns within the Conceptual Model of Information Systems, analyzing existing documents,

Systematic Mapping of Literature on Applicable Patterns

43

thus identifying possible new opportunities for future research. Therefore, the main question that motivates this study is: What is the state of the art of patterns within the Conceptual Model of Information Systems? In this section, we will describe: the foundations, the search strategy, the data extraction process and the synthesis process of this protocol; specifying the research questions that motivate this mapping. 2.1 Foundations When a pattern is applied to Conceptual Modelling fulfills the function of reusing the knowledge and experience of previous systems “encapsulated” in concrete analysis and design solutions, allowing the verification and validation of functional requirements. This concept of a pattern is based on that formulated by the architect Christopher Alexander in his books “A pattern language” [5] and “The Timeless Way of Building” [6]. In his books, he states that “each pattern is a three-part rule, which expresses a relation between a certain context, a problem, and a solution [6]. This idea of a pattern is exploited in software design by Gamma, Helm, Johnson, and Vissides, who published one of the most important work on this subject: “Design Patterns: Elements of Reusable Object-Oriented Software” [7], consistent with the paradigm of Object-Oriented Programming. This work presents a catalogue of twenty-three patterns, all interconnected and classified according to their purpose in three categories, called “Creational”, “Structural” and “Behavioral”, and in two areas of application or scope: “Class Pattern” or “Object Pattern” [7]. After their publication, patterns were incorporated into the different phases of software development creating specific types for each of the phases. The analysis of requirements is the most important phase in the construction of a software, since a correct analysis will depend on the correct implementation of the application [8]. The Software Requirements Specification (SRS) document, is a kind of agreement between the user and the developers in which the former indicates their needs, while the latter simply implements what is stated in document [9]. Using patterns in an initial SRS instance can bring a number of benefits. According to Beck et al. [10] the use of patterns offers the following advantages: • • • •

They help reduce development time. They allow for better communication. They help reduce design errors, showing the essential parts of the design. They are reusable and implement the use of Good Practices.

This research is guided by questions that seek to provide a specific view of the relevant aspects of the patterns used in the Conceptual Modelling of Information Systems. The research questions of this systematic mapping are detailed below: • RQ1: What types of patterns can be applied to the Conceptual Model of Information Systems?

44

O. C. Medina et al.

• RQ2: What types of patterns were used in previous studies on the Conceptual Model of Information Systems? • RQ3: In which real cases were patterns implemented in the Conceptual Modelling of Information Systems? The term, “Pattern Types”, assumes that there is an existing classification of the state of the art of patterns from the approach of Software Engineering. It was found out that there is a general consensus on the types of patterns that are developed by different authors. The ones that stand out are the groupings selected by Pressman and Sommerville. Pressman [9] defines a variety of patterns that include a significant set of applications and cover some aspects of abstractions, which are mentioned below: • Analysis Patterns: These are groups of concepts that together represent a relevant business model within an application domain. • Process Patterns: They allow us to represent process solutions of the problem in the context of software development. • Architectural Patterns: They are used to describe fundamental aspects of the architecture of a software system, identifying subsystems with their responsibilities, and establishing rules for defining relations between them. • Design Patterns: These are related to the development of components and their communication within a system. • Data Patterns: They are used in the modelling of data structures. • Interface design patterns: These are recurring solutions that are used to troubleshoot user interface design problems. These patterns take into account the specific characteristics of end users. • Web Application Patterns: These are used to troubleshoot problems that arise from building web applications using several of the patterns mentioned above. Sommerville [1] also classifies patterns according to specific purposes: objectoriented design patterns [7], patterns for organizational design [11], usability patterns [12], interaction patterns [13], patterns for software configuration management [14] and architectural patterns [15], “POSA series” (Pattern-Oriented Software Architecture) from [16] to [17], among others. In addition, Eriksson and Penker note that each pattern deals with a specific and recurring problem in the design or implementation of a software system [18]. These authors claim that Business Modelling Patterns allow for obtaining the company’s model resources, organizational structures, objectives, processes, and standards. In their work, they present a catalogue of twenty-six business patterns for business modelling represented using UML (Unified Modeling Language). Eriksson and Penker distinguish three types of patterns in relation to the issues they address: Business Patterns, Architectural Patterns and Design Patterns. Business Patterns have the highest level of abstraction which makes it possible to work with business processes. These Business Patterns are classified into three categories:

Systematic Mapping of Literature on Applicable Patterns

45

• Goals or Objectives Patterns • Process Patterns • Resource Patterns and Rules. Another important feature of patterns that model business processes are scenarios. The use of scenarios to model users’ requirements is a strategy set forth by Potts, an authority in the field of pattern modelling, who defines scenarios as “narrative descriptions of interactions between users and proposed systems” [19]. The last type of pattern mentioned completes, prima facie, the list of the main classification of patterns in Software Engineering. It is worth stressing that each question that has been defined for the SML has a specific objective, which, as a whole, contributes to the overall objective of this research. Table 1 below shows the motivation for asking each of the defined questions: Table 1. Motivation behind the research question. Question

Motivation

RQ1

Explore the range of possibilities offered by patterns as reusability models in Conceptual Modelling

RQ2

Inquire the types of patterns used in the Conceptual Modelling of Information Systems phase

RQ3

Learn about the real cases in which patterns have been applied in the Conceptual Model of information systems already implemented

2.2 Search Strategy The systematic mapping of literature was performed using the following indexes and bibliographic databases: • • • • •

Scopus IEEE (IEEE Xplore) ScienceDirect Springer (SpringerLink) ACM (Association for Computing Machinery)

The reason why these indexes and databases were chosen to develop the SML is that these search engines of scientific literature are considered as the most important in the field we work on. Another reason to choose these indexes and databases is that this is a preliminary analysis of the topic we are investigating. SML is carried out using an appropriate methodology to achieve an efficient search strategy. Analysis approaches have been carried out on each database to extract in the most suitable way the documents of interest to validate the use of patterns in the development of Conceptual Models. For

46

O. C. Medina et al.

this reason, through the correct knowledge of the search tools, it was possible to obtain the documents of interest for the subsequent synthesis process. The search was performed taking into account the title only. Therefore, key terms were searched in titles, having in mind the fact that if these key terms are important enough to be included in the title, they signal that a given topic is present in the development of the document. In order to define which papers or scientific reports to include, the following inclusion and exclusion criteria were defined. The inclusion criteria are the ones that follow: 1. Studies carried out from January 1995 to December 2019. 1995 is the starting point for the search due to the fact that it is the year when Fowler’s book on Patterns of Analysis [20] was published. This constitutes the first proposal for incorporating patterns in Conceptual Modelling of software. 2. Studies communicated in English; 3. Studies related to the defined search string; 4. Only the following types of documents are being taken into account: • • • • •

Journal articles, Workshops Conference documents Contributions from industrial/professional workshops Contributions from industrial/professional conferences

The exclusion criteria are the ones that follow: 1. 2. 3. 4. 5. 6.

Documents that do not focus on Software Engineering or Computer Science. Documents that do not focus on patterns in Conceptual Modelling. Documents available in the form of only “abstracts” or PowerPoint presentations. Personal blogs or web pages. Duplications. Documents that lack adequate applicability on the use of patterns and scientific basis.

The terms selected to define the search string are “pattern” and “conceptual model” when they are part of the title. These two terms are also useful to find the terms, “patterns” and “conceptual modelling”, in titles. 2.3 Data Extraction Process A form was designed for data extraction with the aim of facilitating the activity of synthesizing the collected data. The design of the instrument facilitated the classification of works under several categories. The first part is aimed at the metadata of the article (title, author and name of the journal/conference) and the other part is related to the classification of the piece of writing according to the following categories:

Systematic Mapping of Literature on Applicable Patterns

47

1. Year of publication: The year in which the article was published. This field is not related to any RQs but it provides additional results to the study. It is also related to one of the defined inclusion criteria. 2. Pattern Type: The type of pattern referenced by the document, according to the description mentioned above. This data could have the following values: design patterns, creational patterns, structural patterns, behavior, business, processes, rules, objectives, analysis, architecture, scenario and others, including, in this case, the type dealt with in the document. This is related to RQ2. 3. Used in industry: Indicates whether the reference pattern has been used in a real case in the software industry. This category is related to RQ1 and RQ3. 2.4 Synthesis Process For the analysis and synthesis process of the selected pieces of writing, the qualitative and quantitative nature of the research questions was taken into account. This synthesis process was carried out taking into account the following criteria: • Extraction of data, reviewing the articles carefully and in detail observing the existence of patterns which were not included in the proposal. • Systematic coding of data, identifying categories and results across the entire dataset. • Fill in proposed analysis tables and graphs with the encoded data. • Find other topics not analyzed in the questions asked in the research. • Verify the reliability of the synthesis and verify that the questions asked are taken into account in the results obtained.

3 Execution of the SML A systematic search was carried out in the five most recognized electronic library search engines of Software Engineering. A date range, from January 1995 to December 2019, was used for the search. A native search string was created for each search engine. The search strings are seen in Table 2: Table 2. Search strings according to the search engine. Search engine

Search string

Scopus

(TITLE (“conceptual model”) AND TITLE (“pattern”)) AND DOCTYPE (ar OR bk OR cp) AND PUBYEAR > 1995 AND SUBJAREA (comp)

IEEE

((“Document Title”: conceptual model) AND “Document Title”: pattern)) Filters Applied: 1995–2019

ScienceDirect

Year (1995–2019) — Title (“conceptual model” AND “pattern”) - Article type (FLA, CH) (continued)

48

O. C. Medina et al. Table 2. (continued)

Search engine

Search string

Springer

where the title contains “conceptual model” AND pattern Within: Discipline: Computer Science Subdiscipline: Software Engineering Start year 1995, End Year 2019

ACM

((“Document Title”: conceptual model) AND [Publication Title: pattern]) AND [Publication Date: (01/01/1995 TO 04/30/2019)]

When running the search strings in each search engine, a total of sixty-seven (67) documents were found. The documents were read to filter the ones of specific interest for the research. As a result of reading, twenty-six (26) out of sixty-seven (67) documents were selected because they were of interest to the subject of research. The distribution is shown in Table 3 and Fig. 1. Table 3. Documents found and documents of interest per search engine. Search engine

Number of documents found

Scopus

16

7

7

1

IEEE ScienceDirect

Number of documents of interest

6

3

29

11

ACM

9

4

Total

67

26

Springer

Fig. 1. Comparative graph between documents found and documents of interest per search engine.

Systematic Mapping of Literature on Applicable Patterns

49

The documents that were considered texts of interest for this work were temporarily distributed as shown in Fig. 2.

Fig. 2. Temporary distribution of documents of interest.

Based on this research and its subject matter, the search engines performed in different ways when it comes to finding documents of interest. The results are shown in Table 4. The distribution of results indicates that, for example, the search engine, Scopus, selected 43.75% documents of interest out of the original selection derived from the search carried out using the electronic library associated with Scopus. It is noteworthy that, if all searches are considered as a unified set of documents found by all search engines, the average percentage of documents of interest that fulfill the research aim decreases to 38.08%. Table 4. Percentage of documents of interest found in each search engine with respect to the total number of documents filtered using the same search engine. Search engine

Percentage of documents of interest

Scopus

43.75%

IEEE

14.29%

ScienceDirect

50.00%

Springer

37.93%

ACM

44.44%

All search engines

38.08%

50

O. C. Medina et al.

The percentage distribution of the documents of interest found shows the relevance of each search engine with respect to the others. As it can be noticed in Fig. 3 below, the Springer search engine has a relevance of 42.31% of the total:

Fig. 3. Percentage of documents of interest per search engine, based on the total number of documents of interest found among all search engines.

Finally, the analysis of the texts that were found and classified as documents of interest allowed us to identify the pattern types they deal with in their narrative and describe different proposals for use in the Conceptual Modelling of Information Systems (see Fig. 4).

Fig. 4. Number of documents of interest according to the type of pattern they deal with.

Systematic Mapping of Literature on Applicable Patterns

51

The types of patterns that are mentioned in the documents of interest are the following: • • • • • • • • • • • • •

workflow analysis design integration recruitment structural behavior rules anti-patterns processes scenario business architecture

In order to unify criteria and with the intention of interpreting with greater assertiveness the result of the search, each type of pattern listed above should be arranged by Group or Supertype. First, the text about anti-patterns is ruled out since it doesn’t have to do with a specific pattern. Rather, it is an alternative proposal that seeks to help avoid frequent design errors that may converge in a wrong solution for the problem that the information system needs to solve. Second, patterns are assigned as supertypes those pattern types that were effectively called with the same name in the development of the documents, such as the patterns of analysis, design, scenario, business and architecture. Third, the rest of the pattern types are grouped unifying the classification criteria most agreed among the bibliography (as it was mentioned) in the introduction of this work. Structural and behavioral patterns are categories of Design Patterns defined by Gamma et al. [7]. Process Patterns and Rules Patterns, according to Eriksson and Penker [18] are subtypes of Business Patterns. Workflow patterns, as presented by Moscato et al. [21], are a particular case of Process Patterns applied to web services so they are also a subtype of Business Patterns. However, Integration Patterns would be a new subtype of Business Pattern that allow for modelling the interaction between processes in a standardized manner [22] which is an interesting finding. To conclude, the recruitment patterns, as stated by Alamro et al. [23], within the framework of problem-based modelling, are a particular case of Rule Patterns, which are a subtype of Business Patterns, as stated above. The distribution detailed in Table 5 is obtained from the classification of the documents of interest by Group/Patron Supertype:

52

O. C. Medina et al. Table 5. Distribution of documents of interest according to the type of pattern.

Pattern type

Group/pattern supertype

Number of documents

Workflow

Business

1

Analysis

Analysis

3

Design

Design

7

Integration

Business

1

Recruitment

Business

1

Structural

Design

4

Behavior

Design

1

Rules

Business

2

Anti-patterns

N/A

1

Processes

Business

1

Scenario

Scenario

1

Business

Business

1

Architecture

Architecture

2

In this way, as a result of the SML, it can be stated that some of the Pattern Types that can be applied in the Information Systems Conceptual Model are the following: • • • • •

Design Patterns Business Patterns Analysis Patterns Architecture Patterns Scenario Patterns

The abstract of the mapping along with previous publications done by the authors of this work [24–27], constitute an initial approach to the state of the art that underpins the proposal of an analysis model for the definition of patterns in the Conceptual Model of Information Systems. As an appendix to this paper, the list of selected documents of interest and the SML work documents are published on the Research Centre’s website [28].

4 Conclusions As a result of the SML, there were found twenty-six (26) primary studies of interest published between January 1995 and December 2019. It was possible to observe the impact on the use of various types of patterns in the elaboration of the Conceptual Model for the development of efficient software, in different areas depending on the needs of the environment. In this preliminary mapping, there are coincidences in the pattern types that were identified in the classification done prior to the execution of the SML. However,

Systematic Mapping of Literature on Applicable Patterns

53

new pattern types also appeared. This finding would lead to a more detailed review of this bibliography to inquire whether they apply to Conceptual Modelling or to another stage of software development. It was possible to identify the most relevant pattern types that are applied in System Modelling, namely Design, Business, Analysis, Architecture, and Scenario Patterns. Among these, Design Patterns, which are the first patterns defined in Software Engineering, and Business Patterns stand out because they are used for specifying processes in the modelling phase. The subject matter of this study enjoyed higher popularity in 2007, 2011 and 2015 which indicates a fluctuating interest in the problem described herein. As for the search engines, it is noteworthy that 42.31% of the documents found are indexed in the Springer database which is explained by the number of technology publications it has. The integration of patterns with existing analysis and design tools of information systems is also an important challenge that needs to be addressed in this area. In general, these studies are associated with software modelling methodologies, such as UML and others, which provide empirical evidence of the impact of the use of patterns. This field research is quite preliminary and further research efforts would be needed to analyze the impact of this topic. Regarding the search repositories selected to perform the study, it can be stated that some undoubtedly stand out due to the versatility with which a scan can be performed and the process can be repeated with no difference in the outcome. Other search repositories will require less systemic tasks with more manual elaboration and one by one search within results which are a disadvantage to perform heavily delimited searches. In order to develop a complete and updated conceptual framework on this subject, future research should focus on performing a systematic review of the literature encompassing the use of more search engines and types of scientific publications, as well as new conditions for inclusion and exclusion.

References 1. Sommerville I.: Ingeniería de Software (2011) 2. Wohlin, C., Runeson, Fr., Host, M., Ohlsson, M., Regnell, B.: Experimentation in Software Engineering (2012) 3. Genero Bocco, M., Cross-Lemus, M., Piattini, J.A., Velthuis, M.G.: Métodos de investigación en ingeniería de software (2014) 4. Kitchenham, B., Budgen, D., Brereton, Fr.: Using Mapping Studies as the Basis for Further Research - A Participant-Observer Case Study (2011) 5. Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., Angel, S.: A Pattern Language (1977) 6. Alexander, C.: The Timeless Way of Building (1979) 7. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns – Elements of Reusable Object-Oriented Software (1994) 8. Boehm, B., Basili, V.R.: Software defect reduction top 10 list. Computer (2001) 9. Pressman, R.: Ingeniería del software. Un enfoque práctico, en Ingeniería del Software (2005) 10. Beck, K., et al.: Industrial experience with design patterns. In: Proceedings of the 18th International Conference on Software Engineering (1996)

54

O. C. Medina et al.

11. Coplien, J.O., Harrison, N.B.: Organizational patterns of agile software development – Obsah. Incose (2004) 12. Mahemoff, M., Johnston, L.J.: Usability pattern languages: the ‘language’ aspect. In: HumanComputer Interaction: INTERACT’01: IFIP TC, 13 International Conference on HumanComputer Interaction, Tokyo, Japan, 9th−13th July 2001 (2001) 13. Martin, D., Sommerville, I.: Patterns of cooperative interaction: linking ethnomethodology and design. ACM Trans. Comput. Interact. (2004) 14. Berczuk, S.P., Appleton, B.: Software Configuration Management Patterns: Effective Teamwork, Practical Integration (2003) 15. Shaw, M., Garlan, D.: Software Architecture: Perspectives on an Emerging Discipline (1996) 16. Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, P.: Pattern-Oriented Software Architecture: A System of Patterns (1996) 17. Buschmann, F., Henney, K., Schmidt, D.C.: Pattern Oriented Software Architecture Volume 5: On Patterns and Pattern Languages (2007) 18. Eriksson, H.E., Penker, M.: Business Modeling with UML: Business Patterns at Work (2000) 19. Potts, C.: Using Schematic Scenarios to Understand User Needs (1995) 20. Fowler, M.: Analysis Patterns: Reusable Object Models (1995) 21. Moscato, F., Mazzocca, N., Vittorini, V., Di Lorenzo, G., Mosca, P., Magaldi, M.: Workflow pattern analysis in web services orchestration: The BPEL4WS example. In: Yang, L.T., Rana, O.F., Di Martino, B., Dongarra, J. (eds.) HPCC 2005. LNCS, vol. 3726, pp. 395–400. Springer, Heidelberg (2005). https://doi.org/10.1007/11557654_48 22. Grossmann, G., Scherfl, M., Stumptner, M.: Exploiting Semantics of Inter-Process Dependencies to Instantiate Predefined Integration Patterns (2006) 23. Alamro, S., Dogan, H., Phalp, K.: Forming Enterprise Recruitment Pattern Based on ProblemOriented Conceptual Model (2015) 24. Marciszack, M.M., Moreno, J.C., Sánchez, C.E., Medina, O.C., Delgado, A.F., Castro, C.S.: Patrones en la construcción del Modelo Conceptual para sistemas de información (2018) 25. Medina, O.C., Marciszack, M.M., Groppo, M.A.: Aproximación Descriptiva a las Buenas Prácticas de Gobierno Electrónico y a su incorporación en el Modelado Conceptual de Sitios Web Públicos de Argentina (2018) 26. Medina, O.C., Marciszack, M.M., Groppo, M.A.: Un Modelo de Análisis para aplicación de patrones de Buenas Prácticas en el Modelado Conceptual de Gobierno Electrónico (2018) 27. Medina, O.C., Marciszack, M.M., Groppo, M.A.: Proposal for the Patterns Definition Based on Good Practices for the Electronic Government Systems Development (2018) 28. CIDS Centro de Investigación, Desarrollo y Transferencia de Sistemas de Información (2021). http://www.cids.frc.utn.edu.ar. Accessed 1 June 2021

A Brief Review of Vertical Fragmentation Methods Considering Multimedia Databases and Content-Based Queries Aldo Osmar Ortiz-Ballona1 , Lisbeth Rodríguez-Mazahua1(B) , Asdrúbal López-Chau2 , María Antonieta Abud-Figueroa1 , Celia Romero-Torres1 , and Felipe Castro-Medina1 1 Tecnólogico Nacional de México/I.T. Orizaba, Av. Oriente 9 No. 852. Col. Emiliano Zapat,

C.P. 94320, Orizaba, Ver, Mexico {lrodriguezm,cromerot}@orizaba.tecnm.mx, {mabud, dci.fcastro}@ito-depi.edu.mx 2 Universidad Autónoma del Estado de México, Centro universitario UAEM Zumpango, Camino viejo a Jilotzingo continuación calle Rayón, C.P. 55600, Zumpango-Estado de México, Mexico [email protected]

Abstract. Vertical fragmentation is a distributed database design technique widely used in relational databases to reduce query execution costs. This technique has been also applied in multimedia databases to take advantage of its benefits in query optimization. Content-based retrieval is essential in multimedia databases to provide relevant data to the user. This paper presents a review of the literature on vertical fragmentation methods to determine if they consider multimedia data and content-based queries, are easy to implement, are complete (i.e., the paper includes all the information necessary to implement the method), and if they provide a cost model for the evaluation of their results. To meet this objective, we analyzed and classified 37 papers referring to vertical fragmentation and/or content-based image retrieval. Finally, we selected the best method from our comparative analysis and presented a web application architecture for applying vertical fragmentation in multimedia databases to optimize content-based queries. Keywords: Vertical fragmentation · Content-based retrieval · Multimedia database

1 Introduction Finding the right vertical partition scheme for a workload is one of the essential database optimization problems. With proper partitioning, queries and administration tasks can skip unnecessary data, improving performance [1]. Therefore, in the last decade, several vertical fragmentation methods for relational [1–7], and NoSQL [8, 9], databases have been proposed. Fragmentation methods have also been applied in multimedia databases [10, 11] to reduce the cost of the queries. In these databases, content-based retrieval is provided, where concise features vectors of multimedia data are extracted and stored, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 55–68, 2022. https://doi.org/10.1007/978-3-030-89909-7_5

56

A. O. Ortiz-Ballona et al.

then these vectors are used for indexing, retrieval, and comparison during search operations instead of the multimedia objects. Content-based queries are highly adequate for large databases since they allow retrieving multimedia data using features automatically extracted from the data, without the intervention of the user [12]. Content-based image retrieval (CBIR) is a well-known technique that takes an image as a query for obtaining images similar to the query based on visual characteristics such as color, texture, and shape [13]. Since multimedia databases include attributes associated with feature vectors and vertical fragmentation allows reducing the number of irrelevant attributes accessed by the queries, the efficiency of content-based retrieval is considerably increased using this technique. In this paper, we review 37 articles related to vertical fragmentation methods proposed in the last decade to determine if they take into account multimedia data and content-based queries, among other criteria such as the use of a cost model, easy implementation, and completeness. This work has two contributions: 1) A comparative analysis of 37 articles related to vertical fragmentation techniques proposed in the last ten years, and 2) The presentation of an architecture for a web application to implement a vertical fragmentation method for multimedia databases that takes into account contentbased queries. The main contribution of this review regarding other related works [14, 15] is that it focuses on vertical fragmentation, multimedia databases, and content-based retrieval. The rest of this article is organized as follows: Sect. 2 introduces the analysis methodology, Sect. 3 presents the architecture of the web application and, finally, Sect. 4 gives the conclusions and future work.

2 Analysis In the following subsections, an analysis of the state-of-the-art is carried out. The first subsection includes the methodology used for the selection and analysis of each article. The second subsection addresses the classification of articles taking into account the publisher and their year of publication. The third subsection shows an analysis according to the ease of implementation and completeness of the method. In the fourth subsection, an analysis of the content-based image retrieval systems is shown. Finally, the fifth and sixth subsections show an analysis of cost models and types of databases used, respectively. 2.1 Research Methodology An investigation of all the related articles published by the main scientific publishers was carried out according to a methodology for their classification. Figure 1 shows the process used for the selection and analysis of the papers. The first step consisted of a search of the major computer databases with an extensive bibliography of relevant research on vertical fragmentation and/or content-based image retrieval. the digital libraries considered were: 1) ACM digital library, 2) IEEE Xplore Digital Library, 3) Science Direct (Elsevier) and 4) Springer Link.

A Brief Review of Vertical Fragmentation Methods

57

In the second step, an article filter was performed using keywords to select the most relevant papers. The keywords were: 1) Vertical fragmentation or partitioning, 2) Multimedia Database, and 3) Content-Based image retrieval. Publications that met these characteristics were taken into account and works that were not suitable for the study were discarded. The following sentence establishes the criteria considered for the omission of a research paper: • Unpublished working papers, non-peer-reviewed articles, non-English language articles, textbooks, Master and Doctoral dissertations. The selection process resulted in 37 articles, which were analyzed by 6 characteristics in the third step, to check if they complied with them and in what way they did it, for example, the articles that use some method of CBIR (content-based image retrieval) were classified. The properties considered are 1) Vertical fragmentation, 2) Completeness, 3) Ease of implementation, 4) Cost model, 5) CBIR, and 6) Database type. In addition, the publications were also grouped by publisher and year of publication.

Fig. 1. Selection criteria flow diagram

Table 1 shows the record of each article and presents a comparison of all the items found in the digital libraries that take into account the topics of interest. The selected articles were evaluated using a methodology to determine if they have the desired characteristic. Some of this meet most of the qualities [10] and [11], however, [10] does not consider content-based queries and [11] is not oriented towards vertical fragmentation, it proposes a horizontal fragmentation technique.

58

A. O. Ortiz-Ballona et al. Table 1. Comparison of related works.

Article

1

2

Campero Durant et al.[1]

X

X

Mehta et al. [2]

X

Amer et al. [3]

X

Rodriguez and Li [4]

X

X

Amer et al. [5]

X

X

X

Relational

Amossen [6]

X

X

X

OLTP

Amer et al. [7]

X

X

X

Distributed

Chen et al. [8]

X

X

NoSQL

Ho et al. [9]

X

X

X

NoSQL

Rodriguez and Li [10]

X

X

X

X

Multimedia

Rodriguez-Arauz et al. [11]

X

X

X

Rojas Ruiz et al. [16]

X

X

X

X

X

X

Dahal and Joshi [19]

X

Zhan and Zhao [20]

X

Amer and Abdalla [21]

X

X

Abdel Raouf et al. [22]

X

X

Birhanu et al. [24]

X

X

X

Jagannatha et al. [25]

X

Gu et al. [26]

X

Goli and Rouhani Rankoohi [27]

X

Dharavath et al. [28]

X

X

Costa et al. [29]

X

X

Relational

X

Pazos et al. [36]

X

Multimedia

X

Multimedia Distributed Big data Distributed

X

X

Distributed

X

Distributed Distributed X

X

XML Document

X

Distributed

X

Big data

X

Distributed Distributed Big data X

X

X X

Multimedia

X

X X

X

X

Ghorbanian et al. [32]

Huang and Lai [35]

Distributed

X

X

Rahimi et al. [34]

Relational

X

X

Kishore and Rao [33]

6 Distributed

X

Bhardwaj et al. [30]

5

X

Zhao et al. [18]

Amer [31]

4

X

Abdel Raouf et al. [17]

Kim et al. [23]

3

X

Multimedia Distributed

X

Multimedia

X

Multimedia

X

Distributed

X

Distributed

X

Distributed (continued)

A Brief Review of Vertical Fragmentation Methods

59

Table 1. (continued) Article

1

2

Kaur and Laxmi [37]

X

X

Tsuchida et al. [38]

X

Bobrov et al. [39]

X

Buvana et al. [40]

3

4

X

6 Relational Multimedia

X

X

Relational

X

Abdalla and Artoli [41]

5

X

X

X

Multimedia Distributed

1) Vertical fragmentation, 2) Completeness, 3) Ease of implementation, 4) Cost model, 5) CBIR, 6) Database type

2.2 Classification of Articles by Publisher and Year of Publication

Articles quantity

Figure 2 shows the comparison of the number of articles by each publisher: most are concentrated in IEEE; five papers from this editorial meet three characteristics of our interest: [4, 9, 21, 24], and [26], but they do not consider multimedia data; some methods are complete [4–6, 9, 21–23] and/or easy to implement [4, 20, 24, 26], and few have a cost model [5, 6, 8, 9, 20, 21, 24–26]. Regarding ACM, only three articles, [3, 17], and [18], were presented and they take into account relational databases or big data. Springer published the paper that contains most features: vertical fragmentation, completeness, easy implementation, includes a cost model and is oriented to multimedia databases [10], but it does not consider CBIR. With respect to Elsevier, they focus on vertical fragmentation for relational databases [2, 7, 34–36]. Finally, in other editorials, there are approaches for CBIR [16, 30, 32, 33, 40], there is also a paper [11] that meets all the characteristics but it is oriented towards horizontal fragmentation.

20

13

9

3

5

7

ACM

Elsevier

Other

0 IEEE

Springer

Editorial Fig. 2. Number of articles per publisher

Figure 3 shows the papers by year of publication, 2020 was the year with the most publications found, 2017 and 2021, the years in which few papers related to our interest were found. Most of the articles found from the year 2020 use some CBIR method [11, 30, 32, 33]. They have the information necessary for their replication [7, 11, 30, 33, 37]. Also, some articles have a cost model [7, 11, 31]. However, [11] is the only approach that takes into account the partitioning of multimedia databases, but it focuses on horizontal fragmentation which is not the objective of this research.

A. O. Ortiz-Ballona et al.

Articles quantity

60

8 7 6 5 4 3 2 1 0

7 5

5

4

4

2

2

4 2 1

1

2010 2011 2012 2014 2015 2016 2017 2018 2019 2020 2021

Publication year Fig. 3. Number of articles per year

2.3 Completeness and Ease of Implementation

Articles quantity

The graph in Fig. 4 shows a comparison of the articles that present all the information necessary to be replicated, with 24 of 37 articles being analyzed with this characteristic [1, 4–7, 9–11, 16–18, 21–24, 28–30, 33, 36, 37, 39–41]. The rest do not have the necessary information for their implementation, there is a lack of data such as the environment or database management system in which they were developed, algorithm or explanation in some of its steps [2, 3, 8, 19, 20, 25–27, 31, 32, 34, 35, 38].

40

24

13

20 0 Yes

Completeness

No

Fig. 4. Number of articles per completeness

Articles quantity

The graph in Fig. 5 depicts a comparison of the papers that contain an easy implementation, i.e., if they present a clear explanation of the steps of their proposals, including examples and information about the technologies used for their development. 11 out of 37 are considered to have high ease of implementation [2, 4, 10, 11, 16, 18, 20, 26, 34,

40 20

26 11

0 Yes

No

Ease of implementation Fig. 5. Number of articles per ease of implementation

A Brief Review of Vertical Fragmentation Methods

61

36, 39], the rest of the articles do not have the complete information for their implementation or they present a high difficulty for their replication [1, 3, 5–9, 17, 19, 21–23, 25, 27–33, 35, 37, 38, 40, 41]. 2.4 Cost Model

Articles quantity

Figure 6 shows that most methods do not present a cost model, only 21 of the 37 use a cost model, most were published in IEEE with 9 articles [5, 6, 8, 9, 20, 21, 24–26], while Elsevier, ACM, Springer, and other publishers have 4 [7, 34–36], 2 [3, 17], 4 [1, 10, 27, 31], and 2 [11, 41] proposals, respectively. Some cost models consider disk I/O [1, 20, 26, 35], or access to irrelevant attributes [8–10, 24, 27], while other also include the transportation cost [3, 5–7, 11, 17, 21, 25, 31, 34, 36, 41].

21

16

Yes

No

20 0

Cost Model Fig. 6. Number of articles per cost model

2.5 CBIR

Articles quantity

Figure 7 depicts a graph related to the number of articles that take into account some method for content-based retrieval; 7 out of 37 proposed a CBIR method. Although [10] and [11] contain almost all the characteristics of our analysis methodology, only one proposal [11] involves content-based queries, but it is focused on horizontal fragmentation. In [32], a feature selection algorithm is proposed for the classification of hyperspectral images, the similarity index is combined with the k-means clustering algorithm to obtain important bands. The SURF (Speeded Up Robust Feature) descriptor is used in [11, 16, 23]. In [40], the Haralick features are extracted, which is also known as gray level co-occurrence matrix (GLCM) along with local binary pattern (LBP) features and histogram of oriented gradients (HOG). 30

40 20

7

0 Yes

CBIR

No

Fig. 7. Number of articles per content-based image retrieval.

62

A. O. Ortiz-Ballona et al.

2.6 Database Management Systems

Arcles quanty

Figure 8 shows the database management system (DBMS) used by the horizontal fragmentation methods, most articles do not specify it [2–5, 7, 8, 10, 17, 19–25, 27, 28, 30–32, 34–37, 40, 41]. The DBMSs used by the analyzed approaches are PostgreSQL [16, 18, 38, 39], Hadoop [26, 29], HBase [9], H-Store [6], Oracle [1], MongoDB [11], and Corel Paradox [33]. 26

30 20 4

10

2

0

1

1

1

1

1

DBMS Fig. 8. Number of articles per database management systems

2.7 Database Type

Articles quantity

Figure 9 illustrates a graph related to the number of articles where different types of databases were implemented, being the type of distributed database with more articles [2, 3, 7, 17, 19–22, 25, 27, 28, 31, 34–36, 41], 9 proposals were applied in multimedia databases [10, 11, 16, 23, 30, 32, 33, 38, 40], 5 in relational [1, 4, 5, 37, 39], and 7 in other type [6, 8, 9, 18, 24, 26, 29]. 16

20

9

10

5

7

Relational

others

0 Multimedia

Distributed

Types of databases Fig. 9. Number of articles per database type

As a result of this analysis, we conclude that it is necessary to develop a vertical fragmentation method for multimedia databases to reduce the execution cost of contentbased queries. The next section presents the architecture of a web application proposed with this aim.

3 Architecture The architecture of the web application was designed based on the MVC architectural model (Model View Controller) which distributes the system components in a way that

A Brief Review of Vertical Fragmentation Methods

63

facilitates its maintenance and is represented abstractly in Fig. 10. The technologies selected for the development of the web application are the following: Java [42] as a programming language for its robustness, for allowing the creation of web and desktop applications, for having libraries that allow the user to perform common operations and complement the web environment. JSF (Java Server Faces) [43] as a framework for being free, for its ease of project creation and easy development. NetBeans as IDE (Integrated Development Environment) [44] because it offers extensions for development with JSF and because it does not require any payment for its use. UWE (UML-based Web Engineering) [45] for software development methodology, since it provides steps directed to Web applications and their relationship with UML (Unified Modeling Language) diagrams that are considered standard in the OMG (Object management group). Postgres-XL [46] for managing multimedia data, due to its great scalability, power, and robustness; and to store all the information necessary to implement vertical fragmentation, MySQL [47], because it is light and is available on low-resource machines. Lastly, BoofCV [48] was selected because it is an open-source Java library for real-time computer vision that has ease of use and high performance.

Fig. 10. Web application architecture

Figure 10 shows the separation of components that corresponds to the three main parts of the MVC pattern, the model contains the system logic that is represented by the managed beans that have access to the interface components and pass information to model beans, the latter represents the important classes of the domain and use libraries, such as BoofCV for CBIR and JDBC (Java Database Connectivity) to control access to the database management system (MySQL) and manipulate information. The XHTML (eXtensible HyperText Markup Language) files are in the view, where the model is rendered and the interaction with the user is handled; JSF files are used for this purpose, along with style sheets (CSS) to give the user a better display. In the controller part, the Servlet JSF is the link between the model and the view. It is responsible for managing resource requests by accessing the necessary model in the user’s request and selecting the appropriate view to display it.

64

A. O. Ortiz-Ballona et al.

Figure 11 represents the vertical fragmentation workflow that considers contentbased queries. In the first step, the database log file is analyzed, expecting to find contentbased queries. If they are found, then the table is fragmented in the second step separating the key, descriptor (feature vector), and multimedia attribute. Reaching the third step, the table is analyzed to determine if it has atomic attributes (i.e., columns that do not involve descriptors). There are two options: 1) If they do not exist, in the fourth step, it is verified if there are more multimedia attributes (i.e., columns associated with descriptors), if there are, the flow returns to the second step, and if there are none, it continues to the fifth step where the schema is created with the key, attribute, and descriptor. 2) If they exist, in the sixth step, the attribute cost table is created. Afterward, the seventh step consists of creating the schema by assigning attributes to the sites where they are most required. Then, in the eighth step, the fragments are assigned to the sites and the flow is finished. If the table did not receive content-based queries, the flow goes directly to the sixth step.

Fig. 11. The workflow of the vertical fragmentation

4 Conclusion and Future Work Vertical fragmentation is a well-known distributed database design technique for optimizing queries. It also provides an efficient retrieval of multimedia objects in distributed multimedia databases. A review of 37 articles related to vertical fragmentation methods proposed in the last decade was presented in this paper to found a technique that takes into account multimedia data and content-based queries, and at the same time has completeness, easy implementation, and provides a cost model. This review unveils the lack of a technique that meets all these requirements; therefore, we propose to develop a vertical fragmentation method to optimize content-based queries and we presented the architecture and workflow of a web application for this aim. As future work, a web application based on the proposed architecture will be developed. The application will vertically fragment multimedia databases to reduce execution cost and response time for content-based queries. Also, we will modify our proposed method to adapt the vertical fragmentation scheme according to changes in access patterns.

A Brief Review of Vertical Fragmentation Methods

65

Acknowledgments. The authors are very grateful to the Tecnológico Nacional de México (TecNM) for supporting this research work. This work was also sponsored by the Secretaria de Educación Pública (SEP) and by the Consejo Nacional de Ciencia y Tecnología (CONACYT) through the Fondo Sectorial de Investigación para la Educación, grant number A1-S-51808.

References 1. Campero Durand, G., Piriyev, R., Pinnecke, M., Broneske, D., Gurumurthy, B., Saake, G.: Automated vertical partitioning with deep reinforcement learning. In: Welzer, T., et al. (eds.) ADBIS 2019. CCIS, vol. 1064, pp. 126–134. Springer, Cham (2019). https://doi.org/10.1007/ 978-3-030-30278-8_16 2. Mehta, S., Agarwal, P., Shrivastava, P., Barlawala, J.: Differential bond energy algorithm for optimal vertical fragmentation of distributed databases. J. King Saud University – Comput. Inf. Sci., S1319157818302519 (2018). https://doi.org/10.1016/j.jksuci.2018.09.020 3. Amer, A.A., Mohamed, M.H., Al_Asri, K.: On an effective hierarchical clustering based model for data fragmentation and allocation in relational ddbs: review and proposal, In: Proceedings of the 4th ACM International Conference of Computing for Engineering and Sciences on - ICCES’18, Kuala Lumpur, Malaysia, pp. 1–9 (2018). https://doi.org/10.1145/ 3213187.3293604 4. Rodriguez, L., Li, X.: A support-based vertical partitioning method for database design. In: 2011 8th International Conference on Electrical Engineering, Computing Science and Automatic Control, Merida City, Mexico, pp. 1–6 (2011). https://doi.org/10.1109/ICEEE. 2011.6106682 5. Amer, A.A., Mohamed, M.H., Sewisy, A.A., Al Asri, K.: An aggregated similarity based hierarchical clustering technique for relational DDBS design. In: 2018 Fifth International Conference on Parallel, Distributed and Grid Computing (PDGC), Solan Himachal Pradesh, India, pp. 295–299, December 2018. https://doi.org/10.1109/PDGC.2018.8745981 6. Amossen, R.R.: Vertical partitioning of relational OLTP databases using integer programming. In: 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW), Long Beach, CA, USA, 2010, pp. 93–98 (2010). https://doi.org/10.1109/ICDEW.2010.545 2739 7. Amer, A.A., Mohamed, M.H., Al_Asri, K.: ASGOP: an aggregated similarity-based greedyoriented approach for relational DDBSs design. Heliyon 6(1), e03172 (2020). https://doi.org/ 10.1016/j.heliyon.2020.e03172 8. Chen, Z., Yang, S., Zhao, H., Yin, H.: an objective function for dividing class family in NoSQL database. In: 2012 International Conference on Computer Science and Service System, Nanjing, China, pp. 2091–2094 (2012). https://doi.org/10.1109/CSSS.2012.520 9. Ho, L.-Y., Hsieh, M.-J., Wu, J.-J., Liu, P.: Data partition optimization for columnfamily NoSQL databases. In: 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity), Chengdu, China, pp. 668–675 (2015). https://doi. org/10.1109/SmartCity.2015.146 10. Rodriguez, L., Li, X.: A vertical partitioning algorithm for distributed multimedia databases. In: Hameurlain, A., Liddle, S.W., Schewe, K.-D., Zhou, X. (eds.) DEXA 2011. LNCS, vol. 6861, pp. 544–558. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-230912_48 11. Rodríguez-Arauz, M.J., Rodríguez-Mazahua, L., Arrioja-Rodríguez, M.L., Abud-Figueroa, A., Peláez-Camarena, S.G.: Design of a Multimedia Data Management System that Uses Horizontal Fragmentation to Optimize Content-based Queries, 7 (2020)

66

A. O. Ortiz-Ballona et al.

12. Traina, A.J.M., Brinis, S., Pedrosa, G.V., Avalhais, L.P.S., Traina, C.: Querying on large and complex databases by content: challenges on variety and veracity regarding real applications. Inf. Syst. 86, 10–27 (2019). https://doi.org/10.1016/j.is.2019.03.012 13. Tyagi, V.: Content-based image retrieval: an introduction. In: Content-Based Image Retrieval: Ideas, Influences, and Current Trends. Tyagi, V. (ed.) Singapore: Springer, pp. 1–27 (2017). https://doi.org/10.1007/978-981-10-6759-4_1 14. Castro-Medina, F., Rodríguez-Mazahua, L., López-Chau, A., Cervantes, J., Alor-Hernández, G., Machorro-Cano, I.: Application of dynamic fragmentation methods in multimedia databases: a review. Entropy 22(12), Art. no. 12 (2020). https://doi.org/10.3390/e22121352 15. Nashat, D., Amer, A.A.: A comprehensive taxonomy of fragmentation and allocation techniques in distributed database design. ACM Comput. Surv. 51(1), 1–25 (2018). https://doi. org/10.1145/3150223 16. Rojas Ruiz, R., Rodríguez-Mazahua, L., López-Chau, A., Peláez-Camarena, S. G., AbudFigueroa, M. A., Machorro-Cano, I.: A CBIR system for the recognition of agricultural machinery, RCS 147(3), pp. 9–16 (2018). https://doi.org/10.13053/rcs-147-3-1 17. Raouf, A.E.A., Badr, N.L., Tolba, M.F.: An enhanced CRUD for vertical fragmentation allocation and replication over the cloud environment. In: Proceedings of the 10th International Conference on Informatics and Systems - INFOS 2016, Giza, Egypt, pp. 146–152 (2016). https://doi.org/10.1145/2908446.2908480 18. Zhao, W., Cheng, Y., Rusu, F.: Vertical partitioning for query processing over raw data. In: Proceedings of the 27th International Conference on Scientific and Statistical Database Management, La Jolla California, pp. 1–12 (2015). https://doi.org/10.1145/2791347.2791369 19. Dahal, A., Joshi, S.R.: A clustering based vertical fragmentation and allocation of a distributed database. In: 2019 Artificial Intelligence for Transforming Business and Society (AITB), Kathmandu, Nepal, pp. 1–5 (2019). https://doi.org/10.1109/AITB48515.2019.8947444 20. Zhang, S., Zhao, K.: A new method for computation of the node’s estimated_cost of transaction_based on approach in vertical partitioning. In: 2011 International Conference on Internet Technology and Applications, Wuhan, China, pp. 1–3 (2011). https://doi.org/10.1109/ITAP. 2011.6006251 21. Amer, A.A., Abdalla, H.I.: An integrated design scheme for performance optimization in distributed environments. In: International Conference on Education and e-Learning Innovations, Sousse, pp. 1–8 (2012). https://doi.org/10.1109/ICEELI.2012.6360610 22. Abdel Raouf, A.E., Badr, N.L., Tolba, M.F.: An optimized scheme for vertical fragmentation, allocation and replication of a distributed database. In: 2015 IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS), Cairo, pp. 506–513 (2015). https://doi.org/10.1109/IntelCIS.2015.7397268 23. Kim, D., Kim, M., Kim, K., Sung, M., Ro, W.W.: Dynamic load balancing of parallel SURF with vertical partitioning, IEEE Trans. Parallel Distrib. Syst. 26(12), 3358–337 (2015). https:// doi.org/10.1109/TPDS.2014.2372763 24. Birhanu, L., Atnafu, S., Getahun, F.: Native XML document fragmentation model. In: 2010 Sixth International Conference on Signal-Image Technology and Internet Based Systems, Kuala Lumpur, pp. 233–240 (2010). https://doi.org/10.1109/SITIS.2010.47 25. Jagannatha, S., Reddy, S.V.P., Kumar, T.V.S., Kanth, K.R.: Simulation and analysis of performance prediction in Distributed Database design using OO approach. In: 2013 3rd IEEE International Advance Computing Conference (IACC), Ghaziabad, pp. 1324–1329 (2013). https://doi.org/10.1109/IAdCC.2013.6514420 26. Gu, X., Yang, X., Wang, W., Jin, Y., Meng, D.: CHAC: An effective attribute clustering algorithm for large-scale data processing. In: 2012 IEEE Seventh International Conference on Networking, Architecture, and Storage, Xiamen, China, pp. 94–98 (2012). https://doi.org/ 10.1109/NAS.2012.16

A Brief Review of Vertical Fragmentation Methods

67

27. Goli, M., Rouhani Rankoohi, S.M.T.: A new vertical fragmentation algorithm based on ant collective behavior in distributed database systems. Knowl. Inf. Syst. 30(2), 435–455 (2012). https://doi.org/10.1007/s10115-011-0384-6 28. Dharavath, R., Kumar, V., Kumar, C., Kumar, A.: An apriori-based vertical fragmentation technique for heterogeneous distributed database transactions. In: Mohapatra, D.P., Patnaik, S. (eds.) Intelligent Computing, Networking, and Informatics. AISC, vol. 243, pp. 687–695. Springer, New Delhi (2014). https://doi.org/10.1007/978-81-322-1665-0_69 29. Costa, E., Costa, C., Santos, M.Y.: Evaluating partitioning and bucketing strategies for Hivebased Big Data Warehousing systems. J. Big Data 6(1), 1–38 (2019). https://doi.org/10.1186/ s40537-019-0196-1 30. Bhardwaj, S., Pandove, G., Dahiya, P.: A futuristic hybrid image retrieval system based on an effective indexing approach for swift image retrieval. Int. J. Comput. Inf. Syst. Ind. Manage. Appl.12, 1–013 (2020) 31. Amer, A.A.: On K-means clustering-based approach for DDBSs design. J. Big Data 7(1), 1–31 (2020). https://doi.org/10.1186/s40537-020-00306-9 32. Ghorbanian, A., Maghsoudi, Y., Mohammadzadeh, A.: Clustering-based band selection using structural similarity index and entropy for hyperspectral image classification, TS 37(5), 785– 791 (2020). https://doi.org/10.18280/ts.370510 33. Kishore, D., Rao, C.: A Multi-class SVM based content based image retrieval system using hybrid optimization techniques, TS 37(2), 217–226 (2020). https://doi.org/10.18280/ ts.370207 34. Rahimi, H., Parand, F.-A., Riahi, D.: Hierarchical simultaneous vertical fragmentation and allocation using modified bond energy algorithm in distributed databases. Appl. Comput. Inf. 14(2), 127–133 (2018). https://doi.org/10.1016/j.aci.2015.03.001 35. Huang, Y.-F., Lai, C.-J.: Integrating frequent pattern clustering and branch-and-bound approaches for data partitioning, Inf. Sci. 328, 288–301 (2016), https://doi.org/10.1016/j. ins.2015.08.047 36. Pazos, R.A.: Minimizing roundtrip response time in distributed databases with vertical fragmentation. J. Comput. Appl. Math., 9 (2014) 37. Kaur, K., Laxmi, V.: A novel method of data partitioning using genetic algorithm work load driven approach utilizing machine learning. In: Mallick, P.K., Pattnaik, P.K., Panda, A.R., Balas, V.E. (eds.) Cognitive Computing in Human Cognition. LAIS, vol. 17, pp. 49–60. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-48118-6_5 38. Tsuchida, T., Tsuji, T., Higuchi, K.: Implementing vertical splitting for large scale multidimensional datasets and its evaluations. In: Cuzzocrea, A., Dayal, U. (eds.) DaWaK 2011. LNCS, vol. 6862, pp. 208–223. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3642-23544-3_16 39. Bobrov, N., Chernishev, G., Novikov, B.: Workload-independent data-driven vertical partitioning. In: Kirikova, M., et al. (eds.) ADBIS 2017. CCIS, vol. 767, pp. 275–284. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67162-8_27 40. Buvana, M., Muthumayil, K., Jayasankar, T.: Content-Based Image Retrieval based on Hybrid Feature Extraction and Feature Selection Technique Pigeon Inspired based Optimization. Ann. Romanian Soc. Cell Biol. 25(1), 20 (2021) 41. Abdalla, H., Artoli, A.M.: Towards an efficient data fragmentation, allocation, and clustering approach in a distributed environment. Information 10(3), 2 (2019). https://doi.org/10.3390/ info10030112 42. Conozca más sobre la tecnología Java. https://www.java.com/es/about/ 43. JavaServer Faces Technology: Oracle. https://www.oracle.com/java/technologies/javaserve rfaces.html 44. NetBeans IDE – Overview. https://netbeans.org/features/index.html

68

A. O. Ortiz-Ballona et al.

45. Nieves Guerrero, C., Ucán Pech y, J., Menéndez Domínguez, V.: UWE en Sistema de Recomendación de Objetos de Aprendizaje. Aplicando Ingeniería Web: Un Método en Caso de Estudio. Revista Latinoamericana De Ingenieria De Software 2(1), 137–143 (2014) 46. Overview | Postgres-X., https://www.postgres-xl.org/overview/ 47. Deploy Cloud Applications with MySQL Databas., https://www.oracle.com/mysql/ 48. Boofcv.org: BoofCV. https://boofcv.org/index.php?title=Main_Page

A Review of Horizontal Fragmentation Methods Considering Multimedia Data and Dynamic Access Patterns Abraham Castillo-García, Lisbeth Rodríguez-Mazahua(B) , Felipe Castro-Medina, Beatriz A. Olivares-Zepahua, and María A. Abud-Figueroa Division of Research and Postgraduate Studies, Tecnológico Nacional de México, Instituto Tecnológico de Orizaba, Orizaba, Ver., México {lrodriguezm,mabudf}@orizaba.tecnm.mx, {dci.fcastro, bolivares}@ito-depi.edu.mx

Abstract. Horizontal fragmentation methods are widely used in databases to improve query performance. This paper aims to present a systematic literature review on horizontal fragmentation methods applied to databases. Given the importance of multimedia applications today, this review aims to determine whether the methods consider multimedia data to obtain their partitioning schemes; whether they are dynamic, i.e., can adjust their output to access patterns or workload changes; and are based on a cost model focused on reducing query execution cost. We also consider the completeness (i.e., the paper presents all the information needed to implement the method) and implementation easiness. To meet this objective, we analyzed and classified 46 documents on horizontal fragmentation techniques. Subsequently, we selected the best approach from our comparative analysis. Finally, we presented the method workflow and architecture of a web application that implements a dynamic horizontal fragmentation technique for multimedia databases to optimize its performance and information availability. Keywords: Horizontal fragmentation · Multimedia database · Dynamic fragmentation

1 Introduction The invention of new technologies and standardization of industries and devices have produced vast amounts of information requiring systems to manage and manipulate this data [1]. These activities are complex tasks for ordinary servers because they cannot meet high-performance requirements, fast response time, and fault tolerance [2]. Moreover, with the development of cloud computing, more and more applications (e.g., multimedia applications) have to face the challenges of Big Data. Fragmentation or partitioning strategies play a huge role in these applications [3]. Fragmentation is a distributed database design technique widely used in relational and NoSQL databases to improve the performance of the queries. Fragmentation proposals are classified into two main types: horizontal and vertical. The former divides a table into subsets of tuples © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 69–82, 2022. https://doi.org/10.1007/978-3-030-89909-7_6

70

A. Castillo-García et al.

and the latter into subsets of attributes [4]. Horizontal fragmentation methods enable scalability, achieving near-linear acceleration, improving accessibility, and increasing data manageability [5]. For these reasons, several horizontal fragmentation methods have been proposed in the last few years. They also have been adopted in multimedia databases to take advantage of their benefits. However, most of the existing techniques obtain a horizontal partitioning scheme according to a workload, but they do not adapt the scheme when the access patterns change. Multimedia database systems have dynamic characteristics since their workloads are constantly evolving; therefore, static horizontal fragmentation methods are less suitable because these fragments become inadequate through time. The contributions of this research work are twofold: 1) An analysis among 46 papers focused on horizontal fragmentation techniques proposed in the last decades, and 2) The presentation of an architecture for a web application to implement a dynamic horizontal fragmentation technique for multimedia databases. This paper is organized as follows: Sect. 2 defines some relevant concepts; Sect. 3 discusses the analysis methodology defines some relevant concepts; Sect. 4 discusses presents the web application architecture. Finally, Sect. 5 gives conclusions and future work.

2 Theoretical Basis of Fragmentation Fragmentation or Partitioning is the process of splitting relations either through a selection or projection operator, such a process can nest further fragmentation processes [4]. Horizontal fragmentation or (sharding) is the process of dividing a relation using a selection operator in which the selection predicates determine the fragmentation along its tuples (rows of table), there are two versions: (1) primary is performed using predicates that are defined on the relation and (2) derived it is performed through predicates of another relation [4]. Cost model is a set of cost functions to predict the cost of operators (statistics and/or base data) and formulas to evaluate the sizes of intermediate results. The cost is usually in terms of execution time, so a cost function represents the execution time of a query [4].

3 Research Methodology The following subsections provide an in-depth analysis of the state-of-art. The first subsection includes the methodology used and the study of each article. The second subsection deals with the classification of the papers taking into account the publisher and the year of publication. The third subsection shows an analysis focused on completeness and ease of implementation. The fourth subsection addresses the cost models. The fifth subsection indicates the number of articles implementing dynamic horizontal fragmentation, and finally, the sixth subsection presents the types of databases utilized. An investigation of all related articles published by the leading scientific publishers was performed using a methodology to classify them in order to answer the research question: Is there any dynamic horizontal fragmentation method for multimedia databases based on a cost model, complete and easy to implement? Fig. 1 shows the process used to

A Review of Horizontal Fragmentation Methods

71

select and analyze each article, which has three steps. The first step consisted of searching in the most prominent Computer Science databases with an extensive bibliography of relevant research on horizontal fragmentation and improving performance. The digital libraries considered were: 1) ACM digital library, 2) IEEE Xplore Digital Library, 3) Science Direct (Elsevier), and 4) Springer Link. In the second step, a keyword-based search was used to select the most relevant articles. The keywords were: 1) Horizontal fragmentation, 2) Partitioning or Sharding, and 3) Performance. Publications that did not meet these characteristics and papers that were not suitable for the study were discarded. The following sentence establishes the criteria considered for the omission of a research paper: • Unpublished working papers, non-peer-reviewed articles, non-English language articles, textbooks, Master and Doctoral dissertations.

Fig. 1. Selection criteria flow diagram.

The selection process resulted in 46 articles, analyzed by five characteristics in this third step to review if they complied with them and how they did, e.g., the papers were classified into dynamic or static according to if they adapt their fragmentation scheme. The properties considered are 1) Completeness: the paper has all the information needed to implement the method, 2) Ease of implementation: the method is easy to develop or implement, 3) Cost model: the method uses a cost model to determine a horizontal fragmentation scheme, 4) Dynamic: the method is able to adapt the scheme according to access patterns or workload, and 5) Database type: type of database on which the research or method is focused. In addition, the publications were also grouped by publisher and year of publication.

72

A. Castillo-García et al.

Tables 1 and 2 describe the record of the articles and compare all the papers found in the digital libraries that consider the topics of interest. The selected articles were evaluated using a methodology to determine whether they have the desired characteristics. Some of this meet most of the qualities [9, 15, 21, 23, 38], and [41], however, none consider multimedia databases. Table 1. Comparison of related works (1/2). Article

1

2

3

Patil et al. [1]

4 X

Liming et al. [2]

X

Chen et al. [3]

X

Curino et al. [5]

X

5 NoSQL/Relational Distributed

X

X

NoSQL

X

X

OLTP

Ramachandran et al. [6]

X

Guo et al. [7]

X

Kamal et al. [8]

X

Pavlo et al. [9]

X

X

Zamanian et al. [10]

X

X

Castro-Medina et al. [11]

X

X

X

Distributed in the cloud

Rodríguez-Arauz et al. [12]

X

X

X

Multimedia/NoSQL

X

X

OLTP

Quamar et al. [14]

X

X

X

Distributed in the cloud

Abebe et al. [15]

X

X

X

Distributed

Tzoumas et al. [16]

X

Zamanian et al. [17]

X

X

Parallel

Marcus et al. [18]

X

X

X

Distributed

X

X

Distributed in the cloud

X

X

RDF

X

X

OLTP

Taft et al. [13]

Abdalla et al. [19]

X

X

Serafini et al. [21]

X

Goli-Malekabadi et al. [22]

X

X

Distributed in the cloud

X

OLTP

X

OLTP Distributed

X X

X X

Peng et al. [20]

Abdalla et al. [23]

Distributed

X

Relational

X X

Wedashwara et al. [24]

X

Multimedia/NoSQL X

X

Distributed

X

X

Distributed in the cloud

X

Distributed

Amer et al. [25]

X

Lim et al. [26]

X

X X

Multimedia/NoSQL

X

X

NoSQL

Wu et al. [27] Oonhawat et al. [28]

X

Distributed in the cloud

(continued)

A Review of Horizontal Fragmentation Methods

73

Table 1. (continued) Article

1

2

Sauer et al. [29]

3

4

X

Fasolin et al. [30]

X

Lwin et al. [31]

X

5 NoSQL

X

Multimedia X

X

Distributed

1) Completeness, 2) Ease of implementation, 3) Cost model, 4) Dynamic, 5) Database type

Table 2. Comparison of related works (2/2). Article

1

2

3

Kumar et al. [32] Fetai et al. [33] Herrmann et al. [34]

X X

Khan et al. [35] Baron et al. [36]

4

5

X

Distributed

X

Distributed in the cloud

X

X

Distributed

X

X

Relational

X

X

Elghamrawy et al. [37]

X

X

Hauglid et al. [38]

X

X

X

Relational NoSQL

X

Teng et al. [39]

X

Elghamrawy et al. [40]

X

X

X

Distributed In the cloud NoSQL Relational

Abdel et al. [41]

X

X

X

Rodríguez-Mazahua et al. [42]

X

X

X

Feinerer et al. [43]

X

X

Relational

Salmi et al. [44]

X

X

Relational

Olma et al. [45]

X

X

Relational

Liroz-Gistau et al. [46]

X

X

X

Relational

Liroz-Gistau et al. [47]

X

X

X

Relational

X

X

Distributed in the cloud Multimedia

1) Completeness, 2) Ease of implementation, 3) Cost model, 4) Dynamic, 5) Database type

3.1 Classification of Articles by Publisher and Year of Publication Figure 2 shows a comparison of the number of publications by editorial: 16 of these are concentrated in IEEE, where most of papers have a complete description of the technique, ease of implementation, are based on a cost model, and are oriented to static fragmentation (e.g., [11, 26, 34]). In ACM, methods capable of adapting a scheme and simple to execute were presented in great frequency [3, 5, 9, 13–15, 17, 21]. Almost all Elsevier strategies are simple to implement [22–24]. At Springer, the focus was on easy-to-implement and complete procedures [37, 38, 41–44, 46, 47]. Finally, techniques from other publishers used cost models and have implementation easiness [12, 35, 36].

74

A. Castillo-García et al. 10

8

7

6 5

3

3

6

5

3

3

2

0

0 2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

Fig. 2. Number of articles per publisher.

The number of articles by year of publication is plotted in Fig. 3. The graph shows that the highest number of publications is between the years 2014 to 2020. In the year 2020, the papers were mainly focused on horizontal fragmentation methods with easiness of implementation and the use of cost models [11, 12, 15]. 20

16

12

11

10 0

4 ACM

IEEE

Elsevier

3 Springer

Others

Fig. 3. Number of articles per year.

3.2 Completeness and Ease of Implementation The graph in Fig. 4 shows a comparison of articles that present easiness of implementation, considering the specific knowledge to implement techniques of each publication; there 34 out of 46 with this characteristic. Most of these are easy to implement because they are complete, i.e., they show all the process and information to replicate them. The rest of the documents do not have the necessary data to replicate. Figure 5 shows the number of publications that offer completeness in their fragmentation strategy. 40

34

20

12

0 Yes

No

Fig. 4. Number of articles per ease of implementation.

A Review of Horizontal Fragmentation Methods 40

32 14

20 0

75

Yes

No

Fig. 5. Number of articles per completeness.

3.3 Cost Model Figure 6 shows a graph relating the number of articles that take into account a cost model; 23 of 46 methods used a cost model, [2, 9, 15, 16, 18] and [21] from ACM; [11, 19, 20, 26, 31, 33] and [34] from IEEE; [38, 40–42] and [45] from Springer; [23] and [25] from Elsevier; and [12, 35] and [36] from other publishers. Most of the approaches with cost model focus on reducing the access to non-relevant data [2, 9, 11, 12, 15, 18, 20–22, 25, 26, 31, 33, 35, 36, 38, 40–42] and disk I/O [16, 19, 34, 45], which aim to optimize the execution of queries in database management systems.

30

23

23

Yes

No

20 10 0

Fig. 6. Number of articles per cost model.

3.4 Dynamic Horizontal Fragmentation Figure 7 shows a comparison of the number of papers that consider a dynamic horizontal fragmentation method [1, 3, 5, 7–9, 13–15, 17–21, 23, 25, 31–33, 36, 38, 41, 45–47] and those that do not [2, 6, 10–12, 16, 22, 24, 26–30, 34, 35, 37, 39, 40, 42–44]. From the graph, it is observed that dynamic partitioning strategies are taking a great interest, due to the need to automate the implementation of methods that adapt the horizontal fragmentation scheme (HFS) of the database according to changes in access pattern or workload. However, there are articles that still include static fragmentations (i.e., they implement a HFS that does not consider information changes).

76

A. Castillo-García et al. 25

26 24

21

22 20 18

Yes

No

Fig. 7. Number of articles per dynamic fragmentation.

3.5 Database Type Figure 8 compares the types of databases used in each work. The graph shows that most of the methods were applied in: Distributed [2, 6, 10, 15, 18, 23, 24, 31, 32, 34], Cloud Distributed [7, 11, 14, 19, 25, 26, 33, 38, 41] and Relational [16, 35, 36, 40, 43–47] databases. Similarly, there is a tendency to use horizontal fragmentation techniques in NoSQL [1, 3, 28, 29, 37, 39] and Multimedia [12, 22, 27, 30, 42] databases. However, all the strategies that consider multimedia data implement static fragmentation, and only the publications [12] and [42] take into account a cost model to determine the HFS to be used. 15 10

10

9 5

5 0

Distributed

Distributed in the cloud

Mulmedia

6

NoSQL

9 5

OLTP

2 Relaonal

Others

Fig. 8. Number of articles per type of database.

Figure 9 shows the database management systems (DBMS) used in the articles, it can be seen that most of the publications do not mention the DBMS utilized, however, most of the proposed strategies were implemented in MongoDB [1, 12, 22, 27–29, 39], PostgreSQL [6, 14, 16, 18, 34, 45] and MySQL [2, 5, 11, 17]. Of the methods proposed for multimedia content management systems, three out of five used MongoDB [12, 22, 27] and only [30] utilized Oracle.

A Review of Horizontal Fragmentation Methods 30

77

23

20 10

7

3

4

3

MySQL

Oracle

6

0 H-Store

MongoDB

PostgreSQL

Others

Fig. 9. Number articles per database management system.

4 Architecture The previous analysis shows that is a lack of dynamic horizontal fragmentation methods for multimedia databases. Due to the size of multimedia content, and its dynamic behavior, i.e., constantly changing workload, it is desirable to adapt the fragmentation schema to keep the response times of user queries low and decrease the cost of data transmission. Therefore, a partitioning method is proposed that adapts the scheme according to the access patterns; it will be easy to implement through a web application and Jar application (Permanent log analysis, presented in Fig. 10) that will constantly adapt the scheme, it will use an alphanumeric multimedia cost model, which focuses on multimedia content for the databases that contemplate such content, and it will be complete by presenting the entire architecture of the application and the correct documentation of the Jar application. In addition, the architecture of a web application to implement the method is presented. Figure 10 shows the process of the proposed horizontal fragmentation method that consists of nine steps. It starts with the configuration of the database connection. In the second step, the operations and performance thresholds are defined. In step three, the database log is loaded for the analysis of all operations performed. From this, in step 4, the operations cost matrix is created, based on a multimedia cost model. Then, in step 5, the attribute precedence table (ALP) [11, 35, 41] is built. In step 6, an initial schema is created for the database to fragment. In step 7, the schema is applied. In step 8, the fragmentation watchman (FW) is generated, in this step, the database administrator (DBA) can finish the process leaving the static part of the methodology. To implement the dynamic fragmentation, it is necessary to do step 9, which consists of a permanent log analysis, through the execution of the FW; which internally a) will be evaluating its execution (or until the DBA believes necessary), b) will verify that the current operations threshold in the schema is not exceeded, if this is exceeded, then c) starts the analysis of the costs of current schema, and will compare it with the performance threshold, if this is exceeded, d) will fragment again the database and assign the fragments, then e) will create fragmentation watchmen in the secondary nodes, this FW will be executing all these operations until the DBA believes necessary. The web application architecture is based on the MVC (Model View Controller) pattern that distributes the components to allow better maintenance. Figure 10 shows the abstract form of the system structure. The following technologies were selected to

78

A. Castillo-García et al.

Fig. 10. The workflow of dynamic horizontal fragmentation.

develop this proposal: Java [48], as a programming language, since it is compiled language, the execution of the source code is faster being suitable for applications that make multiple operations and during compilation, a large number of checks are made before it can be executed, making the execution more reliable. Java has a more detailed and precise syntax, being more suitable for projects of great complexity, it is multiplatform, has a large library to execute standard operations, facilitating the construction of large-scale applications. JSF (Java Server Faces) [49] is the chosen framework due to its ease of development, program creation, and accessibility. NetBeans for IDE (Integrated Development Environment) [50] because it is free, has a wide range of extensions to improve and integrate with JSF. UWE (UML-based Web Engineering) [51] for software development methodology, as it provides concrete steps aimed at Web applications and uses UML (Unified Modeling Language) diagrams that are considered standard in the OMG (Object Management Group). MongoDB [52] for multimedia data management, for its high availability, scalability, and integrated geographic distribution features that can be very helpful, and finally MySQL [53] to store all necessary database information (e.g., user, database name, password) and configuration for dynamic horizontal fragmentation (e.g., selected table to fragment, predicates, performance threshold and operations threshold), thanks to its high performance and lightness for computers with scarce resources.

Fig. 11. System architecture.

A Review of Horizontal Fragmentation Methods

79

As shown in Fig. 11, the view contains the pages that interact with the user to request information (e.g., database name, user, password, database address, table to be fragment, as well as the initial configuration of the fragmentation method) about the databases and access them. The JSF Managed Beans manage the flow of data that are in the controller. In the model, the business rules are defined, which in this particular case will be the application of dynamic horizontal fragmentation as presented in Fig. 10, in addition to contemplating the cost model, and the generation of a fragmentation vigilant for a given multimedia database. As can be seen, the structure and organization proposed by MVC provide an excellent union between the components, and the changes are evident to parties directly involved.

5 Conclusion and Future Work In this paper, 46 horizontal fragmentation methods were analyzed to discover if one of them considers multimedia data, adapts the horizontal fragmentation scheme to the changes in access patterns, has a cost model, as well as easy implementation and completeness. From our comparative analysis, we obtained only five works that contemplate multimedia content [12, 22, 27, 30, 42]. However, they implement static partitioning. Therefore, we conclude that there is a lack of dynamic horizontal fragmentation techniques for multimedia databases. Therefore, we proposed a web application architecture to implement a strategy that adapts the horizontal fragmentation schemes of multimedia databases through access patterns. Furthermore, we classified the papers by the publisher, year of publication, completeness, ease of implementation, use of cost models, static or dynamic fragmentation, and database type. As future work, a web application based on the proposed architecture will be developed. The application will fragment a multimedia database dynamically to reduce the execution cost and response time of the queries. Acknowledgments. The authors are very grateful to the Tecnológico Nacional de México (TecNM) for supporting this research work. This work was also sponsored by the Secretaria de Educación Pública (SEP) and by the Consejo Nacional de Ciencia y Tecnología (CONACYT) through the Fondo Sectorial de Investigación para la Educación, grant number A1-S-51808.

References 1. Patil, M.M., Hanni, A., Tejeshwar, C.H., Patil, P.: A qualitative analysis of the performance of MongoDB vs MySQL database based on insertion and retriewal operations using a web/android application to explore load balancing — Sharding in MongoDB and its advantages. In: 2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) (2017) 2. Liming, D., Weindong, L., Jie, S.: coexistence of multiple partition plan based physical database design. In: Proceedings of the 5th International Conference on Communications and Broadband Networking, New York, NY, USA (2017)

80

A. Castillo-García et al.

3. Chen, Z., et al.: The data partition strategy based on hybrid range consistent hash in NoSQL database. In: Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, New York, NY, USA (2013) 4. Özsu, M.T., Valduriez, P.: Principles of Distributed Database Systems, Springer International Publishing (2020) 5. Curino, C., Jones, E., Zhang, Y., Madden, S.: Schism: a workload-driven approach to database replication and partitioning. Proc. VLDB Endowment. 3, 48–57 (2010) 6. Ramachandran, R., Nair, D.P., Jasmi, J.: A horizontal fragmentation method based on data semantics. In: 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC) (2016) 7. Guo, M., Kang, H.: The implementation of database partitioning based on streaming framework. In: 2016 13th Web Information Systems and Applications Conference (WISA) (2016) 8. Kamal, J.M.M., Murshed, M., Buyya, R.: Workload-aware incremental repartitioning of shared-nothing distributed databases for scalable cloud applications. In: 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing (2014) 9. Pavlo, A., Curino, C., Zdonik, S.: Skew-aware automatic database partitioning in sharednothing, parallel OLTP systems. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, New York, NY, USA (2012) 10. Zamanian, E., Shun, J., Binnig, C., Kraska, T.: Chiller: contention-centric transaction execution and data partitioning for modern networks. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, New York, NY, USA (2020) 11. Castro-Medina, F., Rodriguez-Mazahua, L., López-Chau, A., Abud-Figueroa, M.A., AlorHernández, G.: Fragment: a web application for database fragmentation. allocation and replication over a cloud environment. IEEE Latin Am. Trans. 6, 1126–1134 (2020) 12. Rodríguez Arauz, M.J., Rodriguez-Mazahua, L., Arrioja-Rodríguez, M.L., Abud-Figueroa, M.A., Peláez-Camarena, S.G., Martínez-Méndez, L.d.C.: Design of a multimedia data management system that uses horizontal fragmentation to optimize content-based queries. In: IMMM 2020, The Tenth International Conference on Advances in Information Mining and Management (2020) 13. Taft, R., et al.: E-store: fine-grained elastic partitioning for distributed transaction processing systems. Proceedings of the VLDB Endowment. 3, 24–256 (2014) 14. Quamar, A., Kumar, K.A., Deshpande, A.: Sword: scalable workload-aware data placement for transactional workloads. In: Proceedings of the 16th International Conference on Extending Database Technology (2013) 15. Abebe, M., Glasbergen, B., Daudjee, K.: MorphoSys: automatic physical design metamorphosis for distributed database systems. Proc. VLDB Endowment. 13, 3573–3587 (2020) 16. Tzoumas, K., Deshpande, A., Jensen, C.S.: Sharing-aware horizontal partitioning for exploiting correlations during query processing. In:Proceedings of the VLDB Endowment. 1, 542--553 (2010). 17. Zamanian, E., Binnig, C., Salama, A.: Locality-aware partitioning in parallel database systems. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, New York, NY, USA (2015) 18. Marcus, R., Papaemmanouil, O., Semenova, S., Garber, S.: NashDB: an End-to-End economic method for elastic database fragmentation, replication, and provisioning. In: Proceedings of the 2018 International Conference on Management of Data, New York, NY, USA (2018) 19. Abdalla, H.I., Amer, A.A.: Dynamic horizontal fragmentation, replication and allocation model in DDBSs. In: 2012 International Conference on Information Technology and e-Services (2012)

A Review of Horizontal Fragmentation Methods

81

20. Peng, P., Zou, L., Chen, L., Zhao, D.: Adaptive distributed RDF graph fragmentation and allocation based on query workload. IEEE Trans. Knowl. Data Eng. 4, 670–685 (2019) 21. Serafini, M., Taft, R., Elmore, A.J., Pavlo, A., Aboulnaga, A., Stonebraker, M.: Clay: finegrained adaptive partitioning for general database schemas. Proc. VLDB Endowment 4, 445– 456 (2016) 22. Goli-Malekabadi, Z., Sargolzaei-Javan, M., Akbari, M.K.: An effective model for store and retrieve big health data in cloud computing. Comput. Methods Programs Biomed. 1, 75–82 (2016) 23. Abdalla, H.I.: A synchronized design technique for efficient data distribution. Comput. Hum. Behav. 1, 427–435 (2014) 24. Wedashwara, W., Mabu, S., Obayashi, M., Masanao, T.: Combination of genetic network programming and knapsack problem to support record clustering on distributed databases. Expert Syst. Appl. 46, 15–23 (2016) 25. Amer, A.A., Sewisy, A.A., Elgendy, T.M.A.: An optimized approach for simultaneous horizontal data fragmentation and allocation in Distributed Database Systems (DDBSs). Heliyon 3 (2017) 26. Lim, L.: Elastic data partitioning for cloud-based SQL processing systems. In: 2013 IEEE International Conference on Big Data (2013) 27. Wu, Q., Chen, C., Jiang, Y.: Multi-source heterogeneous Hakka culture heritage data management based on MongoDB. In: 2016 Fifth International Conference on Agro-Geoinformatics (Agro-Geoinformatics) (2016) 28. Oonhawat, B., Nupairoj, N.: Hotspot management strategy for real-time log data in MongoDB. In: 2017 19th International Conference on Advanced Communication Technology (ICACT) (2017) 29. Sauer, B., Hao, W.: Horizontal cloud database partitioning with data mining techniques In: 2015 12th Annual IEEE Consumer Communications and Networking Conference (CCNC) (2015) 30. Fasolin, K., et al.: Efficient execution of conjunctive complex queries on big multimedia databases. In: 2013 IEEE International Symposium on Multimedia (2013) 31. Lwin, N.K.Z., Naing, T.M.: Non-redundant dynamic fragment allocation with horizontal partition in distributed database system. In: 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS) (2018) 32. Kumar, R., Gupta, N.: An extended approach to Non-Replicated dynamic fragment allocation in distributed database systems. In: 2014 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT) (2014) 33. Fetai, I., Murezzan, D., Schuldt, H.: Workload-driven adaptive data partitioning and distribution — The Cumulus approach. In: 2015 IEEE International Conference on Big Data (Big Data) (2015) 34. Herrmann, K., Voigt, H., Lehner, W.: Cinderella — Adaptive online partitioning of irregularly structured data. In: 2014 IEEE 30th International Conference on Data Engineering Workshops (2014) 35. Khan, S.I.: Efficient partitioning of large databases without query statistics. Database Syst. J. 7, 34–53 (2016) 36. Baron, C., Iacob, N.M.: A new dynamic data fragmentation and replication model in DDBMSs. Cost Functions. Knowl. Horiz. - Econ. 6, 158–161 (2014) 37. Elghamrawy, S.M.: An adaptive load-balanced partitioning module in cassandra using rendezvous hashing. In: Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2016, Cham (2016) 38. Hauglid, J.O., Ryeng, N.H., Nørvåg, K.: DYFRAM: dynamic fragmentation and replica management in distributed database systems. Distrib. Parallel Databases. 1, 157–185 (2010)

82

A. Castillo-García et al.

39. Teng, D., Kong, J., Wang, F.: Scalable and flexible management of medical image big data. Distrib. Parallel Databases 37(2), 235–250 (2018). https://doi.org/10.1007/s10619-0187230-8 40. Elghamrawy, S.M., Hassanien, A.E.: A partitioning framework for Cassandra NoSQL database using Rendezvous hashing. J. Supercomput. 73(10), 4444–4465 (2017). https:// doi.org/10.1007/s11227-017-2027-5 41. Abdel Raouf, A.E., Badr, N.L., Tolba, M.F.: Distributed Database System (DSS) design over a cloud environment. In: Hassanien, A.E., Fouad, M.M., Manaf, A.A., Zamani, M., Ahmad, R., Kacprzyk, J. (eds.) Multimedia Forensics and Security. ISRL, vol. 115, pp. 97–116. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-44270-9_5 42. Rodríguez-Mazahua, L., Alor-Hernández, G., Abud-Figueroa, M.A., Peláez-Camarena, S.G.: Horizontal partitioning of multimedia databases using hierarchical agglomerative clustering. In: Gelbukh, A., Espinoza, F.C., Galicia-Haro, S.N. (eds.) MICAI 2014. LNCS (LNAI), vol. 8857, pp. 296–309. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13650-9_27 43. Feinerer, I., Franconi, E., Guagliardo, P.: Lossless horizontal decomposition with domain constraints on interpreted attributes. In: Gottlob, G., Grasso, G., Olteanu, D., Schallhart, C. (eds.) BNCOD 2013. LNCS, vol. 7968, pp. 77–91. Springer, Heidelberg (2013). https://doi. org/10.1007/978-3-642-39467-6_10 44. Salmi, C., Chaabani, M., Mezghiche, M.: A formalized procedure for database horizontal fragmentation in Isabelle/HOL proof assistant. In: Model and Data Engineering, Cham (2018) 45. Olma, M., Karpathiotakis, M., Alagiannis, I., Athanassoulis, M., Ailamaki, A.: Adaptive partitioning and indexing for in situ query processing. VLDB J. 29(1), 569–591 (2019). https://doi.org/10.1007/s00778-019-00580-x 46. Liroz-Gistau, M., Akbarinia, R., Pacitti, E., Porto, F., Valduriez, P.: Dynamic workload-based partitioning algorithms for continuously growing databases. In: Hameurlain, A., Küng, J., Wagner, R. (eds.) Transactions on Large-Scale Data- and Knowledge-Centered Systems XII. LNCS, vol. 8320, pp. 105–128. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3642-45315-1_5 47. Liroz-Gistau, M., Akbarinia, R., Pacitti, E., Porto, F., Valduriez, P.: Dynamic workload-based partitioning for large-scale databases. In: Liddle, S.W., Schewe, K.-D., Tjoa, A.M., Zhou, X. (eds.) DEXA 2012. LNCS, vol. 7447, pp. 183–190. Springer, Heidelberg (2012). https://doi. org/10.1007/978-3-642-32597-7_16 48. Conozca más sobre la tecnología Java. https://www.java.com/es/about/ 49. JavaServer Faces Technology. https://www.oracle.com/java/technologies/javaserverfaces. html 50. NetBeans IDE – Overview. https://netbeans.org/features/index.html 51. Nieves Guerrero, C., Ucán Pech, J., Menéndez Domínguez, V.: UWE en Sistema de Recomendación de Objetos de Aprendizaje. Aplicando Ingeniería Web: Un Método en Caso de Estudio. Revista Latinoamericana de Ingenieria de Software, 2, 137–143 (2014) 52. What Is MongoDB., https://www.mongodb.com/what-is-mongodb 53. Deploy Cloud Applications with MySQL Database. https://www.oracle.com/mysql/

The State-of-the-Art of Business Intelligence and Data Mining in the Context of Grid and Utility Computing: A PRISMA Systematic Review Ernani Damasceno1,2,3(B) , Ana Azevedo2 , and Manuel Perez-Cota3 1 Department of Computing, Federal Institute of Triangulo Mineiro, Paracatu, Brazil

[email protected]

2 CEOS.PP, ISCAP, Polytechnic of Porto, Porto, Portugal

[email protected]

3 Higher School of Computer Science, Vigo University, Vigo, Spain

[email protected]

Abstract. The development of scientific knowledge is fundamental for the discovery of new knowledge. However, it is necessary to follow certain criteria in the rigor of science to verify what has already been studied to support new knowledge. The systematic review has the attribution, methodically, to guide new directions in the progress of scientific research. Accordingly, this study aims to identify the state-of-the-art of Data Mining and Business Intelligence solutions in the context of Grid and Utility Computing. The systematic review was carried out using the PRISMA method, which was followed in the progress of the study. Thus, it is expected to obtain results for the continuous evolution of science within the scope of the subjects. Keywords: PRISMA · Systematic literature review · SLR · Business intelligence · BI · Data mining · DM · Grid computing · GC · Utility computing · UC

1 Introduction The adoption of new concepts can contribute significantly to the scope and depth of scientific development. Moreover, it is relevant to employ an integrative literature review approach to generate new knowledge on the topic through the process of review, critique, and synthesis of the literature under research [1]. However, when searching for studies in a specific field, it is common to find numerous publications on separate subjects. However, when searching for correlated subjects, very few results are found, and in certain situations, not even a single publication is found. Clarifying that, when the solutions of two or more subjects, such as Business Intelligence in the context of Grid and Utility Computing, are used for a certain purpose, the improvement of an organization’s processes takes place considerably. At the technical level, these include hardware and software components, namely, computers and terminals, smartphones, Internet © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 83–96, 2022. https://doi.org/10.1007/978-3-030-89909-7_7

84

E. Damasceno et al.

infrastructure, wireless communication networks, telecommunication systems, database systems, cloud and fog computing infrastructure, and middleware architecture [2]. Moreover, systematic reviews are essential for healthcare providers, policymakers, and other decision-makers, who would otherwise be confronted by an overwhelming volume of research on which to base their decisions [3]. Considering these aspects, PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) methodology is adopted because it provides a framework, covering the entire literature review process from a systematic search for publications, selection of relevant papers to result in synthesis [1]. Among the areas of computing to bring new solutions for organizations, including the following: Data mining (DM) is used for locating hidden patterns in large datasets using a composition of different methods of machine learning, database manipulations, and statistics. The important features and structures are determined by data analytics; implicit knowledge, rules, and patterns of the data are discovered by DM [4, 5]. The term data mining has mostly been used by statisticians, data analysts, and the management information systems (MIS) communities. It has also gained popularity in the database field [6]. Business Intelligence (BI) can be presented as an architecture, tool, technology, or system that gathers and stores data, analyzes it using analytical tools, and delivers information and/or knowledge, facilitating reporting, querying, and, ultimately, allows organizations to improve decision making [7]. The main purpose of BI is the identification of potential in the data collected during the company operation activities and their subsequent transformation into information necessary for competitive purposes and has tools that help to achieve this goal [8, 9]. Grid computing (GC) is the practice of leveraging multiple computers, often geographically distributed but connected by networks, to work together to accomplish joint tasks. It is typically run on a “data grid,” which is a set of computers that directly interact with each other to coordinate jobs [10]. This enables access to distributed heterogeneous resources using web services [11]. Utility computing (UC) relies on the pay-per-utilization model. It provides computational resources to ask for a metered benefit [12]. It provides more computing power and storage space than a single time-sharing computer and requires infrastructure such as a cloud to provide different services [13]. The main objective of the study presented in this paper is to identify relevant studies that jointly address the topics mentioned above. The specific objectives of this study were as follows: • Identify the state-of-the-art of BI and DM in the context of Grid and Utility Computing (GUC); • Analyze studies in which there is a relationship between BI and GC; • Analyze studies in which there is a relationship between BI and UC; • Analyze studies in which there is a relationship between DM and GC; • Analyze studies in which there is a relationship between DM and UC; • Identify limitations in articles and research gaps in the fields.

The State-of-the-Art of Business Intelligence and Data Mining

85

The remainder of this paper is organized as follows. Section 2 presents the methods adopted in this study, namely the PRISMA statement. Section 3 presents the results obtained after the searches and analysis. Finally, Sect. 4 provides the conclusion and presents directions for future research.

2 Methodological Approach A systematic review methodology has been adapted to a variety of other disciplines, new evidence synthesis methods have been developed, and the standards and quality of systematic reviews have improved [14]. The present systematic review was conducted using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) method. Systematic reviews have many critical roles. They can provide syntheses of the state of knowledge in a field, from which future research priorities can be identified [15]. 2.1 Research Questions Constructing a systematic literature review requires a base set of research questions that drive the research methodology [16]. Considering, it was identified four key research questions to investigate the approaches. Q1. What is the state-of-art of BI and DM in the context of Grid and Utility Computing (GUC)? Q2. Are there many studies concerned with BI and DM in the context of (GUC)? Q3. What are the main areas of study of BI and DM in the context of GUC? Q4. Is the incidence of the words BI and DM the same as GUC to say that they are in the same context? 2.2 Eligibility Criteria This work presents the inclusion and exclusions criteria for the review and how the studies were grouped for the syntheses, in other words, specifying the eligibility criteria used to decide what articles were eligible or ineligible in sufficient detail should enable readers to understand the scope of the review and verify inclusion and exclusion criteria [3]. A. Inclusion Criteria 1. The article must have at least one of the following “utility computing,” “grid computing” associated with “business intelligence” in the title, abstract, keywords, or conclusion; 2. The article must have at least one of the following “utility computing,” “grid computing” associated with “data mining” in the title, abstract, keywords, or conclusion;

86

E. Damasceno et al.

3. Recent articles must be searched, in this case, from 2018 to 2021, because BI and DM in the context of GUC are new concepts; 4. The articles must be written in English; 5. The articles must have a significant number of keywords “business intelligence”, “data mining”, “grid computing” and “utility computing” to consider that the studies of BI and DM in the context of GUC are relevant. B. Exclusion Criteria 1. 2. 3. 4. 5.

BI is not in the context of GUC; DM is not in the context of GUC; The articles are not in English; The articles are not published from 2018 to 2021; There are not significant keywords to consider that BI and DM in the context of GUC.

2.3 Information Sources Almost all sources of data, particularly the Internet, are plenty of human-generated text that requires proper retrieval and processing [17]. The searches were performed in the ScienceDirect1 , Scopus2 , and Web of Science3 databases. These databases were chosen because, first, ScienceDirect contains full-text articles from journals and books, primarily published by Elsevier, but includes some hosted societies. Furthermore, Scopus indexes metadata from abstracts and references of thousands of publishers, including Elsevier, and also builds additional functionality on top of that metadata, such as citation matching, author profiles, and affiliation profiles [18]. Moreover, because Web of Science indexes quality publications from top journals and ranked international conferences [19]. 2.4 Search Strategy The search strategy included the following keywords: • • • •

“data mining” AND “grid computing”; “data mining” AND “utility computing”; “business intelligence” AND “grid computing”; “business intelligence” AND “utility computing.”

An important point to consider is the acronyms; for instance, BI, DM, GC, and UC were not adopted due to misinterpretation of the search engine that could find articles unrelated to the terms of the study. Furthermore, the search was filtered with recent articles, namely 2018 to 2021, since the objective is to know the state-of-the-art of the most recent research. 1 https://www.sciencedirect.com/. 2 https://www.scopus.com/. 3 https://www.webofscience.com/.

The State-of-the-Art of Business Intelligence and Data Mining

87

2.5 Data Collection and Selection Process All available articles were downloaded and imported into Mendeley4 software in separate files according to the keywords. The intention was to organize and delete possible duplicate articles. At this stage of the study, the title, abstract, keywords, and conclusions of the selected articles were read to meet the eligibility criteria, as well as exclusion criteria delimiting the results. Moreover, as the objective of the study is to verify the state-of-the-art of the presented topics, important elements of both the abstract and conclusions were exported in a spreadsheet, in order to identify what has been researched so far, namely their limitations and aspects still open in the field of study. Thus, it enables the creation of graphics, word clouds, and percentages that are relevant to the study.

3 Results In this section, after analysis of the research data, the results of the study will be presented to reveal what was discovered in the research. The demonstration of relevant data will be used to support future research and contribute to science. Figure 1 shows the results of the search in the three databases mentioned previously.

Fig. 1. The number of articles found in each database according to keywords.

As can be seen, Scopus and Web of Science did not bring significant results, for this reason, the research will be conduct through the ScienceDirect database, and additionally, it contains full-text articles from journals and books making it possible a deeper analysis of the articles. However, in Fig. 2, the results of the search and selection process were reported by the PRISMA flow diagram so that readers can understand the flow of retrieved records through inclusion in the review [3].

4 https://www.mendeley.com/download-desktop-new/.

88

E. Damasceno et al.

Fig. 2. Kernel PRISMA 2020 flow diagram for the systematic review including searches of databases, registers, and other sources.

3.1 Study Selection and Characteristics Searches were performed on the ScienceDirect database, resulting in 229 results. Specifically, the strings “data mining” AND “grid computing” returned 165 results, “data mining” and “utility computing” returned 32, “business intelligence” and “grid computing” returned 23 results, and “business intelligence” and “utility computing” returned 9 results. It is appropriate to highlight that it was only possible to download 225 because 4 articles were not available to download. In the next step, 23 articles were manually removed under duplication, resulting in 202 records being screened. This phase shows that 180 reports were excluded because they did not meet the inclusion criteria, and the remaining twenty-two reports were assessed for eligibility, using the exclusion criteria. A fact to call attention to is that no relevant articles were found regarding BI in the context of GUC. However, the selected articles refer to DM in the context of GC, with 19 results, and DM in the context of UC, with 3 results. Table 1 shows the characteristics of twenty-two articles selected for analysis and the intention of this is, to demonstrate that few keywords related to the study were found, therefore, other parts of the articles were analyzed, such as abstract, development, and conclusion. Initially, the study aimed to find the relationships between subjects in the title, abstract, and keywords; however, it was decided to look for the relationships in the entire articles through the searches of the keywords already mentioned, since relationships within the three mentioned parts were not found.

The State-of-the-Art of Business Intelligence and Data Mining

89

Table 1. Characteristics of selected articles. Row Author

Journal

Keywords

Year

1

Mezni, Haithem, et al.

International Journal of Approximate Reasoning

Cloud computing, Uncertainty, Uncertain cloud services, Uncertainty models

2018

2

Dogan, Alican, et al.

Expert Systems with Applications

Machine learning, Data 2021 mining, Manufacturing, Classification, Clustering

3

Zhao, Yaliang, et al.

Information Sciences

CPSS, Big data, Cloud computing, Privacy-preserving, Clustering

4

Zhu, Erzhou, et al.

Neurocomputing

Clustering analysis, Grid 2019 clustering algorithm, Partitional clustering algorithm, Clustering validity index

5

Tavana, Madjid, et al.

Internet of Things

Internet of things, Enterprise resource planning, Cloud computing

2020

6

Vaccaro, A., et al.

International Journal of Electrical Power and Energy Systems

Smart grids, Power distribution systems, Power systems operation, Computational intelligence

2019

7

Tsai, Chun Wei, et al.

Computers in Human Behavior

Clustering, Big data, Cloud computing, Spark

2018

8

Bibri, Simon Elias

Sustainable Cities and Society

Smart sustainable cities, IoT, Big data analytics, Sensor technology, Data processing platforms, Environmental sustainability, Big data applications, Cloud computing, Fog/edge computing

2018

2020

(continued)

90

E. Damasceno et al. Table 1. (continued)

Row Author

Journal

Keywords

Year

9

Liu, Zelei, et al.

Future Generation Computer Systems

Maximal frequent itemsets, Big data, MapReduce, Frequent transactions, Best-transaction-set

2018

10

Blair, Gordon S., et al.

Environmental Modelling Environmental modeling, 2019 and Software Models of everywhere, Grid computing, Cloud computing, Data Science

11

Zhao, Kaiyi, et al.

IFAC-PapersOnLine

Sales data sets, Business 2018 circles, Clustering, Visualization, Interaction

12

Bibri, Simon Elias

Sustainable Cities and Society

Smart sustainable cities, Theories, Academic disciplines, Academic discourses, Multidimensional framework Interdisciplinarity and transdisciplinarity, Systems thinking, Complexity science, Sustainability, Computing, and ICT

2018

13

Martin-Rodilla, Patricia, Data and Knowledge et al. Engineering

Data-analysis, Software-assistance, Data-analysis measurement, Data-analysis performance, Cognitive processes

2018

14

Sciacca, Eva, et al.

Workflow systems, Science gateways, Collaborative environments, Astrophysics, DCIs, Milky way analysis, Infrastructure tests, Monitoring

2019

Future Generation Computer Systems

(continued)

The State-of-the-Art of Business Intelligence and Data Mining

91

Table 1. (continued) Row Author

Journal

Keywords

Year

15

Kumar, Sunil, et al.

Journal of King Saud Big data, FIM, HUIM University - Computer PSPM, Uncertain data and Information Sciences mining

16

Malathi, D., et al.

Computers and Electrical Disease Prediction 2019 Engineering Support System (DPSS), Fog computing, Cloud computing, Hybrid reasoning, Privacy-preserving, Paillier Homomorphic Encryption

17

Ocaña, Kary A.C.S., et al.

Future Generation Computer Systems

Science gateway, Bioinformatics, Data analysis, Computational complexity, High-performance computing

2020

18

Cantabella, Magdalena, et al.

Future Generation Computer Systems

Learning management systems, MapReduce, Apriori algorithm, E-learning analytics, Student behavior, Big data

2019

19

Zhang, Xiang, et al.

Earth-Science Reviews

Collaborative 2018 observation, Focusing service, Geoscience, Geospatial Infrastructure, Integrated management, Scalable processing, Sensor web

20

Ravi, Kumar, et al.

Journal of Network and Computer Applications

Analytics in the cloud, Analytics for cloud, Cloud resource demand prediction. Cloud resource usage optimization, Cloud provisioning

2018

21

Mitra, Amit, et al.

Technological Forecasting and Social Change

Cloud computing, Resource-based view, Unique competencies, Value creation

2018

2019

(continued)

92

E. Damasceno et al. Table 1. (continued)

Row Author

Journal

Keywords

Year

22

Journal of Network and Computer Applications

Big data, Fog data analytics, Data collection, Data storage, Data reduction, Data security, and privacy

2019

Kumari, Aparna, et al.

To illustrate, Fig. 3 shows in a qualitative way, the words highlighted from the point of view of the study subjects, a word cloud was generated showing the words with the highest incidence found in the titles, keywords, and abstracts of all 22 selected articles.

Fig. 3. Word Cloud was generated from the words of titles, keywords, and abstracts of all 22 articles.

The result generated by Word Cloud shows that there are some incidences of words that may be related to the object of study, such as, “data,” “computing,” “clustering” and “cloud.” Therefore, it cannot be said that they are DM in the context of Grid and Utility Computing because de incidence of “data” and “computing” is not the same, in the same way, the incidence of “grid,” “utility” and “computing” are not the same. 3.2 Results of Syntheses The criterion for verifying whether there are studies in BI and DM in the context of GUC was to search for strings in the title, abstract, and keywords of the selected articles. This definition was decided because it is understood that if these terms are in those parts,

The State-of-the-Art of Business Intelligence and Data Mining

93

it means that there are studies in the field. However, after a human analysis of the 22 selected articles, only 8 had strings in the defined locations, six with DM, one with GC, and one with Utility Computing, as can be seen in Fig. 4.

Fig. 4. Results after searching for strings in the title, abstract, and keywords.

Given the above, it can be assertively concluded that there are no in-depth studies on BI and DM in the context of Grid and Utility Computing. The subjects were approached in isolation in the content of the articles without any correlation between the themes, in other words, GUC tools to assist the B and DM solutions. 3.3 Additional Elements Following the study, to present additional elements in the body of evidence of the evaluated results, the selected articles were divided into three groups, such as, “computing” which covers sub-areas such as algorithm, software, models, applications, architecture, and cloud; “disciplines” containing geography, education, astronomy, and urbanism; and finally, “health,” illustrated in Fig. 5. The aim is to demonstrate which areas are doing research related to GUC.

Fig. 5. Articles categorized by groups.

94

E. Damasceno et al.

One aspect that deserves attention is that only one of the 22 articles selected and analyzed in the study proposes offering solutions for the development of routines in the organizational scope. Therefore, it can be said that there are not enough studies that contemplate the organizational branch to offer solutions that increase performance and reduce costs.

4 Conclusions The starting point for the advancement of any study is to define the research questions and the specific objectives of the researcher. Hence, it is possible to define the next steps for developing this research. It is worth remembering that, a scientific study has certain criteria and procedures to be adopted to provide future researchers with clear and concise knowledge. In an in-depth study, some limitations can be highlighted, such as heterogeneity in the research fields, lack of information in the abstracts of selected articles, and unavailability of articles for download. That said, although the searches were made in three databases, it was decided to use only one database, ScienceDirect, to delimit the study and to obtain qualitative data, since it is necessary to have access to the full articles. In this database, it is possible to download, as long as it is logged in by an institution. Scopus and Web of Science did not bring a considerable number of data to conduct the study. However, it was decided to adopt a systematic literature review using the PRISMA methodology, as it understands that it meets the requirements for the progress of scientific research. Through a systematic search of the contents, the results provided by the ScienceDirect database were selected to verify the most recent studies related to BI and DM in the context of Grid and Utility computing. Answering research questions, this study was carried out with the analysis of recent studies, more precisely from 2018 to 2020, as the objective was to verify the stateof-the-art of the subjects. Thus, this state-of-the-art report reflects the low increase of research in this field, since there are not many studies regarding BI and DM in the context of GUC. It was identified that although they are widely known subjects when studied separately, it is concluded that not many advances in the case of studies of the subjects correlated above, in other words, BI and DM in the context of GUC. It was concluded that computing is the more relevant area if we compare it to health and disciplines areas. Finally, a word cloud graphic was created with all 22 articles to check the incidence of words to verify if there are studies in the fields. It was concluded that the incidence of words is not enough to consider that BI and DM are in the context of GUC. To guide future research, it is possible to conclude in this study that there are very few studies in the research object. Thus, it is necessary to expand the scope of the research with more range of years including articles in Portuguese and Spanish. And also, look for articles that present GUC solutions for other areas that are already more consolidated to see what has been studied so far and then adapt to this area, such as BI and DM.

The State-of-the-Art of Business Intelligence and Data Mining

95

References 1. Budrionis, A., Plikynas, D., Daniušis, P., Indrulionis, A.: Smartphone-based computer vision travelling aids for blind and visually impaired individuals: a systematic review. Assist. Technol., March 2020. https://doi.org/10.1080/10400435.2020.1743381 2. Bibri, S.E.: A foundational framework for smart sustainable city development: theoretical, disciplinary, and discursive dimensions and their synergies. Sustain. Cities Soc. 38, December 2017, pp. 758–794 (2018). https://doi.org/10.1016/j.scs.2017.12.032 3. Page, M.J., et al.: PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 372 (2021). https://doi.org/10.1136/BMJ.n160 4. Thakkar, H., Shah, V., Yagnik, H., Shah, M.: Comparative anatomization of data mining and fuzzy logic techniques used in diabetes prognosis. Clin. eHealth 4, 12–23 (2021). https://doi. org/10.1016/j.ceh.2020.11.001 5. Dogan, A., Birant, D.: Machine learning and data mining in manufacturing. Expert Syst. Appl. 166, February 2019, p. 114060 (2021). https://doi.org/10.1016/j.eswa.2020.114060 6. Fayyad, U., Uthurusamy, R.: Data mining and knowledge discovery in databases. Commun. ACM 39(11), 24–26 (1996). https://doi.org/10.1145/240455.240463 7. Azevedo, A., Santos, M.F.: Business intelligence: State of the art, trends, and open issues. In: KMIS 2009 - 1st Int. Conf. Knowl. Manag. Inf. Sharing, Proc., January 2009, pp. 296–300 (2009). https://doi.org/10.5220/0002303602960300 8. Václav, C., Gabriel, F., Blanka, K., Libor, K., Michal, T.: Utilization of business intelligence tools in cargo control. Transp. Res. Procedia 53(2019), 212–223 (2021). https://doi.org/10. 1016/j.trpro.2021.02.028 9. Barrientos Monsalve, E.J., Franco Carreno, M.C., Buelvas Gutiérrez, E.D., Morris Molina, L.H., Franco Garcia, J.C., Bautista Rangel, H.M.: Theorization on case studies in business intelligence management on intellectual capital. J. Phys. Conf. Ser., 1160(1) (2019). https:// doi.org/10.1088/1742-6596/1160/1/012011 10. Liu, H., Shun Chen, R., Lee, C.Y., Cao, W., Chen, L.: Using grid computing architecture in computing resource allocating of IC design. J. Ambient Intell. Humaniz. Comput. (2020). https://doi.org/10.1007/s12652-020-02246-x 11. Liu, J., Pacitti, E., Valduriez, P., Mattoso, M.: A survey of data-intensive scientific workflow management. J. Grid Comput. 13(4), 457–493 (2015). https://doi.org/10.1007/s10723-0159329-8 12. Malik, M.I.: Cloud computing-technologies. Int. J. Adv. Res. Comput. Sci. 9(2), 379–384 (2018). https://doi.org/10.26483/ijarcs.v9i2.5760 13. Bhathal, G.S., Singh, A.: Big data computing with distributed computing frameworks. In: Saini, H.S., Singh, R.K., Kumar, G., Rather, G.M., Santhi, K. (eds.) Innovations in Electronics and Communication Engineering. LNNS, vol. 65, pp. 467–477. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3765-9_49 14. Haddaway, N.R., Macura, B., Whaley, P., Pullin, A.S.: ROSES reporting standards for systematic evidence syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid. 7(1), 4–11 (2018). https://doi.org/10.1186/s13750-018-0121-7 15. Page, M.J., et al.: The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372 (2021). https://doi.org/10.1136/BMJ.n71 16. Jia, R., Yang, Y., Grundy, J., Keung, J., Hao, L.: A systematic review of scheduling approaches on multi-tenancy cloud platforms. Inf. Softw. Technol. 132(October), 2021 (2020). https:// doi.org/10.1016/j.infsof.2020.106478 17. Blazquez, D., Domenech, J.: Big data sources and methods for social and economic analyses. Technol. Forecast. Soc. Change 130, March 2017, pp. 99–113 (2018). https://doi.org/10.1016/ j.techfore.2017.07.027

96

E. Damasceno et al.

18. Elsevier: What is the difference between ScienceDirect and Scopus data?. Elsevier (2018). https://service.elsevier.com/app/answers/detail/a_id/28240/supporthub/agrm/p/ 15838/. Accessed 24 May 2021 19. Zurita, G., Shukla, A.K., Pino, J.A., Merigó, J.M., Lobos-Ossandón, V., Muhuri, P.K.: A bibliometric overview of the Journal of Network and Computer Applications between 1997 and 2019. J. Netw. Comput. Appl. 165, May 2020. https://doi.org/10.1016/j.jnca.2020.102695

Knowledge Management

Towards Optimisation of Electroconvulsive Therapy with a Little Help from Knowledge Management Jorge Rodas-Osollo1(B) , Karla Olmos-Sánchez1 , Alicia Jiménez-Galina1 , Marlene Soltero-Romero2 , and Angélica Pérez-Campos2 1 Instituto de Ingeniería y Tecnología, Universidad Autónoma de Ciudad Juárez, 32584 Juárez,

Chihuahua, México {jorge.rodas,kolmos,alicia.jimenez}@uacj.mx 2 Maestría en Cómputo Aplicado, Universidad Autónoma de Ciudad Juárez, 32584 Juárez, Chihuahua, México [email protected], [email protected]

Abstract. This paper communicates the importance of identifying knowledge to improve medical treatments such as electroconvulsive therapy (ECT). The information analysed comes from patients with severe depressive disorders or schizophrenia, and corresponds to measurements of various psychophysiological parameters taken at different times after the application of electroshock. The paper reports results of applying the KDSM methodology to an Informal Structure Domain, with the presence of very short and repeated serial measures, such as ECT. Keywords: Informal structure domains · Knowledge management · Electroconvulsive therapy · KDSM

1 Introduction For decades, attempts have been made to equip computers with valuable information, knowledge, to help solve problems or needs in various domains that affect human beings. Limited domain knowledge has proven to be a poor substitute for human expertise, so intelligent software tools are only as good as their programming. Clearly, there are major advances in machine learning that allow these tools to model and define rules, instances, knowledge from the information supplied to them. These tools improve accuracy, adapt to the unknowns of a domain and extend their capabilities beyond the original programming. Preliminary results are thus promising, with intelligent applications emerging every day for a wide variety of domains. The challenge of establishing the requirements for the development of such software goes beyond the understanding and application of smart technologies, as it involves understanding the domain and the ability to derive domain knowledge. Often, the construction of an intelligent solution is linked to the knowledge obtained from Informally Structured Domains (ISD) [1] and, in particular, to the a priori knowledge for training many Machine Learning tools. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 99–112, 2022. https://doi.org/10.1007/978-3-030-89909-7_8

100

J. Rodas-Osollo et al.

An ISD has the following characteristics: 1. The presence of one or more specialists who bring a wealth of tacit and explicit, informal or formal knowledge from their particular experiences, backgrounds, perspectives, interests, expectations, and the role they play in the domain. 2. Presence of one or more knowledge requirements engineers, responsible for eliciting knowledge requirements, who are generally not involved in the domain and usually have only general technical knowledge of how to perform their task. 3. Tangible or intangible product or solution that explains a behaviour, or solves or addresses a particular and unrepeatable situation e.g. an intelligent software, or set of knowledge which can be represented as a set of knowledge pieces or even as knowledge requirements. Therefore, it is pertinent to communicate achievements in dealing with case studies of ISD domains where truly useful information is extracted, as it is a complex and challenging task today. Thus, this paper contributes to the state of the art by communicating the result of successfully dealing with one more case study using the Knowledge Discovery in Serial Measurements (KDSM) methodology [2]. The characteristics of the medical domain case study presented below fit an ISD, in addition to having very short and repeated serial measurements, and not enough a priori knowledge to train some powerful machine learning tools. Therefore, KDSM is used as it adapts to these particular characteristics to extract useful information represented as crisp rules, and describe some behaviours to improve a therapy or treatment. The result is presented in this paper, as well as some immediate challenges that motivate further work in this line of research. The structure of the paper is as follows: first, a short background of the case study is presented in Sect. 2. Section 3 briefly describes the data used in the case study. In Sect. 4, a typical ISD problem in the medical field and its main objectives are discussed. Section 5 exemplifies the use of the KDSM methodology through the ECT domain data. Section 6 presents the first set of results of the first rule selection performed by the domain specialist. Finally, in Sect. 7, conclusions on the results and some interesting points for future work are mentioned.

2 Background Electroconvulsive therapy (ECT) is a safe, effective, fast, valuable and widely used treatment for patients with major depression, bipolar disorder, psychosis… in the case of intractable catatonia and neuroleptic malignant syndrome. ECT can be life-saving and the safest treatment option for patients who do not tolerate or respond poorly to medications and who are at high risk of drug-induced toxicity or toxic drug-drug interactions [3, 4]. Efforts are currently underway to optimize the use of ECT and, in synergy with other biological and psychological treatments, to reduce the intensity of side effects, prevent relapses and the recurrence of symptoms. However, several of the brain biological events related to its efficacy remain unknown. The physiological response to ECT has been studied in various ways without promising results. One interesting avenue is the study of the neuropsychological effects of ECT on psychophysiological parameters such

Towards Optimisation of Electroconvulsive Therapy

101

as reaction, decision or motor times. These effects are cognitive changes related to orientation, attention and calculation, memory loss and recall [5] and the aim is to make progress in identifying the factors that cause them by first analysing the effect on reaction times related to visual and auditory stimuli after ECT application. Studies in similar situations in psychiatry, neurology or various areas of medicine have examined everything from simple comparisons of treatments to longer-term effects or pre-post responses to various treatments or medications. Increasingly, there is a need to detect more subtle or domain-specific effects that introduce additional complicating factors. In these situations, condensing the time series data into a summary statistic can simplify the analysis by eliminating the time element. This approach of using summary statistics does facilitate clear communication of the main findings both in simple terms to the public and with a full report of individual responses to the scientific community. But, the graphical presentation of time series data in a line graph does not allow for easy plotting of individual or paired responses. Measures of central tendency certainly have a place to illustrate group effects in graphs and figures, but individual responses to each experimental condition are being presented, and we lose sight of them, especially when sample sizes are minimal and do not facilitate critical evaluation of the data [6].

3 Data Description In this case study, 183 patients with major depressive disorder or schizophrenia were followed according to the ICD-10 research criteria and the DSM-5 criteria and for which the TEC was indicated and which gave their written informed consent for the study and to submit to the TEC. Information on the main characteristics of the patients is available, conditions, somatic state, routine haematological and biochemical tests, chest x-ray, electrocardiograms, anaesthetic risk, comorbidities, age, weight, education… The present standard practice optimizes the therapeutic relationship, in the selection of the parameters of the electrical stimulus such as the energy level, the duration of the stimulus, the pulse width and the pulse frequency. Also, multiple patient responses are monitored by Electroencephalogram (EEG), Electrocardiogram (ECG), and Electromyogram (EMG), includes a rigorous evaluation of the neuropsychological effects of the patient. For the measurement of psychophysiological parameters, the Vienna Protocol Reaction Unit was used at 2, 4, 6, 12 and 24 h after each application of an electroshock (ES). The unit provides visual and audible stimuli and in this study four tests were carried out simple visual (e5), simple audible (e6), complex visual (e7) and complex visual-audible (e8). The following measures were recorded: erroneous decisions, erroneous reactions, absence of reactions, incorrect reactions, correct reactions, decision times, driving times and reaction times [3, 7].

4 Problem Description Presented in This ISD ECT Domain The representation of a set of patients (p1 …pn ) in which ni occurrences of a given ES take place at different time points (ES1 …ESn ). Connected to each electroshock occurrence, there are psychophysiological parameters of denoted by Y (for this particular case the reaction time, RT) which reflects the behaviour of the patient.

102

J. Rodas-Osollo et al.

Hence, a certain small number of measures of Y is taken for each patient and for each occurrence of ES. In this particular case, such a number (r) is fixed for all the occurrences of an ES and the time points where Y will be measured are also fixed. This measure is performed during the first 24 h after the application of each ES, in particular after 2 h, 4 h, 6 h, 8 h, 12 h, and 24 h. Measuring this RT is of special interest for the study of the side effects resulting from an ECT. Such a scenario generates three types of information that can be formalized by 3 matrices: • The matrix X contains a set of quantitative or qualitative characteristics X1 …XK for each patient. • The matrix Y contains groups of very short serial measures of an interest parameter, this case RT, at all the fixed time points for each occurrence of ES. • The matrix Z contains a set of quantitative or qualitative characteristics Z1 …ZL for each ES. The number of ES and the time at which they occur may differ from one patient to another without any other underlying pattern. However, all ES applied to the same patient are influenced by their characteristics, which means that all RT measurements in the same patient are influenced by the same patient. Therefore, in the Y and Z matrices, every patient act as a blocking factor, defining, in the Y matrix, bundles of curves that are not at all independent of each other. A block is constituted by all measurements after the application of each ES on the same patient. Thus, the analysis deals with a set of groups of very short, repeated serial measurements of the parameter of interest in turn with a blocking factor delimited by the patient. Consequently, the characteristics of the matrix structures X (patient characteristics X1 …XK ), Y (serial measurements of the parameter of interest RT), and Z (ECT characteristics Z1 …ZL ) differ and the management of the knowledge interrelating them is not trivial, so a particular treatment must be made for them. These situations are frequent in the medical domains of ISD and serial measurements are often addressed by simplifying to a single series [8].

Fig. 1. VIC graphic of RT curves of the e8 test of the 1st patient.

Towards Optimisation of Electroconvulsive Therapy

103

Figure 1 shows the evolution of the RT through its curves for a 6 ES ECT applied to the 1st patient. The RT correspond to the complex visual-audible (e8) and were measured at 2, 4, 6, 12 and 24 h after applying each ES. Figure 2 shows the evolution of the RT through its curves for an ECT of 5 ES applied to the 4th patient. The RT correspond to the complex visual-audible (e8) and were measured at 2, 4, 6, 12 and 24 h after applying each ES. This would simplify the RT measurements to a single series for each patient, as shown in Figs. 1 and 2 the mean RT curve (black line). Even this action would modify the structure of the matrices to make them compatible and allow for a classical analysis. However, it would lead to loss of potentially relevant information, as the variability depends on both ES and patient characteristics and the efficacy of ECT may be affected. In fact, variability is significant between patient and test curves, making it difficult to find a general pattern. However, for knowledge representation, it is possible that the average curve can communicate a trend of the overall evolution of a patient. Furthermore, there is no standard quantity of ES to be applied to a patient. Consequently, reducing the patient’s information to only one record in the matrix is not the proper way to proceed. Therefore, this patient effect is considered for the analysis by keeping all curves for all patients in the matrices. Given the above, and due to the lack of a priori knowledge for training the machine learning tools, the KDSM methodology was used.

Fig. 2. VIC graphic of RT curves of the e8 test of the 4th patient

5 Knowledge Discovery in Serial Measures KDSM is a methodology for Knowledge Discovery in Informally Structured Domains where very short and repeated serial measurements with a blocking factor are presented (see Fig. 3). For a good knowledge management, it is done in three phases: 1. Individuals Baselines Analysis (BLA): Initially, the baselines and their relationship with matrix X are studied to identify different initial profiles of individuals to be used as a priori knowledge. 2. Event Effects Analysis (EEA): The knowledge induced from the previous phase is used as input to study the effect of a given event E on the attribute of interest Y.

104

J. Rodas-Osollo et al.

Subsequently, different patterns on Y are identified relating (to) the individuals who are affected by E. 3. Knowledge Production (KP): Finally, the results obtained in the previous phase are crossed with matrices X and Z to find relationships between them, and to determine which relevant individual and event attributes constitute the found patterns. In short, the methodology will carry out the next tasks: BLA: - Extraction of a baseline matrix from matrix serial measures - Hierarchical clustering of the patients using baseline matrix - Use of attributes of patients features to interpret the classes obtained - Rules induction from comparison between classes and patients features attributes. A KB is formed by rules obtained EEA: - Blocking factor managing on serial measures for distinguishing the effect of the ES application on RT parameter - Clustering Based on Rules of modified serial measures with a knowledge base KB obtained previously KP:

- Interpretation of resulting classes

5.1 BLA Phase Initial reaction time measurements for the simple visual (e5) and auditory (e6), visual and categorization (e7), visual and auditory and categorization (e8) tests were analysed. A hierarchical clustering was used to classify these reaction times of the patients, as these represent the initial condition of the patients (hierarchical reciprocal neighbours’ method with Ward’s aggregation criterion and Euclidean distance). Subsequently, patient characteristics that are relevant to the domain specialist’s partitioning are distinguished to obtain rules. The classification technique suggested partitions into 2, 3, 4, and 5 classes that group different patients according to their baseline reaction times. In agreement with the domain specialist a cut into 3 classes (C1, C2 and C3 see Table 1) was performed. Table 1. The table shows a partial view of the arrangement of patient records in the three-cluster partition corresponding to the hierarchical cluster. The clustering technique indicates the varying baseline conditions of the patients.

Class

Patients (register)

Class 1 Class 2 Class 3

22 patients: 063059, 499969, … , 997938. 68 patients: 437289, 522175, … , 997877. 85 patients: 505652, 782035, … , 999464.

The features of each group were then analysed with the information obtained from the classification structure. Figure 4 shows the general trend of the basal RT curves for each class. This shows the baseline conditions of the patients regarding the RT. A

Towards Optimisation of Electroconvulsive Therapy

105

consistent response can be seen in class C1 (blue), C2 (orange), and C3 (gray) as a general trend and the reaction times have (largely) the same level for all the tests. The reaction times of classes C2 and C3 show a notorious increment compared to those in class C1 for all tests e5, e6 (single tests), and complex tests e7, and e8 in reaction time.

Fig. 3. Three phases of KDSM methodology for knowledge discovery in informally structured domains where very short and repeated serial measurements with a blocking factor are presented.

Fig. 4. BLA phase, VAC plot showing the variability between classes regarding the basal TRs.

As mentioned above, patient-specific information is available in matrix X (102 attributes), and this information is analysed to find those attributes that identify patterns that clearly characterize the selected partition. Quantitative and qualitative attributes that are significant for the partition are distinguished from the patient characteristics through the non-parametric H-test and the χ2 -test, both (p < 0.05). In addition, multiple box plots and bar charts were used to visualize the distribution of the data and how these attributes of the X matrix interpret the chosen 3-class partition. The information derived from the tests and the graphs were reviewed by the domain specialist to confirm which attributes he was interested in observing, including some that were not statistically significant. Subsequently, rules are obtained after the specialist’s personal selection and interpretation (specialist knowledge) of the graphs. These rules

106

J. Rodas-Osollo et al.

are of crisp type, and represent all the attributes selected by the specialist e.g. height, weight, number of cigarettes per day…. Moreover, they constitute the initial and partial knowledge base (KB) of this domain.

Fig. 5. The distribution of the age attribute in each partition is remarkable for its symmetry and clear delimitation in each partition class. This attribute is extremely relevant because it shows a clear difference in reaction times from younger to older patients. Here it can be seen that the youngest patients are found in BLA Class 1, with lower baseline TR than those in BLA Class 2, up to the mature patients in BLA Class 3.

Figure 5 shows the distribution of the age attribute in the 3-class partition. As can be seen, class BLA C1 includes all baseline curves of youngest patients (up to 29 years) and classes BLA C2 and BLA C3 include baseline curves of patients older than 29 years. The attribute of age was the first of the 40 statistically significant attributes selected, according to the extensive experience of the specialist and the fact that this attribute is one of the attributes that exerts a strong influence on psychophysiological tests [9] consistent with clinical evidence. Therefore, the 3 classes of patients are described by: youngest patients (BLA C1) with shorter and more regular baselines RT for all the tests (e5-e8) and young (BLA C2) and mature (BLA C3) patients with longer baselines RT, in particular, with much longer times in complex tests (e7 and e8) than in single ones (e5 and e6). From this class description, the next step is to integrate the 103 rules obtained from the selected significant attributes into the knowledge base, KB (Table 2). It was decided to include this knowledge for the EEA phase, with separate processes for younger, young and mature patients. Table 2. This table shows some crisp rules contained in the KB. Specifically, the rules for the age attribute. Attribute

Rules

Age

1. If age >= 21 and = 60 and = 30 and =200 and =118 and =130 and =50 and 10 and hamtre17 17 then belongs to C3.

Towards Optimisation of Electroconvulsive Therapy

107

5.2 EEA Phase To study the effect of each ES on reaction time, the existing blocking factor in the Y matrix was treated and transformed into a new matrix without blocking factor. This transformation consisted of performing the differences between post-ES and pre-ES. These data are poorly conditioned for applying classical statistical hypothesis tests. Therefore, the rule-based hierarchical clustering technique (RBHC) [2] is used to overcome this situation. Thus, RBHC based on the KB suggests six classes of curves from effects of each ES applied to reaction times. In three of these classes, the differences between preand post-ES are noticeable to slightly positive, as well as presenting a slightly more uniform pattern, where one class corresponds to younger patients, another to mature patients and the last to older patients. This means that reaction times are slower after ES application and would therefore be considered poor reactions. The remaining three classes present a more variable pattern corresponding to patients with faster reaction times after ES application. The differences between before and after an ES are negative, i.e. they are good reactions (see Fig. 6).

Fig. 6. EEA phase, VAC plots showing the variability between classes regarding the effects of each ES.

108

J. Rodas-Osollo et al.

In Fig. 6 the effect of each ES applied shows a uniform trend in classes EEA C1, C3, and C5 of an increase in the RTs. Thus, there is a tendency towards deterioration of patients in these classes. On the other hand, in classes EEA C2, C4, and C6, we find a variable pattern that the effect tends to decrease the RTs, i.e., a tendency for patients in these classes to improve. 5.3 KP Phase In this Knowledge Production phase, relevant and interesting results were obtained for the domain specialist. The non-parametric H-test and the χ2 test both (p < 0.05) were used and box and bar charts were made to visualise the distribution of the data. Figure 7 shows graphs, of the statistically significant attributes that were of interest to the practitioner, which communicate the distribution of those attributes in relation to

Fig. 7. Boxplots of attributes relevant to the specialist in the KP phase.

Towards Optimisation of Electroconvulsive Therapy

109

the partition. In Fig. 7 (a, e) Enercon (Seizure Energy) for all classes where the patient behaviour is more homogeneous (EEA C1, C3, & C5) is lower than in the classes where the patient behaviour is variable (EEA C2, C4, & C6). For the classes where patients worsen, Fig. 7 (b), EEA C5 the impedance is low. In addition, after the application of an ES, blood pressure the attribute Caidao2t presents a range of lower values for the EEA 4 class of mature patients who improve, and it is a larger range with higher pressure values for the EEA 3 class in which patients worsen. Indeed, Fig. 7 (c, d), classes in which the behaviour of the patients is more variable with a tendency to worsen (EEA 2) show a drop in oxygen (Oximetry, oxygen drop) after application of an ES. Concerning attributes Total seizure time (Tctotal) and Postcri, Fig. 7 (e, f), the classes where patients show variable behaviour (EEA 2) show a lowers level of energy and time. Finally, Fig. 7 (g, h), the lowest doses of both Pentothal and Succinyl were applied in the classes where patient behaviour is most variable.

6 Results For knowledge management in this type of analysis, it was necessary to use KDSM due to the particular characteristics of this ISD domain. KDSM allowed to provide a better structure to the clinical database for a convenient data management. The structure consists of a first matrix containing the patient data, a second matrix containing the initial and post-ES reaction time measurements, and a third matrix containing the electroshock feature data. A three-stage KDSM analysis was carried out with the following results: 1. Individuals Baselines Analysis (BLA): Basal reaction times were analysed because they represent the initial conditions in which patients face the tests. The aim of this task is to find useful information to uncover possible patient profiles before ECT treatment, as well as to look for any a priori structure in the patient set that may determine differences in ES effects. From this, two rules were derived (see Table 2) which form a Knowledge Base and with them the groups of youngest, young, and mature patients were delimited. 2. Event Effects Analysis (EEA): The rule-based hierarchical clustering technique was performed using the results of the previous analysis as a Knowledge-Base. The differences in reaction times were used to eliminate the patients’ effect (characteristics of the patient) and to study the electroshock effect in isolation. The use of KDSM, for this type of intelligent data analysis, improves the quality of the results even when using a small set of rules. The interpretability of the final classes is improved by guiding the clustering process with the knowledge base due to the ability to incorporate attributes verified by the domain specialist as clustering criteria in the reaction time analysis. Furthermore, as mentioned above regarding Artificial Intelligence or Machine Learning techniques… it is impossible to tackle this kind of domains with such a small KB.

110

J. Rodas-Osollo et al.

Finally, 6 classes (EEA) were detected. On the one hand, three groups were shown: very young patients, young patients and mature patients. 3 classes of patients have a tendency of deterioration in their reaction times and great variability of response to treatment. The other 3 showed a more homogeneous response over the time period of the treatment and important level of improvement. Certainly, the relevant characteristics of the electroshocks were identified. 3. Knowledge Production (KP): All the attributes of matrix Z were projected onto the 4 classes obtained in the previous phase. Some relevant attributes were identified using this projection. Some of them show differences between young and mature patients. Others between which of them improve or deteriorate the patients’ condition. Evidently, the last group is more relevant for the expert because it is possible to identify a set of factors associated to the fact that an ES improves patients’ conditions. This result reveals the need to carry out more studies on ECT since, should these attributes trends be confirmed, the efficiency of ECT could improve dramatically.

7 Conclusion As discussed at the beginning of this paper, the main challenge of intelligently analysing an Informal Structure Domain with the characteristics outlined in Sect. 1 lies in the understanding of the domain and the ability to obtain knowledge from it. This action is key, at present, to properly guide the intelligent data analysis or to establish the knowledge requirements for the development of intelligent applications. Extracting knowledge from an ISD is a complex task that often yields very interesting results. Therefore, we consider it pertinent to communicate the use and results, through the use of the KDSM methodology, in a medical domain where very short and repeated serial measurements are presented, and there is not enough a priori knowledge to train some powerful machine learning tools. Thus, this paper reports the results of the analysis of some attributes of interest to the domain specialist, in particular the reaction time of patients undergoing electroconvulsive therapy, at very specific time points and where the measurements constitute blocks relative to each patient. It is worth remembering that KDSM allowed the discovery of knowledge of repeated very short serial measurements from patients as it is able to consider a special type of data management in the presence of a blocking factor. In this sense, some interesting conclusions can be drawn: 1. 103 rules were derived and the groups of very young, young and mature patients are delimited by them with the validation of the specialist; 2. Rule-Based Classification was carried out using the rules selected by the specialist as the Knowledge Base. Differences between RT were used because they measure the effect of electroshock by itself, independently of the patient’s characteristics. Six classes were detected in three groups of patients (youngest, young and mature), patients with different responses, some more positive than others. From a psychiatric point of view, it corroborated observations on the shape of the data. The application of KDSM to ECT measures provided very satisfactory results from

Towards Optimisation of Electroconvulsive Therapy

111

a psychiatric point of view. It has been shown that the behavioural curves of RT in each patient are neither inherent to the patient nor to the overall observation of the whole therapy. On the contrary, all patients may react differently in each ES session (see Table 3). Until now, psychiatrists have treated ECT (the set of all ES applied to the same patient during the whole treatment period) as a single unit, globally analysing the effect of the whole therapy by comparing the values of neuropsychological and psychophysiological measures before and after therapy [9]. In this paper, it is again underlined that too much information has been summarized in past analyses. As a consequence, relevant situations affecting a patient are hidden, as the patient reacts differently after each ES session. If the whole treatment had been summarized to a single observation, this would have never been confirmed. At the moment, the psychiatric specialist, taking these results into account, is working on the identification and analysis of possible causes external or internal to the patient that may or may not be present in each ES session. Thus, the knowledge gained in this analysis is added to that of the specialist and immediately modifies the way he conducts his analysis to face this domain. So far, there is no standardized practice to address these domains of ISD, such as this ECT case study reported in this paper. Thus, direct contributions can be synthesized in that the use of KDSM brought some valuable contributions to the psychiatric domain, in terms of medical praxis, allowing them to reconsider how to study the effects of each ES applied throughout ECT, as these are not monotonic. This type of study raises the risk of the common use of measurement summaries, as valuable information that directly affects treatment efficacy is lost. This paper shows that summaries imply a loss of relevant information, which in most cases is not desirable. Table 3. The table shows that a patient may have different responses positive or negative ones in the same therapy and not keep a specific order of an ES application. Register 63059 437289 499969 505652 522175 542693 682257 782035 783945 789145 818755 822836 824083 … 999464

Class 1

Class 2

Class 3

Class 4

2,3,4

1,5

Class 5

Class 6 1-5

1,3 1-10

2,4

1,2,3,4

5,7

1-4,6 1-5 1,2,5,6,7 3,5

3,4

1,2,4 1-7 1-6

…

…

1,2,8

3-7,9-11

…

…

1,3,4,5 …

2 … 1-9

Therefore, KDSM is a methodology for Knowledge Management in Informally Structured Domains that combines AI and Statistical tools and allows the identification of knowledge, according to the classical requirements of KDD [10].

112

J. Rodas-Osollo et al.

Moreover, it is important to highlight that KDSM is not a simple ad hoc technique to solve an isolated and particular problem, but a general methodology that is useful in any ISD domain where the information is structured in three matrices X, Y and Z that have the properties presented in Sect. 1. Finally, as future work, we are working on other rules of the Knowledge Base and on other forms of representation of the knowledge obtained to improve the interpretation of the knowledge.

References 1. Olmos-Sánchez, K., Rodas-Osollo, J.: Knowledge Management for Informally Structured Domains: Challenges and Proposals. https://doi.org/10.5772/intechopen.7007 (2017) 2. Rodas, J., Rojo, J.: Knowledge discovery in repeated very short serial measurements with a blocking factor. Appl. Psychiatr. Domain, Int. J. Hybrid Intell. Syst. 2(1), 57–87 (2005). https://doi.org/10.3233/HIS-2005-2104 3. Kerner, N., Prudic, J.: Current electroconvulsive therapy practice and research in the geriatric population. Neuropsychiatry 4, 33–54 (2014). https://doi.org/10.2217/npy.14.3 4. Takamiya, A., Hirano, J., Yamagata, B., Takei, S., Kishimoto, T., Mimura, M.: Electroconvulsive therapy modulates resting-state EEG oscillatory pattern and phase synchronization in nodes of the default mode network in patients with depressive disorder. Front. Hum. Neurosci. 13 (2019). https://doi.org/10.3389/fnhum.2019.00001 5. Porter, R.J., et al.: Cognitive side-effects of electroconvulsive therapy: what are they, how to monitor them and what to tell patients. BJPsych Open 6 (2020). https://doi.org/10.1192/bjo. 2020.17 6. Narang, B.J., Atkinson, G., Gonzalez, J.T., Betts, J.A.: A tool to explore discrete-time data: the time series response analyser. Int. J. Sport Nutrition and Exercise Metabolism 30(5), 374– 381, 10 August 2021. https://journals.humankinetics.com/view/journals/ijsnem/30/5/articlep374.xml 7. Schuhfried, G.: Vienna test system: psychophysiological assessment. Modelling, Austria: Schuhfried (2013). 8. Matthews J.N.S.: A refinement to the analysis of serial data using summary measures. Stat. Med., Wiley 12, 27–37 (1993) 9. UK ECT Review Group.: Efficacy and safety of electroconvulsive therapy in depressive disorders: a systematic review and meta-analysis. Lancet. 8 March 361(9360), 799–808 (2003). PMID: 12642045. https://doi.org/10.1016/S0140-6736(03)12705-5 10. Dekhtyar, A., Hayes, J.H.: Automating Requirements Traceability: Two Decades of Learning from KDD (2018). https://doi.org/10.1109/d4re.2018.00009

A Literature Review of Hand-Based Interaction in Virtual Environments Through Computer Vision Cristian Olvera, Graciela Lara(B) , Arturo Valdivia, and Adriana Peña CUCEI of the Universidad de Guadalajara, Av. Revolución 1500, Col. Olímpica, 44430 Guadalajara (Jalisco), México [email protected], {graciela.lara, arturo.valdivia,adriana.ppnegron}@academicos.udg.mx

Abstract. Interaction is an essential feature to achieve immersion in virtual reality. So that, virtual reality applications can achieve greater growth and diffusion by improving user interactions like exploration of virtual environments, select and transform virtual objects and text input. Hand-based interaction can provide naturalness, intuition, and creativity to these interaction techniques. In this paper, we present a literature survey of this broad field, including a comparative analysis of several hand-based interaction proposals through computer vision for virtual reality applications, where we considered the common features of the revised proposals. In addition, we propose two classifications, the first categorizes the interaction gestures, and the second group the used methods. Interaction guidelines are proposed to achieve hand gesture recognition with low computational cost. Keywords: Virtual reality · Virtual environments · Hand-based interaction · Computer vision

1 Introduction In this document, a study in the field of virtual reality (VR) with a specialized focus on hand-based interaction is presented. The existing ways to interact with information systems have learning curves that present difficulties and limitations. Technological progress has allowed the emergence of new ways of interaction that can be more natural and intuitive. Virtual reality presents major usability challenges that can be addressed through the above-mentioned emerging ways of interaction. This can expand the ways of using existing applications and enabling new types of applications. With this, it would be possible significant contributions to accessibility issues and enable applications to be used by people with disabilities. A desired feature of virtual reality is interacting with virtual elements, rather than just observing a passive environment. This is usually achieved through real-world metaphors, such as buttons or body gestures [1]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 113–122, 2022. https://doi.org/10.1007/978-3-030-89909-7_9

114

C. Olvera et al.

In the interaction with virtual environments (VEs), there are several approaches, to present a more specialized and meaningful study, the scope of this review will be limited to hand-based interaction. This approach presents different alternatives, the most common methods are computer vision and the use of wearable devices, such as gloves or bracelets. We focus on analyzing computer vision-based approaches. Concerning communication with devices, body gestures can be considered like natural, intuitive, and creative interfaces. Body gestures have been included under the term natural user interface (NUI). A NUI is a method that enables a user to interact with information systems without additional devices, the known methods involve voice, touch, and body gestures [2–4]. In this sense, it is believed that the key to the success of virtual reality applications is finding the right way to deal with usability issues, solve these problems will allow extending the use of virtual reality not only for entertainment purposes but also for professional applications in fields like engineering and medicine [5]. On another subject, computer vision (CV) involves processing digital images or video. CV includes getting the image data from ordinary cameras or dedicated devices. Before reviewing the existing hand-based interaction proposals, a description of the used dedicated devices is presented below. Leap Motion Controller is a device to recognize and track hands in real-time, providing the estimated position values of the hand parts [6, 7]. This technology works with a combination of two infrared cameras with infrared LEDs. It was developed by the company Ultraleap. Kinect is a device that tracks body parts through depth cameras. HoloLens is a wearable device for augmented reality applications, it also uses depth cameras to track body parts. Kinect and HoloLens are provided by the company Microsoft.

2 Hand-Based Interaction Systems A set of specific hand-based interaction proposals for virtual environments will be described below. The criteria for the selection of hand-based interaction systems were as follows: systems for extended reality (virtual, augmented, or mixed reality), implementation of computer vision, and publication in the last five years. Next, we give a general description of these systems. Fittkau et al. [8] proposed a web-based application called ExplorViz for exploring virtual cities and landscapes in real-time. ExplorViz uses the technologies WebVR API and the Microsoft Kinect v2. Their application includes the next four interaction gestures. The user quickly closes and opens the right hand to select an entity in the city model. The user closes the right hand into a fist and moves it to translate a virtual object. The user closes the left hand into a fist and moves it to rotate a virtual object. Moreover, the user retracts both fists to zoom in and advances both palms to zoom out (like rowing movements). The participants of the experiments rated the interaction gestures for selection, translation, and rotation as highly usable, nevertheless, the zooming gestures were not equally suitable. Melero et al. [9] presented Upbeat, an augmented reality (AR) dance game to enhance rehabilitation therapies in people with amputated upper limbs. In their proposal, the

A Literature Review of Hand-Based Interaction in Virtual Environments

115

patient learns to follow a virtual dance instructor, executing choreographed dance movements. These movements involve hand gestures related to rehabilitation therapy. Upbeat uses a Microsoft Kinect sensor to track patient’s movements and electromyography data from a Myo Armband to analyze hand gestures. The proposed dance choreography includes these hand gestures: spread, wave right, wave left, and fist. The game computes a score according to the number of movements and gestures successfully executed. The game presents a diagnostic of the results and shows the upper arm muscle activity through a video with augmented visualization. They conclude that Upbeat can provide custom feedback, evaluation about the patient’s progress and boost the enthusiasm and willingness of the patient. Du et al. [10] presented a teleoperation method to guide robots step by step with augmented reality (AR). Their proposal uses Microsoft HoloLens. Their method uses hands and speech for interactions. In their proposal, a virtual robot that represents a remote real robot is displayed in the real local environment. For hand recognition and tracking, they combine Leap Motion Controller with an unscented Kalman filter (UKF) and an improved particle filter (IPF) to get higher accuracy. Moreover, Term FrequencyInverse Document Frequency (TF-IDF) and maximum entropy model was implemented to recognize speech and gestures as user instructions. The interaction process includes that the user touch, drag and rotate the mobile parts of the virtual robots, their system will replicate it on the remote robots. They perform experiments with two GOOGOL GRB3016 robots cutting steel plates. The results verified the effectiveness of their proposal in comparison with other three similar methods. These methods are the proposed by Du & Zhang [15], Du et al. [16], and Kruse et al. [17] respectively. Vaitkeviˇcius et al. [7] proposed a system that recognizes American Sign Language (ASL) to interact with virtual environments. Their proposal uses a Leap Motion device to track the user’s hands and generates hands virtual models in a 3D space. The system can learn hand gestures based on the data provided by the Leap Motion API and the Hidden Markov classification (HMC) algorithm. They perform experiments to recognize the 32 gestures of the ASL alphabet, they involve twelve participants with different VR experience levels and with zero experience in ASL. Their proposal accomplishes a sign recognition with a mean accuracy precision of 86% and a mean typing speed of 3.09 words per minute. Rehman et al. [11] presented an interaction technique for virtual environments through hand gestures, these gestures are correctly recognized only with the tip of a finger. Their system uses a low-cost camera to track a colored fingertip coverage. Their proposal uses OpenCV to perform the next three processes: 1. Convert the images from the camera to the HSV model (Hue, Saturation, Value; also known as HSB, for hue, saturation, brightness). 2. Threshold the coverage color in the images. 3. Progressively calculate the 2D position of the coverage during the performance of the gestures. Their interaction technique includes navigation as well as the selection and translation of objects in a virtual environment. They include a comparative analysis between their proposal and the use of Leap Motion Controller to recognize the same gestures.

116

C. Olvera et al.

They perform experiments in a developed VE, where the analyzed parameters were task accuracy, interaction volume, update rate, and spatial distortion of accuracy. Also, they assess the system usability through SUS (Standard Usability Scale). The results obtained report that the ordinary camera approach presents good performance, inferior spatial distortion of accuracy, and larger interaction volume in comparison with Leap Motion controller. However, the use of Leap Motion allows a lower update rate. Also, they mentioned their technique presents high usability. Wang et al. [12] presented a text input technology for virtual environments that uses hand gesture recognition, they detail performance with different input postures and VR experiences. Their implementation uses a Leap Motion controller. They compared results with cameras pointed at both the dorsal side and the palmar side of the hand. The results revealed that the dorsal posture alternative shows a higher performance, higher typing accuracy, higher usability but it requires more physical effort for the user. They conclude that text input with dorsal posture has a greater application potential than with palmar posture. Also, they report that the user VR experience level has a minor effect. Schäfer et al. [6] presented a study of four techniques to control teleportation in virtual environments through hand gestures. This interaction is achieved with a Leap Motion sensor. The four techniques are described below: 1. TwoHandIndex: The right index finger casts out a visible ray that indicates the destination position. Likewise, the user points to the desired position, with the left hand, making a pointing gesture (index finger stretched, the others curled) to trigger teleportation. 2. TwoHandPalm: The open right-hand casts out the ray that indicates the destination, the user rotates the palm to choose the new position, the same pointing gesture with the left-hand triggers teleportation. 3. OneHandIndex: This can be done with either hand. The user points to the front with the index finger, the finger casts out a ray that indicates the destination. If a position is held for more than 1.5 s, teleportation is triggered. 4. OneHandPalm: It also can be done with either hand. The open hand casts out a ray that indicates the destination. If a position is held for more than 1.5 s, teleportation is triggered. Schäfer et al. conducted an experiment in which users completed goals in a VE using the teleportation techniques. Results reveal that the single hand techniques required less teleportations and the palm navigation approaches allowed less time to complete the goals. Also, the single hand techniques had a smaller number of tracking losses. They conclude that is viable to teleport successfully with a single hand only. Raess et al. [13] proposed a technique called TIVE (Thumb-Based Interaction for Virtual Environments). This technique involves an ordinary camera to track hands and thumbs with a low computational cost algorithm. The system considers an Angle of Inclination (AOI) to perform transitions between states, within each state other defined interactions take place. The technique works in two phases, learning and application. In the learning phase, a Support Vector Machine (SVM) is trained to recognize defined fistpostures with different AOI. In the application phase, a learned posture is recognized, and the corresponding state is activated. On input video, thumb trajectory is tracked with

A Literature Review of Hand-Based Interaction in Virtual Environments

117

dynamic mapping, with this process, the user can control a virtual hand. TIVE presents a mean accuracy of 89.7% in a moderate lighting real environment. Koreˇcko et al. [14] proposed three complementary components for applications that use A-Frame and Networked-Aframe technologies. The software framework A-Frame and the extension Networked-Aframe allow the development of custom applications to share virtual environments between multiple users in real-time. They introduce and assess three components for the following functionalities. The first component allows managing users in a shared virtual scene. The second component provides motion tracking for smartphone-based VR headsets. The third component adds hand-based interaction to the Microsoft HoloLens holographic computer. This hand-based interaction is achieved by combining the set of standard HoloLens gestures (tap, hold and drag), defining the following two single hand gestures. The first gesture, known as click-and-push, allows static manipulation of virtual objects. The other gesture, known as tap-and-drag, offers dynamic manipulation of virtual objects. They made experiments to measure performance with and without their introduced components. The interaction component achieved gesture recognition with total accuracy, moreover, the component was evaluated with the Standard Usability Scale. In other interaction results, all the users choose the tap-and-drag gesture as the more convenient because the block was visible during the movement. Next, we will describe two proposals that are not oriented to virtual reality applications, but we considered that their contributions can be implemented with hand-based interaction within virtual environments. Cruz et al. [3] developed a framework with three cameras where the hand is recognized through the background subtraction method. Their framework calculates several features (like arm angle and fingers position) using the variations of Y in the vertical contour image, the wrist is detected by calculating the greatest distance between a baseline and the hand contour. Their method achieves more than 68% successful gesture recognition on about 1800 images. Saboo and Singha [4] proposed an algorithm to detect and track hands. The first step of their system involves detecting hands through information of color and movement. Subsequently, the hand is tracked with the combination of these two algorithms: a modified version of KLT (Kanade–Lucas–Tomasi) feature tracker and color-based tracker Camshift. Their proposal solves common recognition problems like hand shape variation, presence of multiple persons, illumination variation, or occlusion with the background.

3 Classification of Interaction with Hands This section presents two classifications of interaction with hands, the first categorizes the interaction gestures, and the second groups the used methods. – Classification of hand gestures When considering incorporating hand-based interaction into a system, it is essential to choose and define the hand gestures that will allow accomplishing the system’s

118

C. Olvera et al.

requirements or expectations. To achieve a deeper understood of interaction related to hands, we propose six hand gestures categories described in Table 1. The first column specifies the name and the acronym, and the seconds describes the characteristics of the category. Table 1. Proposed classification of hand gestures. Gesture category

Description

Static gestures (SG)

One or both hands remain motionless [3]

Dynamic gestures (DG)

Hands have progressive changes of position and/or orientation following a defined behavior [3]

Palmar posture (PP)

The camera is pointed at the palmar part of the hand during the execution of the gesture. Also known as Hand-down [12]

Dorsal posture (DP)

The camera is pointed at the dorsal part of the hand during the gesture. Also known as Hand-up [12]

Single Hand (1H)

The gesture involves only one of the hands, usually the dominant hand

Both Hands (2H)

The interaction requires the simultaneous use of both hands

It can be observed that a gesture can belong to more than one category. In the interaction flow of an application, SGs are used as a digital input, the system must recognize the difference between the absence and presence of the SG. In the case of DGs, they can be used as both digital and analog inputs, although more benefit can be obtained with the analog inputs. DG recognition requires more computational processing as it is usually important to detect the positions and orientation of the hands or fingers in real-time, the process known as tracking. As some of the systems mentioned in the previous section reveal, gestures with 2H and PP present more recognition failures. Having analyzed the interaction systems in the previous section, the following interaction guidelines are proposed to achieve hand-based interaction with low computational cost and higher recognition accuracy: 1. 2. 3. 4. 5.

Grant preference to hand interaction gestures in dorsal posture over palmar posture. Grant preference to interaction with static gestures over dynamic gestures. Grant preference to interaction with single hand gestures over both hands gestures. Grant preference to 3D control through palm orientation over fingers orientation. Consider using markers on specific parts of the hand (finger coverages or bracelets).

As it is known, virtual reality applications currently require a relatively high computational processing, if there are technological limitations, we point out the feasibility that can be achieved by discarding some gesture categories. We reiterate that the presented guidelines are oriented to achieve hand-based interaction with low computational cost, we also recommend considering other interaction

A Literature Review of Hand-Based Interaction in Virtual Environments

119

guidelines with different purposes. Hereafter, some examples that can be found in the literature are introduced. Wu et al. [18] propose guidelines to address the gesture disagreement problem in hand-based interaction. Bekele et al. [19] present interaction recommendations to set up a contextual relationship between users and achieve cooperation between them in extended reality applications. Zhou et al. [20] provide guidelines to design the layout of interactive components in VEs with hand interaction. – Classification of hand gestures recognition methods The reviewed proposals demonstrate that the process of gesture recognition and hand tracking can be accomplished with different methods. In Table 2 we present a classification, according to the methods used. The first column shows the name and acronym of the category, the second column lists the used methods that belong to the category, and the third column explains for what purposes these methods are used. Table 2. Proposed classification of hand gesture recognition methods. Category

Specific methods

Description

Direct Mapping (DM)

Not Apply

Calculating movement or rotation of the user’s hands (2D or 3D)

Methods based on probability Unscented Kalman filter and statistics (PS) (UKF) Improved particle filter (IPF) Maximum entropy model Hidden Markov classification (HMC)

Filters to improve direct mapping Classification or inference of hand gestures

Machine learning (ML)

Support Vector Machine (SVM)

Recognition of hand gestures through supervised training

Image processing (IP)

Thresholding and tracking of Advanced detection and color on images with HSV tracking of objects or patterns model in a video Sliding scan algorithm Background subtraction method Kanade–Lucas–Tomasi feature tracker Color-based tracker Camshift

4 Comparative Analysis of Hand-Based Interaction Systems This section presents an analysis of the proposals presented in the previous section and identifies some current challenges in the field.

120

C. Olvera et al.

The hand recognition and tracking are a field with many challenges ahead. Most of the proposals present accuracies between 80% and 90%. In addition, these systems require certain appropriate circumstances in the environment such as: illumination, occlusion, among others. Even with these limitations, it can be observed that recent proposals have allowed functional and satisfactory applications in the field of virtual reality. Undoubtedly, computer vision has allowed an obvious advance in hand-based interaction, however, it must be compared with the existing alternatives. Most of the solutions oriented to virtual reality base their interaction on additional devices with buttons. Users are accustomed to button-based input devices with total accuracy. Most of the discussed systems were not able to detect all the users’ gestures, this in a low amount but may be noticeable concerning the known alternatives. For these reasons, achieving hand recognition and tracking systems with accuracies remarkably close to 100% is especially important for the approach to proliferate in virtual reality and, thus, to take advantage of the naturalness and creativity offered by this approach. On the other hand, gestures recognition through computer vision involves established camera hardware, the compiled proposals use these alternatives: ordinary camera (OC), Leap Motion Controller (LM), Microsoft Kinect (MK), and Microsoft HoloLens (HL). The dedicated hardware (LM, MK & HL) is widely used in the field, and the following positive aspects are identified: good accuracy, low latency, good update rate, and API provided. Among its negative points, the following can be considered: limited compatibility, high cost, and limited distribution. The negative aspects make a virtual reality application using LM, MK or HL difficult and expensive to access. In Table 3 are identified the gestures (SG, DG, PP, DP, 1H, 2H) and method categories (DM, PS, ML, IP) involved in each proposal. In addition, the table indicates the camera hardware used (OC, LM, MK, HL), and if the proposal was evaluated with the Standard Usability Scale (SUS). Table 3. Comparative analysis of the key features of the reviewed proposals. Categories and hardware involved in each proposal as well as application of SUS.

A Literature Review of Hand-Based Interaction in Virtual Environments

121

It is not surprising to see that single hand gestures appear in all proposals. Dynamic gestures were used in almost all the proposals. A congruent implication can be identified, if a direct mapping is implemented, dynamic gestures are used. The most common hardware was Leap Motion controller followed by ordinary cameras. Direct mapping and image processing algorithms were the most used methods. SUS was not widely used, perhaps because it provides an evaluation in very general terms.

5 Conclusions Although the field of virtual reality emerged several decades ago, it is not widely used due to several factors: usability, accessibility, cost, technological requirements, prejudices and others. For this reason, the topic presents research opportunities in most of its aspects. Consequently, it was decided to carry out a study on one of its elemental aspects: the interaction with virtual environments, through of a promising approach based on hand gestures. As can be seen in the description of the analyzed systems, there are hand gesture recognition proposals that present good results with low technological requirements, several of them consist of ordinary cameras and/or low computational cost algorithms. These contributions can be particularly significant in driving access and diffusion of virtual reality applications. It is known that virtual reality has limited use due to its high requirements, therefore, research aimed at enabling virtual reality applications with accessible requirements is important and meritorious. In this context, it is considered possible to develop highly accessible applications that offer virtual reality with hand-based interaction. This can be achieved with mobile devices. The interaction can be performed by computer vision using the camera(s) integrated into the device. Virtual reality can be achieved through a low-cost headset for mobile devices that does not obstruct the field of view of the cameras. These headsets allow the user to keep their hands in the field of view of the cameras to interact with virtual environments. The interaction guidelines and methods mentioned in previous sections can be followed to accomplish hand gesture recognition with low computational cost. As future work, it is proposed the development of an application for mobile devices that allows the user to select, move and rotate virtual elements as well as to teleport in a virtual environment. We consider implementing interactions by means of three static gestures and one dynamic gesture based on a single hand with dorsal posture. Computer vision through image processing is intended to be used to recognize the interaction gestures. Experiments are planned to measure recognition accuracy, time to complete defined goals, and evaluate usability using SUS. With the proposed work, is intended to achieve hand-based interaction with virtual environments through low-cost and easily accessible technologies. This research addresses the crucial challenges in the field and can contribute to making virtual reality a part of everyday life in society.

122

C. Olvera et al.

References 1. Bowman, D.A., Hodges, L.F.: An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 Symposium on Interactive 3D Graphics, pp. 35–ff, Apr 1997 2. Petersen, N., Stricker, D.: Continuous natural user interface: reducing the gap between real and digital world. In: ISMAR, pp. 23–26, Oct 2009 3. Cruz Bautista, A.G., González-Barbosa, J.J., Hurtado-Ramos, J.B., Ornelas-Rodriguez, F.J., González-Barbosa, E.A.: Hand features extractor using hand contour–a case study. Automatika 61(1), 99–108 (2020) 4. Saboo, S., Singha, J.: Vision based two-level hand tracking system for dynamic hand gestures in indoor environment. Multimedia Tools Appl. 80(13), 20579–20598 (2021). https://doi.org/ 10.1007/s11042-021-10669-7 5. Caputo, F.M.: Gestural interaction in virtual environments: user studies and applications. Doctoral Dissertation, University of Verona (2019) 6. Schäfer, A., Reis, G., Stricker, D.: Controlling teleportation-based locomotion in virtual reality with hand gestures: a comparative evaluation of two-handed and one-handed techniques. Electronics 10(6), 715 (2021) 7. Vaitkeviˇcius, A., Taroza, M., Blažauskas, T., Damaševiˇcius, R., Maskeli¯unas, R., Wo´zniak, M.: Recognition of American sign language gestures in a virtual reality using leap motion. Appl. Sci. 9(3), 445 (2019) 8. Fittkau, F., Krause, A., Hasselbring, W.: Software landscape and application visualization for system comprehension with ExplorViz. Inf. Softw. Technol. 87, 259–277 (2017) 9. Melero, M., et al.: Upbeat: augmented reality-guided dancing for prosthetic rehabilitation of upper limb amputees. J. Healthc. Eng. 2019, 1–9 (2019) 10. Du, G., Zhang, B., Li, C., Yuan, H.: A novel natural mobile human-machine interaction method with augmented reality. IEEE Access 7, 154317–154330 (2019) 11. Rehman, I.U., et al.: Fingertip gestures recognition using leap motion and camera for interaction with virtual environment. Electronics 9(12), 1986 (2020) 12. Wang, Y., et al.: Investigating the performance of gesture-based input for mid-air text entry in a virtual environment: a comparison of hand-up versus hand-down postures. Sensors 21(5), 1582 (2021) 13. Raees, M., Ullah, S., Rehman, I.U., Azhar, M.: Thumb inclination-based manipulation and exploration, a machine learning based interaction technique for virtual environments. Mehran Univ. Res. J. Eng. Technol. 40(2), 358–370 (2021) 14. Koreˇcko, Š, Hudák, M., Sobota, B., Sivý, M., Pleva, M., Steingartner, W.: Experimental performance evaluation of enhanced user interaction components for web-based collaborative extended reality. Appl. Sci. 11(9), 3811 (2021) 15. Du, G., Zhang, P.: A markerless human–robot interface using particle filter and Kalman filter for dual robots. IEEE Trans. Ind. Electron. 62(4), 2257–2264 (2014) 16. Du, G., Zhang, P., Liu, X.: Markerless human–manipulator interface using leap motion with interval Kalman filter and improved particle filter. IEEE Trans. Industr. Inf. 12(2), 694–704 (2016) 17. Kruse, D., Wen, J.T., Radke, R.J.: A sensor-based dual-arm tele-robotic system. IEEE Trans. Autom. Sci. Eng. 12(1), 4–18 (2014) 18. Wu, H., Zhang, S., Liu, J., Qiu, J., Zhang, X.: The gesture disagreement problem in free-hand gesture interaction. Int. J. Hum.-Comput. Interact. 35(12), 1102–1114 (2019) 19. Bekele, M.K., Champion, E.: A comparison of immersive realities and interaction methods: cultural learning in virtual heritage. Front. Robot. AI 6, 91 (2019) 20. Zhou, X., Jin, Y., Jia, L., Xue, C.: Study on hand-eye cordination area with bare-hand click interaction in virtual reality. Appl. Sci. 11(13), 6146 (2021)

MLV-Viewer: Universal Decision Support System Manuel Pérez Cota1(B) , Carlos Manuel Oliveira Alves2 , and Miguel Ramón González Castro3 1 Universidade de Vigo, Rúa Torrecedeira 86, 36208 Vigo, Spain

[email protected]

2 Instituto Politécnico de Castelo Branco, Av. D. Pedro Álvares Cabral 12,

6000 Castelo Branco, Portugal [email protected] 3 Universidade de Vigo, Rúa Torrecedeira 86, 36208 Vigo, Spain [email protected]

Abstract. A universal decision support system must be supported by a computer in an interactive way and in real time, with the possibility of representing data in a graphical format, supporting the user/decision maker in the cognition process. Through appropriate tools, methods and techniques, it is possible to support the user in understanding the data, especially involving a large volume and multidimensional data. This way, visual and interactive representations enable the decision maker to obtain a more effective view of the data, combining flexibility, creativity and knowledge. The decision maker must always have the chance to choose and customize, according to their needs and preferences. In this article, the MLV-Viewer prototype will be presented, a universal interactive decision support system with real-time response for multilevel data visualization associating a dataset to a symbol. Keywords: UDSS · Data analytics in DSS · Big data visualization

1 Introduction Information systems to support the user’s decision making appeared in the 1960s, with the name of Decision Support System (DSS) [1]. Data visualization in current times can only be seen dynamically, in real time, in virtual, networked environments and supported by computer graphics tools, methods and techniques. Only in this way will the user, that is, the decision maker, obtain a generic and global perception of the problem at hand. Sometimes it is necessary to “forget” the details to focus on the global view of the problem. Data visualization must enable the user to have graphic models adjusted to their needs and realities, as data visualization is nothing more than a “graphical representation of data with the main objective of providing the user with a qualitative and easy understanding of the content of information (which can be data, relationships, perceptions or processes)” [2], transforming data, objects, numbers, concepts, among © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 123–133, 2022. https://doi.org/10.1007/978-3-030-89909-7_10

124

M. P. Cota et al.

others, into representations easily interpreted by the human eye, allowing the user to select, filter and transform these data and choose the correct type of visualization (such as bar, line or bubble charts, among others). Visualization tools have the ability to make data, its representations and relevant information more easily visible by highlighting data that would otherwise be hidden. Therefore, the integration of visualization techniques in dynamic [3, 4] and complex systems makes possible the abstraction of these same data. Visual data analysis is important for the decision maker because it reduces and synthesizes the search for large volumes of data and facilitates the task of exploring and obtaining relevant information from that data. This article will analyze a Universal Decision Support System (UDSS) prototype, called MLV-Viewer, developed for web environments, at no cost to the user, executed in real time, and customizable to the needs of the decision maker. A UDSS should be interpreted as a DSS not specific to an area or activity, as well as to a user without specific computer knowledge. A review of the literature is presented in section “2 Literature review”, in section “3 MLV-Viewer” the prototype MLV-Viewer is described and, finally, in the section “4 Conclusion and Future Work” some conclusions and future work are elaborated.

2 Literature Review A DSS is nothing more than a tool, usually interactive and computer-based, to support decision makers in solving problems in the most diverse areas of business/activity [3– 19]. The main components of a DSS tool are: the decision maker; the database; models and procedures; the program. The system presented at [20], called “Web-based multi-level job assignment DSS” (WMJADSS) is a continuation of previous work (WJADSS - “Web Based Task Assignment Decision Support System”), complementing the process of multi-level work assignment, usually found in a project’s workflow, so that existing models are adjusted to support the single and multilevel job designation. The effectiveness and efficiency of the work done, throughout the project workflow, are increased and the data stored in the repositories are used to produce descriptive and procedural knowledge for the project’s stakeholders. The study [21] analyzes the architecture of the decision support system applied to Tariffs and Trade, facilitating data analysis and trade forecasting, allowing the integration of reporting and information retrieval tools, supporting decision making by providing economic quantitative analysis and system modeling tools. Multidimensional data warehouse modeling techniques as well as artificial intelligence are used. According to [22], the 20 best tools for data visualization are grouped into two sets: those aimed at the end user and those aimed at programmers/developers. Of the first are listed: • Tableau: Big Data visualization tool for companies, allowing, among others, to create tables, graphs, maps and graphs. It can work as a desktop application or as a service hosted on a cloud server;

MLV-Viewer: Universal Decision Support System

125

• Infogram: allows associating visualizations and infographics with Big Data in real time, allowing the user (media editors, journalists, designers and educators) to choose and share between different models of graphs, maps, images and videos; • ChartBlocks: is an online tool that allows the creation of visualizations from spreadsheets, databases and live feeds. A graphics construction wizard, working on HTML5 and the JavaScript library D3.js, creates responsive visualizations for different devices, and also allows the insertion of the graphics created on web pages or to share them on Twitter and Facebook; • Datawrapper: being aimed at editors and journalists, it allows creating, uploading and publishing graphics and maps, with customized layouts, on web sites; • Plotly: it is an easy-to-use tool for the web to create graphics, but if the user has computer skills, he can use an API that includes JavaScript and Python languages; • RAW: based on data from Microsoft Excel, Google Docs, Apple Numbers or a commaseparated list, RAW exports the visualization to Adobe Illustrator, Sketch or Inkscape, thus bridging spreadsheets and vector graphics; • Visual.ly: is a visual content service, in which a team will provide support for the entire duration of the project.

3 MLV-Viewer The present work consists of a universal DSS prototype, called MLV-Viewer, supported in a relational database (currently the SGBD MySQL), in the languages of PHP and

Fig. 1. MLV-Viewer ontology

126

M. P. Cota et al.

JavaScript and the graphic libraries canvasJs, Chart.js, JpGraph, HIGHCHARTS and three.js, enabling the user (who does not need to be an expert in computers) through a web environment to select and interact with the data they want to view and the graphical representation for that visualization. Figure 1 shows the MLV-Viewer ontology. The MLV-Viewer allows the existence of users with different profiles, and, obviously, different privileges to access and view data, they are: “guest”, “viewer”, “decision maker” and “admin”. The admin the profile with the most features, can import and export data, manage users, view data (in 1C, 2C, 3C, 4C, 5C, 6C or 7C formats) and perform data analysis. With the work developed, a universal DSS interactive and in real time, multiplatform, of free access, was built to be used in a web environment and adaptable to the needs of the decision maker (through several user profiles), allowing the user to choose several ways to visualize the same data (dynamically selected by himself) opting for views 1C, 2C, 3C, 4C, 5C, 6C or 7C. The MLV-Viewer uses the concept of layered visualization (also called level) associated with a symbol. The views 1C and 2C can be compared to the views 1D and 2D in which the axes XX and YY are used to represent data. The 3C view is similar to a 3D view, but with the caveat that it is actually a 2.5D view, thus emulating 3D on a device that is a plane/surface and therefore only has two dimensions. In the MLV-Viewer prototype, the 4C visualization (see Fig. 1) adds to the 3C visualization (emulated in 2.5D) a layer to the symbol used in the data representation. This layer is the size of the symbol used to represent the data, which is proportional to the value of the data. The symbol used in the visualization is the circle. The diameter of the circle is proportional to the value of the data to be visualized, and in this way we obtain the 4C visualization in the MLV-Viewer. The same principle is applied in views 5C, 6C and 7C, with new layers being added to achieve these new views. The 5C view is made from the 4C view by adding the color as a new layer. The same color is always associated with the same type/value of data. The 6C view is made from the 5C view by adding the shape as a new layer. The same type/value of data is always associated with the same form. The 7C view achieved by adding a new outer layer (like an onion with the various layers) to the symbol of the 6C view. This new layer is represented by a variable thickness. Figure 2 shows the views in 1C (list in XML format, or list with item figure).

Fig. 2. 1C view

MLV-Viewer: Universal Decision Support System

127

Figure 3 presents several views in 2C format (bars with figures, pyramid, item image with variable size, and graph).

Fig. 3. 2C view

Figure 4 shows the views in 3C format (horizontal bars and lines).

Fig. 4. 3C view

The data analysis is presented to the user either through vertical bars with several sections (corresponding to the quantities of products with the same characteristic) or through horizontal lines (see Fig. 5).

128

M. P. Cota et al.

Fig. 5. Data analysis

In the 4C view the XX, YY and ZZ axes and the radius (of the symbol) are used to obtain the 4 levels/layers. The 5C view adds color to the 4C view. The 6C view adds to the 5C view the shape of the symbol. The 7C view adds to the 6C view an outline outside the symbol with a thickness according to the amount of data to be represented. See Fig. 6.

Fig. 6. 4C, 5C, 6C and 7C views

In order to carry out a critical analysis, the tools described in Sect. 2 were compared, that is, the tools analyzed were Tableau, Infogram, ChartBlocks, Datawrapper, Plotly, RAW and Visual.ly (see Table 1).

MLV-Viewer: Universal Decision Support System

129

Legend: V - means it has the feature X - means it doesn’t have the feature V1 - but also paid V2 - Marketing, Media, ONG, government, reports, teams V3 - only PDF, PNG, MP4 V4 - only PDF, PNG, SVG, EPS, OS X5 - only PNG V6 - is a free library, but it also has a cost in the Dash Enterprise version X7 - only for Clinical Analytics and Government Analytics X8 - only for Sales Forecasting Table 1. Comparison between MLV-Viewer and other tools Visualization Web Free Univ Data Data Data Visua Data Data tool ersal Import Export Visua lization Mining Prediction lization 4C,5C, 6C,7C Tableau

X

X

V

V

V

V

X

V

X

Infogram

V

V1

V2

V

V3

V

X

X

X

ChartBlocks

V

V1

V

V

V4

V

X

X

X

Datawrapper

V

V1

V

V

X5

V

X

X

X

Plotly

V

V6

V

V

X

V

X

X7

X8

RAW

V

V

V

V

V

V

X

X

X

Visual.ly

V

X

V

V

X

V

X

X

X

MLV-Viewer

V

V

V

V

V

V

V

V

V

From the previous table it can be concluded that practically all the tools allow the use in web environment, they are free or have free versions, they are of universal use, they allow the import and export of data, and they allow a diversity of graphs for data visualization. However, for the parameters “Data Mining”, “Data Prediction” and “Data Visualization/View in 4C, 5C, 6C, 7C” the scenario is completely different. Regarding the “Data Mining” parameter, only Tableau and Plotly (with some limitations) allow this functionality. For the “Data Prediction” parameter, only Plotly (with some limitations) allows this functionality. Finally, for the parameter “data visualization/view in 4C, 5C, 6C, 7C” none of the tools, except the MLV-Viewer, allow this functionality. A SWOT analysis of the MLV-Viewer prototype is presented below: • Strengths: • Ability to innovate: the present team of researchers has the necessary skills and abilities to design innovative products;

130

M. P. Cota et al.

• Free technologies and tools: in the development free or tendentially free technologies and tools are used, and this way, the end user will not have costs of acquisition and use; • Weaknesses: • Development team: being a team with a reduced number of elements, it takes time to produce a product or a new feature; • Disclosure capacity: because it is not a company, it does not have marketing resources, making product disclosure more difficult and slow; • Opportunities: • Innovative product: there is nothing similar on the market (as far as it is possible for authors to be aware of non-existence); • Attracting a large and wide target audience: being a product not only focused on an area of activity or business, it makes it more global; • Threats: • attractive and innovative concept that will certainly attract imitation/reproduction attempts.

4 Conclusion and Future Work It is important that the user has the possibility to choose and decide what, when and how he wants to view his data. The system/tool should also advise on the most suitable visualization to the area of business/activity, because a poor choice not only makes an analysis difficult, but it can even make it impossible to obtain the desired information. It must be clear and very explicit that a decision support visualization tool must meet the specifics of each use and situation, not overloading it with data and information that are of no interest in decision making, but, on the other hand, it is required that a good system/tool helps the user to obtain meaningful information in a simple, friendly and customizable way. Visualization and computer graphics must be understood as dynamic, virtual and networked, enabling the user of these tools and systems to obtain a complete idea of a scenario before focusing on the detail, that is, a visualization does not mean just visualization, but it also allows visualizing what is not seen in a natural and immediate way, with the main focus of providing a qualitative, quantitative, easy and quick understanding of the information content (which can be data, relationships, perceptions or processes), transforming objects, numbers and concepts in a way that can be easily captured by the human eye and interpreted by the brain. It is important that the user can see what he wants, when and how he wants. Whenever the views are properly associated, interpreting the data will be facilitated. An incorrect choice of display type can prevent the user from obtaining the desired information.

MLV-Viewer: Universal Decision Support System

131

The development of a UDSS is not easy and must be configurable in order to meet the specificities and needs of each decision maker, because although the data/information may be displayed graphically, users may not be able to understand everything that is being displayed or may be overloaded with too much data to be displayed at the same time. A good tool for DSS should help decision makers to find meaningful information quickly, simply, friendly and customizable, as well as suggest the best view to obtain that information. The visualization of information contained in data using graphs or images is a technology in continuous development, transforming the traditional methods of representation and analysis of relational data into structures and relationships more easily interpreted by humans. These structures and relationships facilitate dialogue, communication, analysis and interpretation in dynamic, complex and real-time environments. The complexity normally associated with a large volume of data can be mitigated through mining algorithms and data visualization to reveal hidden knowledge or patterns. The UDSS prototype presented here, called MLV-Viewer, intends to implement a dynamic and interactive system, in a web environment, using a diverse range of data visualizations and data mining to convert data into meaningful information. Its advantages are: multiplatform; free; customizable to the user; in real time; import and export data; different types of visualizations and with multiple dimensions; data analysis and forecasts; universal. The process of investigating is not easy, it is not enough to just want to. It is a process that, given a lot of work, requires methodology, skill, expertise, dexterity, proficiency, creativity and luck, which, most likely, is not available to everyone. Investigation is not just about having the luck of having an apple dropped on your head. But it is also necessary to know where to go in search of innovation. In the same way that a computer application is never finished, the MLV-Viewer prototype also has many ways to be improved. Among others, the following stand out: • Implement new features for different user profiles such as “create”, “insert”, “change”, “delete” and “view”. These features would be associated with the database, tables, data or types of visualization, that is, in this way, the user could be limited to viewing data (possibly only some data, eg, not being able to perform data prediction), not being able to export data, not being able to create new views on existing data, etc.; • Advise a visualization set that best adapts to the activity/business area and respective data; • Allow the user to create and customize a visualization; • Explore other forms of visualization, e.g., using virtual/augmented reality; • Develop visualizations greater than 7C (8C, 9C, etc.); • Explore video animation associated with data or a symbol (whenever the course is placed over that symbol).

References 1. Power, D.J.: Decision support systems: a historical overview. In: Handbook on Decision Support Systems 1. International Handbooks Information System, pp. 121–140. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-48713-5_7

132

M. P. Cota et al.

2. Kumar, S.M., Belwal, M.: Performance dashboard: cutting-edge business intelligence and data visualization. In: Proceedings of the 2017 International Conference on Smart Technology for Smart Nation, SmartTechCon 2017, pp. 1201–1207 (2018). https://doi.org/10.1109/SmartT echCon.2017.8358558 3. Grignard, A., Drogoul, A., Zucker, J.: A Model-View/Controller approach to support visualization and online data analysis of agent-based simulations. In: The 2013 RIVF International Conference on Computing and Communication Technologies - Research, Innovation, and Vision for Future (RIVF), pp. 233–236 (2013) 4. Cota, M.R.G., Castro, M.P., Domínguez, J.Á.: Importance of visualization usage in enterprise decision making environments. In: 2014 9th Iberian Conference on Information Systems and Technologies (CISTI) (2014) 5. Ellouzi, H., Ltifi, H., Ben Ayed, M.: New Multi-Agent architecture of visual intelligent decision support systems application in the medical field. (2015). https://doi.org/10.1109/AIC CSA.2015.7507135 6. Yan, X., Qiao, M., Li, J., Simpson, T.W., Stump, G.M., Zhang, X.: A work-centered visual analytics model to support engineering design with interactive visualization and data-mining, pp. 1845–1854 (2012). https://doi.org/10.1109/HICSS.2012.87 7. Jorgensen, M., Spohn, J., Bunn, C., Dong, S., Li, X., Kaeli, D.: An interactive big data processing/visualization framework. In: 2017 IEEE MIT Undergraduate Research Technology Conference, URTC 2017, vol. 2018, pp. 1–4, Jan 2018. https://doi.org/10.1109/URTC.2017. 8284188 8. Bencsik, G.,Bacsárdi, L.: Towards to decision support generalization : the universal decision support system concept. In: 2015 IEEE 19th International Conference on Intelligent Engineering Systems (INES), pp. 277–282 (2015) 9. Kozielski, M., Sikora, M., Wróbel, Ł.: DISESOR - decision support system for mining industry. In: Proceedings of the Federated Conference on Computer Science and Information Systems, vol. 5, pp. 67–74 (2015). https://doi.org/10.15439/2015F168 10. Yu, S., Deng, L., Zhang, Y.: Visualization user interface for decision support systems. In: Proceedings of 2009 9th International Conference on Hybrid Intelligent System HIS 2009, vol. 1, pp. 63–66 (2009). https://doi.org/10.1109/HIS.2009.20 11. He, X., Bian, J.: Towards an interactive visualization for dietary supplement knowledge graph. In: 2019 IEEE International Conference on Healthcare Informatics, ICHI 2019, vol. 324, no. 7337, p. 9138 (2019). https://doi.org/10.1109/ICHI.2019.8904618 12. Chamberland, C., Vachon, F., Gagnon, J.F., Banbury, S., Tremblay, S.: Time-oriented visualization and anticipation. In: IEEE Conference on Visual Analytics Science and Technology, VAST 2012 - Proceeding, pp. 215–216 (2012). https://doi.org/10.1109/VAST.2012.6400546 13. Zyanchurin, A.E., Okhtilev, M.Y., Sokolov, B.V.: System of indicators of the quality of humanmachine interaction in three-dimensional visualization in decision support systems. In: Proceedings of 2017 IEEE 2nd International Conference on Control Technology System CTS 2017, pp. 24–25 (2017). https://doi.org/10.1109/CTSYS.2017.8109478B 14. Marques, B., Santos, B.S., Araújo, T., Martins, N.C., Alves, J., Dias, P.: Situated visualization in the decision process through augmented reality. In: Proceedings of International Conference on Information Visualisation, vol. 2019-July, no. iv, pp. 13–18 (2019). https://doi.org/10.1109/ IV.2019.00012 15. Gu, J., Mackin, S., Zheng, Y.: Making sense: an innovative data visualization application utilized via mobile platform. In: Proceedings of 20th International Conference on High Performance Computing and Communications, 16th International Conference on Smart City 4th, International Conference on Data Science System HPCC/SmartCity/DSS 2018, pp. 1105–1109 (2019). https://doi.org/10.1109/HPCC/SmartCity/DSS.2018.00184

MLV-Viewer: Universal Decision Support System

133

16. Sonmez, F.O., Kilic, B.G.: A decision support system for optimal selection of enterprise information security preventative actions. IEEE Trans. Netw. Serv. Manag. XX(XX), pp. 1–20 (2020). https://doi.org/10.1109/TNSM.2020.3044865 17. Cota, M.P., Rodríguez, M.D., González-Castro, M.R., Gonçalves, R.M.M.: Massive data visualization analysis analysis of current visualization techniques and main challenges for the future. In: 2017 12th Iberian Conference on Information Systems and Technologies (2017). https://doi.org/10.23919/CISTI.2017.7975704 18. Cota, M.P., Rodriguez, M.D., Castro, M.R.G., Goncalves, R., Branco, F., Martins, J.: Viewing hidden data using Hadoop. In: 2018 13th Iberian Conference on Information Systems and Technologies (2018). https://doi.org/10.23919/CISTI.2018.8399400 19. Paradice, D.: Two grand challenges for DSS evolution. In: Papathanasiou, J., Zaraté, P., Freire de Sousa, J. (eds.) EURO Working Group on DSS. ISIS, pp. 33–49. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-70377-6_3 20. Vongsumedh, P.: A framework for building a decision support system for multi-level job assignment (2009). https://doi.org/10.1109/ICCGI.2009.19 21. Yu, C.: Architecture research of decision support system for tariff and trade based on the multi-dimensional modeling techniques. In: 2013 IEEE Third International Conference on Information Science and Technology (ICIST), pp. 883–888 (2013) 22. L’Astorina, E.: Review of 20 best big data visualization tools, 19 Sep 2017. https://bigdatamadesimple.com/review-of-20-best-big-data-visualization-tools/. Accessed 12 Dec 2020

Software Systems, Applications and Tools

Towards the Evaluation of Relevant Interaction Styles for Software Developers Adriana Peña Pérez Negrón1 , Mirna Muñoz2 , David Bonilla Carranza1(B) , and Nora Rangel3 1 Computer Science Department, Universidad de Guadalajara CUCEI, Blvd. Marcelino García

Barragán #1421, C.P. 44430 Guadalajara, Jalisco, Mexico [email protected], [email protected] 2 Centro de Investigación en Matemáticas, Unidad Zacatecas, Parque Quantum, Cd. del Conocimiento, Av. Lasec And. Galileo Galilei Mz 3 L7, C.P. 98160 Zacatecas, A.C, Mexico [email protected] 3 Universidad de Guadalajara CEIC, C. Fco. de Quevedo #180, Col. Arcos Vallarta, C.P. 44130 Guadalajara, Jalisco, Mexico [email protected]

Abstract. Software development is the result of human endeavor, then the impact of the human factor would support its success. It has been observed that people consistently behave when facing a similar situation, revealing interaction styles. In the interaction styles observation, the use of games that engage the players facilitates behavior analysis beyond self-evaluations. In this study, video games were applied to analyze selected interactive styles expected to be present during software development, that is, a situation that requires persistency, that present ambiguity, or generate frustration. This study aims to present an approximation for the evaluation of the interactive style, and at the same time to understand the video game characteristics suitable for such situations. Results show that the selected video games are an alternative to evaluate persistency and ambiguity, however for frustration different alternatives should be explored. Keywords: Software engineering · Soft skills · Interactive styles · Human factor · Software development

1 Introduction Software development is a human-centered process, an activity currently hardly grounded in teams. Then, the proper allocation of people in a software development project is crucial for success, productivity control, and improvement. As such, technical factors are decisive, but also non-technical skills which are undergoing in an increasing Please note that the LNCS Editorial assumes that all authors have used the western naming convention, with given names preceding surnames. This determines the structure of the names in the running heads and the author index. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 137–149, 2022. https://doi.org/10.1007/978-3-030-89909-7_11

138

A. P. P. Negrón et al.

number of studies [1–5]. Non-technical skills, denominated also as people skills, transferable skills, social skills, generic competencies, or as will be referred to here, soft skills, have been tried to defined and characterized without reaching consensus [1]. Matturro, Raschetti, and Fontán [1], executed a systematic literature mapping, identifying four components that they used to define soft skills as the combination of abilities, attitudes, habits, and personality traits that allow people to better perform. Human factors in the software development process have been studied and classified from different perspectives such as psychological, cognitive, management, and technical; and from different levels that include organizational, interpersonal, and individual [4], and from a broad Software Engineering (SE) perspective through Project Management, and team organization to software developers [1, 4]. It seems that while hard skills for a particular position are somehow easy to measure according to the project requirements, soft skills categorization is not close to formalization in the software industry, as well as its evaluation [4–6, 8]. In any case, from a granularity point of view, at the individual level, the software developer engineer looks like a proper starting point to study the involvement of soft skills in the software development process. Usually, the set of activities that comprise software development includes analysis, design, coding, and testing; activities that demand a broad set of abilities to carry out them effectively, both hard and soft skills [6]. At the individual level, soft skills in the literature cover psychological issues such as personality types, characteristics, mentality patterns, judgment, bias, and motivations; cognitive issues such as learning skills and types, abstract thinking, knowledge sharing, training; and, management skills like individual process and improvement, estimation, and customer satisfaction [6]. In a team, its members’ personality influences the way they relate to each other, affecting their performance and the team outcome [7]. From an individual and psychological perspective, according to [9], most of the studies carried out on the human factor in information systems concentrate primarily on personality types, mainly to link certain software development roles to certain personality traits. However, these studies do not specify at which phase of the software life cycle they occur or how they are related. In psychology traditional perspectives, personality is understood in terms of traits, assuming that people will behave accordingly to the way they are, regardless, or at least, minimally biased by circumstances. As an alternative, from a behavioral point of view, Ribes [10] changed the emphasis to the interaction of individuals and the situations they faced and proposed the concept of interactive styles. This concept refers to consistent and idiosyncratic ways in which individuals handle situations. Ribes [10] argued that an individual will show a consistent way of behavior in situations involving the same functional structure or contingency arrangement. Following this reasoning, each individual behaves particularly, and its behavior will be constant for similar situations. Built on Ribes’s [10] proposal, a behavioral study is conducted to select functional structures present in the process of software development (i.e. persistency, ambiguity, and frustration). This study aims to achieve an approximation to measure such interactive styles, and at the same time establishing the video game characteristics that present functional structures to observe them.

Towards the Evaluation of Relevant Interaction Styles

139

2 Background Muñoz et al. [11] presented a model to integrate highly effective teams for software development focused on three factors: skills, interactive styles, and knowledge as key aspects to select people according to context and project needs. The model’s first element involves software development phases with the software development cycle using the TSP methodology, which provides guidelines to detail the required activities to be performed on each phase by all the team members. Then, the use of gamification techniques is indicated to construct similar to the first element scenarios, to support the analysis of the individuals’ skills, capacities, and knowledge to integrate the team. Following this model [11], Rangel et al. [12] considered that software developers through the software process could face different situations and express them in terms of the arrangement of contingencies. Individual interactive styles were then linked to Table 1. Evaluation stages for TSPi roles (taken from Rangel et al. [12]) Stage

Role

Interactive style

Interactive style description

1

Team member

Achievement or persistency

Keeping the performance under conditions of growing or higher requirements

2

Team member as responsible for a phase

Decisions/Risk

The possibility of only one response to stimulus that are competitive or uncertain in time

Signal dependency

Adjusting responses to redundant signs in the contingency

Flexibility to change

Changes in responses when facing signaled, not signaled, or unspecified contingencies

Ambiguity

Responses under functional antagonist proprieties, and/or differences between the signal and the supplementation

Conflict reduction

Responding to the concurrent opposite or competitive signs

Frustration

Keeping the performance under conditions with no signals of interference, diminishing, loss or delay of the consequences

3

Team leader

Implementation and Test managers

140

A. P. P. Negrón et al.

similar situations according to the team role activities. Rangel et al. [12] analysis derived in suggesting a three-stage evaluation according to the role, see Table 1. According to the authors, the achievement or persistency evaluation will allow an understanding of how the software engineers will assume requirement increments. The team responsible for a phase involves risk, therefore making decisions and taking risks would be helpful to understand under which parameters team members would or not make decisions or take risks. Also, signal dependency evidence if the team can be altered from events not relevant to the phase criteria. Software projects are dynamic, therefore certain flexibility to change is pertinent. Decision-making involves situations in which people respond under two alternatives, and sometimes, one or both of the alternatives involves the lack of information or relevance discrimination creating ambiguous situations. And the team leader will require to manage conflict, in situations that require a response to signals that might opposite to each other. Finally, how the leader would behave under frustration contingencies will be required given unexpected results. Following this approach [12], towards an evaluation of the interactive styles in Muñoz et al. [13] were proposed a degree of concordance between TSPi roles and interactive styles based on a six-level scale, from 0 (not expected concordance) to 5 (complete concordance). The authors suggested creating the scenarios for evaluation considering three aspects: the users’ enjoyment, the time to perform the activity, and a balanced difficulty of the challenges. They also recommend the first evaluation of concordance to be individual, before collective activities. 2.1 Videogames in the Study of Human Behavior Self-reports, mainly questionnaires, and scales are broadly employed in the study of personality and to identify competencies [11]. This is a non-invasive method that provides information in an easy form, but it can be influenced by personal social desires, memory, and motivation [14]. Instead, situational evaluation of an individual´s behavior can be a reliable tool [10, 15]. In the field of behavior analysis, serious games have allowed observing and analyzing what people do in many situations [17–23]. Games in the study of behavior are also a non-invasive approach based on the observation of the subject of study. When people play, they create a pretended reality that prevents them from the real world [16], engaging them in the game task, reinforcing an authentic behavior [17–23]. In fact, the study of interactive styles has been based on the use of games [17– 23]. However, it is important to highlight that the selected video games need to have mechanics that do not rely on the motor skills that require practice, or with complicated rules hard to understand or to learn, that is, casual games with no motor skills involved that will not affect the outcome in the game achievements in the short time [see for example 17–23].

3 Method Proposal for the Evaluation of Interactive Style The method proposed to define the interactive evaluation is shown in Fig. 1, it involves four phases.

Towards the Evaluation of Relevant Interaction Styles

141

Fig. 1. Method to select and evaluate interactive style.

(1) Selecting the contingency arrangement: providing the reasoning, the contingency arrangements to be evaluated are selected. For this study, three contingency arrangements were selected for the evaluation of participants’ interactive styles, achievement or persistency, ambiguity, and frustration. They were selected because the three arrangements require keeping performance although with growing requirements (persistency), indistinguishable or incongruous events’ proprieties (ambiguity), or under conditions of responses block (frustration), respectively. (2) Identifying reference values: this phase aims to provide the framework or values to be taken as reference. For this study, in Table 2 are the expected values for the role, taken from Muñoz et al. [10]. Only the highest values are represented for the selected interactive styles for evaluation, the roles are Team leader (TL), Planning Manager (PM), Implementation manager (IM), Design Manager (DM), and Test Manager (TM). Also, even though, they were related to a responsible for certain cycle phases in the software development cycle [11], these represent activities that a software developer engineer requires to perform throughout the software development lifecycle using some development methodology. Therefore, column 2 provides the results of the analysis related to the expected concordance degree of roles regarding each interactive style. The degree was established in 5-degree levels, where 5 is the highest expected degree.

Table 2. Evaluation stages for TSPi roles (taken from Rangel et al. [11]) Contingency arrangement- Interactive style

Roles and expected concordance degree

Achievement or persistency

TL (5), PM (4), DM (4), IM (4), TM (4)

Ambiguity

TL (5), PM (4), DM (5), IM (5), TM (5)

Frustration

TL (5), PM (4), DM (5), IM (5), TM(5)

(3) Selecting the games. This phase aims to find the game or games needed to provide an attractive way to evaluate the participants. For this research, three games were selected as next described:

142

A. P. P. Negrón et al.

a. The Hidden Folks™, game, a video game with scenarios in which the players have to locate hidden characters, objects, and animals, the objects in the environment are interactive, when the player clicks on them, they might reveal the object that is been search. b. The Room™, a 3D puzzle game with a story for the player to solve puzzles to unlock a box with a smaller but increasingly complex box puzzle. The player has to manipulate the lens to align secrets into a coherent symbol. c. The Tiny clusters (https://bluelemon.itch.io/tiny-clusters) is a video game that is a 2D puzzle that plays with the scenario formation for the player to find its way. These games have growing requirements as the player advances, compelling the player persistency. They also have periods in which the relevant and functional properties of events cannot be discriminated by the player, because they are incongruous or difficult to differentiate (in this sense, the individual does not know to which object or alternative to responding). And, at some point, the individual’s behavior is blocked or delayed, those are parameters that define ambiguous and frustrating situations. (4) Defining the evaluation instruments. In this phase, the evaluation instruments are designed to collect data regarding the playing game experience. The next section describes the evaluation process design and results.

4 Exploratory Study As a first attempt to evaluate interactive styles, in this exploratory study, we assume that the score of the game could represent a good indicator of how hard was for the participant to achieve the game goals. Also, even though as mentioned self-evaluation can be subjective, we decided to use them, not as the only means for evaluation, but as part of it with a weighted value. Therefore, pre and post-game questionnaires were designed; with them, we can evaluate the correspondence between the individuals’ self-perception and their task performances. 4.1 Participants Thirty-five software developers voluntarily participated called through a Facebook™ publication. Then, they were contacted to ask them if they ever played any of the selected games, only those volunteers who never have played them were selected. Thirty-four males and one female participated, with ages in the range of 20 to 39 years old, 12 of them work in the industry, and 15 are students. All of them studied or are studying a Computer or Informatics career. 4.2 Questionnaires Four questionnaires were designed. A pre-playing questionnaire with questions regarding the players’ software development abilities and their perception about their performance under persistency, ambiguity, and frustration situations. The questionnaires

Towards the Evaluation of Relevant Interaction Styles

143

included a short explanation of the use of the data, and on persistency, ambiguity, and frustration concepts. Table 3 presents the questions as formulated in the first column and the second one the possible answers. Table 3. Prequestionnaire Question

Answers

1.Do you consider yourself a good programmer?

(0 to 4 scale) 0 not so good – 4 very good

2.Do you usually program alone or in a team?

Alone/usually by myself and eventually in a team/usually in a team and eventually by myself/in a team

3.If you generally work in a team, are you in Only certain tasks / all phases charge of certain types of development tasks or do you work on all phases? 4.How good are you at planning a software (0 to 4 scale) development project? estimates (effort-person, 0 bad – 4 very good calendar) establishment of goals, milestones, etc 5.How much do apply software quality assurance? using quality standards and/or models

(0 to 4 scale) 0 not at all – 4 lot

6.How good is usually your design for the software?

(0 to 4 scale) 0 bad – 4 very good

7.How effective are your software tests?

(0 to 4 scale) 0 bad – 4 very good

8.Being persistent means to keep firm or constant, and achievement is to get what you want. How persistent or determined to achievement do you consider yourself?

(0 to 4 scale) 0 nothing – 4 a lot

9.Ambiguity is a situation that can be understood or interpreted in various ways, giving rise to confusion. How tolerant do you think you are to ambiguities?

(0 to 4 scale) 0 nothing – 4 a lot

10.Frustration occurs when it is not possible to (0 to 4 scale) achieve what satisfies a need. Personally, how 0 nothing – 4 a lot much do you tolerate frustration?

The intention of the pre-questionnaire was twofold, first to be sure they understand the meaning of persistency, frustration, and ambiguity, in order to evaluate if there was a probable correlation between their interactive style perception and their software development abilities. And three equal post-questionnaires, each one linked to the game they just played. The questions are presented in Table 4, all of them could be answered on a 0 to 4 scale as

144

A. P. P. Negrón et al.

shown in the Answers column, and a last open question was posted in case they wanted to comment on something. Table 4. Game postquestionnaire Question

Answers (0 to 4 scale)

1.How difficult was the game?

0 not at all – 4 a lot

2.Did you think on stop playing?

0 never – 4 all the time

3.How ambiguous was the game?

0 not at all – 4 a lot

4.Did the game was frustrating?

0 not at all – 4 a lot

5.Do you want to share a comment?

Open question

All the players play each game for 20 min. The given instructions were as follows: “This is a test to understand how software developers face certain situations. Your abilities are not under evaluation. The test consists of playing three video games, 20 min each. Please activate your camera, the video will not be public, only the researchers will see it, do you agree? At the end of the game, I will ask your score and answer a brief questionnaire. You can stop playing at any time, that is to say, you do not have to play the 20 min if you do not want to. Although, you will have to answer the questionnaire.“ The scores were then registered for each game. 4.3 Results Regarding correlations in the pre-questionnaire, only the answer of persistency (question 8, see Table 3) presents correlation with question 1, how good programmer the participant classify him/herself at 51%, and with question 6, how good is your software design. Because the scores of each game are different and to standardize the results with the 5-scale in the questionnaire. The scores were classified into 5 classes, the best scores received 0 and the worst 4. This is because if the player made a great score, it means that the game was not difficult for him/her, then 0 means that the game was not difficult at all, and 4 that the game was very difficult. To evaluate persistency, the score was weighed at 70% and the game participant perception for game difficulty was weighed at 30%, question in Table 4 (Q1); the difficulty of the game was linked to keeping playing although it was difficult for the player. On the contrary, for ambiguity, the score weight was 30% and the participant perception of the game ambiguity was weighted 70%, question 2 in the pre-questionnaire (Q3); in this case, the difficulty of the game or score was also linked, but this time to the perceived ambiguity, because although the participant felt the game was ambiguous, he/she kept playing. Frustration was found to correlate (higher than 50%) with the participant intention of leaving the game (in the three games), then a 30% weight was given to the leaving intention, question 2 in Table 4 (Q2), and 70% to the participant perception about feeling frustrated with the game, question 4 (Q4) in the pre-questionnaire. The formulas

Towards the Evaluation of Relevant Interaction Styles

145

Table 5. Formulas for the evaluation Evaluated item

Formula

Score

5 classes for a 0 to 4 scale best score = 0 to worst score = 4

Persistency

score (.70) + Q1 (.30)

Ambiguity

score (.30) + Q3 (.70)

Frustration

Q2 (.30) + Q4 (.70)

are summarized in Table 5. Although, it is important to underline that the weight can be adjusted. Table 6 shows results for the first game Hidden Folks, columns one to four correspond to the participants answers to the post-questionnaire (D, L, A, F), the fifth column shows the score class obtained (S), and columns six to nine present the results for persistency (P), ambiguity (M) and frustration (F) as explained in the former paragraph, and rounded to one digit. The next columns have the same order, but for the second and third games. In these results, no correlation was found with the participant’s perception of his/her software development abilities. Table 6. Postquestionnaire answers and interactive styles evaluations Hidden Folks game results

The Room game results

Tiny cluster game results

D L A F S P M R D L A F S P M R D L A F S P M R 4

0

4

1

2

3

3

0

4

0

3

0

1

2

2

0

2

2

3

2

3

3

3

2

3

0

3

2

3

3

3

1

3

0

3

3

1

2

2

1

2

1

0

3

0

1

0

2

1

0

1

0

1

1

1

0

3

0

3

0

0

1

2

0

3

0

2

3

2

2

2

1

2

1

1

1

1

1

1

1

3

1

2

1

1

2

2

1

1

0

0

0

2

2

1

0

2

0

2

2

2

2

2

1

2

1

2

2

1

1

2

1

2

0

2

0

2

2

2

0

2

0

2

0

2

2

2

0

2

0

3

0

0

1

2

0

2

2

0

3

0

1

0

2

0

0

2

1

2

1

2

0

0

0

0

0

1

1

0

0

0

0

1

0

3

2

2

0

2

0

2

0

3

3

2

0

3

1

1

3

1

2

1

2

1

0

0

0

3

2

1

0

3

0

3

1

3

3

3

0

3

0

1

2

0

1

1

1

2

0

4

0

4

3

4

0

2

2

2

3

3

3

2

2

4

2

4

3

0

1

3

2

2

1

2

1

4

3

3

1

0

0

2

1

1

1

2

0

0

0

2

1

1

1

2

0

4

4

4

4

3

3

4

4

2

2

1

1

2

2

1

2

3

2

3

2

1

2

2

2

2

2

0

1

2

2

1

2

3

0

3

3

3

3

3

1

3

0

3

3

2

2

3

1

2

0

2

2

2

2

2

1

(continued)

146

A. P. P. Negrón et al. Table 6. (continued)

Hidden Folks game results

The Room game results

Tiny cluster game results

D L A F S P M R D L A F S P M R D L A F S P M R 1

0

0

1

3

2

1

0

3

1

1

1

0

1

1

1

1

3

1

3

3

2

2

3

2

0

2

0

2

2

2

0

4

0

2

1

1

2

2

0

3

0

1

0

4

4

2

0

3

0

3

1

0

1

2

0

3

0

3

1

1

2

2

0

2

0

1

0

1

1

1

0

3

1

1

2

0

1

1

1

3

0

1

0

2

2

1

0

1

0

1

1

4

3

2

0

3

0

2

1

4

4

3

0

3

3

4

3

1

2

3

3

4

1

1

1

3

3

2

1

3

0

2

1

3

3

2

0

3

0

2

0

0

1

1

0

3

0

2

1

3

3

2

0

2

1

2

1

3

3

2

1

3

0

1

0

1

2

1

0

3

1

4

3

3

3

4

2

1

0

1

0

1

1

1

0

2

0

3

1

0

1

2

0

2

0

0

1

2

2

1

0

3

0

2

1

1

2

2

0

3

0

3

2

4

4

3

1

3

0

2

1

4

4

3

0

3

1

1

2

4

4

2

1

4

0

1

2

3

3

2

1

4

0

0

0

2

3

1

0

3

0

2

1

3

3

2

0

3

0

1

0

2

2

1

0

2

0

0

0

3

3

1

0

2

1

3

2

2

2

3

1

4

3

2

3

1

2

2

3

2

1

2

2

3

3

2

1

2

0

1

1

1

1

1

0

4

4

4

4

2

3

3

4

2

0

3

1

2

2

3

0

3

1

2

2

2

2

2

1

3

2

3

2

3

3

3

2

0

4

0

0

4

3

1

3

3

1

1

3

1

2

1

2

3

0

1

2

0

1

1

1

2

3

2

2

3

3

2

3

3

0

2

2

3

3

2

1

4

0

2

2

1

2

2

1

1

0

2

2

2

2

2

1

1

0

1

1

2

2

1

0

1

2

0

2

3

2

1

2

1

0

1

2

4

3

2

1

4

2

2

3

3

3

2

2

4

0

0

1

2

3

1

0

3

0

0

0

2

2

1

0

3

0

1

1

0

1

1

0

3

1

2

3

2

2

2

2

1

0

2

1

2

2

2

0

2

2

3

2

3

3

3

2

3

1

3

1

2

2

3

1

2

2

2

3

3

3

2

2

1

2

1

2

3

2

2

2

4

0

1

2

1

2

1

1

2

0

0

1

3

3

1

0

Towards the Evaluation of Relevant Interaction Styles

147

5 Discussion Even though, an evaluation on each interactive style was accomplished for all the participants; according to the results, it can be argued that only those participants challenged by the video game could be classified as persistent, that is, they kept their performance even if they found the game difficult to play. However, for those participants that found three games easy to play, this interactive style might not be evidently revealed. One solution could be to enlarge the list of games, in order to find one that could provide enough challenge, but this can get out of control. Instead, the game can be somehow tricked in order to make the participant make use of other resources to solve the challenge, which denotes the participants’ performance even with the increase in the requirements. Another option might be to use two different types of games, one with increasing requirements and one without this feature, and observe which one the participant prefers to play. The same can be applied to ambiguity and frustration. Because ambiguous situations can be easy to figure out for some people, then ambiguity should be explicit, disconcerting the player in order to observe if the player decides that the game somehow makes no sense and quits or not. Finally, frustration could be managed by sudden stopping the game several times or making the game present failures. Following this reasoning, by taking up at least two evaluations, it can be said that almost all participants were somehow evaluated for persistency and ambiguity, however, for frustration 12 of them did not get any evaluation. Adding the results of the evaluation of all the participants for each game, in the first one Hidden Folks results are for persistency 74 (column P in Table 6), for ambiguity 66 (column A in Table 6), and for frustration 26 (column F in Table 6), for the Room, the additions are 59, 62, and 33, and in the third game Tiny cluster, 84, 60, and 32 in the same order. From here it could be concluded that some types of games are better to evaluate persistency, and none of them are completely proper to evaluate frustration.

6 Conclusions and Future Work Software development is mainly a team effort. Working on a team conveys interaction with others, and in turn, such interaction is shaped by the way people face certain situations among other factors. The interaction styles proposal states that people show a persistent way of interaction when facing similar contingency situations. Interactive styles have been proposed as an alternative to personality traits with an emphasis on the context, and for software development candidate selection. However, the interactive styles have never been evaluated with that purpose. In this paper, we present an approximation to evaluate interactive styles aimed to form software development teams. The evaluation method considers the use of video games because they engage participants, and they can present similar to real-life situations. The diversity of video games presents an opportunity to explore different contingency arrangements. For the evaluation, post-questionnaires regarding the interactive styles were applied; and using a weighted combination of the game outcomes with the game score, an evaluation was conducted. The persistency and ambiguity interactive styles could be somehow evaluated, however, frustration might require another type of game or some type of manipulation in the

148

A. P. P. Negrón et al.

application. This exploratory study allowed us to understand, at least as a first attempt, how video games can be applied for the evaluation of interactive styles. These results have to be confirmed by other studies, as well as using other types of games or strategies for other interactive styles.

References 1. Matturro, G., Raschetti, F., Fontán, C.: A systematic mapping study on soft skills in software engineering. J. UCS 25(1), 16–41 (2019) 2. Lacher, L., Walia, G., Fagerholm, F., Pagels, M., Nygard, K., Münch, J.: A behavior marker tool for measurement of the non-technical skills of software professionals: an empirical investigation. In: the 27th International Conference on Software Engineering Knowledge Engineering. Wyndham Pittsburgh University Center, Pittsburgh, USA, pp. 409–414 (2015) 3. Wagner, S., Ruhe. M.: A systematic review of productivity factors in software development. arXiv preprint arXiv:1801.06475 (2018) 4. Pirzadeh, L.: Human factors in software development: a systematic literature review. Unpublished Master’s Thesis. Chalmers University of Technology, Göteborg, Sweden (2010) 5. Prashandi, W.A.C., Kirupananda, A.: Automation of team formation in software development projects in an enterprise: what needs to improve? In: International Conference on Advanced Computing and Applications (ACOMP). IEEE (2019) 6. Bourque, P., Fairley, R.: Swebok. Nd. IEEE Computer Society (2004) 7. LePine, J.A., et al.: A review of research on personality in teams: accounting for pathways spanning levels of theory and analysis. Hum. Resour. Manag. Rev. 21(4), 311–330 (2011) 8. Pavlenko, M., Pavlenko, L.: Formation of communication and teamwork skills of future IT-specialists using project technology. J. Phys. Confer. Ser. 1840.1. IOP Publishing (2021) 9. Ahmed, F., Capretz, L. F., Bouktif, S., Campbell, P.: Soft skills and software development: a reflection from the software industry. arXiv preprint arXiv:1507.06873 (2015) 10. Ribes, E.: Psicología General. Editorial Trillas, México (1990) 11. Muñoz, M., Mejia, J., Peña, A., Rangel, N.: Establishing effective software development teams: an exploratory model. In: Kreiner, C., O’Connor, R.V., Poth, A., Messnarz, R. (eds.) EuroSPI 2016. CCIS, vol. 633, pp. 70–80. Springer, Cham (2016). https://doi.org/10.1007/ 978-3-319-44817-6_6 12. Rangel, N., Torres, C., Peña, A., Muñoz, M., Mejia, J., Hernández, L.: Team members’ interactive styles involved in the software development process. In: Stolfa, J., Stolfa, S., O’Connor, R.V., Messnarz, R. (eds.) EuroSPI 2017. CCIS, vol. 748, pp. 675–685. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-64218-5_56 13. Muñoz, M., Peña Pérez Negrón, A., Mejia, J., Gasca-Hurtado, G.P., Gómez-Alvarez, M. C., Hernández, L.: Applying gamification elements to build teams for software development. IET Softw. 13.2, 99–105 (2019) 14. Arce, R., Velasco, J., Novo, M., Fariña, F.: Elaboración y validación de una escala para la evaluación del acoso escolar. Revista Iberoamericana de Psicología y Salud 5, 71–104 (2014) 15. Mischel, W.: Toward a cognitive social learning reconceptualization of personality. Psychol. Rev. 80(4), 252–283 (1973) 16. Adams, E.: Fundamentals of Game Design. Pearson Education, Berkeley, CA, USA (2014) 17. Ribes-Iñesta, E., Sánchez, S.: Individual behavior consistencies as interactive styles: their relation to personality. Psychol. Rec. 42, 369–387 (1992) 18. Ribes-Iñesta, E., Contreras, S.: Individual consistencies in behavior: achievement persistence interactions as personality styles. Psychol. Rep. 101, 365–377 (2007)

Towards the Evaluation of Relevant Interaction Styles

149

19. Ribes-Iñesta, E., Martínez-Montor, D.R.: Individual consistencies as interactive styles under decision and ambiguity contingencies. Psychol. Rec. 69(1), 131–142 (2018). https://doi.org/ 10.1007/s40732-018-0315-y 20. Ribes-Iñesta, E., Contreras, S., Martínez, C., Doval, E., Viladrich, C.: Individual consistencies across time and tasks: a replication of interactive styles. Psychol. Rec. 55, 619–631 (2005) 21. Quiroga, M.Á., Martínez-Molina, A., Lozano, J.H., Santacreu, J.: Reflection-impulsivity assessed through performance differences in a computerized spatial task. J. Individ. Differ. 32(2), 85–93 (2011) 22. Merchán H.L., Torres Ceja C.J., Rangel Bernal, N.E.: Modulación del estilo interactivo: efectos de logros y preferencias de otros individuos. Mexican J. Behav. Anal.47.1 (2021). https://doi.org/10.5514/rmac.v47.i1.79749 23. Hernández López, J.M., Lozano Bleda, J.H., Santacreu Mas, J.: La evaluación de la persistencia basada en una tarea de aprendizaje adquisición-extinción. Escritos de Psicología (Internet) 4(1), 25–33 (2011)

Software Implementation for Multivariable Control of Dividing Wall Columns Reyna R. Díaz-Vásquez, Galo R. Urrea-García(B) , Guadalupe Luna-Solano, and Denis Cantú-Lozano Tecnológico Nacional de México/Instituto Tecnológico de Orizaba, Av. Instituto Tecnológico No. 852, C.P. 94320 Orizaba, Veracruz, Mexico [email protected]

Abstract. Multivariable control performance is evaluated for the separation of a ternary mixture in dividing wall distillation column, indirectly controlling the composition of the products by temperature control. Control development implements a simulation using Fortran language and PI controllers. The resulting structure was evaluated using an economic function, and adequate control performance was demonstrated since the cost remained stable and minimal during control process dynamic assessment; while the compositions of the output currents confirm the adequate performance, since each product stream maintained a purity around 98% with respect to its main component. Keywords: Control · Multivariable · Dividing wall columns

1 Introduction Distillation is an operation of recognized relevance in the chemical processes developed worldwide, therefore optimizing this operation results in a consequent economic and environmental benefit. The implementation of dividing wall columns (DWCs) and their variants, in the chemical industry represents an opportunity to achieve energy savings of 30 to 50%, compared with conventional distillation technology, as these columns are able to concentrate various distillation sequences in a single shell, employing a single reboiler and a single condenser [1]. Figure 1 shows an outline of a dividing wall column, which points to key components that characterize the equipment. The illustration shows the feed of a ternary mixture (ABC) where A represents the most volatile component, B the intermediate volatility component, and C to the lowest volatility component. The benefits that DWCs can mean will only be achieved if adequate process control is available; which often implies an arduous search as a result of the complexity of operations on these columns [2]. Proportional Integral (PI) control has proven to be an adequate control mode for regulation of dividing wall column output variables [3–5]; and dynamic process simulation turns out to be an essential tool for the study of operation and analysis of control results, because of its reliability and low cost compared with experimental studies; proof of this can be found in the reviews and researches around DWCs, where mathematical © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 150–162, 2022. https://doi.org/10.1007/978-3-030-89909-7_12

Software Implementation for Multivariable Control

151

Fig. 1. Characteristics of a dividing wall column.

modeling and its consequent computer simulation is advised and used, not only for the control of the process, but from the design stage of the equipment [1, 6–9]. And although limited, some studies have validated the simulations with experimental data [5, 10, 11]. In addition, the present study and its background, add value taking into account that commercial process simulators do not include particular subroutines for DWCs [1]. In this work, a multivariable (MIMO) three-loop temperature control structure for a DWC is selected based on the overall minimization of a cost function. The control structure is intended to indirectly regulate the composition of output streams through specific column tray temperature, using PI controllers. The performance assessment of control loops and final structures was carried out using an objective economic function. For this purpose, a previously developed dynamic simulation program code was used for the design and evaluation of various alternatives of the control structures [12]. Regarding the strategy for the design of the software, Fig. 2 illustrates the cascading design methodology selected for this project; where requirements are the data output that the software must provide; the code design includes the simulation of the system in a stable state and under perturbations with its corresponding controllers; the implementation involves generating files with the data required to establish each control loop; in the integration the three stages to be controlled and the most efficient controllers are chosen to add them to the code definitively; finally in the last cascade sequence the completed MIMO structure is executed and evaluated economically. The cascading design methodology was selected because one of the conditions for its viability is that the requirements are well understood and unlikely to change radically during the development of the system [13]; and since the user is also the designer and the objectives of the research are well defined, it was possible to organize the design tasks according to this methodology. In addition to the control of a DWC, the cascading model is very appropriate since the work progresses in a linear way [14]; that is, there are no simultaneous objectives to program and then couple, but it is a process made up

152

R. R. Díaz-Vásquez et al.

of consecutive tasks; where it is not possible to design a later stage without the correct completion of the stage in progress.

Fig. 2. Cascade design methodology [13].

2 Dynamic Modeling and Process Control The system studied corresponds to a DWC, for the separation of a mixture of benzene (B), toluene (T) and ortho-xylene (X), which is fed in a single non-equimolar flow in stationary state with composition of 30/30/40% respectively; in order to obtain three output currents with 99% purity of each component as appropriate. Mathematical modeling of the system consists of a set of ordinary differential equations (ODEs), the number of which depends on the number of components of the mixture and the number of plates (stages) of the column; but in general it consists of equations of: total continuity per stage, continuity by component per stage, energy balance, hydraulic ratio per stage, liquid density, liquid and vapor enthalpy, and liquid-vapor balance per component. Continuity and energy equations for the prefractionator and for the main column were set, taking into account that the set of equations are coupled by the vapor and liquid exchange streams between prefractionator and main column. Table 1 sets out the column parameters required for the resolution of the equation system. For multivariable control structure design purposes, the column can be considered as divided into four sections, each one related with one manipulated variable. The sections are identified with roman numerals in Fig. 3, and are also named as: prefractionator

Software Implementation for Multivariable Control

153

Table 1. Column parameters [12]. Parameter

Values (units)

Number of components

3

Liquid fraction (β)

0.322

Vapor fraction (α)

0.631

Prefractionator Number of stages

24

Feeding stage

13

Column diameter

5.63 (m)

Main column Number of stages

44

Liquid feed stage

12

Vapor feed stage

37

Side output stage

26

Column diameter

5.63 (m)

Reflux flow rate

0.6672 (kmol/s)

Bottom pressure

67.89 (kPa)

Pressure in the dome

37.49 (kPa)

Reboiler heat input

127.24 (106 kJ/h)

Stage efficiency

100 (%)

section (I), bottom section (II), lateral stream section (III) and dome or distillate section (IV). In addition, the letters: F, D, LSS and B correspond to the four streams involved in the process: feed, distillate product, intermediate product, bottom product, respectively. Three manipulated variables are considered: the reflux flow rate, the heat input rate of the reboiler, and the flow rate of the lateral stream. It is proposed to carry out indirect control of the three product streams composition through the control of the temperature of three column trays, each one located at different section. For the selection of controlled temperatures, characterization of the open loop dynamic response of trays temperature to step changes in each one of the manipulated variables was performed. PI controllers for the candidate trays were tuned and their closed loop performance under feed composition disturbances was compared. The disturbances considered are shown in Table 2 where disturbances manifest at three different times in the process. Finally, it should be mentioned that the numerical solution of the dynamic model, as well as its subsequent control, was carried out in the FORTRAN programming language.

154

R. R. Díaz-Vásquez et al.

Fig. 3. Sections of the column defined for the study.

Table 2. Disturbances in the feed composition. Time (h) 20

Composition B/T/X (%) 30/35/35

60

30/30/40

100

35/30/35

2.1 Algorithms for Dynamic Simulation and Process Control Dynamic simulator that provides the numerical solution of the DWC mathematical model was further developed for allowing the implementation of the indirect composition control of the DWC product streams. Figure 4 shows the flowchart that represents the algorithm described; where a variable n is set, as a counter to the number of times the process must be repeated based on the number of manipulated variables set; and t corresponds to a timer. 2.2 Characterization of Dynamic Process Response (FOPDT Parameters) Process characterization is performed by considering a first order plus dead time (FOPDT) model, widely used for its practicality to capture the dynamic response of many processes [15], where it is necessary to calculate the gain in stationary state, the constant of time and delay time. For the calculation of the above parameters, a step change is required, which is a supervised change in the manipulated variable, by which the response of the controlled variable is evaluated. The step change made in each manipulated variable corresponds to 1% of the magnitude of the steady state variable. The implemented code is then exemplified to make that change.

Software Implementation for Multivariable Control

155

Fig. 4. Process flow diagram. Right: Simulation of the process. Left: Process control.

QRnom=QR IF (TIME > = 20.0D0) QR = 1.01D0*QRnom IF (TIME > = TPAUSE) THEN TPAUSE = TPAUSE + 6.0D0 END IF

Once the system response to a step change has been obtained, the calculation of the parameters is developed: the steady state gain of the process (K), the time constant (τ)

156

R. R. Díaz-Vásquez et al.

and the dead time (t0 ), whose equations are shown below. cs . m

(1)

3 (t2 − t1 ). 2

(2)

K= τ=

t0 = t2 − τ.

(3)

Where cs corresponds to the change in the magnitude of the output variable and m to the change in the magnitude of the manipulated variable; t1 and t2 , correspond to the time when the system response reaches values corresponding to 0.283 and 0.632, respectively, of the total increase in the magnitude of the controlled variable (cs ). 2.3 Design and Tuning of Controllers The design of the controllers involves the determination of the parameters: gain of the controller (Kc ) and integral time (τI ). kc =

1 τ . k τc + t0

τI = min{τ, 4(τc + t0 )}.

(4) (5)

The calculation of which requires the prior estimation of the closed loop time constant (τc ), an adjustment variable that is usually used to specify how aggressive the controller should be. It is recommended [16] to match this parameter with the timeout value, however, depending on the type of tuning: smoother or more aggressive, τc may vary, right in the order in which the equations are exposed. τc = t0 .

(6)

τc = 1.5t0 .

(7)

τc = 0.5t0 .

(8)

The following is an example of code used for implementing controllers.

QRnom=QR IF (TIME > = 5.0D0) THEN error38 = Tref – T(38) sum38 = sum38 + DELTA*error38 QR = QRnom + KC38*(error38 + sum38 / taoI38) END IF

Software Implementation for Multivariable Control

157

2.4 Economic Objective Function The performance of the final control configurations is evaluated using a cost function, given by Eq. (9), for which three conditions are assumed in this study [17]: 1. A process where operating power is not a limiting, therefore, the DWC can operate at its maximum capacity. 2. The price of the products depends on their purity. 3. Unit costs are the same for all three components.

  J = xD,B D + xS,A + xS,C S + xP,B P.

(9)

Where D, S and P represent the flow rate of the distillate, side product and bottom product, respectively; variables x indicate the composition of the product stream (i) with respect to its key impurity component (j). Components A, B and C are considered to correspond in this order to the components of the mixture with high, medium and low volatility; and therefore each one is the main component in distillate, lateral current and bottoms, respectively. Thus the cost function is expressed as the total sum of impurities in the output streams; where the key impurity in distillate (D) stream is component B; in lateral stream (S), the key impurities are A and C; and in the bottoms stream (P), the key impurity is component B.

3 Multivariable Control Results The control structure that resulted with the lowest value in the cost function under the sequence of feed disturbances given in Table 2 consist in controlling temperature of stages 38 (in section IV), 7 (in section II) and 22 (in section III) using the heat input to reboiler, the reflux flow rate and lateral stream flow rate, respectively, as manipulated variables, and establishing loops in that order; it is worth mentioning that in a MIMO control structure, the relationship between variables such as the order of establishment of loops is relevant. Similar studies with distillation of ternary mixtures with traditional PI control or MPC (Model Predictive Control), establish three [9], four [18, 19], five [20] or six [5, 21] loops, to perform a direct control of compositions or indirect through temperature, as in this study. Where the manipulated variables are in addition to the heat input to reboiler, the reflux flow rate and lateral stream flow rate; includes: distillate stream flow rate, bottom stream flow rate and additional ones that will not be required for comparisons (level of the reflux drum, level of bottoms and liquid split). So when we compare the first three loops of the structures mentioned with that of the study, we find that regularly the controlled temperature or composition, as the case may be, is coupled to a nearby manipulated variable; that is, the composition of the distillate, or temperature of section IV, is controlled by the reflux flow rate or the distillate stream flow rate; the composition of the side current or the temperature of section III,

158

R. R. Díaz-Vásquez et al.

is controlled by the lateral stream flow rate; and the composition of the bottoms or the temperature of section II, is controlled by the heat input to reboiler; and just in the order described. Only a background investigation presents a different coupling and order [20]; in turn, the present study proposes an original structure. Table 3 reports the parameters used to control the indicated stages; where Tref denotes the reference temperature to be maintained, of the controlled stages. Table 3. Control parameters Parameter

Controlled stages 38 *(Btu/°Fh)

kc (*) τi (h) Tref (ºF)

0.7391 3.9157 154.315

7 *(lbmol/°Fh) 34.503 4.5667 233.737

22 *(lbmol/°Fh) 941.3906 198.3600 0.2612

Figures 5, 6 and 7 show the behavior of the controlled temperatures of these stages 38, 7 and 22, respectively, throughout the 150 h evaluated, considering the effect of the three disturbances in the feed composition. The control performed in the temperatures of these stages can be compared with studies of indirect control of the composition through the control of temperatures [5, 9, 18]; however, in these, the disturbance in composition happens only once; and therefore its study period represents only one hundredth of that evaluated by the present; also for this reason, although the control carried out manages to stabilize the behavior of the temperatures; it presents more deviations and a longer time of settlement than the reference investigations.

Fig. 5. Stage 38 controlled temperature, manipulating the heat rate of the reboiler (QR).

Software Implementation for Multivariable Control

159

Fig. 6. Stage 7 controlled temperature, manipulating the reflux rate (R).

Fig. 7. Stage 22 controlled temperature, manipulating the flow of the side current (LSS).

On the other hand, Fig. 8 shows the behavior of the main component composition in each one of the three product streams. It can be observed that distillate stream maintains a higher purity of its main component during the development of the process. Although bottoms and lateral streams show a significant decrease in composition after the first disturbance, they exhibit good regulation capacity after a transient period, and keep composition of toluene and o-xylene respectively, about 98%, most of the process. Similar studies of control of distillation of ternary mixtures in DWCs [20–22]; show in their product stream, a purity of their main components of between 95% to 98%, observing slight oscillations and in [20] and [22] times of settlement similar to those obtained in this control structure; it should be noted that the studies mentioned only control one disturbance.

160

R. R. Díaz-Vásquez et al.

Fig. 8. Composition behavior in product streams: distillate (D), bottoms (B) and lateral current (LSS).

Finally, Fig. 9 presents the economics of the resulting control configuration. It is perceived that during the first disturbance the cost increases significantly, but as the process continues and the actions of the control structure are carried out, the costs decrease and remain below 125 units per hour.

Fig. 9. Cost response in the process with three control loop.

4 Conclusions Implementing software to simulate the control of a DWC is convenient by reducing the costs and time investment required to conduct experimental studies, while obtaining

Software Implementation for Multivariable Control

161

reliable results; in addition, the software presented has two advantages compared to the simulators found in the aforementioned research: to work with equipment of minimum specifications and to control more disturbances. A previous modeling of the process, guarantees results with acceptable approximations. Economic evaluation through the cost function; in addition to being novel for studies of DWCs; reveals that the resulting control structure is adequate, since the cost remained stable and minimal during most of the process; which mean that the key impurity in each product stream can be kept under pre-established values, even under the effect of feed composition disturbances. Acknowledgments. Authors acknowledge Tecnológico Nacional de México (TecNM) for Research Project Number 10105.21-P.

References 1. Lorenz, H.M., Staak, D., Grützner, T., Repke, J.U.: Divided wall columns: Usefulness and challenges. Chem. Eng. Trans. 69, 229–234 (2018) 2. Tututi-Avila, S., Jiménez-Gutiérrez, A., Hahn, J.: Analysis of multi-loop control structures of dividing-wall distillation columns using a fundamental model. Processes 2(1), 180–199 (2014) 3. Arjomand, A., Fanaei, M.A.: Optimal operation of a three-product dividing wall column with self-optimizing control structure design. IJCCE 34(1), 107–117 (2015) 4. Rodríguez-Hernández, M., Chinea-Herranz, J.A.: Decentralized control and identified-model predictive control of divided wall columns. J. Process Control 22(9), 1582–1592 (2012) 5. Buck, C., Hiller, C., Fieg, G.: Decentralized temperature control of a pilot dividing wall column. Chem. Eng. Process 50(2), 167–180 (2011) 6. Henrique, J.P., Secchi, A.R., Ravagnani, M.A.S.S., Barbara, A., Costa, C.B.B.: Divided wall column modeling and simulation in an open-source environment. Chem. Biochem. Eng. Q. 34(3), 149–167 (2020) 7. Tututi-Avila, S., Domínguez-Díaz, L.A., Medina-Herrera, N., Jiménez-Gutiérrez, A., Hahn, J.: Dividing-wall columns: design and control of a kaibel and a satellite distillation column for BTX separation. Chem. Eng. Process 114, 1–15 (2017) 8. Yildirim, Ö., Kiss, A.A., Kenig, E.Y.: Dividing wall columns in chemical process industry: a review on current activities. Sep. Purif. Technol. 80(3), 403–417 (2011) 9. Dohare, R.K., Singh, K., Kumar, R.: Modeling and model predictive control of dividing wall column for separation of Benzene–Toluene-o-Xylene. Syst. Sci. Control Eng. 3(1), 142–153 (2015) 10. Li, C., Zhang, Q., Xie, J., Fang, J., Li, H.: Design, optimization, and industrial-scale experimental study of a high-efficiency dividing wall column. Sep. Purif. Technol. 247, 116891 (2020) 11. Niggemann, G., Hiller, C., Fieg, G.: Experimental and theoretical studies of a dividingwall column used for the recovery of high-purity products. Ind. Eng. Chem. Res. 49(14), 6566–6577 (2010) 12. Matla-González, D., Urrea-García, G., Alvarez-Ramirez, J., Bolaños-Reynoso, E., LunaSolano, G.: Simulation and control based on temperature measurements for Petlyuk distillation columns. Asia-Pac. J. Chem. Eng. 8(6), 880–894 (2013) 13. Pressman, R.: Ingeniería del Software: Un Enfoque Práctico. 7ma edn. McGraw-Hill, México (2010)

162

R. R. Díaz-Vásquez et al.

14. Sommerville, I.: Ingeniería del software. 7ma edn. PEARSON EDUCACIÓN, Madrid (2005) 15. Muresan, C.I., Ionescu, C.M.: Generalization of the FOPDT model for identification and control purposes. Processes 8(6), 682 (2020) 16. Grimholt, C., Skogestad, S.: Optimal PI and PID control of first-order plus delay processes and evaluation of the original and improved SIMC rules. J. Process Control 70, 36–46 (2018) 17. Khanam, A., Shamsuzzoha, M., Skogestad, S.: Optimal operation and control of divided wall column. Comput. Aided Chem. Eng. 33, 673–678 (2014) 18. Ling, H., Luyben, W.L.: Temperature control of the BTX divided-wall column. Ind. Eng. Chem. Res. 49(1), 189–203 (2010) 19. Ling, H., Luyben, W.L.: New control structure for divided-wall columns. Ind. Eng. Chem. Res. 48, 6034 (2009) 20. Koko, I.O.M., Barakat, T.A.M.: Modelling and control analysis of dividing wall distillation columns. Proceeding. 1, 60–68 (2012) 21. Kiss, A.A., Rewagad, R.R.: Energy efficient control of a BTX dividing-wall column. Comput. Chem. Eng. 35(12), 2896–2904 (2011) 22. Van Diggelen, R.C., Kiss, A.A., Heemink, A.W.: Comparison of control strategies for dividing-wall columns. Ind. Eng. Chem. Res. 49(1), 288–307 (2010)

Polyglot Programming with GraalVM Applied to Bioinformatics for DNA Sequence Analysis José Antonio Romero-Ventura1 , Ulises Juárez-Martínez1(B) , and Adolfo Centeno-Téllez2 1 Technological Institute of Orizaba, National Technological of México, Oriente 9, Emiliano

Zapata, 94320 Orizaba, Veracruz, México [email protected] 2 Waves Lab, Camerino Z. Mendoza, Veracruz, México [email protected]

Abstract. Currently, application development requires multiple execution environments to meet the needs of various programming languages used. Each execution environment requires specific characteristics that lead to complexity, high consumption of time and money for maintenance activities. To overcome these drawbacks, polyglot programming with GraalVM allows the development of applications using several programming languages simultaneously from a single development environment. The main key under the polyglot approach lies in the interoperability of programming languages. This paper presents a bioinformatics case study applying polyglot programming with GraalVM for DNA sequence analysis. Java and Python languages are used and two solutions with Dockers are provided, one of them with native image, and performance data for both solutions are provided. Keywords: GraalVM · Polyglot programming · Native image · Docker · Bioinformatics

1 Introduction Modern application development involves the use of several programming languages regardless of the type of application: web, desktop, mobile or cloud. Each language used requires a specific environment to run, for example, a browser, an application server, a virtual machine, or a set of interpreters, as well as the use of separate environments for their execution or compilation characteristics. Consequently, there is the use of different servers, applications on separate computers due to the program’s own characteristics and the use of several APIs (Application Programming Interface). All the above is required to allow communication between programs for sending messages or executing tasks. Additionally, specific and particular configurations are required because the APIs used are developed by third parties. As an alternative to avoid the use of different development and execution environments, as well as different configurations, GraalVM [1] (Graal Virtual Machine) © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 163–173, 2022. https://doi.org/10.1007/978-3-030-89909-7_13

164

J. A. Romero-Ventura et al.

emerges as a unique high-performance environment to accelerate the development and execution of applications made in Java with support for interoperability with other programming languages and with native compilation capabilities. Like the JVM, GraalVM preserves its compatibility with various operating systems, positioning it as a universal, high-performance, multi-language platform for compiled application development. This article presents the creation of a service application that allows the analysis of DNA sequences using the polyglot properties of GraalVM. This application is generated in Java language as Host language of the polyglot environment and uses as Guest language the Python language whose purpose is to analyze DNA sequences. The application is generated as a Java application with native image inside a Docker container. A brief analysis of the execution times of the different applications is presented, and it is analyzed which is the best option to implement in a project. This type of programming benefits in the development of applications, since combining different programming languages maximizes the features of each language and thus optimizes the implementation of a system, reduces the use of tools and with native properties maximizes the use of memory thanks to the JIT and AOT properties, resulting in a much faster application. This article is structured as follows: Sect. 2 comprises the background that discusses the techniques and technologies that allow polyglot programming in GraalVM; Sect. 3 presents the state of the art; Sect. 4 describes everything related to the implementation of the system for the analysis of DNA sequences using polyglot programming with GraalVM; Sect. 5 discusses the results obtained in the experimentation of the application that was developed; finally, Sect. 6 mentions the conclusions and Sect. 7 presents the future work.

2 Background Oracle’s Java HotSpot VM [2] is equipped with a highly advanced Just-In-Time (JIT) compiler. This means that class files (which are compiled from Java source code) are usually compiled at runtime and can be converted into highly optimized machine code. This optimized code generally runs as fast as (and, in certain cases, faster than) compiled C/C++ code. The JIT compiler is therefore one of the most important parts of the Java HotSpot VM and yet many Java developers do not know much about it or how to check that their applications work well with the JIT compiler. The AOT [3] (Ahead-Of-Time) compiler dynamically compiles Java methods into native AOT code at runtime and stores them in the shared class cache. This activity allows the virtual machine to start an application faster the next time it runs because it does not need to spend time interpreting Java methods. The virtual machine automatically chooses which methods to compile with AOT based on heuristics that identify the startup phase of large applications. AOT code is always used in combination with class data option is enabled exchange and is automatically enabled when the via the command line. When a cached AOT method is executed, the JIT compiler also enables its optimization. GraalVM is a universal virtual machine for running applications written in languages such as JavaScript, Python, Ruby, R, JVM-based languages such as Java, Scala, Groovy, Kotlin, Clojure and LLVM-based languages such as C and C++. GraalVM eliminates

Polyglot Programming with GraalVM Applied to Bioinformatics

165

isolation between programming languages and enables interoperability in a shared runtime. It runs standalone or in the context of OpenJDK, Node.js or Oracle Database. Any JVM-based application [4] running on Java HotSpot VM can run on GraalVM. GraalVM is based on Java HotSpot VM but integrates an advanced JIT compiler. At runtime, the application loads and runs normally on the JVM. The JVM passes the bytecodes to the GraalVM compiler, compiles them to machine code and returns them to the JVM. Thanks to these features, GraalVM’s dynamic compiler makes it possible to improve the efficiency and speed of applications written in the various languages mentioned above through unique approaches to code analysis and optimization. For example, it ensures performance advantages for highly abstracted programs due to its ability to eliminate costly object allocations. Native Image [5] is a technology to precompile Java code into a standalone executable. This executable includes the application classes, their dependency classes, runtime library classes and native code statically linked by the JDK. It does not run on the JVM, but includes necessary components such as memory management, thread scheduling, etc. from a different runtime system, called “Substrate VM”. Substrate VM is the name of the runtime components (such as deoptimizer, garbage collector, thread scheduling, etc.). The resulting program has a faster startup time and lower runtime memory overhead compared to a JVM. A dynamic programming language [6] is a programming language in which operations performed at compile time can be performed at runtime. For example, with JavaScript it is possible to change the type of a variable or add new properties or methods to an object while the program is running. This is opposed to so-called static programming languages, where such changes are normally not possible. GraalVM offers support for several dynamic programming languages. Although GraalVM has support for the Python language, it does not mean that all Python-based tools are directly compatible. In many cases it is necessary to install and configure additional virtual environments to achieve compatibility, such is the case with BioPython [7], a toolkit for biological computation processing and analysis written in the Python language. As an alternative for the context of the presented case study, a Python implementation was made from scratch of the main operations required for DNA string processing.

3 State of the Art The Escape Analysis technique [8] allows the compiler to determine whether an object is externally accessible from a method or a thread. This information was used to perform optimizations such as Scalar Replacement, Stack Allocation and Lock Elision, allowing modern compilers to remove abstractions introduced by advanced programming models. In binding processes, virtual objects become escape objects that allow invalidating supposed virtual objects in previous processes of the same branch. As a result, it is obtained that a high number of placed bytes also demonstrate a high number of memory spaces. A significant reduction in the number of memory locks was also realized. For most of the benchmarks used, a performance improvement of 10% was observed. It is undeniable the use of databases in every system, or at least a form of alternative storage for the data used. Within this context, Stored Procedures (SP) are used for

166

J. A. Romero-Ventura et al.

centralized management of business logic in a system running within the same database. SP avoid excessive use of network, memory, and direct management of data in the system. However, despite these benefits SPs also have features that are considered detrimental to use such as: some SPs have proprietary vendor specifications; programmers specialized in some SPs are hard to find; some SPs are difficult to find in the database even with modern managers; and finally, SP support programs lack support for other programming languages. The consideration of working SPs in conjunction with GraalVM is highly attractive so in [9] a GraalVM embedded model is presented with Oracle Database and MySQL Database that allows running SPs and some user-created functions. JavaScript is used as SP to solve the disadvantages mentioned above. GraalVM has a wide support of programming languages [10] including dynamically typed languages such as JavaScript, Ruby, R, and Python, and statically typed languages such as Java, Kotlin, Scala or applications compiled with LLVM. The interoperability of languages and the independence of their tools are considered two peculiar forms of an internal mechanism of virtual machine meta-programming. Interoperability of two or more programming languages in GraalVM share object instances with wide scope and minimal execution effort. The programming language exports any of its objects by message resolution. Subsequently when an object is shared, the language sends a message describing the specific operation to be executed. If the message is successful, the functionality is offered to the external language that executes the code fragment to complete the operation. For a visual analysis of languages performance in the context of GraalVM [11], it is necessary to look at the implementations done through the Truffle framework. For this purpose, Flame Graphs was used, and stack calls were timed. The Linux perf tool and the JVM agent, perf-mapagent, were used, along with the Graal JIT compiler that maps the stack calls to the JVM, which handles the different programming languages. The analysis shows how easy and flexible it is to use modified tools with low resource consumption, and the application of techniques that allow understanding the behavior of polyglot applications. Low overhead techniques using standard Linux tools for profiling GraalVM and Truffle implementations were presented. The approach does not require any modification to the Truffle framework implementation, unlike previous profiling. It is noted that Truffle framework implementations in languages such as TruffleRuby and FastR have been shown to be faster than existing implementations. On the other hand, the JavaScript implementation with Truffle showed that it is in a reasonably level performance configuration with the version of JavaScript V8 implemented by Google. Šipek and collaborators [12] reported that contemporary software commonly becomes quite complex, and the use of multiple technologies and different programming languages is necessary for its development. Even, interoperability between programming languages causes a considerable drop in software performance, therefore, it is important to know how polyglot programming with a high-level language-oriented compiler performs these processes. With the current version of GraalVM, without any additional configuration, compared to the standard JDK, the results were better than expected, and both versions, Graal CE (Community Edition) and Graal EE (Enterprise Edition), proved to be better in different tests. This proves that the next versions of GraalVM will have better performance, better optimization, and better results.

Polyglot Programming with GraalVM Applied to Bioinformatics

167

Sending data between different programming languages by means of polyglot adapters is a code execution tool that is widely used in GraalVM [13]. Polyglot adapters are the mechanism to make polyglot execution possible, i.e., to execute separately or in combination the different programming languages supported by GraalVM. However, there are also problems in such executions, where data passing between languages is a major issue. Although GraalVM provides interoperability between programming languages by means of a polyglot API and allows message passing between programming languages, it is repeatedly necessary to pass non-primitive type values between languages, which complicates communication between the languages involved. Some tests to show the benefits of polyglot adapters have been developed using the Python language and the GraalVM shell. It was observed that polyglot adapters provide support for sending non-primitive objects between programming languages from one programming language to another. They also allow the sending of messages between languages, as well as the modification of language behavior. Finally, another palpable benefit is the possibility of using adapters frequently and automatically by means of frameworks. One of the frequent challenges when working with polyglot application development is to solve an algorithmic problem by applying different programming languages on data structures. The purpose is to make these solutions effective and make the code more portable. GraalVM supports developers with user definition mapping [14], which allows to reduce mental complexity to focus on business logic. The mapping consists of an adapter between programming languages that make use of runtime messages to enable binding to the expected interface [14]. This mapping also allows to improve programming productivity by making more flexible the selection of the right tools, suitable languages, appropriate libraries, and frameworks available according to the programming language(s) used. In the context of polyglot programming, it is important to have tools that allow timely error detection. In dynamic execution environments such as GraalVM, the verification of error states within the application is done by means of SymJEx [15], a tool that finds non-trivial bugs in existing software systems. Since GraalVM offers the feasibility of working with polyglot ecosystems, it is necessary to implement the same concepts for several languages and to make their interoperability more flexible. This implementation is achieved through Example-Based Live Programming (ELP) [16]. Finally, to test the stepwise optimization of GraalVM, a large-step and a small-step operational semantics are used by applying the polyglot capabilities in Isabelle/HOL [17] (higher order logic).

4 Polyglot Programming with GraalVM to Analyze DNA Sequences As a case study to show the benefits of polyglot programming, as well as the advantages of using GraalVM, an application for the analysis of DNA sequences (Deoxyribonucleic Acid) was developed. The case study allows to display the results of the analysis through two solutions, one is a web application and the other is an application for mobile devices. The case study implements a set of solutions that considers the basic operations of DNA strand analysis such as complement, reverse complement, obtaining RNA (Ribonucleic Acid) and its respective codons and proteins. Additionally, support is offered to consider FASTA and GenBank files.

168

J. A. Romero-Ventura et al.

4.1 System Architecture Figure 1 shows the SOA architecture of the software solution applied to the case study. The right side shows the “BioGraalVM” application, which is a REST API of services where the polyglot programming was implemented with GraalVM and the business logic of the DNA sequence analysis was also implemented internally. On the left side is presented the responsive web application “BioGraal” that was made with Angular. The model is the part that consumes the web services of the “BioGraalVM” application. The components of the MVC architecture work together to show, through a web channel from a browser, the results of the DNA sequence analysis to the user.

Fig. 1. System architecture.

4.2 Implementation “BioGraalVM” is the name given to the application containing the REST services, which contains the internal processes for the analysis of DNA sequences implementing the polyglot properties of GraalVM. class. This class implements The following code presents the the execution of the code [18] in Python SeqComplement.py (line highlighted in bold) that performs the DNA sequence analysis. The execution is performed through the GraalVM context using its polyglot execution and interpretation properties.

Polyglot Programming with GraalVM Applied to Bioinformatics

import import import import import

169

io.micronaut.core.annotation.Introspected; org.graalvm.polyglot.Context; org.graalvm.polyglot.HostAccess; org.graalvm.polyglot.PolyglotAccess; org.graalvm.polyglot.Value;

@Introspected public class Complement extends SequenceResourses implements ISequence{ @Override public String run(String sequence) { String complement = ""; String pythonscript=""; //Create context instance Context context = GraalVMContext.getInstance(); //Get python script from resources pythonscript = readResource("SeqComplement.py"); //Execute eval to load python function context.eval("python",pythonscript); //Execute python function setting sequence Value pyfunction = context.getPolyglotBindings() .getMember("complement"); Value sqcomplement = pyfunction.execute(sequence); //Casting result as String complement = sqcomplement.asString(); return complement; } }

In [18] the script that performs the complement operation for a DNA sequence is presented. The first two lines define the directives that allow communication between Java and Python via GraalVM. class that implements the The code [19] presents the Singleton design pattern to create a single GraalVM context. In such context polyglot environments are executed using different programming languages, in this case properties are specifically Python and Java. In this context the values of the defined, these properties are assigned to locate internally the GraalVM libraries needed when using a specific language. The definition also allows localization externally to the native image. Docker was used as a container for the “BioGraalVM” application. The container has a configuration and software necessary for the execution of each of the internal functions file is presented. of the application. In the code [20] the content of the The file contains the lines to be executed for the creation of the image to be executed in the docker and the commands that will create the native image from GraalVM. These commands install the necessary components for the creation and execution of the native

170

J. A. Romero-Ventura et al.

image with GraalVM and the necessary configuration to be used as the REST API application “BioGraalVM” from port 8080. The implementation of the web application “BioGraal” was done with Angular CLI 11.0.6 framework. This web application is the graphical interface that the user uses to upload DNA sequences, or search for one by means of an NCBI identifier. The complement, reverse complement, RNA, codons, and proteins are obtained by submitting a DNA sequence for analysis to the REST API “BioGraalVM” from “BioGraal”. In [21] a code block in Angular is presented to make a POST request to “BioGraalVM” sending a DNA sequence as parameter. Figure 2 shows the graphical interface of the “BioGraal” web application. This interface allows the user to perform DNA sequence analysis through the application working with the REST API “BioGraalVM”.

Fig. 2. Graphical interface of the “BioGraal” web application.

5 Results and Discussion This section presents the results obtained from the execution of the “BioGraalVM” application from different environments such as JVM, Native Image and Docker. The application was executed by making an HTTP GET call to the REST service of the “BioGraalVM” application from a terminal to request the analysis of a DNA base sequence. The DNA sequence corresponding to the Covid-19 virus [22] (Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1) was used. The experiment was performed a total of 100 times for each environment (web and mobile device). What was measured was mainly the response time of each HTTP call to the “BioGraalVM” application service. 5.1 Execution Time The data obtained from the experiments allow us to observe the difference in time when using different solutions with GraalVM and Docker technology. Table 1 shows the comparison of results where it is observed that the native image has the shortest execution time. This shorter time is because the code is compiled in

Polyglot Programming with GraalVM Applied to Bioinformatics

171

the executable format of the platform used. It is also possible to observe that the native image when using Docker increases the execution times due to the management layers of the container itself. In many cases the increase in execution times using Docker may not be relevant compared to the advantages gained in terms of modularity and control. Table 1. Comparative table of execution time results. Result (s)

JVM

Native image

Native image Docker

Total

7.164

0.6690

27.973

Range

0.047 to 0.239

0.006 to 0.015

0.235 to 0.361

Average

0.071

0.0066

0.2797

5.2 Polyglot Implementation Based on the results of the implementation using polyglot programming, the ease and flexibility of using several programming languages is evident, especially in terms of interoperability. Interoperability techniques stand out by making the programmer eliminate the need to use several execution environments, which translates into savings of time, money, and effort. From a practical point of view, development focuses on the solution, applying the most convenient programming language to each part of the problem. The coding is a combination of direct use of files and text strings, allowing a form of reuse comprising complete modules and key code scripts for effective development. The native property offered by GraalVM is very attractive because execution times are greatly reduced by eliminating distinctions between compiled and interpreted languages.

6 Conclusions One of the remarkable features of using the Graal Virtual Machine is the native property. This property allows generating a native image of the application in Java and taking advantage of the AOT and JIT properties to make the application much faster than its counterpart in a traditional JVM. As could be seen in Table 1, the native image shows its superiority in terms of execution time by running the “BioGraalVM” application in an average time of 0.00669 s. This time is extremely fast compared to running on JVM and the native image inside a Docker container. Based on this, the following points can be mentioned: the execution of the native image is much faster than the Java application using JVM directly or contained in a Docker; the implementation of the native image inside the Docker increases the execution times due to the management and control layers; and finally, the implementation of the Python scripts also required a series of internal directives in the Python code, allowing polyglot communication between the Guest language (Python), with the Host language (Java). Despite the existence of BioPython, the “BioGraalVM” application is necessary because it has the polyglot properties of GraalVM and the potential of its native image

172

J. A. Romero-Ventura et al.

property, which allows it to be a much faster and powerful application. Also, the architecture that was implemented allows the use of the application as a REST API. The use of polyglot programming with GraalVM leads to a significant improvement in software processes since it allows handling different programming languages in the same environment using a single virtual machine. Polyglot programming makes the development of systems much faster. It also reduces costs in terms of component infrastructure for specific implementations that allow the execution of one programming language per environment. With GraalVM the infrastructure is polyglot in the same environment. The design of the applications is more homogeneous because the polyglot environment allows the different programming languages used to communicate and to share primitive and non-primitive data types. Finally, the native property, as seen in the runtimes, makes the applications much faster.

7 Future Work Future work will continue with the improvement of the DNA sequence analysis application by applying polyglot programming. New ways of applying GraalVM properties to optimize the application will also be investigated. More programming languages will be selected to add to the polyglot environment and solve new functions to be added for DNA sequence analysis such as alignments. We will also study how to integrate BioPython to the developed system “BioGraalVM”.

References 1. GraalVM. https://www.graalvm.org/ 2. Understanding Java JIT Compilation with JITWatch, Part 1. https://www.oracle.com/techni cal-resources/articles/java/architect-evans-pt1.html 3. Ahead-Of-Time (AOT) compiler. https://www.eclipse.org/openj9/docs/aot/ 4. Oracle GraalVM Enterprise Edition 20 Guide. https://docs.oracle.com/en/graalvm/enterprise/ 20/docs/reference-manual/jvm/ 5. GraalVM Native Image. https://www.graalvm.org/reference-manual/native-image/ 6. Dynamic programming language. https://developer.mozilla.org/en-US/docs/Glossary/Dyn amic_programming_language 7. Biopython. https://biopython.org/ 8. Stadler, L., Würthinger, T., Mössenböck, H.: Partial escape analysis and scalar replacement for Java. In: Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, New York, pp. 165–174 (2014) 9. Brantner, M.: Modern stored procedures using GraalVM: invited talk. In: Proceedings of The 16th International Symposium on Database Programming Languages, New York, p. 1 (2017) 10. Bonetta, D., Graal, V.M.: Metaprogramming inside a polyglot system (invited talk). In: Proceedings of the 3rd ACM SIGPLAN International Workshop on Meta-Programming Techniques and Reflection, New York, pp. 3–4 (2018) 11. Gaikwad, S., Nisbet, A., Luján, M.: Performance analysis for languages hosted on the truffle framework. In: Proceedings of the 15th International Conference on Managed Languages and Runtimes, New York, pp. 1–12 (2018)

Polyglot Programming with GraalVM Applied to Bioinformatics

173

12. Šipek, M., Mihaljevi´c, B., Radovan, A.: Exploring aspects of polyglot high-performance virtual machine GraalVM. In: International Convention on Information and Communication Technology, Electronics and Microelectronics, pp. 1671–1676 (2019) 13. Niephaus, F., Felgentreff, T., Hirschfeld, R.: Towards polyglot adapters for the GraalVM. In: Proceedings of the Conference Companion of the 3rd International Conference on Art, Science, and Engineering of Programming, New York, pp. 1–3 (2019) 14. Riese, A., Niephaus, F., Felgentreff, T., Hirschfeld, R.: User-defined interface mappings for the GraalVM. In: Conference Companion of the 4th International Conference on Art, Science, and Engineering of Programming. pp. 19–22, New York (2020) 15. Kloibhofer, S., Pointhuber, T., Heisinger, M., Mössenböck, H., Stadler, L., Leopoldseder, D.: SymJEx: symbolic execution on the GraalVM. In: Proceedings of the 17th International Conference on Managed Programming Languages and Runtimes, New York, pp. 63–72 (2020) 16. Niephaus. F., et al.: Example-based live programming for everyone: building languageagnostic tools for live programming with LSP and GraalVM. In: Proceedings of the ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, New York, pp. 1–17 (2020) 17. Webb, B.J., Utting, M., Hayes, I.: A formal semantics of the GraalVM intermediate representation. J. ArXiv (2021) 18. BioGraalVM_Project/SeqComplement.py. https://github.com/JAntonioRV/BioGraalVM_P roject 19. BioGraalVM_Project/GraalVMContext.java. https://github.com/JAntonioRV/BioGra alVM_Project 20. BioGraalVM_Project/Dockerfile. https://github.com/JAntonioRV/BioGraalVM_Project 21. biouv-front/graalv.service.ts. https://github.com/adsoftsito/biouv-front 22. Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1, complete genome. http://www.ncbi.nlm.nih.gov/nuccore/NC_045512.2

Virtual Assistant as Support for People Visually Impaired Liliana E. Olguín-Gil1(B) , Francisco Vázquez-Guzmán1 , Eduardo Vázquez-Zayas1 , Jezreel Mejía2 , and Itzel Blanco-Cruz1 1 Tecnológico Nacional de México/Instituto Tecnológico de Tehuacán, Libramiento

Tecnológico S/N, Tehuacán, Puebl, México {lilianaelena.og,francisco.vg,eduardo.vz, l17361099}@tehuacan.tecnm.mx 2 Centro de Investigación en Matemáticas, A.C., Jalisco S/N, Col. Valenciana, Guanajuato, Gto, México [email protected]

Abstract. With the emergence of virtual assistants and their increased popularity, the possibility of generating one of our own has raised; using neural network algorithms and artificial intelligence to recognize commands and execute the actions commanded through natural language, allowing accessibility and inclusion for people with visual disabilities. This article describes the result of three years of research into using Information and Communication Technologies (ICT) to generate tools that enable visually impaired people to integrate into the world more independently. Therefore, the software presented here is an application that will allow people with visual disabilities to be independent on a companion or to have knowledge of braille language. Keywords: Virtual assistants · Visual impairment · Mobile apps · Inclusion · Mobile devices

1 Introduction People with a disability problem face the social and ideological challenge of the loss of self-esteem and the lack of knowledge about the technological tools and methods that support the inclusion of this group of people. In addition, the international and national laws that protect their rights are unknown by most people. According to the 2020 Mexican Population and Housing Census [1], there are 6,179,890 people with some disability in Mexico, representing 4.9% of the country’s total population. Of these, 53% are women, and 47% are men. In terms of the type of disability, walking problems are the disability with the highest presence (48%), followed by difficulties in seeing (44%), hearing (22%), speaking or communicating (15%), and others (60%) see Fig. 1. As COVID-19 continues to have far-reaching impacts worldwide, it is crucial to point out that people with disabilities are significantly more affected by the pandemic crisis in health, education, and transportation issues. According to the World Bank [2], 15% © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 174–188, 2022. https://doi.org/10.1007/978-3-030-89909-7_14

Virtual Assistant as Support for People Visually Impaired

Walking up or down

See, even wearing glasses

Hear, even wearing hearing aids

Bathing, dressing, or eating

Remember or focus

175

Talk or communicate

Fig. 1. Percentage of population by disability. Source: INEGI, Population and Housing Census 2020

of the world’s population, or 1000 million people, experience some form of disability, with a higher prevalence in developing countries. Between 110 million and 190 million people, or one-fifth of the world’s population, are affected by significant disabilities. According to the United Nations [3], about 10% of the world’s population, or 650 million people, lives with a disability. They constitute the most significant minority in the world. This number is increasing due to population growth, advances in medicine, and the aging process. In countries where life expectancy is over 70 years, on average, about eight years or 11.5% of an individual’s life is spent with disabilities of some kind. 80% of people with disabilities live in developing countries, according to the United Nations Development Programme (UNDP). In the Organisation for Economic Co-operation and Development (OECD) countries, disability rates are significantly higher among groups with lower educational attainments. The average is 19%, compared to 11% among those with more education. In most OECD countries, women are reported to have a higher incidence of disabilities than men. Therefore, it is necessary to incorporate information and communication technologies (ICT) to develop hardware and software applications or prototypes to support people with disabilities to help them to be included in society. The role of ICT concerning people with disabilities is fundamental since it will enable communication with other people and their environment, facilitate their learning, or integrate them into the world of work. Some examples of current applications are: • Seeing AI [4]: This is a Microsoft research project that combines the power of the cloud with artificial intelligence to deliver an intelligent application designed to help you perform your everyday actions. • Brailliac [5]: Braille Tutor. Aimed at people with visual disabilities, it is an application that aims to facilitate the learning of reading and tactile writing for blind people. • iDentifi [6]: It is an application that allows visually impaired people to gain more independence in daily tasks such as shopping, browsing themselves in indoor environments, and reading. It uses artificial intelligence to allow a visually impaired user to click on a photo and recognize virtually any object, brand, color, facial expression,

176

L. E. Olguín-Gil et al.

handwriting, and text, and subsequently provide an audible description of the image content to the user. • Say it in signs [5]: It is created not only for people with hearing disabilities but also for all those who want to learn sign language. It offers the learning of sign language by resorting to a game. The access of people with disabilities to the so-called Knowledge Society is crucial for their integration, development, and participation in society. Many people with disabilities would not access information or communicate with others if it was not due to specific technologies. Among the possibilities and advantages offered by ICT in the field of diversity, we can highlight the following [7]: • They help overcome the limitations that derive from the students’ cognitive, sensory, and motor disabilities. • They favor the students’ autonomy, being able to adapt it to the needs and demands of each student in a personalized way. • They favor the students’ synchronous and asynchronous communication with the rest of their classmates and teaching staff. • They promote individualized training and a trundle at their own pace, which is extremely important for those with some disability [8]. However, none of the existing applications are conformed entirely to all the fundamental principles of perceptibility, operability, comprehensibility, and robustness for blind people. Moreover, almost all of them use a graphical interface that the user does not understand thoroughly. This is why at the Tecnológico Nacional de México campus Tehuacán, a group of researchers have been focused, for more than three years now, on developing applications that allow the inclusion of people with hearing and visual disabilities through image processing. This article presents the results of their research and the development of a solution for image processing, which results in one software and two mobile applications that seek to position themselves as a technology option to support people with visual disabilities. The ultimate goal of this research is to integrate new technologies such as virtual assistants into software developments to allow natural and straightforward interaction with the environment to achieve independence and inclusion into the world. Therefore, after this introduction, Sect. 2 presents the development methodology, the research phase, and the research background. Next, Sect. 3 addresses the discussion of development alternatives and testing of the previous work. Then, Sect. 4 presents the proposed solution showing the final application and the tests performed. Finally, the last section sets out the conclusions and references.

Virtual Assistant as Support for People Visually Impaired

177

2 Methodology 2.1 Development Methodology The methodology used was Checkland’s Soft Systems (MSS) (1990). This methodology aims to study the so-called soft systems from which it is possible to observe processes, interactions, and human and social order interpretations. “The human world is in which nothing happens the same twice in an exact way” [9]. Therefore, soft problems are challenging to define, and instead of raising a problem itself, a problem situation is proposed. In this research, the problem situation is the use of information technologies so that people with visual disabilities can have access to elements that are in their environment, such as objects, books, street signals, medicines, restaurant menus, etc. Peter Checkland’s original idea is to describe in the form of a list the six elements that must be investigated to reach a satisfactory conclusion: Customers, Actors, Transformation process, Global vision, Owner, and Environmental constraints (CATWOE) [10], Table 1. Table 1. Agents involved in the process, represented by the methodology of soft systems Initial Elements

Description

C

Customers

Visually impaired people

A

Actors

Software developers

T

Transformation

Using the technology of virtual assistants to develop and include an application that allows digital image recognition (OCR) to recognize images, texts, and elements and be reproduced through voice

W

World View

Providing people with visual disabilities information technologies in order to have inclusion and independence in general

O

Owner

Software developers

E

Environmental constraints Promoting the use of the software and device within institutions that support people with visual disabilities

Root definition: Visually impaired people who do not have access to written material, images, medical prescriptions, or medications if they are not written in braille; if the person is unaware of braille; or if being in an unknown and challenging environment (with the possibility to trip over objects in their path or not knowing which person is in front of them). This is followed by analyzing CATWOE’s responses, which leads us to identify underlying problems that can impact research (such as the user’s resistance to using an application -for fear that it will be more complicated than the traditional way-). Finally, once the application is developed, a test plan is proposed with the end-user (visually impaired people) to achieve feedback and an improvement process to details that were not considered in the initial development and research stage. This is one of the main

178

L. E. Olguín-Gil et al.

reasons why MSS was chosen since this methodology is recursive and allows to rethink elements that were not initially considered. 2.2 Phases of the Investigation The project has been divided into six phases; the procedure designed at each stage is described below: • In the first phase, an investigation of image recognition algorithms of people, text, and objects is carried out to define the state of the art of the research. • In the second phase, the requirements for the design of the application are lifted. • In the third phase, tests are carried out with different training or learning algorithms to determine the best results presented in the capture, processing, and recognition of the image. • In the fourth phase, the integration of image recognition is carried out, with pattern recognition and LNP (Natural Language Processing) to achieve the implementation of the virtual assistant. • In the fifth phase, we want to test the new application with visually impaired people integrating the software and their quotidian activities and evaluating its effectiveness and functionality. All the observations on the impact of inclusion and independence were collected to improve future interaction. • In the sixth phase, the investigation would be completed, and an assessment will be done of whether the objectives set at the beginning were achieved or not. 2.3 Background of the Investigation In recent times with the rise of information technologies, applications that allow people with disabilities to expand their skills and being included in the world have been developed. Virtual assistants are software that allows interaction in a simple way between people and technological devices to obtain information, perform tasks, or offer services to an individual. These tasks or services are based on data input by the user, location recognition, and the ability to access information from various online resources: Weather, traffic, news, stock prices, user hours, etc. The success of virtual assistants is the result of the integration of three proven technologies: • Messaging or chat apps (Facebook, Messenger/Workplace chat, Skype/Skype for Business/Slack, etc.). • Natural Language Processing (PLN) engines. • Robotic Process Automation (RPA) -also a blend of three technologies: robotics, artificial intelligence, and machine learning-. RPA is a technology that can automatically execute the transactions assigned to it, work with data, initiate response actions, and collaborate with other systems when and as needed.

Virtual Assistant as Support for People Visually Impaired

179

This integration of technologies allows virtual assistants to contribute information or perform tasks at the request of users, such as make queries on the internet, read, write and send emails, call and send messages to contacts in the calendar, write an appointment in the agenda generating automatically a reminder alarm. In addition, to indicate the better route from one point to another by using a map application, make purchases, turn on, program or turn off home technology devices (light, heating), etc. More and more visually impaired people are using virtual assistants to improve their autonomy in daily life, and the fact that they do not have to interact with touch screens or keyboards to perform a task makes it even easier for them. For example, asking verbally if the lights in the house are on or asking to run the blinds are tasks that seem trivial, but for people with disabilities, those are quite a challenge [11]. According to a study carried out by Fundación ONCE within the Discapnet’s Observatory of ICT Accessibility, the possibility offered by intelligent speakers in terms of home automation control “becomes the service that currently contributes the most and that will contribute to the autonomy of people with disabilities and the concept of independent living” [12]. Currently, we have our virtual assistant for each mobile operating system with a significant market share: Siri on Apple, Cortana on Microsoft, and Google Assistant on Android. Also, the e-commerce company Amazon has created a virtual assistant, Alexa, for improving the user experience in the use of its services. However, the great reception that Alexa has had in the market has evolved its functionality to levels comparable to those of the other three assistants linked to their respective operating systems. Even today, it integrates naturally with Cortana from Microsoft. Each of these assistants is in continuous evolution to improve its functionalities and user experience. Alexa is the voice-controlled virtual assistant created by Amazon and launched in November 2014 alongside its Echo line of smart speakers. Initially, the wizard was only tied to smart speakers created by Amazon. However, a few months later, its software development kit was opened so that other manufacturers and developers could start working on it. Since then, the assistant has been included in many devices, from wall clocks to microwaves. Alexa’s features depend on two key elements: 1. The integrated voice commands with which can be made a wide variety of requests. 2. The skills which are add-ons installed to affix even more functionalities. The Amazon Echo Plus comes with the integrated smart home hub, which can automatically detect compatible devices (smart bulbs, surveillance cameras, thermostats, etc.), thus eliminating complicated configuration processes. These companies offer natural language processing platforms such as Google with Dialogflow, Amazon with Lex, IBM with Watson, each with the same concept but with a different approach [13]. Skills are third-party applications designed specifically for Alexa. With them, you can expand the possibilities of the assistant, adding more voice commands or different sources with which to make its functionalities more complete. Worldwide there are more than 25,000 skills available, although most are not compatible with the Spanish language. However, this circumstance is changing very quickly: the number of skills

180

L. E. Olguín-Gil et al.

grows exponentially, and the same happens with those designed for our language, since every day, more firms are interested in offering their services through Alexa. The study of facial recognition techniques using the neural network is essential in the authentication of people. Since it has many applications such as security systems, criminal identifications, teleconferences, etc., it is worth having further studies about it [14]. There are different mobile applications used as aids for low vision, one of the most used and best-valued applications is Seeing AI. According to the study conducted in “The Use of Mobile Apps as Aids to Low Vision: A Pilot Study”, where approval was obtained by the Massachusetts Eye and Ear Hospital and the INSIGHT Association, patient contributions yielded a list of 14 mobile apps where the best score was from Seeing AI. The free mobile app Seeing AI uses artificial intelligence to describe nearby people, text, and objects, take screenshots and use photos for patients who begin to incorporate low vision tools [15]. Below is the comparison of Seeing AI and the application proposed in this research as support for the visually impaired: Table 2. Comparison of applications for the visually impaired Characteristics applications

Free

IOS and Android

Reading short text and document, barcode scanning

Facial recognition and currency

Virtual assistant

Voice manipulation

Seeing AI

Yes

Yes

Yes

Yes

No

No

Application proposed

Yes

Android

Yes

Yes

Yes

Yes

Other authors are developing virtual assistants for computer-assisted living environments for people with physical disabilities, using Amazon’s Alexa technology that enables automatic speech recognition and natural language comprehension with Artificial Intelligence using skills [16].

3 Discussion The development of image processing applications on Android devices is beneficial for mobility, portability, and low cost. However, portable devices typically provide limited computing power and memory capacity [17]. With the opening of the SDK of development of skills for Alexa by Amazon, our research group considered the possibility of implementing an application that averts that the blind person should interact physically with devices to be able to access the

Virtual Assistant as Support for People Visually Impaired

181

software, either through a keyboard or a computer or from the touch screen of a mobile. Thereby, the person activates by voice a series of commands that allow controlling both the device’s camera to take a photo or image as the speaker device, indicating with voice the object that has been captured by the camera. The project is ambitious and goes beyond just reading text. For example, it aims, through the use of artificial intelligence, the recognition of the denomination of paper currency, which could be complicated to do for people with disabilities since there is no Braille version of these; or the identification of faces, so when the blind person is in a room he or she can recognize if there are more people in there and, if the face of the person has been previously registered (as is done in biometrics), the device will say with a voice who it is. Within the research that has been carried out for three years, has been developing new technologies, such as the APP LECROXVOZ [18], carried out by students of the Tecnológico Nacional de México campus Tehuacán, and that it is compatible with the technology design for this project. The application uses OCR (optical character recognition) on images captured through the application and the camera of the mobile device. In digitizing text from the image, it can be identified and stored characters using the Google Text-to-Speech API and play in audible form. This application was tested with restaurant menus, medicine packaging, and traffic signs, obtaining satisfactory results. Within the assays, it was observed that due to the interaction that the person with disabilities had to do with the touch screen of the mobile device to activate the application, focus the camera and take the photo, makes the process complicated, especially considering that the objective of these applications is to achieve the independence of physical interaction. Thinking about the applications that virtual assistants have in everyday life (like the activation through the voice of the digital surveillance cameras, turn on or off lights, and open or close blinds), a virtual assistant that allows to activate and control the camera of the device through the voice has been proposed, avoiding the physical interaction with the touch screens of the devices, since this is one of the disadvantages of the Seeing AI app, especially with the camera function when manipulating manually.

4 Description of the Proposed Architecture Currently, the project is in the design and implementation phase. In this section, the details of the concept, architecture, and the implementation of the proposal are described. Although during the research, different elements of tiflo technology have been found to support the activities of people with visual disabilities, the concept faces the development of new technologies and its adaption to provide accessibility to existing technologies for use by people with visual disabilities. The new technologies applied to visual impairment are speech synthesizers, screen readers, optical character readers, braille keyboard, text and image magnifiers, and talking scanners [19]. Screen readers are software installed on a computer. Their objective is to reproduce, through a synthetic voice, the information displayed in the monitor, replacing the

182

L. E. Olguín-Gil et al.

keyboard and mouse. In addition, they contain functions that allow reading from characters to full texts, enabling access to the internet browsers and web pages that have included web accessibility within, to access the chat and email, and various other online applications [20]. After analyzing the difficulty that represents for people with disabilities to found within a flat touch screen the location of the control buttons, even with the functionality provided by screen readers, our research group proposes a virtual assistant application that could be downloaded and installed into the mobile device, to control its camera through the voice of the user, avoiding physical interaction with the touch screen. It is also designed to add a face and voice recognition functionality, as some biometric devices allow recognizing a person who has previously been registered in a database. Another element that could bring additional benefits to the final user is recognizing the paper money denominations. This proposal is taking the image of the bill with the camera, and that through the voice, the device will tell which is its denomination—an outline of how the application works is shown in Fig. 2.

Fig. 2. Scheme of operation of the application

In the application development, two technologies were employed to recognize text images using OCR and facial recognition algorithms such as Hopfield neural networks or the PCA/2DPCA (Principal Component Analysis/2 Dimensional PCA) technique. Artificial intelligence is derived from the premise that systems based on these algorithms can understand and process orders using natural language. Facial recognition systems are mainly based on two techniques: Biometrics and PCA. Biometrics is based on calculating distances between significant parts of the face, such as the distance between eyes, dimensions of the nose, size of the mouth, etc. [21]. This calculation of distances would fail if the cameras are uncalibrated, misfocused, or with recurrent movements of the lenses; obtaining blurry images not suitable for a reasonable calculation of distances. Gesturing is another possible variation that can interfere with measuring distances on a face; hence, the dimensions of the mouth and

Virtual Assistant as Support for People Visually Impaired

183

eyes can be substantially modified. Therefore, if good measurements of the distance between characteristic points of the face are needed, we cannot rely solely on a 2D projection. PCA-based systems rely on the appearance of the images to proceed with the comparison; this could be a disadvantage due to the gesturing influence at the measurement moment that for the recognition of a person’s face is made. This problem is solved by enriching the database with enough gestures from the same person. In these two facial recognition techniques, the problem of scene lighting arises, causing that the colors of some regions of the face could vary depending on where the focus is. This alignment is not easily eliminated [22]. Once the algorithms to be used in the process of facial and bill recognition were established, the development of coding four methods was proposed: first, a method that allows the capture of faces, a second one that processes images, a third method that recognizes the image, and the fourth one that reproduces through voice, the name of the person or the denomination of the bill (if those have been previously registered). Important: there is a limit to the number of faces or images stored, considering the internal storage capacity of the mobile device. However, the processing of text images

Fig. 3. Diagram of the architecture and technologies used in the development stage

184

L. E. Olguín-Gil et al.

and their reproduction in voice using OCR has already been solved by our research group, based on the algorithms developed in the previous applications. On the other hand, language recognition used with a database of basic commands, so the application understands a verbal command to activate or execute an action (just as Siri or Alexa do), is being developed using natural language processing techniques. Figure 3 shows the scheme that specifies the technologies used in developing each element of the application and how it works. Although during the documental research applications such as Seeing AI (Microsoft) were found (with similar characteristics to our proposal), we developed some features that offer advantages to the end-user experience. First, the use of natural language processing algorithms avoids the blind person having to press buttons to interact with the application, and second, the fact that our application is focused on the Mexican market. 4.1 Case Study The difficulty of carrying out daily activities for a blind person is the main problem that our research is seeking to solve. Our concern is making actions easier for the blind person, such as recognizing people, knowing money denomination, or reading medicine labels. As mentioned early in this paper, several commercial applications were analyzed. Those devices are an important contribution, but interaction with a screen is required. A blind person is unable to see the display, the buttons, and the menus. These represent the main limitation for the user; since the screen is flat, without any borders, and locating those elements by touch is not possible. Our proposal for a better user experience is to activate the commands by voice. “Face,” “currency,” and “text” are the words that activate the action needed. The functionality sought is similar to that presented by Siri or Alexa. The application interface, because it is focused on the visually impaired, is straightforward, as shown in Fig. 4. It consists only of a large button that occupies almost the entire screen in order to be easy for the blind person to find it on the display.

Fig. 4. Initial application screen

Virtual Assistant as Support for People Visually Impaired

185

The functionality of the application consists of: 1. Installing the application on the device (with bills images preloaded in its database) 2. Uploading up to five faces with the corresponding person’s name 3. The button on the screen surface must be pressed for the application to “listen” a command (Fig. 5)

Fig. 5. Launching the app’s in “listening” mode

4. Verbal commands are short words already defined within the application, such as: “face,” “currency,” “text.“

Fig. 6. Recognizing image

5. The application defines the actions to be performed accordingly to every order or command (see Fig. 6) 6. In the “Face” command, the application opens the device’s rear camera and focuses on the person’s face to identify. Then, a review of the preloaded pictures is quickly made. If the face is registered, the person’s name will be said verbally; if not, the message “person not registered” will be pronounced. 7. With the “Text” option, the camera is opened to take the image of the message, and it is played with voice—Fig. 7. 8. In the action of “Divisa,” the device’s rear camera opens, the image of the bill is taken, and the denomination of the currency is issued with voice (at the moment, it only works for Mexican currencies).

186

L. E. Olguín-Gil et al.

Fig. 7. The user is listening to the text captured in the image

Trials have been made with blindfolded people to simulate blindness. Due to the COVID19 pandemic and the healthy distance restrictions, we used volunteers. We tested the applications available on the market and the one that we developed. Volunteers said to have problems when using commercial applications. Positioning themselves on the screen was the principal issue. They used the device’s edges as a reference, guessing the distance from the edge to the buttons, using both hands trying to locate and manipulate the buttons with their fingers as a length unit. Thus, the maneuver turned of very hard to do. With our proposed application, the orders are voice commands. Volunteers referred to an easier manipulation to carry out the trial. They alluded to a better experience without the need to search within the screen’s surface. There is no need to find by hand the buttons or menus, as done in other applications. Hence, our application has a better interaction. It only takes one hand to zoom in or focus on the object that needs to be recognized. This feature beats any commercial device. Table 2 compares the functions of the proposed application versus the commercial one.

5 Conclusions This application aims to be a companion for people with visual disabilities, which allows them to interact naturally with their environment, without the need to acquire or have additional elements such as an Amazon echo dot or any other; hence, having a smart mobile device would be enough. It is still in development in the improving image processing stage since it has been detected that angles and lighting conditions affect the explicit recognition of the images. The final product will be a downloadable app in Play Store, installable on a smartphone, with an initial setup requirement and the solicitude of input of the images to be recognized by de device - speaking specifically of faces-. It is essential to consider the internal storage of the device since it will be necessary to limit the number of images to be recorded or, if needed, the option of using a cloud for additional storage (which is not considered viable yet, since the application should have a permanent internet connection to be able to access to this external storage). The tests with end-users will be carried out as soon as the engagement in public activities is permitted due to the confinement restrictions caused by the COVID pandemic.

Virtual Assistant as Support for People Visually Impaired

187

These interactions will allow having feedback from blind and visually impaired people concerning future improvements that achieve a satisfactory experience when integrating it into their day-to-day experience.

References 1. cuentame.inegi.org. http://cuentame.inegi.org.mx/poblacion/discapacidad.aspx?tema=P 2. Banco-Mundial. https://www.bancomundial.org/es/topic/disability 3. Naciones-Unidas. Algunos Datos sobre las Personas con Discapacidad. https://www.un.org/ development/desa/disabilities-es/algunos-datos-sobre-las-personas-con-discapacidad.html 4. Microsoft: Aplicación de Cámara que Habla para las Personas con Deficiencias Visuales. https://www.microsoft.com/es-mx/ai/seeing-ai/ 5. Adecco, F.: Las Apps que toda Persona con Discapacidad debe Conocer (2019). https://fun dacionadecco.org/blog/las-apps-que-toda-persona-con-discapacidad-debe-conocer/ 6. Identifi: Tecnología que Empodera a las Personas con Discapacidades visuales (2017). http:// getidentifi.com/ 7. Correa, M.R., González, M.J.A.: Las TIC al Servicio de la Inclusión Educativa. Digit. Educ. Rev. 25, 108–126 (2014) 8. Almenara, J., Córdoba, M.: Inclusión Educativa: Inclusión Digital. Revista Educación Inclusiva 2, 61–77 (2009) 9. Checkland, P., Poulter, J.: Soft systems methodology. In: Reynolds, M., Holwell, S. (eds.) Systems Approaches to Managing Change: A practical guide, pp. 191–242. Springer London, London (2010). https://doi.org/10.1007/978-1-84882-809-4_5 10. Cerem Comunication. CATWOE para tus Problemas (2018). https://www.cerem.es/blog/cat woe-para-tus-problemas 11. ONCE. Informe de Asistentes Virtuales (2019). https://cti.once.es/noticias/informe-sobre-asi stentes-virtuales-y-altavoces-inteligentes 12. Discapnet: Observatorio Accesibilidad de Asistentes Virtuales (2019). https://www.discap net.es/sites/default/files/areas-tematicas/tecnologia/informe_observatorio_asistentes_virt uales_2019_revisado_415862.pdf 13. Manjarrés-Betancur, R.A., Echeverri-Torres, M.M.: Asistente Virtual Académico Utilizando Tecnologías Cognitivas de Procesamiento de Lenguaje Natural Revista. Politécnica 16, 31– 85(2020) 14. Dai, Y., Nakano, Y.: Recognition of facial images with low resolution using a Hopfield memory model. Pattern Recogn. 31(2), 159–167 (1998). https://doi.org/10.1016/S0031-3203(97)000 40-X 15. Dockery, D.M., Krzystolik, M.G.: The use of mobile applications as low-vision aids: a pilot study. R. I. Med. J. 103, 69–72 (2020) 16. Palacios, J., Bosquez, V., Palacios, Á.: Integración de un asistente virtual en ambientes de vida asistida por computador para personas con discapacidad fisica. Revista de Investigación Talentos 7(1), 48–61 (2020). https://doi.org/10.33789/talentos.7.1.122 17. Phuoc Phat Dat Le, D., Tran, D.N., Hussin, F.A., Yusoff, M.Z.: Acceleration of Retinex algorithm for image processing on Android device using Renderscript. In: Mat Sakim, H., Mustaffa, M. (eds.) The 8th International Conference on Robotic, Vision, Signal Processing & Power Applications. Lecture Notes in Electrical Engineering, vol. 291. Springer, Singapore (2014). https://doi.org/10.1007/978-981-4585-42-2_16 18. Vázquez-Guzmán, F., Olguín-Gil, L.E., Vázquez-Zayas, E., Nicanor-Pimentel, B.J.: Mobile app with reading speech-translated ocr images for visually impaired people. ECORFAN J. Mex. 7, 26–31 (2020)

188

L. E. Olguín-Gil et al.

19. Aguirre, E.R., Galindo, R.: Posibilidades de las TIC para la Inclusión Educativa en Educación Superior. Memorias del Encuentro Internacional de Educación a Distancia (2014). http://www. udgvirtual.udg.mx/remeied/index.php/memorias/article/view/11 20. Aquino-Zúñiga, S., García-Martínez, V., Izquierdo, J.: Tiflotecnología y Educación a Distancia: Propuesta para Apoyar la Inclusión de Estudiantes Universitarios con Discapacidad Visual en Asignaturas en Línea. Apertura. Revista de Innovación Educativa (2014). http:// www.udgvirtual.udg.mx/apertura/index.php/apertura/article/view/516/360 21. Bronte, S., Bergasa, L.M., Nuevo, J., Barea, R.: Sistema de Reconocimiento Facial de Conductores (2008). https://tv.uvigo.es/uploads/material/Video/2661/P04.pdf 22. Núñez-Cárdenas, F., Tomás-Mariano, V., Crespo-Hernández, J., Andrade-Hernández, E.: Reconocimiento Facial mediante Redes Neuronales HopField, Backpropagation y Algoritmo PCA, un Enfoque Comparativo (2017). https://www.uaeh.edu.mx/scige/boletin/huejutla/n9/ a4.html#:~:text=Los%20sistemas%20de%20inteligencia%20artificial,%C3%B3rdenes% 20empleando%20el%20lenguaje%20natural.&text=Los%20sistemas%20de%20reconocimi ento%20facial,Analysis%20%2F2%20Dimensional%20PCA

A Novel Method Based on Numerical Iterations for Joint-Space Trajectory Generation with a via Point Yadira Quiñonez1 , Oscar Zatarain1 , Carmen Lizarraga1 , Raquel Aguayo1 , and Jezreel Mejía2(B) 1 Universidad Autónoma de Sinaloa, Mazatlán, México {yadiraqui,carmen.lizarraga,raquelaguayog}@uas.edu.mx, [email protected] 2 Centro de Investigación en Matemáticas, A.C., Universidad Autónoma de Sinaloa, Mazatlán, México [email protected]

Abstract. At present, several methods can be found for the trajectory generation in manipulators, where the objective is to obtain a smooth and continuous motion. This paper presents a novel method based on numerical iterations for modeling the trajectory planning in the joint-space with via point. Its proof is presented, and some examples are described to create the trajectory and comparing to the sixthorder polynomial method. This new method is inspired for avoiding infinite spikes at the beginning and the end of the velocity-acceleration-jerk function. Also, this function is to get better accuracy in matching the via point and the final point. Keywords: Trajectory generation · Manipulators · Numerical iterations · Velocity · Acceleration · Jerk · Via point

1 Introduction In recent years, various researchers have grown interested in industrial robotics [1, 2] and robotic medical systems [3, 4], particularly in planning and generating trajectories. Determining position, velocity, and acceleration over time is crucial to achieving control of a robotic arm. For example, for a manipulator to perform complex and flexible tasks such as painting, polishing, or welding, optimal and complex methods are required [5–9]. The most used methods to calculate trajectories in robotic arms are cubic polynomials [10], trapezoidal trajectory [11], and Euler angles [12]. With these methods, it is possible to obtain smooth movements in the joint space, which are predictable in the task space. Joint-space trajectory generation plays an essential role in each joint of a robotic arm because it provides a continuous and smooth motion to set different joint positions. Exist many different methods to obtain the trajectory for moving between one Cartesian pose to another using joint-space trajectories. In general, every method is straightforward in predicting and calculating its constraints. Nevertheless, these methods sometimes require many calculations to obtain a smooth motion and accuracy. Besides, many of © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 189–204, 2022. https://doi.org/10.1007/978-3-030-89909-7_15

190

Y. Quiñonez et al.

these methods present significant spikes at the begging and end of the jerk function. For example, the third-order, fourth-order, and fifth-order polynomial function presents infinite spikes at the beginning and end of the jerk function. The sixth-order polynomial function is presented by [13]. However, this method presents discontinuity at the beginning and end of the jerk. Therefore, it requires many calculations and presents little inaccuracy at the via point. The final point or oscillations are presented near the via point when the distance between positions is too small; this inaccuracy can be improved. In this work, a trajectory function presented in [14, 15] is used to obtain a new trajectory function with a via point, and it will be tried to solve the problems stated in the past paragraphs. The remainder of this paper is organized as follows: Sect. 2 describes previous required results and a brief explanation that is necessary to introduce this method. Section 3 explains the method to obtain the trajectory with via point, the proof of this method, and one example of generating the trajectory by iterations. Section 4 presents the velocity, acceleration, and jerk function with the results obtained, making brief comparisons with sixth-order polynomial method. Section 5 presents a discussion and a brief conclusion of the paper and indicates further work.

2 Some Crucial Properties This method has been presented and tested in [14, 15] as a single point trajectory, which can be obtained different velocity, acceleration, and jerk shapes using and changing the parameters a, n, ts and constraint ζ , where n is a real fixed number such that n > 1, a is the parameter for changing the shape of the trajectory, velocity, acceleration, and jerk, ts so that nx+ x −1 = na never happens; and ts is the sample time of the trajectory or the final time to get the trajectory at the desired final point. This function is presented below: ⎧ ⎪ q0 if x ≤ 0 ⎪ ⎪ ⎪ ⎨  nt x−ζ  + q0 if 0 < x ≤ ts and q0 ≤ Sf  x+ xs −1 a  −n  n (1) Sζ ats (x) = ⎪ ⎪ nx−ζ ⎪   ⎪ ⎩ − nx+ txs −1 −na  + q0 if 0 < x ≤ tS and Sf < q0 where Sf is the final point, and q0 is the initial point. Both in this case are presented in degrees, and ζ is calculated in [14, 15]. The function (1) is an interpolation for every x ≥ 0 which represents the trajectory time for this application. To present the trajectory function with a via point, using function (1), it is essential to consider the following property of this function. Changing the parameter a such that a < ts to obtain a trajectory without spikes at infinity through the time, a can be manageable to get a trajectory that increases little by little for a specific time; in other x−ζ x−ζ words,  x+ nts −1  (or −  x+ nts −1  depending on the case) would be too small until n

x

−na 

n

x

−na 

x gets to a specific time to increase faster and Sζ ats (x) gets to Sf . Then, this property is used to get the trajectory with a via point. Figure 1 is an example of the property that just was mentioned:

A Novel Method Based on Numerical Iterations

191

Fig. 1. Different trajectory shapes with the same parameters but different a and ζ .

The trajectory in red has a greater parameter a than the blue one, so, in the case of x−ζ the red one, the part  x+ nts −1  is smaller than the blue one for every x ≥ 0. This is n

x

−na 

also true for the case where Sf < q0 [14, 15]. In simple words, the parameter a can be used to change the velocity through time, obtaining many different forms of trajectory between the initial and final point, such as was presented in Fig. 1. Knowing this, in the next section, the trajectory with a via point is presented.

3 Trajectory Function with a via Point The trajectory functions with a via point, denoted as S(ζ ats )v (x), is defined as a summation of the function (1) with different parameters, constraints, and different points to intersect, using the property mentioned before. The values to obtain the trajectory are taken from [13] to obtain and compare results. First, the function is defined below:

192

Y. Quiñonez et al.

⎧ q0 if x ≤ 0 ⎪ ⎪ ⎧ ⎪ x−ζvi ⎪ ⎪ x−ζi ⎪ ⎪ n ⎪ n v ⎪  + q0 if 0 < x ≤ tv < ts and q0 ≤ Sv ≤ Sf ⎪   +  ⎪ ⎪ ⎪  x+ tv −1   x+ ts −1  i ⎪ ⎪ ⎪  ⎪ a  ⎪ −na  ⎪ ⎨ n x −nv v  nv x ⎪ ⎪   ⎪ ⎪ 1st cond ⎪ ⎪ tv −ζvi ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ nx−ζi ⎪   +  nv  ⎪ ⎪ ⎪ ⎪ t  t −1 av  + q0 if 0 < tv < x ≤ ts and q0 ≤ Svi ≤ Sf   s ⎪ v ⎪ ⎪ −nv  ⎩ nx+ x −1 −na  nv ⎪ ⎪   ⎪ ⎪ ⎪ ⎧ x−ζv i ⎪ ⎪ nv ⎪ ⎪ nx−ζi ⎪  + q0 if 0 < x ≤ tv < ts and q0 < Sv , Sf < Sv ⎪  +  −  ⎪ ⎪ ts −1 ⎪  x+ tv −1   i i ⎪ ⎪ ⎪ x+  a a    ⎪ ⎪ −n  ⎪ ⎨ n x nv x −nv v   ⎪ ⎪   ⎪ ⎪ 2nd cond ⎪ ⎪ tv −ζv i ⎪ ⎪ ⎪ ⎪ ⎪ nx−ζi ⎪ ⎪ −   +  nv ⎪ ⎪ ⎨  tv −1 av  + q0 if 0 < tv < x ≤ ts and q0 < Svi , Sf < Svi  x+ ts −1  ⎪ ⎩ −nv  a n n  x v −n   S(ζ ats )v (x) = ⎪ ⎧ ⎪ x−ζvi ⎪ ⎪ ⎪ nv ⎪ ⎪ nx−ζi ⎪  + q0 if 0 < x ≤ tv < ts and Sv < q0 , Sv < Sf ⎪  −  ⎪ ⎪  t t    ⎪ i i ⎪ v s ⎪ ⎪ x+ −1 −na   x+ x −1 av  ⎪ ⎪ ⎪ ⎨ n x −nv  nv  ⎪ ⎪   ⎪ 3rd cond ⎪ ⎪ tv −ζvi ⎪ ⎪ ⎪ ⎪ ⎪ ⎪  nx−ζi ⎪ ⎪  nv   ⎪ ⎪ ⎪ ⎪  −  tv −1 av  + q0 if 0 < tv < x ≤ ts and Svi < q0 , Svi < Sf  x+ ts −1 ⎪ ⎪ −nv  ⎪ a ⎩ n   x v n −n  ⎪  ⎪ ⎪ ⎪ ⎧ ⎪ x−ζv i ⎪ ⎪ nv ⎪ ⎪ nx−ζi   ⎪ ⎪   ⎪ ⎪ ⎪ −  x+ ts −1 a  −  x+ tv −1 a  + q0 if 0 < x ≤ tv < ts and Svi < q0 , Sf < Svi ⎪ ⎪ ⎪ ⎪ ⎪ −n  ⎨ n x ⎪ −nv v  nv x ⎪   ⎪ ⎪ 4th cond ⎪ ⎪ tv −ζv i ⎪ ⎪ ⎪ x−ζi ⎪ n ⎪ ⎪ n ⎪ −  ⎪  −  v ⎪ ⎪  tv −1 av  + q0 if 0 < tv < x ≤ ts and Svi < q0 , Sf < Svi   x+ ts −1 ⎪ ⎪ ⎩ ⎩ −nv  nv n x −na 

(2)





where, for any iteration i ∈ {0, 1, 2, . . . } at tv , Svi is defined as ⎧ ntv −ζi   ⎪ ⎪ ⎪ θv − ntv + ttvs −1 −na  for the 1st and 3rd cond ⎨   Svi = ntv −ζi   ⎪ + θ ⎪ v  tv + ts −1  for the 2nd and 4th cond ⎪ ⎩ n tv −na  

(3)



where θv is the desired via point at the time tv . For the initial iteration Sv0 , it should be chosen ideally to lie close to the desire via point, Sf is the final point and q0 is the initial point. nv , tv , ζvi and av corresponds to the part of (2) where the trajectory to get the via point is calculated, and n, ts , ζi and a for calculating the function from the via point to the final point in (2). To calculate ζi and ζvi the same process taken from [14, 15] is used for each condition. It is got: ⎧   tv a ⎨ tv − log if q0 < Svi nv Svi − q0 nv − nv   ζv i = (4) tv a ⎩ tv − log if Svi < q0 nv q0 − Svi nv − nv   

ts − logn Sf − Svi nts − na  if Svi < Sf ζi = ts − logn Svi − Sf nts − na if Sf < Svi   ts   Taking in consideration that nx−ζ /nx+ x −1 − na  is changing while getting to the via point (when x ≤ tv ), this can affect the trajectory for getting the via point and the endpoint. For example, if Sv1 = 180 at tv = 1.5 and Sf = 120 at ts = 3, the trajectory obtained with these values is presented in Fig. 2.

A Novel Method Based on Numerical Iterations

193

Fig. 2. Trajectory with Sv1 = 180 at tv = 1.5 and Sf = 120 at ts = 3.

3.1 Iteration Method to Get the Desired via Point It is easy to observe in Fig.2 the trajectory does not match with the via point at 1.5 s, due ts   to the increment of nx−ζi /nx+ x −1 − na  has not been considered yet. Thus, to reduce this error, it must find Svi for making the iterations. This method is presented below: Let q0 the initial point of the trajectory, θv the via point, and Sf the final point. A trajectory is sought for a specific tv and ts , where S(ζ ats )v (tv ) = θv and S(ζ ats )v (ts ) = Sf , S(ζ ats )v (x) is defined as in Eq. (2). Starting from any Sv0 , then, exist an unique Svi = θv −

tv −ζi−1  n   tv + ts −1  n tv −na   

or Svi = θv +

tv −ζi−1  n   tv + ts −1  n tv −na   

at the i’th step, given Svi−1 , compute Svi

calculating ζi−1 using Eq. (4), that makes S(ζ ats )v (tv ) = θv and S(ζ ats )v (ts ) = Sf . Proof: considering the first case and for when x is 0 < x ≤ ts for any Sv0 , Sf and q0 chosen, such that q0 ≤ Sv0 ≤ Sf , then it is had a ζv0 and ζ0 by Eq. (4), and x−ζv

nv 0 nx−ζ0 +  + q0 S(ζ ats )v (x) =  t t v  x+ xs    − na  nvx+ x −1 − nav v  n  

(5)

For when x = tv , S(ζ ats )v (x) must be our desired via point which is θv , so, S(ζ ats )v (x) gets θv for some x ∈ (0, ts ], but, it is wanted for when x = tv , it must be S(ζ ats )v (tv ) = θv . Then tv −ζv

nv 0 ntv −ζ0   + q0 = S(ζ ats )v (tv ) = θv   +   tv + ttvs −1   − na  ntvv − nav v  n

(6)

194

Y. Quiñonez et al.

And, making Sv1 =

tv −ζv0

 nv   tv av  nv −nv 

+ q0 , it is obtained

ntv −ζ0  Sv1 = θv −   tv + ttvs −1  − na  n Then, we can choose Sv1

=

θv −

tv −ζ0  n   tv + ts −1  n tv −na   

(7)

to obtain ζv1

=

      log nv Sv1 − q0 ntv − na and ζ1 = ts − log n Sf − Sv1 nts − na and get: ⎛ ⎞ tv −ζv1 t −ζ v 1 n n  + q0 ⎠ = θv  + ⎝ v S(ζ ats )v (x) =   tv  tv + ttvs −1  a  a −n  nv − nv v  n Making Sv2 =

tv −ζv1

 nv   tv av  nv −nv 

+ q0 = θv −

tv −ζ1  n   tv + ts −1  n tv −na   

tv −

(8)

and repeating the same procedure

 

(i − 1) - times, it is obtained ζvi−1 = tv − log nv Svi−1 − q0 ntvv − nav and ζi−1 =    ts − log n Sf − Svi−1 nts − na ; then the Eq. (9) is obtained. tv −ζvi

ntv −ζi−1  + q0 = θv −   Svi =   tv  tv + ttvs −1  a  − na  nv − nv v  n nv

(9)

Since S(ζ ats )v (x) is always Sf at x = ts , this means S(ζ ats )v (ts ) = Sf for any Svj ; j ∈ N by [2, 3], then, Svj is a finite number for any j ∈ {0, 1, 2, 3, . . . }, this       mean ζi = ts − log n Sf − Svi nts − na and ζvi = tv − log nv Svi − q0 ntv − na mustconverge for (i → ∞),  then ζi−1 = ts −  when i−  Times tends to infinity  when i tends log n Sf − Svi−1 nts − na = ζi = ts − log n Sf − Svi nts − na for  = ζvi = to infinity and the same for ζvi−1 = tv − log nv Svi−1 − q0 ntv − na    tv − log nv Svi − q0 ntv − na . This mean that Svi and Svi+1 are doing equal trough i tends to infinity, and due to ζi−1 , ζi and ζvi−1 , ζvi converge, then: tv −ζv

nv i−1 ntv −ζi−1   + q0  = Svi = θv −   tv  tv + ttvs −1  a  − na  nv − nv v  n

(10)

tv −ζv

Svi+1

i n ntv −ζi  + q0 = v = θv −  t s  tv  tv + tv −1  a  − na  nv − nv v  n

   Converge for then ζi−1 = ts −log n Sf − Svi−1  nts − na  =  when i tends  t to infinity,  ζi = ts − log n Sf − Svi n s − na and ζvi−1 = tv − log nv Svi−1 − q0 ntv − na =    ζvi = tv − log nv Svi − q0 ntv − na , obtaining the definitive Svi+1 for when i− times tends to infinitive and where S(ζ ats )v (tv ) = θv finally. This mean, when the procedure is made i− Times and for when i tends to infinity (the method is made infinity times

A Novel Method Based on Numerical Iterations

195

or in other words, infinity iterations are made), then ζi−1 = ζi and ζvi−1 = ζvi , and Svi converges to Svi+1 , making to Svi unique and therefore: ⎛ ⎞ tv −ζi n ntv −ζi−1  + q0 ⎠  + ⎝ v S(ζ ats )v (tv ) =   tv  tv + ttvs −1  a  a −n  nv − nv v  n ⎛ ⎞ t −ζ t −ζ v v i−1 i n n  + ⎝θv −  ⎠ =  tv + ttvs −1   tv + ttvs −1  a −n  − na  n n ntv −ζi−1  + Svi+1 = θv =  tv + ttvs −1  − na  n

(11)

An analog proof can be made using the other conditions. For when the 3rd case is considered, it is got a: tv −ζv

i n ntv −ζi  + q0  = − v Svi+1 = θv −    tv + ttvs −1  t a  − na  nvv − nv v  n

(12)

For the 2nd condition: tv −ζv

i n ntv −ζ  + q0 = v Svi+1 = θv +  t s  tv  tv + tv −1  a  − na  nv − nv v  n

(13)

For the 4th condition, it is got tv −ζv

nv i ntv −ζ   − q0  = Svi+1 = θv +   tv  tv + ttvs −1  a  − na  nv − nv v  n

(14)

Fortunately, infinity iterations are not required to get exact via point position; finite iterations can be applied to get a good approximation to the desired via point. For example, the method presented in the proof is used to get a trajectory with an initial position equal to 2°  (q0 = 2), a via point in 8° (θv = 8) in 1.5 s (tv = 1.5) and a final position equal to 7 Sf = 7 in 3 s (ts = 3). The method starts with Sv0 = 2π , emphasizing that the first iteration can be started with any Sv0 . However, it is recommended to start using a Sv0 not too different to θv , so Sv0 can be an excellent guess to get faster the desired via point θv in tv . Also, in example 1, a q0 = 2, θv = 8 and Sf = 7 are considered to show that the final trajectory obtained does not oscillate for when these three positions are close (see Fig. 3, 4 and 5). Example 1: Let n = 2, a = 1, ts = 3 and nv = 4, av = 1.05 and tv = 1.5, first, ζ0 and ζv0 are calculated with Sv0 = 2π with the initial position q0 = 2 and Sf = 7 and the desired via point θv = 8. Using Eq. (4), then ζ0 = −0.8317 and ζv0 = −0.4528. The trajectory obtained in the first iteration is shown in Fig. 3.

196

Y. Quiñonez et al.

Fig. 3. First iteration depicted, using Sv0 = 2π and obtained as a result ζ0 = −0.8317, ζv0 = −0.4528.

Then,

 n

ts  nx−ζ0 / x+ x −1

  ≈ −1.0587 and making Sv1 = θv −

 − na



 nx−ζ0

   x+ txs −1 a  −n  n

≈

8 − (−1.0587). Continuing with the next iteration, using Sv1 = 8 − (−1.0587), ζ1 and ζv1 are calculated, then, by Eq. (4), ζ1 = −0.8560 and ζv2 = −0.6267, and the next trajectory is obtained (see Fig. 4).

Fig. 4. Start trajectory and first iteration trajectory are depicted, using Sv1 = 8 − (−1.0587)

  ts   Then nx−ζ1 /nx+ x −1 − na  ≈ −1.1942 and for the next iteration, Sv2 = θv −   x−ζ  nt 1   x+ xs −1 a  −n  n

≈ 8 − (−1.1942). Repeating the method 14 times again, a good approx-

imation to the via point θv = 8 is obtained where Sv16 = 8 − (−1.3814), ζ16 = −0.8882

A Novel Method Based on Numerical Iterations

 and ζv16 = −0.8368, then for a i getting bigger, Svi = θi −

197

 nx−ζi−1

   x+ txs −1 a  −n  n

start to

converges in a value closed to 8 − (−1.3814). Obtaining the following trajectories in each iteration is shown in Table 1 and depicted in Fig. 5. Table 1. Number of iterations to calculate the trajectory with a desired via point θv = 8 at tv = 1.5 s and Sf = 7 at ts = 3 s. θvi represents the via point obtained in each iteration, it means S(ζ ats )v (tv ) = θvi . Svi

Sf

ntv −ζi    tv + ts −1  n tv −na   

θvi

1

2π

7

−1.0587

7.7663

2

8 − (−1.0587)

7

−1.1942

7.8645

3

8 − (7.8645)

7

−1.2728

7.9214

4

8 − (−1.2728)

7

−1.3184

7.9544

5

8 − (−1.3184)

7

−1.3449

7.9735

6

8 − (−1.3449)

7

−1.3603

7.9846

7

8 − (−1.3603)

7

−1.3692

7.9911

8

8 − (−1.3692)

7

−1.3744

7.9948

Iteration i

9

8 − (−1.3692)

7

−1.3791

7.9983

10

8 − (−1.3791)

7

−1.3801

7.9990

11

8 − (−1.3801)

7

−1.3807

7.9994

12

8 − (−1.3807)

7

−1.3810

7.9997

13

8 − (−1.3810)

7

−1.3812

7.9998

14

8 − (−1.3812)

7

−1.3813

7.9999

15

8 − (−1.3813)

7

−1.3814

7.9999

16

8 − (−1.3814)

7

−1.3814…

7.99999…

Example 2: Given q0 = 30, Sv0 = −285, Sf = 120, n = 2, nv = 4, tv = 1.5, ts = 3, a = av = 0.7 and a desired via point θv = −105. Table 2 presents the iterations obtained and Fig. 6 shows the trajectories obtained in each iteration.

4 Velocity, Acceleration, and Jerk Functions The velocity, acceleration, and jerk functions obtained through function (2) are convenient because each function is finite and does not present infinite spikes at the beginning

198

Y. Quiñonez et al.

Fig. 5. Trajectories with several iterations equal to 16 and q0 = 2, θv = 8, Sf = 7.

Table 2. Number of iterations to calculate the trajectory with via point θv = −105 at tv = 1.5 s and Sf = 120 at ts = 3 s. Iteration i

Svi

Sf

ntv −ζ    tv + ts −1  n tv −na   

θvi

0

−285

120

51.4189

−58.6055

1

−105–226.3945

120

252.3289

−79.0656

2

−105–252.3289

120

266.8262

−90.5027

3

−105–266.8262

120

274.9302

−96.8960

4

−105–274.9302

120

279.4603

−100.4699

…

…

…

…

…

12

−105–284.8891

120

285.0273

−104.8618

13

−105–285.0273

120

285.1046

−104.9227

14

−105–285.1046

120

285.1478

−104.9568

…

…

…

…

…

21

−105–285.2009

120

285.2016

−104.9993

22

−105–285.2016

120

285.2020

−104.9999

and end. The velocity, acceleration, and jerk profiles can be changed through the parameters. Therefore, it can take the most convenient profiles according to get a smooth, long live, and long live and reliable robot system.

A Novel Method Based on Numerical Iterations

199

Fig. 6. Different trajectories obtained in each iteration. The total number of iterations obtained is 22 and q0 = 30, θv = −105, Sf = 120.

The functions are presented below: first, considering the derivatives of and

x−ζv i

nv    x+ txv −1 av  , nv −nv  

x−ζ  nt i   x+ xs  −na  n

these are: x−ζv +1

ts

x+ tv

ln|n|nx−ζi +1 (ts nx+ x − na+1 x2 ) ˙ ln|nv |nv i (tv nv x − nav +1 x2 ) S˙ 1 (x) = , S = (x) 2

  2 2 ts x+ txv av +1 nx+ x − na+1 x2 nv − nv x2

S¨ 1 (x) =

ln(n)nx−ζi +1

     2ts ts ts n2x+ x ln(n) + nx+ x +a+1 ln(n) ts2 + nx+ x +a+1 ln(n) + n2a+2 ln(n) x4  3 ts nx+ x − na+1 x4

   ts 2ts ts 2nx+ x +a+1 − 2n2x+ x ts x − 4nx+ x +a+1 ln(n)ts x2 +  3 ts nx+ x − na+1 x4    x−ζvi +1

ln(nv )nv

2x+ 2txv

nv

x+ txv +av +1

ln(nv ) + nv

S¨ 2 (x) =



x+ txv

nv 

x+ txv +av +1

2nv +

(15)

2x+ 2txv

− 2nv 

x+ txv

nv



  2a +2 ln(nv ) + nv v ln(nv ) x4

3 − nav +1

x4 

ln(nv )tv x2

(17)

3 − nav +1

x+ txv +av +1

ln(nv ) tv2 + nv

x+ txv +av +1

tv x − 4nv

(16)

x4

200

Y. Quiñonez et al.

Then, the velocity function is:

⎧ S˙ 1 (x) + S˙ 2 (x) if 0 < x ≤ tv < ts and q0 ≤ Svi ≤ Sf ⎪ ⎪ 1st cond ⎪ ⎪ S˙ 1 (x) if 0 < tv < x ≤ ts and q0 ≤ Svi ≤ Sf ⎪ ⎪

⎪ ⎪ ˙ ⎪ −S1 (x) + S˙ 2 (x) if 0 < x ≤ tv < ts and q0 < Svi , Sf < Svi ⎪ ⎪ ⎪ 2nd cond −S˙ (x) + S˙ (x) if 0 < t < x ≤ t and q < S , S < S ⎪ ⎨ 2 v s 0 vi f vi

1 ˙ s ) (x) = ˙ 1 (x) − S˙ 2 (x) if 0 < x ≤ tv < ts and Svi < q0 , Svi < Sf S(ζ at S v ⎪ 3rd cond ˙ ⎪ ⎪ S − S˙ 2 (x) if 0 < tv < x ≤ ts and Svi < q0 , Sv < Sf ⎪

1 (x) ⎪ ⎪ ˙ ⎪ − S˙ 2 (x) if 0 < x ≤ tv < ts and Svi < q0 , Sf < Svi − S (x) ⎪ 1 ⎪ 4th cond ⎪ ⎪ ˙ −S1 (x) − S˙ 2 (x) if 0 < tv < x ≤ ts and Svi < q0 , Sf < Svi ⎪ ⎪ ⎩ 0 if x ≤ 0 (18)

⎧ S¨ (x) + S¨2 (x) if 0 < x ≤ tv < ts and q0 ≤ Svi ≤ Sf ⎪ ⎪ 1st cond 1 ¨ ⎪ ⎪ S1 (x) if 0 < tv < x ≤ ts and q0 ≤ Svi ≤ Sf ⎪ ⎪

⎪ ⎪ ¨ ⎪ −S1 (x) + S¨2 (x) if 0 < x ≤ tv < ts and q0 < Svi , Sf < Svi ⎪ ⎪ 2nd cond ⎪ ⎪ −S¨ (x) + S¨2 (x) if 0 < tv < x ≤ ts and q0 < Svi , Sf < Svi ⎨

2 ¨ S(ζ ats )v (x) = S¨ (x) − S¨2 (x) if 0 < x ≤ tv < ts and Svi < q0 , Svi < Sf ⎪ 3rd cond ¨1 ⎪ ⎪ S (x) − S¨1 (x) if 0 < tv < x ≤ ts and Svi < q0 , Sv < Sf ⎪

1 ⎪ ⎪ ¨ ¨ ⎪ ⎪ ⎪ 4th cond −S1 (x) − S2 (x) if 0 < x ≤ tv < ts and Svi < q0 , Sf < Svi ⎪ ⎪ ¨ −S1 (x) − S¨2 (x) if 0 < tv < x ≤ ts and Svi < q0 , Sf < Svi ⎪ ⎪ ⎩ 0 if x ≤ 0 (19) In the same way, the jerk function is defined. 4.1 Trajectory Planning and Results Now, using the values to create the trajectory and compare to the sixth-order polynomial presented in [13], with q0 = 30, θv = 180 at tv = 1.5 s, and Sf = 120 at ts = 3 s. It is presented below: θ (t) = −9.22t 6 + 85.19t 5 − 265.56t 4 + 282.22t 3 + 30

(20)

Note for when t = 1.5 s, then θ (1.5) = 179.985 and for when t = 3 s, then θ (3) = 119.37, then, for better results, more precise calculations must be performed. Also, the author in [13] states that a maximum angle θMAX = 185.6, occurring at t = 1.70 s. These values can be a disadvantage because a robotic arm could be delimited by the workspace or its degrees of freedom. Besides, it presents large values at the start and end of the jerk. Therefore, these disadvantages can be eliminated by using the function proposed. Taking the same values to get the trajectory planning. The following results have been obtained, taking nv = 10, av = 0.7, tv = 1.5, ts = 3, n = 8, a = 0.7, ζvi = −2.1257 and tv −ζi 1.5−(−2.0302)  = 180 +  8  = 180 + 8.7254 ζi = −2.0302 then Svi = θv +  nts    3 ntv + tv −1 −na   

81.5+ 1.5 −1 −80.7   

A Novel Method Based on Numerical Iterations

201

(some iteration has been made for finding the values of ζvi , ζi , Svi and getting the desired trajectory). The final trajectory is obtained because of some iterations realized: ⎧ x+2.0302 10x+2.1257  + 30 if 0 < x ≤ tv , q0 < Sv , Sf < Sv  +  ⎪ −  x+8 3 −1 ⎪ i i    ⎨ 8 x −80.7  10x+ 1.5 x −1 −100.7    S(ζ ats )v (x) = 1.5+2.1257 8x+2.0302  ⎪ 10  ⎪ ⎩ −  x+ 3 −1  + 101.5 −100.7  + 30 if tv < x ≤ ts , q0 < Svi , Sf < Svi 8

x

−80.7 

(21)

Fig. 7. (a) represents the trajectory with a via point equal to 180 in 1.5 s (line in red) and the final position equal to 120 in 3 s. (b) represents the velocity of the trajectory, (c) represents the acceleration obtained in the trajectory, and (d) represents the jerk obtained in the trajectory. (Color figure online)

Note that in Fig. 7, the velocity, acceleration, and jerk functions do not match at tv = 1.5 s. Nevertheless, this does not present infinite spikes in acceleration and jerk, also, the velocity, acceleration and jerk remain finite. This miss-matched can be fixed by finding the proper parameters and constraints, but other techniques are required to obtain these parameters. On the other hand, the jerk is always finite and starts at zero and ends at zero. The same happens to acceleration. The velocity, acceleration, and jerk

202

Y. Quiñonez et al.

do not present large values through the trajectory planning, unlike other methods such as the sixth-order polynomial presented in [13]. Making the same trajectory, but softer, the next values are used: n = 8, a = 1.5, ts = 3 and nv = 4, av = 1.207509959, tv = 1.5, then, ζi = −2.0178 and ζvi = −2.8662 with Svi = 180 − (−9.4882) with a number of iteration i = 9 where the method was started with Sv0 = 180. The following trajectory and profiles are obtained and are shown in Fig. 8.

Fig. 8. Softer trajectory obtained using different parameters. (a) represents the trajectory with a via point equal to 180 in 1.5 s (line in red) and the final position equal to 120 in 3 s. (b) represents the velocity of the trajectory, (c) represents the acceleration obtained in the trajectory, and (d) represents the jerk obtained in the trajectory. (Color figure online)

In this trajectory depicted in Fig. 8, the maximum angle of S(ζ ats )v (x), denoted as θMAX , is θMAX = 180.7822, unlike to sixth-order polynomial presented in [13], where the maximum angle was θMAX = 185.6. Also, the velocity is almost continuous in this trajectory; perhaps, a continuous velocity, acceleration, and jerk can be obtained if θMAX is increment or finding better set values of S(ζ ats )v (x) where these functions are continuous and θMAX is not very increased.

5 Discussion and Conclusion The contribution of this paper is to present a new method based on numeric iterations to get the desired value as a via point and a final point, avoiding some disadvantages that

A Novel Method Based on Numerical Iterations

203

in other methods are presented. Most of the nth -order polynomial methods presented in books and different papers, although these are suitable solutions, cannot be applied in some circumstances, such as having restrictions in the workspace or in the degrees of freedom of the robot system. Also, precision is crucial in some applications, and these methods show some inaccuracy at these so-called via points and final points. Another disadvantage presented in this nth-order polynomial is that trajectories with small distances using these methods show big oscillations. An example of this is presented in [13], the θMAX is 185.6°, and this happens for a great difference of the via point and final point; that means, for a via point so close to the final point, greater and many oscillations can be presented. These oscillations are also solved by the function that has been proposed in this paper. In addition, some disadvantages presented in this paper have been solved. The method presented is easy to use, does not require many calculations and iterations to find the best solutions, and does not present oscillations in small distances. The velocity, acceleration, and jerk are always finite through the trajectory. Nevertheless, these present functions miss-matched at tv and this could be solved in future works. The ideas on this paper have been inspired in solving some of the issues remarked in [13]. The method presented is an excellent solution to obtain accuracy in the trajectories, the smooth, and the oscillation avoidance. Also, the Velocity-Acceleration-jerk function can be modified to choose a better solution according to the application. As future work of this investigation, it is necessary to solve some problems that may arise. For example, some of these works could be simulated to see how the motion planning behaves in the robotic arm. Another work could be discovering some arithmetic properties of this function and techniques that can help us to find a method without the issues that continue to arise.

References 1. Grau, A., Indri, M., Bello, L.L., Sauter, T.: Industrial robotics in factory automation: from the early stage to the Internet of Things. In: 43rd Annual Conference of the IEEE Industrial Electronics Society, Beijing, pp. 6159–6164, IEEE Press (2017) 2. Yenorkar, R., Chaskar, U.M.: GUI based pick and place robotic arm for multipurpose industrial applications. In: 2nd International Conference on Intelligent Computing and Control Systems, Madurai, India, pp. 200–203, IEEE Press (2018) 3. Burgner-Kahrs, J., Rucker, D.C., Choset, H.: Continuum robots for medical applications: a survey. IEEE Trans. Rob. 31(6), 1261–1280 (2015) 4. Murali, A., et al.: Learning by observation for surgical subtasks: multilateral cutting of 3D viscoelastic and 2D orthotropic tissue phantoms. In: IEEE International Conference on Robotics and Automation, Seattle, WA, pp. 1202–1209. IEEE Press (2015) 5. Chen, W., Li, X., Ge, H., Wang, L., Zhang, Y.: Trajectory planning for spray painting robot based on point cloud slicing technique. Electronics 9, 908 (2020) 6. Lan, J., Xie, Y., Liu, G., Cao, M.: A multi-objective trajectory planning method for collaborative robot. Electronics 9, 859 (2020) 7. Mohsin, I., He, K., Li, Z., Du, R.: Path planning under force control in robotic polishing of the complex curved surfaces. Appl. Sci. 9, 5489 (2019) 8. Park, S.-O., Lee, M.C., Kim, J.: Trajectory planning with collision avoidance for redundant robots using Jacobian and artificial potential field-based real-time inverse kinematics. Int. J. Control Autom. Syst. 18(8), 2095–2107 (2020). https://doi.org/10.1007/s12555-019-0076-7

204

Y. Quiñonez et al.

9. Cimurs, R., Suh, I.H.: Time-optimized 3D path smoothing with kinematic constraints. Int. J. Control Autom. Syst. 18(5), 1277–1287 (2020). https://doi.org/10.1007/s12555-019-0420-x 10. Sidobre, D., Desormeaux, K.: Smooth cubic polynomial trajectories for human-robot interactions. J. Intell. Rob. Syst. 95(3–4), 851–869 (2018). https://doi.org/10.1007/s10846-0180936-z 11. Hong-Jun, H., Yungdeug, S., Jang-Mok, K.: A trapezoidal velocity profile generator for position control using a feedback strategy. Energies 12(7), 1–14 (2019) 12. Dong, M., Yao, G., Li, J., Zhang, L.: Research on attitude interpolation and tracking control based on improved orientation vector SLERP method. Robotica 38(4), 719–731 (2020) 13. Williams, R.L.: Simplified robotics joint-space trajectory generation with a via point using a single polynomial. J. Robot. 2013, 1–6 (2013) 14. Quiñonez, Y., Mejía, J., Zatarain, O., Lizarraga, C., Peraza, J., Estrada, R.: Algorithm to generate trajectories in a robotic arm using an LCD touch screen to help physically disabled people. Electronics 10, 104 (2021) 15. Quiñonez, Y., Zatarain, O., Lizarraga, C., Mejía, J.: Proposal for a new method to improve the trajectory generation of a robotic arm using a distribution function. In: Mejia, J., Muñoz, M., Rocha, Á., Quiñonez, Y. (eds.) CIMPS 2020. AISC, vol. 1297, pp. 213–231. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-63329-5_15

Main Technological Tools in Virtual Educational Environments According to Different Strategies and Learning Styles to Improve the e-learning Process Raquel Aguayo, Carmen Lizarraga(B) , and Yadira Quiñonez(B) Universidad Autónoma de Sinaloa, Mazatlán, México {raquelaguayog,carmen.lizarraga,yadiraqui}@uas.edu.mx

Abstract. At present, it has considerably increased technological advances and innovations, which has allowed the development of new technological tools and virtual environments to optimize the teaching-learning process. This work aims to classify the different technological tools according to teaching strategies and learning styles to integrate various strategies in the subject’s content and thus facilitate the teaching-learning process in virtual environments. The applied methodology was a non-experimental quantitative cross-sectional study with a descriptive scope. A digital questionnaire was applied in Google format to know the didactic resources, tools, and strategies in response to virtual education. Finally, an analysis was carried out on the learning strategies used by teachers during virtual classes, and a set of learning tools and strategies is proposed according to the learning styles of neuro-linguistic programming. Keywords: Technological tools · Teaching strategies · Learning styles · Virtual educational environments · Neuro-linguistic programming · Didactic resources

1 Introduction Currently, people live immersed in the so-called information society because technology has had a considerable increase. With these advances, education has been favored, and the traditional model has evolved with the use of new Information and Communication Technologies (ICT) [1]. The application of virtual educational spaces; has undoubtedly been marking another era [2]. When it comes to technology, it refers to a set of applied knowledge and techniques, which facilitate the creation of tools and virtual environments to exchange and collaborate to achieve the optimization of the teaching-learning process [3]. For this reason, constant technological change leads society to apply technology in various areas, such as medicine, communications, the environment, industry, and education [4]. Technology applied to education is of great help for students to positively and self-taught the teaching and learning process [5]. Due to the importance and current teaching conditions at different educational levels, some authors [6] propose to take advantage of the current situation of students who © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 205–217, 2022. https://doi.org/10.1007/978-3-030-89909-7_16

206

R. Aguayo et al.

have grown up under the influence of interactive technology, creating communities of knowledge based on learning combined. With the use and incorporation of emerging technologies, humanity has been provided with new channels of communication and immense sources of information that disseminate models of social behavior, attitudes, values, and forms of organization. As a result, students acquire more responsibility in the teaching-learning process and improve abilities both in individual and group study [7]. The virtualization of the teaching-learning process before the COVID-19 pandemic was considered an optional modality. However, in the last year, it became an obligation forcing educational institutions of all academic levels to change learning environments. The main drawback that most of the institutions had was the lack of a contingency plan or adequate technology to carry out all the online courses. In [8] the author presents a concentrate of different works related to the experience of some post-COVID-19 universities, focusing on applying virtual environments, highlighting the advantages and suitability of the technological tools used in learning activities online. On the other hand, Kumar et al. [9] present a work related to assistive technology tools available to deliver quality education in online environments. The authors focus on Webinars as a teaching tool for teachers to conduct their classes during this period of COVID-19. Among the platforms used for Webinars, the following stand out Zoom [10], GoToWebinar [11], Google Meet [12], Cisco Webex [13], and Microsoft Team [14]. The authors have compared each of these platforms with their main characteristics: free version available, the number of meeting participants, screen sharing, whiteboard, meeting recording, security (E2E encryption), payment plans, mobile application, and collaborative workspace. They highlight the importance of exploring different platforms, training teachers and students, conducting demonstration sessions, and introducing assessment platforms. This work aims to classify the different technological tools according to teaching strategies and learning styles to facilitate the teaching-learning process in virtual environments. The remainder of this paper is organized as follows: Sect. 2 presents the theoretical context underlying the research process concerning educational paradigms, learning styles, learning strategies, and the Neuro-Linguistic Programming model. Section 3 describes the methodological strategy applied in this work. In Sect. 4, the results obtained with the applied digital questionnaire and analysis of the technological tools are presented. Finally, Sect. 5 presents a summary and a critical analysis of the conclusions obtained with this research work.

2 Conceptual Fundamentals Learning occurs empirically in all aspects of life and begins at an early age. Learning has been studied by different disciplines that are based on various theories that explain different learning styles. There are different educational paradigms; Fig. 1 shows five models that support this research. Learning styles focus on how knowledge is received or understood, and it is generally used as a learning method or strategy [15]. These learning styles allow optimizing the

Main Technological Tools in Virtual Educational Environments

207

teaching process for the teacher and the learning process for the student since it helps to understand why some strategies may work perfectly for a group of students and others not. For this reason, once the teacher knows the learning styles that exist, the teacher can identify which ones predominate in the group and, in this way, be able to implement the most appropriate learning strategies that strengthen all the students’ learning channels [16]. According to Gómez et al. [17], Table 1 shows some of the most known and used learning models.

Fig. 1. Most common educational paradigms in the teaching-learning process tools.

Table 1. Characteristics of learning styles. Model

Learning style

Characteristics

Herrmann’s brain quadrants

Left cortical, left limbic, right This style represents four limbic, and right cortical different ways of operating, thinking, creating, learning, and, in short, living with the world

Felder and Silverman

Sensitive/intuitive, visual/verbal, inductive/deductive. sequential/global, active/reflective

It conceives learning styles as the preferences that a subject has to receive and process information

Kolb

Active, reflective, pragmatic, and theoretical

This style identifies two main dimensions of learning: perception and processing, where the perceived learning is obtained, and how it processes what it has perceived. Extremes are raised to measure strengths and weaknesses (continued)

208

R. Aguayo et al. Table 1. (continued)

Model

Learning style

Characteristics

Neurolinguistic programming Visual, auditory, and of Bandler and Grinder kinesthetic

It defines three elements as crucial constituents of human behavior: Visual that understands the world as it sees it. Auditory: excellent conversationalist; remember what it hears. Kinesthetic: Process by associating with the body; remember what it does

Gardner’s multiple intelligences

It mentions that intelligence cannot be measured with a number like IQ. Intelligence is the ability to order thoughts and coordinate them with actions

Logical-mathematical, linguistic-verbal, corporal-kinesthetic, spatial, musical, interpersonal, intrapersonal, and naturalistic

Neuro-Linguistic Programming (NLP) is also known as the VAK model by its initials Visual, Auditory, and Kinesthetic. According to Bandler [18], it is the best way to learn from a student, filtering information through a predominant channel, facilitating the assimilation of knowledge; this does not mean that only tasks of the predominant style must be assigned. However, it is also convenient to strengthen the other channels through various activities to train students more receptive to the acquisition of new knowledge in different contexts. In addition, it is based on three components, as mentioned by Díaz-Barriga [19]. First, programming refers to the human ability to produce and use behavioral programs; second, the neurological component is made up of the sensory perceptions that determine the emotional state of each person; third, the linguistic component refers to verbal and non-verbal communication. According to the information mentioned above, in this work, it is proposed to use the NLP model to know the students’ predominant learning styles to help the teacher make a better selection of the methodology used. According to Gallego [20], the results obtained in the group depend on this methodological selection. Above all, it is avoided that they work only through the teacher’s learning channel or the channel that the teacher considers most appropriate. Table 2 shows some examples of activities adapted to each style [21], which help identify which technological tool can be used for each learning style according to the VAK model. In general, visual people understand the world as they see it, and when they remember something, they do so in the form of images. Auditory people focus on listening, are very talkative, and have a remarkable ability to organize ideas. Kinesthetic people can explore the environment and objects through touch and movement; they usually have an easier time learning through direct experience. Once it is understood that each person learns differently from others, teachers must contemplate different strategies considering each

Main Technological Tools in Virtual Educational Environments

209

Table 2. Activities adapted to each learning style. Style

Learning skills

Students prefer

Students learn better

Visual

Ability to read, to write, Think with pictures, read, to plan, to abstract and and write, memorize to memorize information photos, images, and faces, read, and build cognitive maps

Reading, listening, watching, writing, discussing, working with images, and building cognitive maps

Auditory

Ability to understand, assimilate and retain information, speak, and listen

Audio content such as podcast or audiobook, metaphors to remember words, reading and writing aloud, videos to listen to, recorded classes, debate or discuss topics

Listening to audiobooks, creating rhythmic patterns, playing background music, recording themselves reading, listening to music, and hearing new information orally

Kinesthetic

Ability to do physical activities and manipulation of objects, they are usually very participative and very expressive

Prefer to work on projects, Interspersing study study on your feet, and sessions with exercise, move building things, and working with your hands

learning style to increase the students’ cognitive abilities and achieve more significant learning. The learning strategies involve different tasks and different degrees of responsibility of the teachers to achieve the effective transmission of the contents and proactive attitudes of the students. Some authors mention that teaching strategies are related to repetition, preparation, organization, and task self-regulation [22, 23]. According to the author Pimienta-Prieto [24], in his work, he defines a set of strategies and methodologies that favor the teaching-learning process of university students. This classification is related to strategies for investigating previous knowledge, strategies for promoting understanding through the organization of information, group strategies, and active methodologies to contribute to the development of competencies. In this sense, Table 3 describes ten teaching strategies that favor both superficial and deep learning [24], in addition to identifying the most effective strategies for the teaching-learning process. Therefore, it is essential to know the learning strategies for virtual educational environments, the main characteristics, and the learning channels that they strengthen according to the learning styles of the VAK model; these strategies are briefly described below.

210

R. Aguayo et al.

Table 3. According to Pimienta-Prieto [24], these learning strategies improve student performance more effectively. Learning strategies

Description

Interactive presentations

It refers to presenting information using text, images, sound, animations, and videos. As an expository method, it allows students to listen, read and observe. In addition, to facilitate the acquisition of conceptual knowledge and the interaction between the teacher and the students. When interactive presentations are used as a strategy in the teaching-learning process, students tend to be more active and exchange information, experience, and knowledge

Cognitive maps

Cognitive maps are advanced graphic organizers representing a series of ideas, concepts, and themes with meaning and relationships, framing all of them in a scheme or diagram. These maps are helpful in the teaching-learning process to acquire and assimilate any information

Gamified questionnaires

The application of gamified questionnaires focuses on different approaches. First, they can investigate previous knowledge, second, reactivate or evaluate acquired knowledge, and third, as an activity to reinforce some information. Regardless of the focus, the main goal is to increase student engagement to evoke positive experiences and motivations. This strategy helps develop critical and logical thinking; in addition, it analyzes information, delves into a topic, generates ideas, or challenges that can be faced, stimulates new ways of thinking, develops metacognition, and enhance learning

Video editing

An educational video is a didactic medium that aims to reinforce and obtain new knowledge; as it is a motivating strategy, it facilitates the student’s discovery and assimilation because it integrates images and sound that allow attention to be captured. The videos help consolidate knowledge thanks to the audiovisual and auditory stimuli it contains and be important in reducing the digital divide between the world of school and technology

Collaborative virtual walls This strategy allows students to share information and work collaboratively on specific topics to facilitate interaction and group tasks so that students can share objectives and distribute responsibilities. Using this strategy, the teacher stimulates group activities, encourages collaborative work, and promotes other skills such as creativity and the ability to analyze and synthesize (continued)

Main Technological Tools in Virtual Educational Environments

211

Table 3. (continued) Learning strategies

Description

Animated videos

It is an excellent alternative to achieving meaningful learning; it allows students to listen, see, read, and interact, facilitating communication and retention. It attracts attention due to the combination of stimuli by integrating images with movement and sound. Usually, this strategy is used to capture the total attention of the students; through the animated videos, visual and auditory learning is potentialized

Cloud storage

It is a service that allows data to be stored over the internet. The integration of this strategy in the learning process motivates the development of digital and collaborative learning skills. It allows students to upload and share content generated by them with shared images, videos, and documents. This didactic resource is viable because it strengthens knowledge acquisition interactively and allows access to information from anywhere with an internet connection

Timelines

Timelines are graphic representations intended to present a chronological sequence of events, events, or processes, highlighting the most relevant aspects. This strategy favors the assimilation of information in a more meaningful way

Podcast

It is an excellent strategy that promotes reading and writing skills, promotes analysis, and collaborates to generate other forms of communication between teachers and students. Furthermore, it offers new ways of learning educational content, awakens interest, creativity, autonomy, and motivates problem-solving

Debate

It refers to the confrontation of different points of view to reach a solid judgment, promotes communication and research skills, and facilitates analysis, interpretation, and synthesis. Therefore, the debate as a strategy allows students to be more participatory and thus enhance the teaching-learning process

212

R. Aguayo et al.

3 Methodology A quantitative non-experimental cross-sectional study of descriptive scope was carried out [25]. The population of this research was the professors at the Autonomous University of Sinaloa (UAS). The sample was for convenience with professors from three different faculties, the Academic Unit of Mazatlán Gastronomy and Nutrition, the Mazatlán Faculty of Informatics, and the Mazatlán Faculty of Economic-Administrative Sciences. These three faculties were chosen mainly to analyze the didactic resources, tools, and strategies used by teachers from different areas of specialization during virtual classes. The survey technique was used to collect the information and analyze the tools most used by teachers during virtual classes; the instrument used for this work was a digital questionnaire carried out in a Google form. A population of 250 professors from the three UAS faculties was considered, and the resulting sample was 137 professors from different teaching areas at the undergraduate level. According to the instrument applied, the following variables were teaching resources, tools, and strategies related to virtual education. This research was carried out in an organized and systematic manner. First, the corresponding permits were requested from the authorities of the faculties and the professors participating in this research. Second, the information was collected by applying the instrument in a google form. Third, the results obtained were analyzed according to the variables used.

4 Analysis and Presentation of the Results Incorporating technological tools in virtual educational environments is essential since the students acquire the necessary technical skills and favor interaction with teachers during the teaching-learning process. In this work, a study was carried out to analyze the learning strategies most used by teachers during virtual classes. 4.1 Results of the Digital Questionnaire The participants were 137 undergraduate-level professors from different areas. According to the instrument applied, the variables of interest for this study were: the videoconferencing platforms used in virtual classes, the platforms used for learning management, and the main technological tools used in virtual classes. When the COVID-19 pandemic began, it directly affected education at all educational levels and forced institutions to change learning environments. In this sense, the university organized refresher courses for all teachers focused mainly on virtual platforms for videoconferences (Zoom) and learning management (Classroom). According to the results obtained in Fig. 2, it can be observed that 100% of the teachers used platforms for videoconferences, the most used was Zoom with 68% of the teachers, and for learning management, the Classroom platform was the most used with 62% of teachers. Once it has been verified that teachers use the two minimum platforms to interact with students and carry out the teaching-learning process, this research’s objective was to inquire about the teaching strategies used by teachers during virtual classes. According

Main Technological Tools in Virtual Educational Environments

213

to the results of the question main teaching strategies used in virtual classes, in Fig. 3, it can be clearly observed that teachers use very few learning strategies during virtual classes. According to the results, 100% of the teachers responded that during the virtual classes, they use the interactive presentation strategy to impart the thematic content of the subject; in addition, 49.64% confirmed that they use cloud storage tools to share documents for collaborative work with students.

Fig. 2. (a) Videoconferencing platform for virtual classes, (b) Platform used for learning management.

On the other hand, only 14.6% and 13.14% of the teachers indicated that they had used gamified questionnaires and cognitive maps as a learning strategy. These figures are considered very low for all the advantages offered by the elaboration of questionnaires and diagrams. With lower percentages, 3.65% and 5.11% of the teachers indicated that they had used video editing tools, animated videos, and collaborative virtual walls as a strategy during virtual classes; these strategies are highly interactive and suitable for

Fig. 3. Learning strategies used in virtual classes.

214

R. Aguayo et al.

all learning styles are little used. Finally, the teachers have confirmed that the timeline, podcast, and debate strategies have not yet been used. 4.2 Analysis of Technological Tools Technological tools favor creating learning spaces; in these times of pandemics where technology was the only link between the teacher and the student, there was no alternative but to get involved in mastering specific platforms and tools to achieve the objective of transmitting knowledge to students in a virtual environment. The teachers applied some strategies as needed; others were acquired through courses and tool recommendations. However, in addition to learning to manipulate the tools, it is necessary to know the learning style that predominates in the students and the learning strategies that motivate and facilitate the students’ interaction with the content shared by the teacher. Figure 4 presents some technological tools that have been selected and analyzed for the main characteristics, such as sound and video integration, interactive activities, and collaborative participation. It was determined which tools were appropriate for each learning style based on activities related to learning skills, students’ learning preferences, and how students learn best (see Table 2). In addition, each tool has been classified according to the different learning strategies mentioned in Table 3. PowToon Prezi Vyond ProProfs Moovly Educandy

Interactive Presentations

Animated Videos Videoscribe

Genially Animaker EducaPlay Rawshorts Mindomo Drive Popplet Dropbox Coggle

Cognitive Maps One Drive

Cloud Storage

Mindmeister

MediaFire Mindmup Mega Kahoot!

ICloud

Socrative Timeline Quizizz

Gamified Questionnaires

Ahaslides

Technological Tools

Tiki-Toki

Timelines

ISpring

Zzish Timetoast Kinemaster

Visme

Inshot Ivoox Vivavideo

Video Editing Spreaker

Spliceapp

Podcast

Audacity

Quik PodBean Glogster

Talkshoe

Flinga Flipgrid Lino

Collaborative Virtual Walls

Loomio

Padlet

Debate

Slack

Mural Twist Wakelet Blogger

Fig. 4. Classification of technological tools according to the learning strategies mentioned in Table 3.

Main Technological Tools in Virtual Educational Environments

215

Figure 5 shows a classification of learning strategies according to learning styles; this classification has been made based on the activities defined in Table 2 related to learning skills, the student’s learning preferences, and how students learn best. As shown in Fig. 5, it was obtained that some learning strategies can favor the three learning styles. According to the characteristics of some tools, it is possible to integrate sound, video, interactive activities, and collaborative participation. This means that teachers can use the same strategy and tool for a particular activity and favor all students regardless of the predominant learning style.

Learning Strategies

Visual Learning

Auditory Learning

Cognitive maps

Podcast

Cognitive maps

Interactive presentations

Debate

Video editing

Gamified questionnaires

Cognitive maps

Collaborative virtual walls

Video editing

Interactive presentations

Animated videos

Collaborative virtual walls

Video editing

Cloud storage

Cloud storage

Collaborative virtual walls

Time lines

Time lines

Animated videos

Kinesthetic Learning

Cloud storage

Fig. 5. Classification of learning strategies according to learning styles.

5 Discussion and Conclusion Incorporating technological tools into the educational field is of great importance because the tools support both the student and the teacher to acquire the technological skills necessary to face the world of work today. In this sense, to create an effective online learning environment and efficiently transmit knowledge, the teacher must have technical skills and know the appropriate teaching strategies and the technological resources for each learning style (VAK model) to achieve the teaching-learning process successfully. With the development of this research, exciting information was obtained; on the one hand, it was verified that the teachers adapted to the conditions presented by the COVID19 pandemic. According to the results, 100% of the teachers used videoconferencing platforms to teach virtual classes. The Zoom platform was the most used with 68% and Google Meet only 28% of the teachers who participated in this research. In addition,

216

R. Aguayo et al.

it was corroborated that platform for learning management were also used, 62% of the teachers used the Classroom platform, and 38% used Moodle. On the other hand, regarding the use of teaching-learning strategies during virtual classes, it was observed that teachers continue to use the same strategies as in face-to-face classes. 100% of the teachers confirmed that they use the interactive presentation as a strategy to impart the thematic content of the subject. In lower percentages, the strategies that have been used are cloud storage, gamified questionnaires, and cognitive maps. It is known that education requires several elements that allow the consolidation of learning. Regarding virtual educational environments, the use of technology is crucial to guarantee the success of the teaching-learning process; not only is it enough to move the teacher in front of a computer. In this sense, it is essential to mention various technological tools that teachers can use to carry out the teaching-learning process. However, to choose the appropriate tool, it is necessary to know the learning styles of the students, which are directly related to skills and the way they learn, to establish the correct strategy that enhances the teaching-learning process. Therefore, a scheme was designed with different technological tools related to teaching-learning strategies in this research work. In addition, a classification of the strategies was proposed according to each learning style. This research focused on knowing the technological tools used by teachers during virtual classes, and classification of the different technological tools according to teaching strategies and learning styles was proposed. However, it would be interesting to analyze the opinions of students based on technological tools used by teachers to improve the teaching-learning process in virtual environments.

References 1. Seitebaleng, S.D.: Information and communication technology use in higher education: perspectives from faculty. Int. J. Educ. Dev. Using Inf. Commun. Technol. 14(2), 121–166 (2018) 2. Zambrano, L.L.: Use of information and communication technology in virtual education and its correlation with the emotional intelligence of teachers in Ecuador in a COVID-19 context. RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação 40, 31–44 (2020) 3. Lizarraga, C., Quiñonez, Y., Aguayo, R., Peraza, J.: Incorporation of robots in the teachinglearning process for the different educational levels. In: Proceedings of the 9th International Conference on Education and New Learning Technologies, Barcelona, Spain, pp. 5459–5457. Iated Digital Library (2017) 4. Srivastava, T., Waghmare, L.S., Jagzape, A., Rawekar, A., Quazi, N., Mishra, V.P.: Role of information communication technology in higher education: learners perspective in rural medical schools. J. Clin. Diagn. Res. 8(6), XC01–XC06 (2014) 5. Lekka, A., Tsironi, M., y Pange, J.: New trends of information and communication technologies in education. In: IEEE International Conference on Interactive Mobile Communication Technologies and Learning, Thessaloniki, Greece, p. 389 (2015) 6. Hui, T., Yan-Jun, Y.: Knowledge building community in blended learning environments. In Proceedings of the 8th International Conference on Information Technology in Medicine and Education, Fuzhou, pp. 581–583. IEEE Publisher (2016) 7. Lillejord, S., Børte, K., Nesje, K., Ruu, E.: Learning and Teaching with Technology in Higher Education – A Systematic Review. Knowledge Centre for Education, Oslo (2018)

Main Technological Tools in Virtual Educational Environments

217

8. Attallah, B.: Post COVID-19 higher education empowered by virtual worlds and applications. In: Proceedings of the 7th International Conference on Information Technology Trends (ITT), Abu Dhabi, United Arab Emirates, pp. 161–164. IEEE Publisher (2020) 9. Kumar, A., Malhotra, S., Katoch, A., Sarathkar, A., Manocha, A.: Webinars: an assistive tool used by higher education educators during Covid19 case study. In: Proceedings of the 12th International Conference on Computational Intelligence and Communication Networks (CICN), Bhimtal, India, pp. 1–6. IEEE Publisher (2020) 10. Zoom. https://zoom.us/meetings 11. Goto Webinar. https://www.gotomeeting.com/en-in 12. Google Meet. https://meet.google.com/ 13. Cisco Webex Meeting. https://www.webex.com/ 14. Microsoft Team. https://www.microsoft.com/es-mx/microsoft-teams/online-meetings 15. Pantoja, M.A., Duque, L.I., Correa, J.S.: Learning styles models: an upgrade for their revision and analysis. Revista Colombiana de Educación 64, 79–105 (2013) 16. Bustos-López, M., Alor-Hernández, G., Sánchez-Cervantes, J.L., Salas-Zárate, M.P., ParedesValverde, M.A.: EduRP: an educational resources platform based on opinion mining and semantic web. J. Univ. Comput. Sci. 24(11), 1515–1535 (2018) 17. Gómez, L., Aduna, A., García, E.: Manual de estilos de aprendizaje. Secretaría de Educación Pública, México, DF (2004) 18. Bandler, R.: Use su cabeza para variar, submodalidades en PNL. Cuatro Vientos (1988) 19. Díaz-Barriga, F.: Experiencias de aprendizaje mediadas por las tecnologías digitales: pautas para docentes y diseñadores educativos. Universidad Nacional Autónoma de México (2015) 20. Gallego, D.: Diagnosed learning styles of my students and now what do I do? Revista de Estilos de aprendizaje 11(12), 1–13 (2013) 21. De la Parra Paz, E.: Herencia de vida para tus hijos: crecimiento integral con técnicas PNL. Grijalbo (2004) 22. Hattie, J., Donoghue, G.: Learning strategies: a synthesis and conceptual model. npj Sci. Learn. 1, 16013 (2016) 23. García-Pérez, D., Fraile, J., Panadero, E.: Learning strategies and self-regulation in context: how higher education students approach different courses, assessments, and challenges. Eur. J. Psychol. Educ. 36(2), 533–550 (2020). https://doi.org/10.1007/s10212-020-00488-z 24. Pimienta-Prieto, J.H.: Estrategias de enseñanza-aprendizaje, docencia universitaria basada en competencias. Pearson Educación, Mexico (2012) 25. Creswell, J.W., Creswell, J.D.: Research Design. Qualitative, Quantitative, and Mixed Method Approaches, 5th edn. Sage, Thousand Oaks, California (2017)

User Research Techniques for User Interface Design of Learning Management Systems: A Decade Review Luis Raúl Méndez-Becerra1(B) , Viviana Yarel Rosales-Morales2 , Giner Alor-Hernández3 , and Carmen Mezura-Godoy1 1 Facultad de Estadística e Informática, Universidad Veracruzana, Xalapa, México

[email protected]

2 Facultad de Estadística e Informática, Cátedras CONACyT - Universidad Veracruzana,

Xalapa, México [email protected] 3 Tecnológico Nacional de México/I.T. Orizaba, Orizaba, México [email protected]

Abstract. From several years ago, learning management systems (LMS) have been widely adopted in higher education institutions around the world. There are some cases in which universities have decided to develop their own LMS, so the system’s user interface design turns out to be a key factor. It is desirable for the LMS to have a user friendly interface design that promotes easy, natural, and engaging interaction, allowing users to perform their required tasks. Through user research, which focuses on understanding user’s behaviors, needs, and motivations, it is possible to improve a user interface by employing techniques such as surveys, usability tests, user observations, interviews, among others. The review’s goal is to collect and classify research papers found in the literature focused in using user research techniques to improve user interfaces of learning systems in order to identify the most suitable ones, as well as to discover the ones that require a more extensive research. Keywords: Learning management systems · User interface design · User research techniques · Systematic literature review

1 Introduction In recent years, systems that facilitate teaching and learning in schools, such as learning management systems (LMS), were developed. LMS have been widely adopted in higher education institutions around the world [1–5], so there is a high demand in developing these systems. Nowadays there are commercial, free and open-source solutions of LMS, however there are some cases in which universities have decided to develop their own systems [6–11], therefore, in the development of these systems there are design aspects that are relevant, such as the user interface. Being as the perceived ease of use of a learning system dramatically impacts the perception of utility, and affect students’ acceptance and satisfaction [12–16], LMS © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 218–232, 2022. https://doi.org/10.1007/978-3-030-89909-7_17

User Research Techniques for User Interface Design

219

must have a user interface usable and attractive so that users are interested in using it and to be focused on performing tasks instead of learning on how to use the system. Hence, when designing a user interface, it is necessary to consider the user, and this is achieved through user research. Since user research focuses on understanding user behaviors, needs, and motivations, by applying user research techniques during the design phase of learning systems, it is possible to identify common errors in the user interface design, find opportunities for improvement, as well as to choose the design solutions that best suit the user. In this review, research papers found in the literature that applied user research techniques to improve user interfaces of LMS were collected and classified to identify the most suitable ones, as well as to discover the ones that require a more extensive research. This document was structured as follows: Sect. 2 gives a brief context on the related research topics, Sect. 3 describes the review process, Sect. 4 discusses the results, and Sect. 5 presents the conclusion.

2 Theoretical Background In this section, a context is provided concerning to the main topics covered herein as follows: 1) user research techniques, 2) user interface design and 3) learning management system. 2.1 User Research Techniques The user research is the process of finding out how people interpret and use products and services [17], it focuses on understanding the behaviors, needs, and motivations of the user through certain techniques. Some user research techniques, such as surveys, interviews, or focus groups, are best suited for gathering information about users’ general attitudes and perceptions; as for some others, such as usability testing and user observations, are more appropriate to understand specific aspects of user behavior and interaction with the system [18]. Table 1 lists the user research techniques and their description. Although there are more techniques, only those that were applied in the papers included in the review are presented. Table 1. User research techniques and their descriptions. User research techniques Description Persona

Fictional character constructed to represent the needs of a whole range of real users [18]

Survey

Set of questions that allows a large group of people to describe themselves, their interests, and their preferences in a structured way [17] (continued)

220

L. R. Méndez-Becerra et al. Table 1. (continued)

User research techniques Description Usability testing

Measured observation of user’s behavior as they engage in the use of a software application [19]

Task analysis

It is carry out to closely examining the precise steps users go through in accomplishing their tasks [18]

Questionnaire

Form designed to get quantitative data from participants [20]

Interview

It is conducted to get user perspectives and draw broader conclusions about a system and its use [21]

User observation

Involves to watch and listen carefully to users as they work with a product [22]

Focus group

Structured, attentively moderated group discussions that reveal a target audience’s conscious preferences, recalled experiences and stated priorities [17]

Eye tracking

An individual’s eye movements are measured so that the researcher knows both where a person is looking at any given time and the sequence in which their eyes are shifting from one location to another [23]

2.2 User Interface Design The user interface is the point of interaction between the user and the system, is the visual part of an application, so it is a fundamental part of any software product. User interface design is a subset of the Human-Computer Interaction (HCI) field [24], and it refers to choose the right interface elements for the task the user is trying to accomplish and arranging them on the screen in way that will be readily understood and easily used [18]. A good user interface design encourages easy, natural, and attractive interaction, allowing users to perform their required tasks [25]. 2.3 Learning Management System LMS is an e-learning system that manages instructional content, collects and presents data to monitor the learning process of an organization [26], is a web-based system that shows courses and content for students and teachers. The contexts in which LMSs are implemented include institutions of higher education, elementary and secondary school systems, private companies, and the Army [27]. LMSs are known in the literature by different names such as course management systems, virtual learning environments, e-learning platforms [28], among others. Another learning system with similar characteristics to LMS is the MOOC (Massive Open Online Course) platform, which revolutionized universities and the corporate educational landscape, attracting considerable public attention due to their sudden increase and disruptive potential [29, 30].

User Research Techniques for User Interface Design

221

3 Research Method This systematic review of the literature was based on Kitchenham procedures [31] and was carried out through the following phases: 1) Search Planning, 2) Search Report and 3) Analysis of Results. 3.1 Search Planning According to the protocol of this review, research questions were proposed, search strategies were defined, and inclusion and exclusion criteria, as well as a quality assessment, were established. Search Planning. To accomplish this review, the following research questions were proposed: 1. How many papers related to user research techniques for user interface design of learning management systems were proposed from 2011 to 2021? 2. What user research techniques were applied to improve the user interface design of learning management systems? 3. What standardized questionnaires were applied to evaluate the user interface design of learning management systems? The first question was proposed to obtain quantitative information from papers related to user research techniques applied to improve user interfaces of learning management systems in the last ten years. The second question was to know the user research techniques that were applied in existing papers in the literature to improve user interfaces of learning management systems. Lastly, the third question was specifically to find out about standardized questionnaires that were applied to evaluate user interfaces of learning management systems to improving them. In addition, other web-based learning systems that share similar characteristics to LMS, such as e-learning systems, and MOOC platforms, were considered for a greater scope. Search Strategy. To perform the search in digital databases, a set of search strings were constructed from the research field keywords. As can be seen in Table 2, the words were “user research”, “user interface design” and “learning management system.” Besides, synonyms, similar terms, and acronyms were used. Thus, an example of the search string was ‘(“user research” OR “user experience research”) AND (“user interface” OR “UI”) AND (“learning management system” OR “LMS” OR “educational platform” OR “e-learning” OR “virtual learning environment”).’ However, these search strings had some variations that depended on the digital library database systems. ACM Digital Library, Emerald Insight, IEEE Xplore, ScienceDirect, SpringerLink, and Willey Online Library were the digital library databases considered.

222

L. R. Méndez-Becerra et al. Table 2. Keywords with similar terms.

Keywords

Similar terms

User research

User experience research

User interface design

Interface design, UI

Learning management system

Learning system, educational platform, e-learning, virtual learning environment, LMS

Inclusion and Exclusion Criteria. For the selection of papers, the following inclusion criteria were defined: • • • • •

Papers published from 2011 to June 2021 Papers published in international conferences, journals, and book chapters Papers in English Papers focused on improving user interfaces of web-based learning systems Papers where user research techniques were applied.

Papers from ten years ago were included. Papers published in international conferences, journals, and book chapters were included since they were subjected to rigorous peer review. For feasibility reasons, only papers written in English were included. Finally, only papers that applied user research techniques to improve user interfaces of web-based learning systems were included. Furthermore, papers that did not match the inclusion criteria were excluded. Quality Assessment. To determine the quality of the included papers, the following criteria had to be met: • Paper was cited by other authors • Paper had specified goal, problem, and contribution • Paper findings had substantiated with objective criteria. Citations of other authors were assessed to determine the relevance of the paper. It was examined that the paper had specified its goal, problem and solution. Finally, the results presented in the papers had to be objectively validated. Once the search was executed, the title, keywords, and abstract of the papers obtained by the search strings were read to check that they met the inclusion and exclusion criteria. Subsequently, it was verified that the papers met the quality assessment criteria. 3.2 Search Report The search was conducted in June 2021. A total of 445 papers were found, where 20 papers were included according to the inclusion and exclusion criteria, discarding 425 papers. Then, these 20 papers were reviewed, excluding seven papers that did not meet the quality assessment criteria. In the end, 432 papers were excluded, obtaining as result

User Research Techniques for User Interface Design

223

13 papers. A flow chart representing the search process is shown in Fig. 1. Further, Emerald Insight and Wiley Online Library databases were discarded due no papers were found that matched the established criteria. (“user research” OR “user experience research”) AND (“user interface design” OR “UI”) AND ((“learning management system” OR “LMS”) OR “educaonal plaorm” OR “e-learning” OR “virtual learning environment”)

ACM Digital Library Emerald Insight IEEE Xplore ScienceDirect SpringerLink Willey Online Library

Begin

Digital Library Databases

No

Perform Search 445 found

Inclusion and Exclusion Criteria

425 excluded

Exclusion

Yes 20 included

7 excluded

No

From 2011 to 2021 Internaonal conferences and journals English User research techniques related Focused on improving user interface design of learning systems

Quality Assessment

Cited by other authors Results substanated with objecve criteria

Yes 13 results

Results

End

Fig. 1. Search process flow chart.

3.3 Analysis of Results In the analysis of results, both quantitative and qualitative approaches, the research questions proposed were answered. Papers Included from 2011 to 2021. As presented in Table 3, 13 papers were considered, according to the inclusion and exclusion criteria, as well as the quality assessment criteria, among them, seven papers were focused on LMS, four papers focused on systems and platforms of e-learning, and two papers were aimed at MOOC platforms. Also,

224

L. R. Méndez-Becerra et al.

nine papers carried out evaluations of user interfaces of learning systems to improve them, three papers came up with design proposals of learning systems, and one paper conducted an exploration of the user interface tasks complexity of an LMS. Besides, ten papers are from conference proceedings, two papers are from indexed journals, and one paper is from a book chapter. Lastly, papers [32] and [41] are the most cited according to Google Scholar, whereas [42] and [44] are the least cited. Additionally, case studies are described in all papers. Table 3. List of included papers. Papers

System type

Contribution type

Publication

Citations

[32]

LMS

Evaluation

Journal

40

[33]

E-learning platform

Design

Conference

10

[34]

LMS

Evaluation

Conference

11

[35]

Assessment system

Evaluation

Conference

3

[36]

LMS

Evaluation

Conference

5

[37]

LMS

Exploration

Conference

5

[38]

LMS

Evaluation

Conference

8

[39]

E-learning platform

Evaluation

Conference

3

[40]

E-learning system

Design

Conference

2

[41]

MOOC platform

Evaluation

Conference

48

[42]

LMS

Design

Conference

1

[43]

LMS

Evaluation

Book chapter

2

[44]

MOOC platform

Evaluation

Journal

1

Moreover, Fig. 2 shows papers classified by year and digital database. In ACM Digital Library three papers were found; in IEEE Xplore eight papers were found; in ScienceDirect one paper was found; lastly, in SpringerLink one paper was found. From another perspective, 23% of papers were found in ACM Digital Library, 61% IEEE Xplore, 8% in ScienceDirect, and 8% in SpringerLink. In 2012, 2015, and 2016 the publication of papers was the lowest with one paper per year, while in 2017 and 2020 the highest publication number was reached with three papers per year. User Research Techniques. In the 13 papers, user research techniques were applied, through which it was possible to understand the behaviors, needs and motivations of the user to improve the user interfaces of learning management systems. Personas: In [33], Personas were built to create user roles, for instance, student and teacher roles, in order to identify their needs; besides, in [40], was mentioned that Personas were used to reflect the user’s characteristics and requirements based on learning styles.

User Research Techniques for User Interface Design ACM Digital Library

IEEE Xplore

ScienceDirect

SpringerLink

225

1 3 1

1

1

2012

2015

2016

2017

1

1

1

1

2018

2019

2 2020

Fig. 2. Papers classified by year and digital library database.

Survey: In [40] and [41], surveys were applied to obtain demographic information from users; in [41] surveys were applied to collect antecedents on the use of learning systems; in [33, 35] and [43], surveys were applied to find out the needs and perceptions about the usage of learning systems; finally, in [40] and [41], surveys were applied to conduct evaluations of learning systems. Usability testing: In [32] and [33], usability testing was performed in learning systems to identify design issues; in [35–37] and [44], usability testing was performed to ensure that their users could complete the required tasks; in [35], usability testing was applied to determine the user experiences of users interacting with the system, providing the causes of usability problems in the system and suggesting possible solutions; in [44], the level of success and mistakes made by each user was measured when performing the tasks and problems they faced; in [41] and [43], new learning system designs were tested by applying usability testing; finally, in [36, 39] and [40], usability testing was carried out to evaluate the ease of usage of learning systems. Although some papers did not perform usability testing in the learning systems, other methods were proposed, for instance, in [41], the learning platform itself was not considered, but images of the system were presented to their users; moreover, in [34, 38, 41] and [43], only the previous experience of users who had used the learning systems was considered. Task analysis: In [32, 33, 36, 37] and [44], task analysis was conducted based on user observations to create scenario tasks, which were used in usability testing to measure the level of success, time, and errors that the user made when trying to complete them. Further, in [33], task analysis was based on Personas. Meanwhile, in [37], levels of complexity in the tasks were examined. Questionnaire: In [34–42] and [44], questionnaires were applied to measure quality attributes of the interaction between the user and the learning system, such as usability and user experience. Standardized questionnaires were applied in most of the papers, but in [38], a personalized one was applied instead. Interview: In [32, 33] and [44], users were interviewed after completing tasks to identify their experiences and behaviors using a learning system. Besides, in [44], interviews were applied to identify typical tasks and user experience indicators.

226

L. R. Méndez-Becerra et al.

User observation: In [36] and [43], observations of facial expression, screen navigation, and user speech interacting with the user interface were carried out. Some observations were direct, whereas others were recorded. Focus group: In [43], a focus group was carried out for users to discuss their feelings, attitudes, and thoughts about the learning system, thereby revealing their motivations and preferences. Eye tracking: In [32], eye tracking was applied to collect data on students’ eye movements to measure their performance towards an LMS to identify design issues. For the aforementioned, Table 4 lists the included papers where user research techniques were applied. Usability testing and questionnaires were applied in nine of the papers; surveys and task analysis were applied in five papers; interviews were carried out in three papers; Personas were built in two papers; user observations were done in two papers; lastly, focus group and eye tracking were applied in one paper. Usability testing and questionnaires were the most applied techniques, while the least applied techniques were focus group and eye tracking. Table 4. User research techniques applied in papers. User research techniques

Included papers

Quantity

Personas

[33, 40]

2

Surveys

[33, 35, 40, 41, 43]

5

Usability testing

[32, 33, 35–37, 39, 40, 42, 44]

9

Task analysis

[32, 33, 36, 37, 44]

5

Questionnaires

[34–42, 44]

9

Interviews

[32, 33, 44]

3

User observations

[36, 43]

2

Focus group

[43]

1

Eye tracking

[32]

1

Standardized Questionnaires. Among the 13 included papers, standardized questionnaires were applied to evaluate the user interfaces of learning systems to identify common errors in user interface design and to find opportunities for improvement. A standardized questionnaire is a questionnaire designed for repeated use, generally with a specific set of questions presented in a specific order, using a specific format, with specific rules to produce metrics based on the responses of the respondents [20]. In [37, 39] and [40], the SUS was applied to measure the overall perceived usability, whereas in [36] and [41], SUS was applied to quantify the results of satisfaction with the user interface design of a learning system. In [35], the PSSUQ was adapted to develop a web evaluation usability test of a learning system. In [44], the CSUQ was applied to measure the usability of the overall user experience of a MOOC platform; also, the ASQ was applied to test the task-based user experience of a MOOC platform. In [39] and [42],

User Research Techniques for User Interface Design

227

the UEQ was applied to measure the overall user experience based on its dimensions; In [34], the UEQ was applied to obtain a comprehensive perception of the user and a relative quality level of an LMS; lastly, in [37], the UEQ was applied to measure user experience in an LMS environment with low levels of task complexity. In [41], the UMUX-Lite was applied to measure satisfaction with the interface design of a MOOC platform. Finally, in [44], the VisAWI was applied to measure the overall user experience of the visual design of a MOOC platform. Table 5 presents a list of standardized questionnaires that were applied in the included papers to evaluate the user interfaces of learning systems. The SUS questionnaire was applied in five papers, being the most used questionnaire, followed by the UEQ, applied in four of the included papers. The other questionnaires were only applied in one paper. Table 5. Standardized questionnaires in papers. Standardized questionnaires

Authors

Included papers

Quantity

SUS

[45]

[36, 37, 39–41]

5

PSSUQ

[46]

[35]

1

CSUQ

[47]

[44]

1

ASQ

[48]

[44]

1

UEQ

[49]

[34, 37, 39, 42]

4

UMUX-Lite

[50]

[41]

1

VisAWI

[51]

[44]

1

4 Discussion By applying user research techniques such as surveys, usability tests, questionnaires, interviews, among others, it is possible to identify common errors in the process of user interface design in order to find opportunities for improvement, as well as to select the design solutions that best suit the user. From the 13 papers presented in this review, user research techniques were applied to improve user interfaces of learning systems. Although usability testing was one of the most applied techniques since the interaction between the interface and the user is carried out, in some papers it was not applied, for instance, previous experience of users using learning systems was just considered, while in other papers only images of the user interface were shown to users. Another of the most used techniques were questionnaires, which were applied with standardized questionnaires to measure quality attributes such as usability and user experience, however, there were some papers that applied personalized questionnaires instead. Task analysis, applied to create scenario tasks, which were applied in usability testing, was another of the most applied techniques; some papers based their task analysis on Personas, while others examined levels of complexity in the tasks. Another technique that was mostly applied in the papers was

228

L. R. Méndez-Becerra et al.

the survey, which made it possible to obtain demographic information and comments on user experiences by using learning systems. Moreover, the least applied techniques, but no less important ones were interviews, which allowed to receive feedback from users; Personas, which were built to create user roles, such as student and teacher, to identify their needs; user observations, which allowed to analyze the user interacting with the interface, either direct or recorded; focus group, in which users discussed their feelings, attitudes, and thoughts about the learning system; and lastly, eye-tracking, which made it possible to collect data on the students’ eye movements in order to measure their performance towards an LMS. As for the application of standardized questionnaires, SUS was the most applied to determine the usability of a learning system, as well as to obtain quantitative metrics of satisfaction with the user interface design of a learning system. UEQ was another of the most applied questionnaires since it allowed to evaluate the user experience and observe how users were satisfied when using a learning system. Furthermore, the least applied questionnaires were PSSUQ and CSUQ, which were applied to measure usability; ASQ applied to measure the satisfaction of each task; UMUX-Lite, which was applied to measure the satisfaction with the interface design; lastly, VisAWI was applied to measure aesthetics. Some papers applied more than one standardized questionnaire and others applied none.

5 Conclusions In this review, papers related to user research techniques for user interface design of LMS were presented. By following a search strategy, 445 papers were found, of which 13 met the established criteria. From the thirteen papers, a quantitative and qualitative analysis was performed to answer the four proposed research questions. Nevertheless, it was estimated that this review could have the following limitations: • • • •

The choice of databases The effectiveness of the search string The lack of papers related to the research topic The unavailability of the papers.

Although digital databases containing papers that were subjected to rigorous review were chosen, there may be other databases to consider. As each digital database system works differently, the search string had some variations, so its effectiveness could be affected. Since there were few papers related to the user interface design of LMS, other web-based learning systems that share similar characteristics were considered. Lastly, there were papers that were inaccessible. However, despite the limitations, it is considered that this research work was successfully completed as planned. After the discussion, it was concluded that the user interface design of LMS can be improved by employing a combination of various user research techniques such as surveys, usability testing, questionnaires, task analysis, user observation, and interviews; applying standardized questionnaires such as SUS and UEQ; and lastly, following a proper procedure. In addition, it was discovered that eye-tracking and focus group are techniques that require a more extensive research in this context.

User Research Techniques for User Interface Design

229

In future work, we are interested in creating a guide based on user research techniques, design principles and iterative testing to support developers, even if they are not experts in the field of HCI, in designing the user interface of LMS. This guide will produce a user interface that will be usable, in terms of satisfaction, for the end-user. Acknowledgements. This work was partially developed under the support of the National Council of Science and Technology (CONACYT) in the scope of the Catedras CONACYT project “Infraestructura para Agilizar el Desarrollo de Sistemas Centrados en el Usuario”, Ref. 3053. In addition, we thank CONACYT for the scholarship number 745392, as well as at Universidad Veracruzana for the support in the development of this research.

References 1. Aldiab, A., Chowdhury, H., Kootsookos, A., Alam, F., Allhibi, H.: Utilization of learning management systems (LMSs) in higher education system: a case review for Saudi Arabia. Energy Procedia 160, 731–737 (2019) 2. Al-Sharhan, S., Al-Hunaiyyan, A., Alhajri, R., Al-Huwail, N.: Utilization of learning management system (LMS) among instructors and students. In: Zakaria, Z., Ahmad, R. (eds.) Advances in Electronics Engineering. LNEE, vol. 619, pp. 15–23. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-1289-6_2 3. Liu, J.C., Brantmeier, N., Wilcox, D., Griffin, O., Calcagno-Roach, J., Brannon, R.: Faculty perceived functionality of learning management system: development and validation of a scale. In: Ma, W.W.K., Chan, W.W.L., Cheng, C.M. (eds.) Shaping the Future of Education, Communication and Technology. ECTY, pp. 165–177. Springer, Singapore (2019). https:// doi.org/10.1007/978-981-13-6681-9_13 4. Rhode, J., Richter, S., Gowen, P., Miller, T., Wills, C.: Understanding faculty use of the learning management system. Online Learn. 21(3), 68–86 (2017). https://doi.org/10.24059/ olj.v21i3.1217 5. Alberto Aning, A.B.: E-learning design in Malaysian higher educational institutions: principles and guidelines. J. Hunan Univ. Nat. Sci. 47, 123–130 (2020) 6. Por, F.P., Mustafa, Z., Osman, S., Phoon, H.S., Fong, S.F.: Design and development of multimedia pronunciation learning management system for non-native English speakers. Procedia Soc. Behav. Sci. 64, 584–593 (2012) 7. Almarashdeh, I., Elias, N.F., Sahari, N., Zain, N.A.M.: Development of an interactive learning management system for Malaysian distance learning institutions. Middle East J. Sci. Res. 14, 10–5829 (2013) 8. Kotsifakos, D., Adamopoulos, P., Douligeris, C.: Design and development of a learning management system for vocational education. In: Proceedings of the SouthEast European Design Automation, Computer Engineering, Computer Networks and Social Media Conference on SEEDA-CECNSM 2016. ACM Press, New York (2016) 9. Johnson, M.C., Seawright, L.L., McDonald, J.K.: A design case of an enterprise-wide learning management system. In: Bishop, M.J., Boling, E., Elen, J., Svihla, V. (eds.) Handbook of Research in Educational Communications and Technology, pp. 675–688. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-36119-8_31 10. Muhardi, M., Gunawan, S.I., Irawan, Y., Devis, Y.: Design of web based LMS (learning management system) in SMAN 1 Kampar Kiri Hilir. J. Appl. Eng. Technol. Sci. (JAETS) 1, 70–76 (2020)

230

L. R. Méndez-Becerra et al.

11. Rabiman, R., Nurtanto, M., Kholifah, N.: Design and development e-learning system by learning management system (LMS) in vocational education. Online Submission 9, 1059– 1063 (2020) 12. Ghazal, S., Aldowah, H., Umar, I.: Critical factors to learning management system acceptance and satisfaction in a blended learning environment. In: Saeed, F., Gazem, N., Patnaik, S., Balaid, A.S.S., Mohammed, F. (eds.) IRICT 2017. LNDECT, vol. 5, pp. 688–698. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59427-9_71 13. Ohliati, J., Abbas, B.S.: Measuring students satisfaction in using learning management system. Int. J. Emerg. Technol. Learn. 14, 180 (2019) 14. Ghapanchi, A.H., Purarjomandlangrudi, A., McAndrew, A., Miao, Y.: Investigating the impact of space design, visual attractiveness and perceived instructor presence on student adoption of learning management systems. Educ. Inf. Technol. 25(6), 5053–5066 (2020). https://doi. org/10.1007/s10639-020-10204-5 15. Amsal, A.A.: Meuthia: Students’ online learning satisfaction on learning management system. In: Proceedings of the 3rd International Conference on Educational Development and Quality Assurance, ICED-QA 2020, Paris, France. Atlantis Press (2021) 16. Nguyen, N.-T.: A study on satisfaction of users towards learning management system at International University – Vietnam National University HCMC. Asia Pac. Manag. Rev. (2021) 17. Goodman, E., Kuniavsky, M., Moed, A.: Observing the User Experience: A Practitioner’s Guide to User Research. Morgan Kaufmann, Oxford, England (2012) 18. Jesse, J.G.: The Elements of User Experience: User-Centered Design for the Web and Beyond. New Riders Publishing (2011) 19. Lowdermilk, T.: User-Centered Design: A Developer’s Guide to Building User-Friendly Applications. O’Reilly Media, Sebastopol, CA (2013) 20. Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann, Oxford, England (2012) 21. Raita, E.: User interviews revisited: identifying user positions and system interpretations. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction Making Sense Through Design - NordiCHI 2012. ACM Press, New York (2012) 22. Gomoll, K., Nicol, A.: Discussion of guidelines for user observation. User Observation: Guidelines for Apple Developers (1990) 23. Poole, A., Ball, L.J.: Eye tracking in HCI and usability research. In: Ghaoui, C. (eds.) Encyclopedia of Human Computer Interaction, pp. 211–219. IGI Global (2006) 24. Galitz, W.O.: The Essential Guide to User Interface Design: An Introduction to GUI Design, Principles and Techniques. Wiley (2007) 25. Stone, D., Jarrett, C., Woodroffe, M., Minocha, S.: User Interface Design and Evaluation. Morgan Kaufmann, Oxford, England (2005) 26. Szabo, M.: CMI theory and practice: historical roots of learning management systems. In: E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, pp. 929–936.Association for the Advancement of Computing in Education, Montreal, Canada (2002) 27. Stone, D.E., Zheng, G.: Learning management systems in a changing environment. In: Handbook of Research on Education and Technology in a Changing Society, pp. 756–767. IGI Global (2014) 28. Kats, Y., (ed.): Learning Management System Technologies and Software Solutions for Online Teaching: Tools and Applications. Information Science Reference, Hershey, PA (2010) 29. Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., Emanuel, E.J.: The MOOC phenomenon: Who takes massive open online courses and why? SSRN Electron. J. (2013). https://doi.org/10.2139/ssrn.2350964 30. Kaplan, A.M., Haenlein, M.: Higher education and the digital revolution: about MOOCs, SPOCs, social media, and the Cookie Monster. Bus. Horiz. 59, 441–450 (2016)

User Research Techniques for User Interface Design

231

31. Kitchenham, B.: Procedures for performing systematic reviews, vol. 33, pp. 1–26. Keele University, Keele, UK (2004) 32. Ramakrisnan, P., Jaafar, A., Razak, F.H.A., Ramba, D.A.: Evaluation of user interface design for leaning management system (LMS): investigating student’s eye tracking pattern and experiences. Procedia Soc. Behav. Sci. 67, 527–537 (2012) 33. Ostrowski, S., Rolczy´nski, R., Pniewska, J., Garnik, I.: User-friendly e-learning platform: a case study of a design thinking approach use. In: Proceedings of the Mulitimedia, Interaction, Design and Innnovation on ZZZ, MIDI 2015. ACM Press, New York (2015) 34. Sahid, D.S.S., Santosa, P.I., Ferdiana, R., Lukito, E.N.: Evaluation and measurement of learning management system based on user experience. In: 2016 6th International Annual Engineering Seminar (InAES). IEEE (2016) 35. Lai, T.-L., Chen, P.-Y., Chou, C.-Y.: A user experience study of a web-based formative assessment system. In: 2017 International Conference on Applied System Innovation (ICASI). IEEE (2017) 36. Srimarong, S., Achalakul, T.: Usability evaluation of outcome-based education tool. In: 2017 IEEE 9th International Conference on Engineering Education (ICEED). IEEE (2017) 37. Yulianandra, P.V., Wibirama, S., Santosa, P.I.: Examining the effect of website complexity and task complexity in web-based learning management system. In: 2017 1st International Conference on Informatics and Computational Sciences (ICICoS). IEEE (2017) 38. Hasan, L.: Usability problems on desktop and mobile interfaces of the Moodle learning management system (LMS). In: Proceedings of the 2018 International Conference on EBusiness and Applications, ICEBA 2018. ACM Press, New York (2018) 39. Wang, T.Y., Wang, C.H.: E-learning platform of STEAM aesthetic course materials based on user experience. In: 2018 1st International Cognitive Cities Conference (IC3). IEEE (2018) 40. Hasani, L.M., Santoso, H.B., Isal, R.Y.K.: Designing alternative interface design of e-learning modules based on Felder-Silverman learning styles and user centered design approach. In: 2019 International Conference on Advanced Computer Science and information Systems (ICACSIS). IEEE (2019) 41. Korableva, O., Durand, T., Kalimullina, O., Stepanova, I.: Studying user satisfaction with the MOOC platform interfaces using the example of coursera and open education platforms. In: Proceedings of the 2019 International Conference on Big Data and Education – ICBDE 2019. ACM Press, New York (2019) 42. Kirana, K.C., Wibawanto, S., Cahyono, G.P.: Design of teleconference-based learning management system for a learning tool in the co-19 pandemic. In: 2020 4th International Conference on Vocational Education and Training (ICOVET). IEEE (2020) 43. Andone, D., Vert, S., Mihaescu, V., Stoica, D., Ternauciuc, A.: Evaluation of the virtual mobility learning hub. In: Zaphiris, P., Ioannou, A. (eds.) HCII 2020. LNCS, vol. 12205, pp. 20–33. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50513-4_2 44. Liu, S., Liang, T., Shao, S., Kong, J.: Evaluating localized MOOCs: the role of culture on interface design and user experience. IEEE Access 8, 107927–107940 (2020) 45. Brooke, J.: SUS: a ‘quick and dirty’ usability scale. In: Usability Evaluation in Industry, vol. 189 (1996) 46. Lewis, J.R.: Psychometric evaluation of the post-study system usability questionnaire: the PSSUQ. Proc. Hum. Factors Soc. Annu. Meet. 36, 1259–1260 (1992) 47. Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7, 57–78 (1995) 48. Lewis, J.R.: Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. SIGCHI Bull. 23, 78–81 (1991) 49. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 63–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89350-9_6

232

L. R. Méndez-Becerra et al.

50. Lewis, J.R., Utesch, B.S., Maher, D.E.: UMUX-LITE: when there’s no time for the SUS. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013. ACM Press, New York (2013) 51. Moshagen, M., Thielsch, M.T.: Facets of visual aesthetics. Int. J. Hum. Comput. Stud. 68, 689–709 (2010)

New Quantum Image Manipulation System (NQIMS) Rasha F. Aly(B) Department of Computers and Systems Engineering, Faculty of Engineering, Helwan University, Cairo, Egypt

Abstract. Quantum Image Processing (QIP) is focused on extending conventional image processing tasks and operations to quantum computing framework. It is utilizing quantum computing technologies to capture, manipulate, and recover quantum images in different formats and purposes. This paper proposes new technique using quantum principles and Bloch’s sphere applied on the color image directly using HSV color system. This new mapping technique between HSV color system and Bloch’s sphere uses for object detection. Also, this paper shows how to use usefully this new technique solving different image processing applications and important problems such as one of the recent challenges for finding 3D image from single 2D color image. The main challenge of this proposes technique that the image is not 3-D image and it avoiding the quantum measuring mathematical complexity. Keywords: Quantum Image Processing (QIP) · Bloch’s sphere · Classical image processing · Color systems · Quantum computing · 3D image construction · Depth algorithm · Quantum computing

1 Introduction Just as classical bit has state - either 0 or 1. A qubit is represents using super position principle using Eq. (1) and the bit state depends on probability natural. In general, |α|2 + |β|2 = 1 [1, 2]. Qubits could be represented also by points on Bloch’s sphere [3]. Figure 1 [4] using the spherical coordinate system. This representation based on the fact that any qubit could be represented as shown in Eq. (2). |ψ > = α|0 > + β|1 >

(1)

Where, numbers α and β are complex numbers θ θ φ |ψ > = cos |0 > + ei sin |1 > 2 2 − → The Bloch vector for state (2) is r = (x, y, z), where: x = sin θ cos φ, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. Mejia et al. (Eds.): CIMPS 2021, AISC 1416, pp. 233–244, 2022. https://doi.org/10.1007/978-3-030-89909-7_18

(2)

234

R. F. Aly

y = sin θ sin φ, z = cos θ.      1 1 0 √ 1 North Pole = , South Pole = for z- axis, Y-axis ranges from 2 i to 0 1          1 √ 1 . 1 √ 1 , and X-axis ranges from 1 √ 1 2 −i 2 −1 to 2 i

Fig. 1. Qubit representation using Bloch’s sphere [4].

or the quantum image processing, a good deal of operations is done by relying on the corresponding applications on classical image processing as reference. The flexible representation for quantum images, FRQI, which is similar to the pixel representation for images on conventional computers, captures the essential information about the colors as well as the corresponding positions of every point in an image and integrates them into a quantum state [5, 6]. In the proposes technique it will use the HSV color system representation of 2D color image to map image in the Bloch sphere by calculating θ and φ represented in Eq. (2). Then this mapping formats applies to finds objects, and the 3D color image.

2 Related Works This paper proposes new black and white image detection technique using quantum computing principles instead of classical computer and using new mapping technique that maps the HSV color image to Bloch’s sphere then uses spin up ↑ and spin down ↓ principle for each pixel finding the black and white of the color image, then this b/w image will use to detect objects or 3D color image. The propose system block diagram shows in Fig. 2.

New Quantum Image Manipulation System (NQIMS)

235

Fig. 2. The proposed system block diagram

Noted that, the reason for chosen the HSV color space is the similarity of axes shapes with the Bloch’ s sphere axes, which make mapping technique easy to manipulate. 2.1 Image Color Conversion The captured color image must be converting to the HSV color system. The HSV type color spaces are deformations of an RGB color cube. If you imagine the RGB cube tipped cube onto the black corner, then the line through the cube from black to white defines the lightness axis. The color is then defined as a place on a circular plane around the lightness axis. Hue is the angle from a nominal point around the circle to the color while saturation is the radius from the central lightness axis to the color. Alternative examples to HSL include HSV (Hue Saturation Value), HSI (Hue Saturation Intensity) and HCI (Hue Chroma Intensity) [7, 8] shows in Fig. 3.

Fig. 3. The HSV color system

2.2 Mapping Technique A new technique proposes in this paper to mapping HSV color image to Bloch’s sphere. This’s by extracting angles θ, and φ for each pixel in the HSV color image then putting each pixel’s angles in Bloch’s sphere equation form shown in Eq. (2) as seen in Fig. 4. In Fig. 4 shows the proposes technique finding the same angles represent each pixel in two dimensions’ image using the HSV color system. Finding angles θ and φ as follows in the proposes Eq. (3):   (3) θij = sec Sij /Vij

236

R. F. Aly

i = 0, . . . , n − 1, j = 0, . . . , m − 1 Where n, m image dimensions. Figure 4 shows that how the HSV color system mapping to simulate the Bloch’s sphere. φ angle as angle of H matrix the only difference is that H matrix is from 0 to 360° taken from center (S matrix) but φ taken from axis perpendicular to y-axis (perpendicular to S matrix). So φ angle easy to find from H matrix. The angle φ depends on the pixel place to x-axis.

Fig. 4. The proposes Mapping the HSV color system to Bloch’s sphere

2.3 Quantum Computer Finally, image pixels in Bloch’s sphere format enter the quantum computer which will spin up pixel to one or spin down it to zero producing black and white image depending on the probabilities of α = cos 2θ and β = eiφ sin 2θ in Eq. (2) of the image directly, which acts as object detector.

New Quantum Image Manipulation System (NQIMS)

237

Algorithm 1: Quantum Manipulation System

1. Input: 2-D color image 2. im = Read image; // read color image data using imread command 3. im1= convert color image to HVS color image; //using rgb2hsv command 4. [m,n] = size (im1); 5. For i = 1 to m 6. For j = 1 to n 7. Calculate θ(i,j) = sec (im1(i,j,2)/im1(i,j,3)); // S is second matrix of HSV image // V is third matrix of HSV image 8. Calculate ϕ(i,j) = im1(i,j,1) ; // H is first matrix of HSV 9. if 0 < = ϕ