Emerging Technologies in Data Mining and Information Security: Proceedings of IEMIS 2022, Volume 3 (Advances in Intelligent Systems and Computing, 1348) 9811946752, 9789811946752

Citation preview

Advances in Intelligent Systems and Computing 1348

Paramartha Dutta Abhishek Bhattacharya Soumi Dutta Wen-Cheng Lai   Editors

Emerging Technologies in Data Mining and Information Security Proceedings of IEMIS 2022, Volume 3

Advances in Intelligent Systems and Computing Volume 1348

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Nikhil R. Pal, Indian Statistical Institute, Kolkata, India Rafael Bello Perez, Faculty of Mathematics, Physics and Computing, Universidad Central de Las Villas, Santa Clara, Cuba Emilio S. Corchado, University of Salamanca, Salamanca, Spain Hani Hagras, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK László T. Kóczy, Department of Automation, Széchenyi István University, Gyor, Hungary Vladik Kreinovich, Department of Computer Science, University of Texas at El Paso, El Paso, TX, USA Chin-Teng Lin, Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan Jie Lu, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia Patricia Melin, Graduate Program of Computer Science, Tijuana Institute of Technology, Tijuana, Mexico Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro, Rio de Janeiro, Brazil Ngoc Thanh Nguyen , Faculty of Computer Science and Management, Wrocław University of Technology, Wrocław, Poland Jun Wang, Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing such as: computational intelligence, soft computing including neural networks, fuzzy systems, evolutionary computing and the fusion of these paradigms, social intelligence, ambient intelligence, computational neuroscience, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, Web intelligence and multimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results. Indexed by DBLP, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and Technology Agency (JST). All books published in the series are submitted for consideration in Web of Science. For proposals from Asia please contact Aninda Bose ([email protected]).

Paramartha Dutta · Abhishek Bhattacharya · Soumi Dutta · Wen-Cheng Lai Editors

Emerging Technologies in Data Mining and Information Security Proceedings of IEMIS 2022, Volume 3

Editors Paramartha Dutta Department of Computer and System Sciences Visva-Bharati University Santiniketan, India Soumi Dutta Institute of Engineering and Management Kolkata, India

Abhishek Bhattacharya Department of Computer Application and Science Institute of Engineering and Management Kolkata, West Bengal, India Wen-Cheng Lai Department of Electronic Engineering National Taiwan University of Science and Technology Taipei, Taiwan

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-981-19-4675-2 ISBN 978-981-19-4676-9 (eBook) https://doi.org/10.1007/978-981-19-4676-9 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

Welcome to the 3rd International Conference on Emerging Technologies in Data Mining and Information Security (IEMIS 2022) which was held on 23–25 February 2022 in Kolkata, India. As a premier conference in the field, IEMIS 2022 provides a highly competitive forum for reporting the latest developments in the research and application of Information Security and Data Mining. We are pleased to present the proceedings of the conference as its published record. The theme this year is Crossroad of Data Mining and Information Security, a topic that is quickly gaining traction in both academic and industrial discussions because of the relevance of Privacy Preserving Data Mining (PPDM model). IEMIS is a young conference for research in the areas of Information and Network Security, Data Sciences, Big Data and Data Mining. Although 2018 was the debut year for IEMIS, it has already witnessed significant growth. As evidence of that, IEMIS received a record 610 submissions. The authors of the submitted papers come from 35 countries and regions. Authors of accepted papers are from 11 countries. We hope that this programme will further stimulate research in Information Security and Data Mining and provide practitioners with better techniques, algorithms and tools for deployment. We feel honoured and privileged to serve the best recent developments in the field of Data Mining and Information Security to you through this exciting programme. Satyajit Chakrabarti President of IEM Group, India Chief Patron, IEMIS 2022 Kolkata, India

v

Preface

This volume presents proceedings of the International Conference on Emerging Technologies in Data Mining and Information Security IEMIS2022, which took place in the Institute of Engineering and Management in Kolkata, India, from 23 to 25 February 2022. The volume appears in the series “Advances in Intelligent Systems and Computing” (AISC) published by Springer Nature, one of the largest and most prestigious scientific publishers, in the series which is one of the fastest growing book series in their programme. The AISC is meant to include various high-quality and timely publications, primarily conference proceedings of relevant conference, congresses and symposia but also monographs, on the theory, applications and implementations of broadly perceived modern intelligent systems and intelligent computing, in their modern understanding, i.e. including tools and techniques of artificial intelligence (AI), computational intelligence (CI)–which includes Data Mining, Information Security, neural networks, fuzzy systems, evolutionary computing, as well as hybrid approaches that synergistically combine these areas–but also topics such as—multi-agent systems, social intelligence, ambient intelligence, Web intelligence, computational neuroscience, artificial life, virtual worlds and societies, cognitive science and systems, perception and vision, DNA and immunebased systems, self-organizing and adaptive systems, e-learning and teaching, human-centred and human-centric computing, autonomous robotics, knowledgebased paradigms, learning paradigms, machine ethics, intelligent data analysis, various issues related to “Big Data”, security and trust management, to just mention a few. These areas are at the forefront of science and technology and have been found useful and powerful in a wide variety of disciplines such as engineering, natural sciences, computer, computation and information sciences, ICT, economics, business, e-commerce, environment, health care, life science and social sciences. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and worldwide distribution. This permits a rapid and broad dissemination of research results. It is indexed by DBLP, INSPEC, WTI Frankfurt eG, zbMATH and Japanese Science and Technology Agency (JST). All books published in the series are submitted for consideration in Web of Science. IEMIS 2022 is an vii

viii

Preface

annual conference series organized at the School of Information Technology, under the aegis of the Institute of Engineering and Management. Its idea came from the heritage of the other two cycles of events: IEMCON and UEMCON, which were organized by the Institute of Engineering and Management under the leadership of Prof. (Dr.) Satyajit Chakrabarti. In this volume of “Advances in Intelligent Systems and Computing”, we would like to present the results of studies on selected problems of Data Mining and Information Security. Security implementation is the contemporary answer to new challenges in threat evaluation of complex systems. Security approach in theory and engineering of complex systems (not only computer systems and networks) is based on multidisciplinary attitude to information theory, technology and maintenance of the systems working in real (and very often unfriendly) environments. Such a transformation has shaped natural evolution in topical range of subsequent IEMIS conferences, which can be seen over the recent years. Human factors likewise infest the best digital dangers. Work force administration and digital mindfulness are fundamental for accomplishing all-encompassing cybersecurity. This book will be of extraordinary incentive to a huge assortment of experts, scientists and understudies concentrating on the human part of the Internet and for the compelling assessment of safety efforts, interfaces, client-focused outline and plan for unique populaces, especially the elderly. We trust this book is instructive yet much more than it is provocative. We trust it moves, driving per user to examine different inquiries, applications and potential arrangements in making sheltered and secure plans for all. The Programme Committee of the IEMIS 2022 Conference, its organizers and the editors of these proceedings would like to gratefully acknowledge the participation of all reviewers who helped to refine the contents of this volume and evaluated conference submissions. Our thanks go to all respected Keynote Speakers: Prof. Seyedali Mirjalili, Prof. Md. Abdur Razzak, Prof. Rafidah Md. Noor, Prof. Xin-She Yang, Prof. Reyer Zwiggelaar, Dr. Vincenzo Piuri, Dr. Shamim Kaiser and to our all session chairs. Thanking all the authors who have chosen IEMIS 2022 as the publication platform for their research, we would like to express our hope that their papers will help in further developments in design and analysis of engineering aspects of complex systems, being a valuable source material for scientists, researchers, practitioners and students who work in these areas. Santiniketan, India Kolkata, India Kolkata, India Taipei, Taiwan

Paramartha Dutta Abhishek Bhattacharya Soumi Dutta Wen-Cheng Lai

About This Book

This book features research papers presented at the International Conference on Emerging Technologies in Data Mining and Information Security (IEMIS 2022) held at the Institute of Engineering and Management, Kolkata, India, on 23–25 February 2022. Data Mining is a current well-known topic in mirroring the exertion of finding learning from information. It gives the strategies that enable supervisors to acquire administrative data from their heritage frameworks. Its goal is to distinguish legitimate, novel, possibly valuable and justifiable connection and examples in information. Information Mining is made conceivable by the very nearness of the expansive databases. Information Security advancement is an essential part to ensure open and private figuring structures. Notwithstanding how strict the security techniques and parts are, more affiliations are getting the chance to be weak to a broad assortment of security breaks against their electronic resources. Network-intrusion area is a key protect part against security perils, which have been growing in rate generally. This book comprises high-quality research work by academicians and industrial experts in the field of computing and communication, including full-length papers, research-in-progress papers and case studies related to all the areas of Data Mining, machine learning, Internet of Things (IoT) and Information Security, etc.

ix

Contents

Computational Intelligence Empowering Indian Citizens Through the Secure E-Governance: The Digital India Initiative Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alka Agrawal, Raees Ahmad Khan, and Md Tarique Jamal Ansari

3

A Performance Evaluation of Genetic Algorithm and Simulated Annealing for the Solution of TSP with Profit Using Python . . . . . . . . . . . Neha Garg, Mohit Kumar Kakkar, Gourav Gupta, and Jajji Singla

13

Performance Assessment of DVR in a Solar-Wind Hybrid Renewable Energy Connected to Grid Using ANN . . . . . . . . . . . . . . . . . . . . S. K. Mishra, Kombe Shweta, and Behera Chinmay

27

Development of IoT Middleware Broker Communication Architecture for Industrial Automation with Focus on Future Pandemic Possibilities: Industry 5.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sujit Deshpande and Rashmi Jogdand Graceful Labeling of Hexagonal and Octagonal Snakes Along a Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lalitha Pattabiraman and Nitish Pathak Comparison and Analysis of Various Autoencoders . . . . . . . . . . . . . . . . . . . Aziz Makandar and Kanchan Wangi A New Delta (δ)-Doped Partly Insulated SOI MOSFET for Analogue/RF Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jay Prakash Narayan Verma and Prashant Mani Energy Monitoring with Trend Analysis and Power Signature Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S. Divya, Akash Murthy, Siva Surya Babu, Syed Irfan Ahmed, and Sudeepa Roy Dey

47

59 67

77

89

xi

xii

Contents

Analysis of NavIC Signal Data for Topological Precision . . . . . . . . . . . . . . 103 Raj Gusain, Anurag Vidyarthi, Rishi Prakash, and A. K. Shukla Advance Computing Experimental Study on Resource Allocation for a Software-Defined Network-Based Virtualized Security Functions Platform . . . . . . . . . . . . . . 115 S. D. L. S. Uwanpriya, W. H. Rankothge, N. D. U. Gamage, D. Jayasinghe, T. C. T. Gamage, and D. A. Amarasinghe Smart Intelligent Drone for Painting Using IoT: An Automated Approach for Efficient Painting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 P. Vidyullatha, S. Hrushikesava Raju, N. Arun Vignesh, P. Haran Babu, and Kotakonda Madhubabu Auto-Guide for Caroms Playing Using IoT . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 CH. M. H. Saibaba, S. Hrushikesava Raju, P. Venkateswara Rao, S. Adinarayna, and M. Merrin Prasanna Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage . . . . . . 151 Anupam Tiwari and Usha Batra Utilizing Off-Chain Storage Protocol for Solving the Trilemma Issue of Blockchain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Saha Reno and Md. Mokammel Haque An Empirical Evaluation to Measure the Blended Effect of Test-Driven Development with Looped Articulation Method . . . . . . . . 181 Nidhi Agarwal, Tripti Sharma, Sanjeev Kumar Prasad, Kumud Kundu, and Prakhar Deep Impact of Security Challenges Over Cloud Adoption—A Comparative Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Md Nurul Islam, S. M. K.Quadri, and S. K. Naqvi A Passive Infrared-Based Technique for Long-Term Preservation of Forensic Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Vijay A. Kanade Smart Contract Assisted Public Key Infrastructure for Internet of Things . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Pinky Bai, Sushil Kumar, and Upasana Dohare IoTFEC-19: Internet of Things-based Framework for Early Detection of COVID-19 Suspected Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Mahmood Hussain Mir, Sanjay Jamwal, Shahidul Islam, and Qamar Rayees Khan IoT-Based ECG and PCG Monitoring Systems . . . . . . . . . . . . . . . . . . . . . . . 243 V. A. Velvizhi, M. Anbarasan, S. Gayathri, K. Jeyapiriya, and S. Rajesh

Contents

xiii

Data Science and Data Analytics Comparison Based Analysis and Prediction for Earlier Detection of Breast Cancer Using Different Supervised ML Approach . . . . . . . . . . . 255 Soumen Das, Siddhartha Chatterjee, Debasree Sarkar, and Soumi Dutta Smart Crop Prediction and Assistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Archana Gupta and Ayush Gupta An Exploration of Machine Learning and Deep Learning-Based Diabetes Prediction Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Atiqul Islam Chowdhury and Khondaker A. Mamun Machine Learning-Based Smart Tourist Support System (Smart Guide) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 Prasanna Vikasitha Rathnasekara, Anuradha Sadesh Herath, Avishka Heshan Abeyrathne, Prasadini Ruwani Gunathilake, and Samantha Thelijjagoda Analysis of User Inclination in Movie Posters Based on Color Bias . . . . . 303 Harshita Chadha, Deeksha Madan, Deepika Rana, and Neelam Sharma The Changing Role of Information Technology Management in the Era of Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Abeer A. Aljohani Study of Document Clustering Algorithms Applied on Covid Data . . . . . 323 Sweety Suresh, Gopika Krishna, and M. G. Thushara Course and Programme Outcomes Attainment Calculation of Under Graduate Level Engineering Programme Using Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 S. Manikandan, P. Immaculate Rexi Jenifer, V. Vivekanandhan, and T. Kalai Selvi Multilayer Communication-Based Controller Design for Smart Warehouse Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Ngoc-Huan Le, Minh-Dang-Khoa Phan, Duc-Canh Nguyen, Xuan-Hung Nguyen, Manh-Kha Kieu, Vu-Anh-Tram Nguyen, Tran-Thuy-Duong Ninh, Narayan C. Debnath, and Ngoc-Bich Le Development of DeepCovNet Using Deep Convolution Neural Network for Analysis of Neuro-Infections Causing Blood Clots in Brain Tumor Patients: A COVID-19 Post-vaccination Scenario . . . . . . 355 Kunal S. Khadke Machine Learning Models to Predict Girl-Child Dropout from Schools in India . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Annapurna Samantaray, Satya Ranjan Dash, Aditi Sharma, and Shantipriya Parida

xiv

Contents

Keeping the Integrity of Online Examination: Deep Learning to Rescue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Towhidul Islam, Bushra Rafia Chowdhury, Ravina Akter Youki, and Bilkis Jamal Ferdosi Steering Wheel Angle Prediction from Dashboard Data Using CNN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Manas Kumar Rath, Tanmaya Swain, Tapaswini Samanta, Shobhan Banerjee, and Prasanta Kumar Swain Implementation of a Multi-Disciplinary Smart Warehouse Project with Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Ngoc-Huan Le, Manh-Kha Kieu, Vu-Anh-Tram Nguyen, Tran-Thuy-Duong Ninh, Xuan-Hung Nguyen, Duc-Canh Nguyen, Narayan C. Debnath, and Ngoc-Bich Le Machine Learning Approach Based on Fuzzy Logic for Industrial Temperature Regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Sunanda Gupta, Jyoti Verma, Shaveta Thakral, and Pratima Manhas Predictive Analytics of Logistic Income Classification Using Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 S. Beski Prabaharan and M. N. Nachappa Improvement of Speaker Verification Using Deep Learning Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Kshirod Sarmah Hybrid Texture-Based Feature Extraction Model for Brain Tumour Classification Using Machine Learning . . . . . . . . . . . . . . . . . . . . . . 445 Ishfaq Hussain Rather, Sonajharia Minz, and Sushil Kumar Intensity and Visibility of PSO-Based Waveguide Arrays with an Overview of Existing Schemes and Technologies for Multi-beam Combination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 Simarpreet Kaur, Mohit Srivastava, and Kamaljit Singh Bhatia Traffic Sign Recognition Using CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 Kavita Sheoran, Chirag, Kashish Chhabra, and Aman Kumar Sagar Motion Controlled Robot Using Multisensor and Powered by AI Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477 M. K. Mariam Bee and B. Bhanusri Pattern Recognition Smart Health Care System for Elders’ Home to Monitor Physical and Mental Health in a Controlled Environment . . . . . . . . . . . . . . . . . . . . . 487 Abhilash Krishan, Chinthaka Ashoda, Dilini Madhumali, and Gayan Pradeep

Contents

xv

Privacy Conserving Using Fuzzy Approach and Blowfish Algorithm for Malicious Personal Identification . . . . . . . . . . . . . . . . . . . . . . 503 Sharmila S. More, B. T. Jadhav, and Bhawna Narain Career Advisor Using AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 Harish Natarajan, Dereck Jos, Omkar Mahadik, and Yogesh Shahare A Novel Blood Vessel Parameter Extraction for Diabetic Retinopathy Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519 R. Geetha Ramani and J. Jeslin Shanthamalar Identifying Criminal Communities in Online Networks via Non-negative Matrix Factorization-Incorporated Sentiment Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 Shafia and Manzoor Ahmad Chachoo Fuzzy Logic-Based Disease Classification Using Similarity-Based Approach with Application to Alzheimer’s . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 Ankur Chaurasia, Priyanka Narad, Prashant K. Gupta, Mahardhika Pratama, and Abhay Bansal WhyMyFace: A Novel Approach to Recognize Facial Expressions Using CNN and Data Augmentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 Md Abu Rumman Refat, Soumen Sarker, Chetna Kaushal, Amandeep Kaur, and Md Khairul Islam Location Accuracy and Prediction in VANETs Using Kalman Filter . . . . 565 Ritesh Yaduwanshi and Sushil Kumar Early Detection of Breast Cancer Using CNN . . . . . . . . . . . . . . . . . . . . . . . . 577 S. Gayathri, K. Jeyapiriya, V. A. Velvizhi, M. Anbarasan, and S. Rajesh Analysis of Misogynistic and Aggressive Text in Social Media with Multilayer Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 Sayani Ghosal and Amita Jain Information Retrieval Development of Android Applications and Its Security . . . . . . . . . . . . . . . . 599 Hridya Dham, Tushar Dubey, Kunal Khandelwal, Kritika Soni, and Pronika Chawla Smart COVID Simulation App for Tracking of Area-Wise COVID Patients: Aware of Number of Patients While Moving on the Places . . . . 611 S. Janardhan Rao, S. Hrushikesava Raju, K. Yogeswara Rao, S. Adinarayna, and Shaik Jemulesha Automatic CAD System for Brain Diseases Classification Using CNN-LSTM Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623 Deipali Vikram Gore, Ashish Kumar Sinha, and Vivek Deshpande

xvi

Contents

Implementation of E2EE Using Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635 Suprit Dusane, Md Iliyaz, and A. Suresh Study of Spike Glycoprotein Motifs in Coronavirus Infecting Animals and Variants of SARS-CoV-2 Observed in Humans Across Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Akhbar Sha and Manjusha Nair A Systematic Review on Approaches to Detect Fake News . . . . . . . . . . . . . 651 Shashikant Mahadu Bankar and Sanjeev Kumar Gupta An Analysis of Semantic Similarity Measures for Information Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665 Preeti Rathee and Sanjay Kumar Malik Recommendation Engine: Challenges and Scope . . . . . . . . . . . . . . . . . . . . . 675 Shikha Gupta and Atul Mishra Learning Impact of Non-personalized Approaches for Point of Interest Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681 Rachna Behl and Indu Kashyap Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

About the Editors

Dr. Paramartha Dutta is currently Professor in the Department of Computer and System Sciences in, Visva-Bharati University, Shantiniketan, India. He did Bachelors and Masters in Statistics from ISI, Kolkata, India. Subsequently, he did Master of Technology in Computer Science from ISI, Kolkata, India. He did Ph.D. (Engineering) from BESU, Shibpore, India. He is Co-author of eight authored books apart from thirteen edited books and more than 200 and 40 research publications in peerreviewed journals and conference proceedings. He is Co-inventor of 17 published patents. He is Fellow of IETE, Optical Society of India, IEI, Senior Member of ACM, IEEE, Computer Society of India, International Association for Computer Science and Information Technology, and Member of Advanced Computing and Communications Society, Indian Unit of Pattern Recognition and AI—the Indian Affiliate of the International Association for Pattern Recognition, ISCA, Indian Society for Technical Education, System Society of India. Dr. Abhishek Bhattacharya is Assistant Professor at Institute of Engineering and Management, India. He has completed his Ph.D. (CSE), BIT, Mesra. He is certified as Publons Academy Peer Reviewer, 2020. His research interests are data mining, cyber security, and mobile computing. She has published 25 conference and journal papers in Springer, IEEE, IGI Global, Taylor & Francis, etc. He has 3 chapters in Taylor & Francis Group EAI. He is Peer Reviewer and TPC Member in different international journals. He was Editor in IEMIS 2020, IEMIS 2018, and special issues in IJWLTT. He is Member of several technical functional bodies such as IEEE, IFERP, MACUL, SDIWC, Internet Society, ICSES, ASR, AIDASCO, USERN, IRAN, and IAENG. He has published 3 patents. Dr. Soumi Dutta is Associate Professor at Institute of Engineering and Management, India. She has completed her Ph.D. (Engineering), IIEST, Shibpur. She received her B.Tech. (IT) and M.Tech. (CSE) securing 1st position (Gold medalist), from MAKAUT. She is certified as Publons Academy Peer Reviewer, 2020, and Certified Microsoft Innovative Educator, 2020. Her research interests are data mining, OSN data analysis, and image processing. She has published 30 conference and journal xvii

xviii

About the Editors

papers in Springer, IEEE, IGI Global, Taylor & Francis, etc. She has 5 chapters in Taylor-&-Francis Group and IGI-Global. She is Peer Reviewer and TPC Member in different international journals. She was Editor in CIPR-2020, CIPR2019, IEMIS2020, CIIR-2021, and IEMIS-2018 special issues in IJWLTT. She is Member of several technical functional bodies such as IEEE, ACM, IEEE, IFERP, MACUL, SDIWC, Internet Society, ICSES, ASR, AIDASCO, USERN, IRAN, and IAENG. She has published 4 patents. She has delivered more than keynote talks in different international conferences. Dr. Wen-Cheng Lai has been working in the field of radio frequency circuits, analog IC integrated design, microwave antenna, computer, and communication for more than 20 years. He completed his Ph.D. from National Taiwan University of Science and Technology. He has authored/co-authored more than 200 SCI indexed journals and IEEE conference/EI papers. He received the Ph.D. degree from the National Taiwan University of Science and Technology. He is Assistant Professor at the National Yunlin University of Science and Technology. He also worked as Executive Director of China Radio Association, Assistant Professor of National Yunlin University of Science and Technology, Assistant Professor of National Penghu University of Science and Technology, and Director of AsusTek Computer Inc.

Computational Intelligence

Empowering Indian Citizens Through the Secure E-Governance: The Digital India Initiative Context Alka Agrawal, Raees Ahmad Khan, and Md Tarique Jamal Ansari

Abstract In India, e-governance has progressed from the computerized system of government units to programmes that encompass the finer elements of governance, including the citizen-centric approach, responsiveness, and accountability. Lessons learned from prior e-Government programmes have helped to shape the country’s advanced e-Government policy. Although policymakers have been persuaded to speed up the deployment of e-Government across the different bodies of government at the national, state, and municipal level, yet there is a need for preventative steps to minimize cyber-attacks. In other ways, cyber-security issues appear to be roadblocks to e-governance achievement. These challenges may include socio-economic, religious, and technological limits, as well as privacy and security implications. Despite the numerous obstacles and limits, the government is confident in its ability to overcome these obstacles and pave the road for the success of e-government. Different significant initiatives of e-governance, security issues-challenges, and prospects of e-governance in the Digital India context are discussed in this study. Keywords Digital India · Web application · E-Governance · Cyber-security · Digital platform · Durable security

1 Introduction The term “e-governance” is now becoming increasingly popular. We’ve been seeing e-governance all around the world. Since socio-economic concerns have become more prevalent, governments have begun to extend their management from top to bottom levels in order to address a variety of socio-economic, research and engineering, and other challenges through the extensive use of digital devices. In fact, both established and emerging economies around the world use a variety of technological technologies to make public administration more efficient, transparent, and A. Agrawal · R. A. Khan · M. T. J. Ansari (B) Department of Information Technology, Babasaheb Bhimrao Ambedkar University, Lucknow, Uttar Pradesh 226025, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_1

3

4

A. Agrawal et al.

responsible [1–3]. In the Indian scenario, which is one of the world’s most democratic, based on demographics, as well as geographically dispersed countries; there is still a disparity in the use of government services among its huge population. Moreover, there are still concerns associated with the nation’s socio-economic position, such as unemployment, economic hardship, education, wellbeing, banking, as well as business, to name a few. As a consequence, the Indian government has been introducing number of innovative approaches in order to address these issues with optimum e-governance through extensive use of electronic devices [4–7]. In the Indian economy, e-governance is viewed as a supplement to information security, with the main emphasis on e-service implementation and IT policy creation. Initiatives in the E-Governance structure component are significant for incorporating procedures as well as security protocols in various organizations and firms, in addition to providing important strategies. The domain of cyber-security governance is particularly demanding in the context of attaining India’s sustainability targets since it must create a security IT strategy and assist the implementation of e-services to citizens. Different implementation models and pathways have been designed as methods to cover the current gaps in public developments, where numerous stakeholders are involved in the design of the E-Government procedure. In recent times of e-Governance, major stakeholders such as the Ministry of Electronics and Information Technology (MeitY) as well as the Government of India (GoI) have implemented a number of regulatory steps. These steps are critical for realizing the vision and goals of the Digital India initiative. The programme is regarded as one of the most important factors in achieving long-term growth [8]. For IT systems, especially Digital India, cyberspace security is critical. A security layer should be included in every e-governance initiative. The security component of any e-governance initiative should be considered during the design phase. The administration’s cyber-security unit must play a key role in monitoring the initial design. With the advancement of technology as well as digitalization, the view of security has experienced a significant transition. Authorities must take further safeguards by incorporating different government-led steps to ensure the integrity of an important country’s infrastructure [9–12]. The Government of India’s main programmes, the National e-Governance Plan as well as Digital India, aim to provide smooth government operations as well as accountable and productive solutions free of security risks. In other words, the goal of Digital India’s e-governance strategy effort is to offer real-time governance to everyone by ensuring easy, fast, and responsible delivery of public services. The rest of the paper is systematized as follows: Sect. 2 discusses the different egovernance initiatives in India. Section 3 discusses the various security issues in the way of efficient e-governance. Section 4 discusses the ways to enhance the security durability for e-Governance services. Finally, Sect. 5 concludes the paper.

Empowering Indian Citizens Through the Secure …

5

2 E-Governance Initiatives in India At the central as well as state levels, there seem to be a plethora of e-Government initiatives presently. The National e-Governance Plan (NeGP) was developed by the Department of Electronics and Information Technology and Administrative Reforms and Public Grievances with the goal of making all public services available to the general public, ensuring efficiency, visibility, and serviceability of such services at reasonable costs in order to meet the basic requirements of the general public. Several e-governance initiatives have been made possible because of the NeGP. Some of the popular e-Governance initiatives are listed below in Table 1.

3 Security Issues in E-Governance The implementation of an e-governance paradigm in India, and also on a worldwide scale, is fraught with difficulties. The real difficulty is figuring out how to create and sustain effective e-governance projects that provide residents with cutting-edge e-services. Nevertheless, developing an e-governance portal as a service delivery method is not as simple. Efficient e-governance initiatives cannot be implemented in a hurry. In the case of India, e-Governance must allow for increased access to information and sharing of information between the state and central governments. The following subsections discuss the different security challenges faced by the e-governance initiatives.

3.1 Usability Usability is concerned with making services and applications simple to use. Usability is tied to security considerations since efforts to improve data security may reduce their usability. In the context of software development, usability also refers to how and by whom data will be used. As a result, usability necessitates a major focus on challenges of e-Governance confidence elicited by interactions among actors who govern, provide, or profit from the services. Usability issues in electronic procurement arise from national regulations requiring company intelligence reports or eSignature solutions [10]. Distinct health systems in eHealth have diverse record mechanisms, and even within these processes, different record-keeping systems may exist. There’s also the issue of fully digitizing record-keeping systems and ensuring that professionals and patients understand how to operate them.

6

A. Agrawal et al.

Table 1 Initiatives taken for e-Governance in India S. No.

E-governance initiatives

Details

1

Digital India

The Government of India started the Digital India attempt to assure that all citizens have access to government services via the internet

2

myGov.in

myGov.in is a nationwide citizen participation platform where citizens may contribute ideas and participate in policymaking and governance discussions

3

Aadhaar

Aadhaar is a UIDAI-issued unique identity card that serves as verification of person and residence using biometric information. It can be used to deliver substantial advantages to Indian citizens. Aadhaar can be used to e-sign documents

4

UMANG

UMANG is a unified mobile application that gives users access to a variety of central as well as state public services, such as Aadhaar, Digital Locker, PAN, as well as Employee Provident Fund facilities

5

PayGov

PayGov is a secured and efficient electronic portal for government to citizen engagement that has been approved by the government. PayGov allows users to make online payments to any public or private bank

6

Digital Locker

Citizens can use Digital Locker to electronically store sensitive documents such as mark sheets, PANs, Aadhaar cards, and degree credentials

7

Mobile Seva

The goal of Mobile Seva is to deliver government services via smartphones and tablets. Numerous live applications are available in the m-App store that can be utilized to access different governmental facilities

8

e-Hospital-online registration framework (ORF)

It is an endeavour to make it easier for patients to schedule Out Patient Department (OPD) visits with public hospitals via the internet. A patient care, clinical services, and health record administration are all included in this architecture

9

DARPAN

It is an online application that may be used to track and analyse the progress of the state’s important and rising initiatives. It allows for actual data on Key Performance Indicators (KPIs) of specified schemes/projects to be presented to top State Government and district administration officials (continued)

Empowering Indian Citizens Through the Secure …

7

Table 1 (continued) S. No.

E-governance initiatives

Details

10

Common services centres 2.0 (CSC 2.0)

CSCs are being applied to encourage and assist the adoption of information systems in the nation’s rural regions

3.2 Network Security With the rise of electronic government, communications infrastructure security is becoming more important, and resilience to network assaults (access, manipulation, and denial of service) is critical [11]. Threats to information security (cyber-terrorist activity, cyberwarfare, blended attacks, and so on) are constantly evolving as weaknesses in both existing and newly acquired systems are uncovered, necessitating solutions to address those threats. Methods to maintain network security comprise firewalls as well as proxy to keep undesired individuals out, antivirus applications and Internet Security Solution packages, anti-malware, encoding as well as enhanced computer designs, etc.

3.3 Access Control Individuals who crave to use sensitive data for malicious reasons will be interested in all technological systems containing it. As a consequence, permissions to such systems are required to prevent unauthorized access to the data held. Generally, access control has a broad meaning, encompassing everything from your automobile lock to one’s payment card pin code. However, the primary function is to prevent unauthorized access [12]. The systems range from data sources of citizen details, medical histories, banking information, and agreements to control of facilities such as electricity, roadways, and terminals, and the implies of access control will primarily be online or physical (walls, vouchers, tamper protective devices) in the space of electronic government.

3.4 Identification The problem of identification poses a number of pertinent questions in current situations. The issue of validating a business’s identification in technological capability is vital not only for ensuring that the business is who it claims to be when completing a contract, as well as in the long run. Issues have been expressed about the reliability of biometric information and whether it would be secured against fraudsters attempting to falsify the data and biometric passports. As a result, the efficacy of biometric data

8

A. Agrawal et al.

would be a topic of discussion. The topic of how patients, physicians, and other medical professionals would recognize themselves in the e-healthcare environment is a challenge.

3.5 Interoperability The potential of systems or operational processes to collaborate to accomplish a common goal has long been a goal in developing as well as even developed nations. The ability of the products they use to share and exchange data is required for effective interaction among government, industry, and individuals. The ineffectiveness of the electronic government system will be harmed by a lack of interoperability caused by language, a poor infrastructure, and diverse classification approaches.

4 Enhancing Security Durability for e-Governance Services Given the country’s growing IT sector, aggressive ambitions for rapid modernization and equitable progress, and India’s significant impact on the worldwide IT market, putting the proper kind of emphasis on developing a safe computing platform with sufficient trust as well as assurance in online transactions, applications, operations, devices, and infrastructure became one of the nation’s most pressing issues. Such planning is based on the country developing an appropriate cyber-security environment that is compatible with the globally distributed system. The digital world is susceptible to a wide range of tragedies, whether deliberate or unintentional, man-made or environmental, and data transferred in virtual worlds can be used for malicious ends by both country and anonymous actors. Secure cyberspace is defined by the protection of personal information architecture and the restoration of data’s confidentiality, integrity, and availability. Application security durability is often expressed as the software’s expected serviceability. Since new security threats emerge every day, the requirement for software updates grows as time passes. If these risks become active, security will be compromised, and the application will malfunction as a consequence. The security durability properties have already been recognized and categorized by the authors. Figure 1 shows several security durability traits [13, 14] that should be significant to maximizing security durability, comprising dependability, trustworthiness, as well as human trust in order to empower Indian citizens with efficient e-Governance initiatives.

Empowering Indian Citizens Through the Secure …

9

Fig. 1 Different factors of security durability

4.1 Dependability The degree to which stakeholders believe a crucial system is referred to as dependability. The most significant system feature of a crucial system is generally dependability. To be effective, a system does not need to be recognized. The user’s faith in the product’s capacity to function normally is reflected in its dependability. Due to dependable software frequently necessitates certification, both procedure and product documents must be created. A prior requirements elicitation is also necessary to identify needs and requirements incompatibilities that could jeopardize the system’s stability and security. This is at odds with the agile approaches of co-developing requirements and systems while minimizing paperwork. The agile approach, testfirst progression, and user interaction in the development team are all strategies that can be included in an agile approach. Agile approaches can be employed as long as the software development group follows the procedure and logs their activities. However, because more documentation and preparation are required for trustworthy system design, “pure agile” is unfeasible.

4.2 Trustworthiness The extent to which the system is supposed to work as intended in the presence of emerging interruptions, loss of execution completeness and consistency, human mistakes, system malfunctions, and cyber-attacks are referred to as trustworthiness. The accuracy of results one has in this anticipation is referred to as confidence of trustworthiness. For a business enterprise to have faith in a system, it must be proven to be trustworthy. Only purposeful process improvement for the construction of the solution at every level can ensure software’s trustworthiness. A firm’s ability to balance opposing aspects and provide the requisite performance all through the lifecycle from start to the finish is based on objective, clearly specified, quantifiable criteria and their prioritization. This specifies the expected level of quality, as well as how it would be maintained and controlled throughout the software engineering

10

A. Agrawal et al.

and assurance testing processes. It is frequently necessary to consider who produced and administered the programme, as well as how this happened.

4.3 Human Trust Human trust is generally described as a critical matter in which the person or group has an ethical value to the receiving entity in human–human contact. Customer experience in software producers is referred to as human trust in software. Whenever it comes to software, users’ trust is based on the product’s security architecture and the assurance that it will perform properly and protect their confidential information and data. Digital devices can be networked everywhere and at any time due to significant advancements in wireless networking technology. While travelling, these gadgets’ apps dynamically identify hosts and resources with which they can initiate connections. However, the concern of being exposed to potentially dangerous transactions involving unknown parties may stymie interaction. An innovative approach towards the establishment of trust-based interactions is required to reduce this vulnerability.

5 Conclusion Numerous policy initiatives and infrastructure improvements have been devoted to developing administration and support facilities in order to encourage e-Governance holistically in India. This article examined concerns linked to information security in e-governance, such as risks, attacks, and susceptibility, as well as success variables that may affect the number of threats, challenges, and weaknesses in the e-governance system in the context of the Digital India Initiative. The researcher believes that robust cyber-security practices must be used to enhance the confidentiality of egovernment systems. Security measures, methods, and protocols, as well as the usage of access control, must be established in order to protect e-government platforms prevent the attack. This study also indicates that having a proper infrastructure that provides the required standard of durable security authentication, as well as having a continuous information security awareness programme to ensure that people are concerned about the security consequences, comprehend how to identify possible problems, and continue to maintain a secure e-government platform, are significant considerations. Acknowledgements The authors gratefully acknowledge the support from Council of Science and Technology, Uttar Pradesh (UPCST); Letter No. CST/D-2300.

Empowering Indian Citizens Through the Secure …

11

References 1. Y. Goswami, A. Agrawal, A. Bhatia, E-Governance: a tendering framework using blockchain with active participation of citizens, in 2020 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS) (IEEE, December, 2020), pp. 1–4 2. B. Warf, Urban informatics and e-governance, in Handbook of Urban Geography (Edward Elgar Publishing, 2019) 3. M. Roy, E-Governance in the age of globalization: challenges ahead for India. SOCRATES 5(3 and 4) (2017); September and December, 5, 88 (2018) 4. S. Sen, S. Mukhopadhyay, S. Karforma, A blockchain based framework for property registration system in e-governance. Int. J. Inf. Eng. Electron. Bus. 13(4) (2021) 5. M.T.J. Ansari, D. Pandey, M. Alenezi, STORE: security threat oriented requirements engineering methodology. J. King Saud Univ. Comp. Inf. Sci. (2018) 6. M. Shah, E-governance in India: dream or reality. Int. J. Educ. Develop. ICT. 3(2) (2007) 7. P. Malik, P. Dhillon, P. Verma, Challenges and future prospects for e-governance in India. Int. J. Sci. Eng. Technol. Res. 3(7), 1964–1972 (2014) 8. V. Shubha, Deploy right, develop right: analysis and recommendation on using e-governance competency (EGCF) framework for digital India, in Proceedings of the 10th International Conference on Theory and Practice of Electronic Governance (March, 2017), pp. 39–42 9. P. Rossel, M. Finger, Conceptualizing e-governance, in Proceedings of the 1st International Conference on Theory and Practice of Electronic Governance (December, 2007), pp. 399–407 10. M. Bhuvana, S. Vasantha, Assessment of rural citizens satisfaction on the service quality of common service centers (CSCs) of e-governance. J. Crit. Rev. 7(5), 302–305 (2020) 11. R. Kumar, M. Alenezi, M.T.J. Ansari, B.K. Gupta, A. Agrawal, R.A. Khan, Evaluating the impact of malware analysis techniques for securing web applications through a decision-making framework under fuzzy environment. Int. J. Intell. Eng. Syst. 13(6), 94–109 (2020) 12. R. Betala, S. Gawade,.Usability analysis of e-governance applications, in International Conference on Advanced Informatics for Computing Research (Springer, Singapore, December, 2020), pp. 637–648 13. R. Kumar, M. Zarour, M. Alenezi, A. Agrawal, R.A. Khan, Measuring security durability of software through fuzzy-based decision-making process. Int. J. Comput. Intell. Syst. 12(2), 627–642 (2019) 14. T.J. Ansari, D. Pandey, An integration of threat modeling with attack pattern and misuse case for effective security requirement elicitation. Int. J. Adv. Res. Comp. Sci. 8(3) (2017)

A Performance Evaluation of Genetic Algorithm and Simulated Annealing for the Solution of TSP with Profit Using Python Neha Garg, Mohit Kumar Kakkar, Gourav Gupta, and Jajji Singla

Abstract The Traveling Salesman Problem with profit (TSPP) is defined on an graph G = (V, E). Traveling salesman problems with profit (TSPP) is a generalization of the traveling salesman problem (TSP), with one condition that it is not required for a sales person to travel all cities of the network. Our main purpose is to optimize the total profit and cost of the traveling. Here we are focusing on that in the case of TSPs; in real world scenario it is not always relevant for the salesman to visit each and every customer or city. To solve the problem, a GA with special mutation operators has been presented. Here we have utilized and compared different heuristic techniques genetic algorithm (GA) and simulated annealing (SA). Numbers of tours plots are generated for the comparison of performance of both the algorithm implemented for the solution of TSPP. These plots are beneficial for route planner and for those who want to apply this concept of TSPP. It has been observed that SA performs better than GA, as the response consumed less time than GA. Keywords Genetic algorithm (GA) · Simulated annealing (SA) · Traveling salesman problems with profits (TSPP) · Python

1 Introduction Let G = (V, E) be an undirected graph with sets V = {v1 , v2 , v3, …, vn } of n vertices (cities) and E{e1 , e2 , …} of edges between vertices of graph. Let profit Pi be associated with each vertex vi ∈ V and a distance cij (cost) be associated with each edge between two vertices. In the recent years Nature inspired algorithm for optimization have progressively attracted the interests of many researchers, such as GA, ACO, particle swarm optimization (PSO), SA. All these nature inspired N. Garg · M. K. Kakkar (B) · G. Gupta · J. Singla Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, Punjab, India e-mail: [email protected] G. Gupta e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_2

13

14

N. Garg et al.

algorithms playing important role in solving NP hard type problems like TSP. TSP is considered as an NP hard problem, because, for a large number of cities or customers, it is not possible to find every optimized route under feasible time limit. Consequently, TSPs are well suited to solving using randomized optimization algorithms. In this study we are focusing on TSPP, traveling salesman problem with profit. The TSP is very popular studied problems in tour cost minimization and also it has the flavor of variety of applications as described by Erol and Bulut [8]. Similarly Hashim and Ismail [15] gave the applications of Traveling Salesman Problem in Tourist Visit in Langkawi. Iscan and Gunduz [17] provided an application of beautiful fruit fly algorithm for the solution of TSP. Xu et al. [29] studied the application of genetic algorithms (GA) on TSP. Raman and Gill [25] gave the Review of different heuristic algorithms (HA) on solving TSP. Almufti et al. [1] explained an ACO for Solving Symmetric TSP. Liu and Li [21] discussed a method called Greedy permuting for solution of traveling salesman problem based on GA. Hacizade and Kaya [14] provided the concept which was on GA-based TSP solution and also explained the application of TSP for transport routes optimization. Kaspi et al. [19] discussed the approach for maximizing the profit per unit time for the TSP problem. Ayon et al. [3] discussed the novel algorithm Spider monkey optimization (SMO) to solve TSP. Ha, et al. [16] provided the min-cost TSP with drone. An application of improved ACO was given by the Yang and Wang [30] for traveling salesman problem. Zhang et al. [31] discussed the problem related to the TSP with profits for stochastic customers. Namazi et al. [23] provided the concept of profit guided heuristic for traveling problems, which was basically the combination of two concepts, TSP and knapsack problem (KP). Santini et al. [26] explained the approach of Heuristic and ML to TSP with Crowdsourcing. Qin et al. [24] solved the TSP with profits under time dependent cost (TDC) and multiple tour mode. Bouziaren and Aghezzaf [5] explained An Improved method of Branch-and-Cut for the solution of the TSPP. Gelareh et al. [12] discussed the new concept of draft limits for the solution of traveling salesman problem. Beraldi et al. [4] discussed about risk-averse TSPP. Lahyani et al. [20] explained the metaheuristic algorithm for solving multiconstrained TSPP. For the easiness they have assumed that service time included in the traveled duration. Gansterer et al. [11] discussed the MVPPD problem which is based on profitable multi-vehicle delivery problem. Eskandari et al. [9] provided a modified and enhanced ant colony optimization algorithm (ACO) for TSP. Yang and Wang [30] demonstrated an application of improved ant colony optimization algorithm on traveling salesman problem. Costa et al. [6] presented a heuristic approach for TSP based on Christofides’s heuristic. Filippi and Stevanato [10] studied the two-phase method for bi-objective combinatorial optimization problems. Angelelli et al. [2] presented a new approach for solving TSP with profits (TSPP). Gottlieb et al. [13] presented an approach where total focusing on the prizes cities or customers where salesman decides that where he should visit with respect to the benefit of prizes. Kang et al. [18] delivered an algorithm which is based on time dependent profit. Derya et al. [7] proposed a new approach based on mixed integer programming for solving SGTSP. Mathur et al. [22] discussed a new strategy for the

A Performance Evaluation of Genetic Algorithm and Simulated …

15

Table 1 Summary of TSP with profits Problem detail

Objective function based on

Constraints

Servers/Repairman/Salesman

Orienteering problem (OP) (selective TSP, maximum collection problem (MCP))

Maximize P (where P = profit)

Route duration

Single

Profitable route/tour problem (PTP)

Maximize (P − C)

NA

Single

Profit collecting traveling repairman problem (TRP)

Minimize C (where C = cost)

Route profit

Single

Team OP (TOP) (maximum profit based problem on multiple tour)

Maximizing P (where P = profit)

Route duration

Multiple

solution of (SA) Sales Augmentation using algorithm based on TSP and policy of Time Bound Marginal (TBM) Discount. Valdez et al. [28] discussed the comparison of Ant colony optimization with GA and SA also for the solution of TSP problem. Sarin et al. [27] discussed the multiple TSP with and without effect of precedence constraints. In this paper, we will take care of the sales representative issue with benefits where the sales representative gathers some benefit for visiting each customer. As contradicted to the traditional TSP, there is no prerequisite to visit all the clients. The target of TSPP is to decide the best subset of clients to be visited in order to expand the all-out net benefit, which is equivalent to the all-out benefit earned from visited clients less the absolute expense of the visit. The last can be taken as the complete length of the visit determined as the Euclidean separation. In Table 1 we summarize the classification of different kind of problems based on TSP and their characteristics.

2 Mathematical Modeling A complete graph G: (V, E) is given where V = set of customers/cities including depot and E is the set of edges. For any subset S of V, we define X + (S) = {(i, j):i ∈ S}. Traveling cost and time (cij , t i,j ) both may be associated with every edge (i, j) ∈ E. A profit pi which is (a positive value may also zero) is fixed with each customer i which can be collected one time maximum. The traveling time t i,j is connected with each edge (i, j) ∈ A. single salesman is present here at the depot with T max Which is the maximum time limit associated with salesman for his journey. Let us introduce the problem variables:

16

N. Garg et al.

• ai j = {0, 1} , 1 if edge (i, j) ∈ A is related to the salesman, else 0. • bi = {0, 1} , 1 if customer or city i ∈ X is being served by the salesman, else 0. Max

 i,∈X





pi bi −

ci j al

(1)

(i, j)∈A

ai j = bi ∀i ∈ V

(2)

a ji = bi ∀i ∈ V

(3)

j∈V

 j∈V

b0 = 1 

ai j ≥ bl , ∀S ⊂ X, ∀l ∈ X/S

(4) (5)

(i, j)∈X + (s)

ai j , bl ∈ { 0, 1} , ∀i ∈ X

(6)

The objective function (1) focusing on maximizing (Collected profit − Traveling cost). Thus, max. (Collected profit − traveling cost) = min. (traveling cost + uncollected profit).

3 Proposed Strategy Here we have considered three data sets (eil51, eil76, eil101) as a benchmark dataset each of these data sets are designed for two types of benefits first one is for high customer profit (P-I) and second one is for relatively low customer profit (P-II), so total 6 datasets are existing with us for experiments. Therefore, the number of customers is equal to 50, 75, and 100 for eil51, eil76, and eil101, respectively. Hence, the input data consisting two attributes one is the locations of city and the profit regarding each city or customer. This paper mainly uses genetic algorithm (GA) and simulated annealing algorithm (SA) to solve the TSP problem with profit. The code is written in python language.

3.1 Solution Representation We used a simple representation (optimum tour traveled by salesman) for the output of the program which is basically a list of cities or customers like [2, 6, 7, 4, 5, 1, 3,

A Performance Evaluation of Genetic Algorithm and Simulated …

17

9, 8, 2], here 2 is considered as depot or starting point of the salesman which would be the ending point as well. The pseudo code of the Genetic Algorithm: Start Initial population generated of genes of some finite size; Calculate fitness value for every gene; for i: = 1 to n do start pick the gene i from other i − 1 genes using tournament selection with the group size; divide genes into the pairs; start crossover for every pair; start mutation; End; select the best gene as the result; End; Operator like mutation can affect the searching process. So we should be careful about these operators’ values, because it might also divert the convergence of the algorithm toward local maxima or minima. Here in this paper we are considering following mutation operators. (a) TWORS Mutation (TM): It is a strategy where two genes picked up randomly and then position of them exchanged with each other. (b) Reverse Sequence Mutation (RSM): In this strategy a sequence is identified after fixing two positions in the chromosome and the order of genes is reversed in between these two positions. (c) Center Inverse Mutation (CIM): In this concept the chromosome is partitioned into two sets. Genes in each set are reversed and new chromosome generated. From Table 2 it is clear that best objective values of the tour is more in ExperimentI as compared to the other Experiments, here we can see that for higher mutation rate, best objective values is comparatively higher (if all the other parameters remains same in both Experiments I and II).

Table 2 Best objective value for different parameters for eil51 Parameters

Experiment-I

Experiment-II

Experiment-III

Experiment-IV

Crossover rate

0.90

0.90

0.70

0.70

Mutation rate (RSM)

0.10

0.15

0.30

0.55

Selection rate

0.95

0.95

0.80

0.80

Best objective value

8.92

8.75

7.82

7.46

18

N. Garg et al.

The pseudo code of the Simulated Annealing Start Start with some initial T and alpha Generate and score a random solution (score_old) Generate and score a solution with “neighboring"parameters (score_new) Compare score_old and score_new: If score_new > score_old: move to neighboring solution If score_new < score_old: maybe move to neighboring solution Decrease T: T * = alpha; Repeat the above steps until one of the stopping conditions met: T < T _min; n_iterations > max_iterations; total_runtime > max_runtime; Return the score and parameters of the best solution End; The decision to move to a new solution from an old solution is probabilistic and temperature dependent. Specifically, the comparison between the solutions is performed by computing the acceptance probability a = exp((score_new − score_old)/T ). The value of ‘a’ is then compared to a randomly generated number in [0, 1]. If ‘a’ is greater than the randomly generated number, the algorithm moves to the hyper parameters of the neighboring solution. This means that while T is large, almost all new solutions are preferred regardless of their score. As T decreases, the likelihood of moving to hyper parameters resulting in a poor solution decreases.

4 Results and Discussion We have implemented simulated annealing SA and GA to solve the Traveling Salesperson Problem (TSP) with profit by using the PYTHON 3 programming language. The operating environment is Windows 7, Intel Core i5-2520 2.50 GHz, RAM 4 GB, language Python 3.7.6 (Anaconda-3) (Table 3). Figures 1, 2, 3 depict the optimized tour of the salesman for all three data sets with respect to P-I. And Similarly Figs. 4, 5, 6 showcasing the optimized tour for P-II for all three data sets eil-51, 76, 101. From Figs. 1, 2, 3, 4, 5 and 6, we can see that out of 50 cities (one city is considered as depot or origin) salesman just travels 25 different cities and gets maximum benefit in both cases (P-I, II) but when we compared for eil51, these both algorithm (from Tables 4 and 5) in terms of CPU time (in seconds), SA is faster than the GA. But for large size of data set (eil101) GA is performing

A Performance Evaluation of Genetic Algorithm and Simulated … Table 3 Best objective value for different mutation operator (Crossover rate = 0.85, Selection rate = 0.95, Mutation rate = 0.1)

19

Operator TM Operator RSM Operator CIM Best objective 7.74 value (eil51)

8.92

8.52

Best objective 6.76 value (eil76)

6.94

6.80

Best objective 4.69 value (eil101)

5.53

5.44

Fig. 1 TSPP for eil-51 (25 cities, P-I)

well. It can be seen that SA performs better than GA for small size data set (eil-51, eil-76), as the response consumed less time than GA.

5 Conclusion In this problem of TSPP a salesman has to visit less number of cities in order to maximize his profit. This work shows that TSP with profit which is an NP hard problem can be solved by using GA and SA, both are heuristic optimization algorithms. Experiments with simulated annealing showed that increase in number of

20

N. Garg et al.

Fig. 2 TSPP for eil-76 (38 cities, P-I)

iteration gives better result only in combination with increase in cooling ratio. GA performs exceptionally with RSM (mutation operator). So we can add here that for the TSP with profit, the operator of mutation with the best solutions is Reverse Sequence Mutation. Based on this study we can conclude that reverse sequencing strategy for mutation operator performs better than the mixing of sequence. The time spent mainly depends on the scope of the neighborhood, the initial temperature, and the cooling strategy as far as SA is concerned. Therefore, more debugging is needed to get the best parameters for both GA and SA.

A Performance Evaluation of Genetic Algorithm and Simulated …

Fig. 3 TSPP for eil-101 (50 cities, P-I)

Fig. 4 TSPP for eil-101 (50 cities, P-II)

21

22

Fig. 5 TSPP for eil-76 (38 cities, P-II)

Fig. 6 TSPP for eil-51 (25 cities, P-II)

N. Garg et al.

A Performance Evaluation of Genetic Algorithm and Simulated …

23

Table 4 Comparison of GA and SA for TSPP for P-I Instance Best objective Number of Sequence of value customers customers visited (depot visited excluded)

CPU time (s): CPU time (s): (for Python and (for Python and C) [For GA] C) [For SA]

Eil-51

8.92

25

[1, 32, 27, 48, 45.50 (P) 26, 7, 23, 6, 14, 15.80(C) 18, 13, 4, 44, 47, 51, 46, 38, 34, 50, 16, 2, 3, 28, 31, 8, 22, 1]

41.04(P) 14.66(C)

Eil-76

6.94

38

[1, 33, 49, 50, 135.72(P) 25, 32, 44, 3, 22.64(C) 16, 51, 17, 40, 72, 10, 12, 26, 35, 46, 34, 67, 76, 75, 6, 68, 4, 30, 21, 74, 2, 73, 62, 22, 64, 42, 43, 41, 56, 23, 63, 1]

127.42(P) 21.35(C)

Eil-101

5.53

50

[1, 69, 31, 10, 271.39(P) 90, 32, 30, 70, 34.82(C) 33, 81, 9, 51, 20, 66, 71, 35, 34, 78, 79, 3, 77, 68, 80, 29, 24, 39, 4, 40, 58, 13, 97, 95, 6, 89, 84, 83, 18, 82, 48, 19, 11, 62, 7, 52, 27, 101, 53, 28, 12, 76, 50, 1]

286.71(P) 35.08(C)

24

N. Garg et al.

Table 5 Comparison of GA and SA for TSPP for P-II Instance Best objective Number of Sequence of value customers customers visited (depot visited excluded)

CPU time (s): CPU time (s): (for Python and (for Python and C) [For GA] C) [For SA]

Eil-51

22.94

25

[1, 27, 32, 11, 54.29(P) 38, 16, 2, 22, 3, 15.80(C) 28, 31, 26, 7, 23, 43, 24, 14, 25, 18, 17, 47, 46, 51, 6, 48, 8, 1]

41.04(P) 15.66(C)

Eil-76

28.61

38

[1, 73, 33, 63, 126.68 (P) 16, 3, 44, 32, 22.64(C) 51, 6, 2, 62, 22, 28, 61, 71, 74, 30, 4, 68, 75, 76, 67, 26, 17, 40, 12, 72, 9, 50, 18, 24, 49, 23, 56, 41, 64, 42, 43, 1]

124.42(P) 21.35(C)

Eil-101

27.53

50

[1, 69, 52, 18, 242.13(P) 60, 61, 85, 98, 34.82(C) 37, 93, 59, 99, 96, 94, 6, 89, 27, 28, 101, 53, 58, 13, 95, 97, 57, 2, 74, 73, 21, 40, 26, 4, 54, 80, 68, 12, 76, 50, 77, 3, 79, 78, 81, 33, 51, 9, 71, 65, 66, 30, 70, 1]

256.71(P) 362.08(C)

References 1. S.M. Almufti, A.A. Shaban, U-turning ant colony algorithm for solving symmetric traveling salesman problem. Acad. J. Nawroz Univ. 7(4), 45–49 (2018) 2. E. Angelelli, C. Bazgan, M.G. Speranza, Z. Tuza, Complexity and approximation for traveling salesman problems with profits. Theoret. Comput. Sci. 531, 54–65 (2014) 3. S.I. Ayon, M.A,H. Akhand, S.A. Shahriyar, N. Siddique, Spider monkey optimization to solve traveling salesman problem, in International Conference on Electrical, Computer and Communication Engineering (ECCE) (2019), pp. 1–5 4. P. Beraldi, M.E. Bruni, D. Laganà, R. Musmanno, The risk-averse traveling repairman problem with profits. Soft. Comput. 23(9), 2979–2993 (2019) 5. S.A. Bouziaren, B. Aghezzaf, An improved augmented ε-constraint and branch-and-cut method to solve the TSP with profits. IEEE Trans. Intell. Transp. Syst. 20(1), 195–204 (2018)

A Performance Evaluation of Genetic Algorithm and Simulated …

25

6. A. Costa, J.F. Cordeau, G. Laporte, Steiner tree problems with profits. INFOR 44(2), 99–115 (2006) 7. T. Derya, E. Dinler, B. Keçeci, Selective generalized travelling salesman problem. Math. Comp. Modell. Dyn. Syst., 1–39 (2020) 8. M.H. Erol, F. Bulut, Real-time application of travelling salesman problem using Google Maps API, in Electric Electronics, Computer Science, Biomedical Engineerings’ Meeting (EBBT) (2017), pp. 1–5 9. L. Eskandari, A. Jafarian, P. Rahimloo, D. Baleanu, A modified and enhanced ant colony optimization algorithm for traveling salesman problem, in Mathematical Methods in Engineering (Springer, Cham, 2019), pp. 257–265 10. C. Filippi, E. Stevanato, A two-phase method for bi-objective combinatorial optimization and its application to the TSP with profits. Algorithmic Oper. Res. 7(2) (2012) 11. M. Gansterer, M. Küçüktepe, R.F. Hartl, The multi-vehicle profitable pickup and delivery problem. ORSpectrum 39(1), 303–319 (2017) 12. S. Gelareh, B. Gendron, S. Hanafi, R.N. Monemi, R. Todosijevi´c, The selective traveling salesman problem with draft limits. J. Heuristics, 1–14 (2019) 13. L.A. Gottlieb, R. Krauthgamer, H. Rika, Faster algorithms for orienteering and $ k $-TSP. arXiv preprint arXiv:2002.07727 (2020) 14. U. Hacizade, I. Kaya, Ga based traveling salesman problem solution and its application to transport routes optimization. IFAC-Papers OnLine 51(30), 620–625 (2018) 15. Z. Hashim, W.R. Ismail, Applications of travelling salesman problem in optimizing tourist destinations visit in Langkawi, in Regional Conference on Science, Technology and Social Sciences (RCSTSS2014) (Springer, 2016), pp. 265–273 16. Q.M. Ha, Y. Deville, Q.D. Pham, M.H. Ha, On the min-cost traveling salesman problem with drone. Transp. Res. Part C: Emerg. Technol. 86, 597–621 (2018) 17. H. Iscan, M. Gunduz, An application of fruit fly optimization algorithm for traveling salesman problem. Procedia Comp. Sci. 111, 58–63 (2017) 18. M. Kang, J. Kyung, C. Lee, An algorithm for TSP with time dependent profits (TSPwTDP). 대한산업공학회추계학술대회논문집, 1–22 (2019) 19. M. Kaspi, M. Zofi, R. Teller, Maximizing the profit per unit time for the travelling salesman problem. Comp. Indus. Eng. 135, 702–710 (2019) 20. R. Lahyani, M. Khemakhem, F. Semet, A unified matheuristic for solving multi-constrained traveling salesman problems with profits. EURO J. Comput. Optimization 5(3), 393–422 (2017) 21. J. Liu, W. Li, Greedy permuting method for genetic algorithm on traveling salesman problem, in International Conference on Electronics Information and Emergency Communication (ICEIEC) (2018), pp. 47–51 22. S. Mathur, V.K. Mittal, N. Upadheye, K. Mathur, V. Singh, SCM strategy for sales augmentation using TSP algorithm and time bound marginal discount utility. Int. J. Supply Chain Manage. 7(1), 87–94 (2018) 23. M. Namazi, M.H. Newton, A. Sattar, C. Sanderson, A profit guided coordination heuristic for travelling thief problems, in Twelfth Annual Symposium on Combinatorial Search (2019) 24. S. Qin, S. Liu, Q. Zhang, The traveling salesman problem with profits considering timedependent cost and multi-trip mode, in 2017 36th Chinese Control Conference (CCC) (2017), pp. 2899–2903 25. V. Raman, N.S. Gill, Review of different heuristic algorithms for solving travelling salesman problem. Int. J. Adv. Res. Comp. Sci. 8(5) (2017) 26. A. Santini, A. Viana, X. Klimentova, J.P. Pedroso, Exact, heuristic and machine learning approaches to the probabilistic travelling salesman problem with crowdsourcing (2020) 27. S.C. Sarin, H.D. Sherali, J.D. Judd, P.F.J. Tsai, Multiple asymmetric traveling salesmen problem with and without precedence constraints: performance comparison of alternative formulations. Comput. Oper. Res. 51, 64–89 (2014) 28. F. Valdez, F. Moreno, P. Melin, A comparison of ACO, GA and SA for solving the TSP problem, in Hybrid Intelligent Systems in Control, Pattern Recognition and Medicine (Springer, Cham, 2020), pp. 181–189

26

N. Garg et al.

29. J. Xu, L. Pei, R.Z. Zhu, Application of a genetic algorithm with random crossover and dynamic mutation on the travelling salesman problem. Procedia Comp. Sci. 131, 937–945 (2018) 30. X. Yang, J.S. Wang, Application of improved ant colony optimization algorithm on traveling salesman problem, in 2016 Chinese Control and Decision Conference (CCDC) (2016), pp. 2156–2160 31. M. Zhang, J. Qin, Y. Yu, L. Liang, Traveling salesman problems with profits and stochastic customers. Int. Trans. Oper. Res. 25(4), 1297–1313 (2018)

Performance Assessment of DVR in a Solar-Wind Hybrid Renewable Energy Connected to Grid Using ANN S. K. Mishra, Kombe Shweta, and Behera Chinmay

Abstract The accessibility of renewable energy (RE) resources is getting cheaper day by day using technological advancement and the connectivity with the conventional grid is very common nowadays. The hybrid power generation has been rising exponentially in order to meet the power demand. In such constraint, the distribution lines are becoming more complex to analyse the real-time operation when there is any disturbances occur. It not only initiate several power quality problem at industrial and utility consumer end but also disturbs the performance of the line. Therefore, study of performance assessment is important in a grid connected solar-wind hybrid renewable energy in the presence of dynamic voltage restorer (DVR). In this paper, DVR is connected to the grid under abnormal condition as the generation of solar-wind hybrid power varies from time to time. To access the system reliability and effectiveness of the scheme, different types of shunt fault cases are conducted with and without DVR. The result of this paper shows that during small transient disturbances or the fault cases, DVR helps to mitigate the sudden decrease of voltage magnitude by injecting the reactive power. The wind turbine (WT) and solar photovoltaic (PV) duo system when connected to the conventional grid (CG) should follow standard grid codes (GC). Further to validate the fault case, an attempt is considered using artificial neural network (ANN). Keywords Dynamic voltage restorer (DVR) · Hybrid PV-WT · Fault assessment · GC · ANN

S. K. Mishra (B) · K. Shweta G H Raisoni University, Amravati, India e-mail: [email protected] B. Chinmay C.M.R.I.T, Bengaluru, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_3

27

28

S. K. Mishra et al.

1 Introduction The stochastic behaviour of solar energy has an adverse impact when it is grid connected due to the dynamic behaviour solar irradiation from time to time throughout the day. The grid stability disturbs during the sudden transient disturbance or in fault condition. As the solar alone cannot generate power throughout the day at all the time. Therefore, an attempt has been taken to access the fault including hybrid PV-WT. The merit of considering wind here is to have power generation when solar power is unable to produce during critical weather condition. On the reverse way when there is some disturbance of wind speed that time solar can manage the power generation alone. This hybrid PV-WT has been considered to provide the power to the grid in a coordinate manner so as to maintain the power to the grid. However, considering hybrid PV-WT the system complexity goes on increasing the reactive power may play a vital role when there is a disturbance on the line. In order to compensate the reactive power, a DVR is included in the line to manage the reactive power and maintain stability of the line. However, fault analysis in a coordinated system is very important in order to improve the power quality. In a structured power system, industrial customer fed power directly from a sub transmission or distribution system. Therefore, whenever there is a fault in the transmission system, it needs to be clear as soon as possible; otherwise, the system’s reliability will decrease and damage various equipment at the user end. In [1] during various power system fault condition doubly fed induction generator, WT-based DVR system is controlled by using phase compensation and fuzzy control technique to mitigate voltage sag, undervoltage and other shortcircuit-related issues. In WT system at the rotor side converter, the current should be at low transient condition, therefore, the stator voltage increases in order to maintain the field flux. Therefore, a series compensator is implemented to inject the voltage at the stator side which helps in monitoring grid voltage and provide reactive power support whenever it is required [2]. Therefore, the DVR plays an important role in supplying voltage at PCC to maintain the WT stator side voltage. In [3], DVR provides better transient and steady state response when it is controlled by feedback based voltage control technique. In feedback-based voltage control of DVR helps to enhance the simulation performance and quick response during instability condition of WTs. Unsymmetrical faults are taken into consideration to analyse the transient condition for a supercapacitor based WT. The fixed WT and doubly fed WT with series and shunt compensation helps in retain voltage stability, improve voltage sag, lower harmonics and flickers. To provide all day power at rural and remote location, the PV-WT hybrid system are proposed. In [4], the capacitance of a supercapacitor for doubly fed WT is evaluated by interpolation technique by using the voltage capacity relationship. A prototype DVR is developed and demonstrate in [5] for a PV-WT hybrid system. In [6], system efficiency is boosted by RE resources which is further compared two topologies (i.e. inverter with boost circuit and inverter with step up transformer) of PV GC system. In [7], model for PV system, inviters for grids, buck and boost converters are well explained. During fault recovery, the asymmetrical and symmetrical voltage sag as well voltage swell are analysed at PCC [8] and it is found

Performance Assessment of DVR in a Solar-Wind …

29

that voltage swell along with some transient occurs at the DC link of RE resources based power system. Owing to its wind speed fluctuation, the voltage at the grid imbalance from its desired value therefore in [9] DVR is adopted for a RE-based conventional GC system to manage the reactive power support. In [10], WTs with DVR are modelled in PSCAD/EMTDC and it is found that during faulty condition at point of common coupling, there is no variation in voltage and current. Thus, we give a scope of integrating WTs with CG. The fault detection approach using ANN-wavelet has been developed in order to detect various shunt fault including wind. This approach is very simple to detect various fault with lesser detection time [11, 12]. The DVR is basically connected between load and sources. During faulty condition at WT-PV GC, hybrid system with DVR can be helpful. Therefore, in this paper, DVR is adopted for WT-PV based conventional hybrid system to mitigate the voltage-related issues. In this paper, an approach of fault assessment study has been conducted with and without DVR. The PV cell alone cannot generate power at all the time due to weather condition. Therefore, combined PV-WT can generate power at all time and connected to grid through DVR. Combined approach of ANN and fuzzy is discussed in [13] for classification and detection of fault, but it fails to produce the precise results because of inaccuracies in input phasor data. The paper is organized in six sections. The introduction part has been discussed in Sect. 1. Section 2 briefs PV system. Section 4, the proposed structure of DVR, the simulation results are discussed in Sect. 5 and conclusion in Sect. 6.

2 PV System Over the past few decades, a lot of research has been done on renewable energy generation such as solar energy, wind energy, fuel cell energy, due to environmental and economic implications. Figure 1 shows the equivalent circuit of single diode PV cell. In this paper, 36 solar cells having nine modules (85 W each) are connected in series and parallel connection to form a PV array with 1000 w/m2 isolation using MATLAB/SIMULINK. The mathematical modelling part is referred from paper [7]. A summary of various PV system literatures are discussed in the Table 1 from the referred paper [14, 15]. The PV and IV standard characteristics of PV module at different irradiation level (200, 400, 600, 800 and 1000 W/m2 ) of power 285 W solar cell is shown in Fig. 2, and the Table 2 presents the specification of 5 KW power solar array system.

3 WT System Wind power provides the major share of renewable energy generation in all over the world. The wind exposed the huge potential of the various regions in matter of wind energy where mountain chains on the coasts create a natural corridor that

30

S. K. Mishra et al.

Fig. 1 Equivalent circuit of single diode solar cell

enhances the stability of winds. The most of the areas of coasts have the benefit of being next to where electricity is most demanded. The wind generator considered here is a gearless direct-driven PMSG. The production of a single FSIG cannot be controlled [16]. The summary of some review papers of WT system is presented in Table 3 from the referred paper [17–19]. Table 4 presents the parameter specification of WT subsystem model.

4 Proposed Structure of DVR Power and control circuits are two main basic blocks of DVR. The frequency, phase shift and magnitudes are the complex parameters which are injected to the system by DVR control unit. The voltage-dependent signals are generated by the switches of power circuit. Supercapacitors, fly wheels, batteries, etc., are used as energy storage system for DVR unit. Therefore, during voltage disturbances, the storage unit provides the compensation due to the reactive power, which is a primary function of DVR. Figure 3 depicts the proposed structure of PV-WT hybrid system with DVR. In this figure, we have simulated various shunt fault (L-G, LL-G, LLL-G and LLL) in the presence of DVR.

4.1 Operating Mode with DVR See Fig. 3.

Microgrid

Power plant

Approach

Reconfigurable solar converter

Author

Kim et al. [14]

Table 1 Summary of various PV system

Implementation

Hardware

Operation mode PV-to-Grid/PV-to-Battery/PV_Battery-to-Grid

Advantages A single electrical conversion system was used to operate in different operation modes. The solar plant is more easily controlled and due to the ease of operation its maximum energy can be transferred at low cost

Contribution

(continued)

Additional AC inductors have been added if AC filter inductance is not available for charging. Frame ratio-integral current has been proposed for energy control

Performance Assessment of DVR in a Solar-Wind … 31

Microgrid

PV solar plant

Approach

Reconfigurable solar converter/PV battery

Author

Perera et al. [15]

Table 1 (continued)

MATLAB

Implementation PV-to-Grid/PV-to-Batteries/Batteries-to-Grid/PV-Battery-to-Grid

Operation mode Slowly energy variation is achieved Various PV modules and small energy storage techniques were used. Maximum power conversion losses is achieved

Advantages

Ramp rate control technique used for power controlling The battery is used to control the ramp rate

Contribution

32 S. K. Mishra et al.

Performance Assessment of DVR in a Solar-Wind …

33

Fig. 2 PV and IV characteristics of PV module at different irradiations of 285 W solar cell

Table 2 Specification of 5KW power solar array system

Parameters

Specification

Module manufacture with model code

SUNTECH, STP285-20/Wfh

Power output of PV module

285 W

Module voltage (V oc )

38.3 V

Module short circuit current (I sc )

9.08 A

No. of series mode connected modules/string

4 Nos

No. of parallel mode connected/string

4 Nos

5 Simulation Result and Discussion Fault cases are considered with DVR and without DVR. The fault performance of the system has been improved with the DVR. Case-1: L-G Fault With DVR The fault study is an essential aspect in a transmission line, including DVR, to verify the system stability. In this section, faults are considered between DVR and grid

34

S. K. Mishra et al.

Table 3 Summary of some review papers of WT system Author

Approach

Implementation

Advantages

Contribution

Benkahla et al. [17]

DFIG/PI/SMC/AFLC

Hardware

Use of robust decoupled DFIG gives better performance

Designed the electrical power conversion system based on DFIG connected to the grid using stator and fed

Benbouhenni [18]

NSVM/FSVM/NSOSMC

MATLAB

Low ripple factor Low stator current harmonic distortion

Presented study is on fuzzy space vector modulation (FSVM) and neural space vector modulation (NSVM) inverter based on neuro second order sliding strategy in wind system

Chong et al. [19]

PMSG/SCIG/DFIG/SRG

–

This study showed that a permanent magnet synchronous generator is most suitable for wind power systems

Presented comparative analysis of various generators of wind turbine and discussed the various component required for wind energy system

shown in Fig. 3. Figure 4 depicts L-G fault occurs between the point 1 and point 2 (i.e. 0.08–0.15 s). Figure 4 is divided into four sub figures. Sub figure (i) depicts the normal V abc supply voltage to the grid. Sub figure (ii) shows the RMS voltage. A L-G fault (green colour) occurs in one phase, whose magnitude is decreased during the fault. However, the other two phases (red and blue colour is slightly disturbed) and behaves as normal. This clearly indicates the fault is in green phase, i.e. L-G fault. Sub figure (iii) shows the load voltage. Sub figure (iv) shows the injected voltage during the fault by means of DVR during the time from 0.08 to 0.12 s. The magnitude of voltage at 1 and 2 has been shown at the right side of Fig. 4 and

Performance Assessment of DVR in a Solar-Wind …

35

Table 4 Parameter specification of WT subsystem model Sl. No. Name of Simulink block

Parameters

1

Wind turbine

Nominal power output, mechanical Pm = 200 W; Base power, generator Pg = 222 VA; Base speed, wind N s = 15 m/s; Base speed, wind at Pm = 0.73 pu; Base rotational speed, N r = 1.2 pu

2

Permanent magnet synchronous machine Total no. of phases, T ph = 3; Size of the (PMSG) rotor = Round; Mechanical input, Pm in Torque; Present model: 0.8 Nm, 300 V DC, 3000 RPM

3

LC filter

Inductance L = 20 mH; Capacitive load C: Nominal voltage (Ph. to ph.) V nom. = 380 V; Nominal freq. f nom. = 50 Hz; Reactive power (Capacitive) Qc = 3 KVAr

Fig. 3 Proposed structure of PV-WT hybrid system with DVR

recorded as −182.917 and −162.914 and time as 0.050 and 0.150 s, respectively, and corresponding frequency is recorded as 10 kHz. The corresponding value ∆v/∆T has been recorded as 34.468 VKs. This has been verified that the L-G fault cleared within short span of time with the help sudden injecting voltage to that phase. Case-2: LL-G Fault With DVR In the figure shown in 5, a LL-G fault with DVR is considered. The sub figure (i) shows the three phase system voltage. The LL-G fault occurs between the points 1 and 2 of time from 0.080 to 0.15 s. This clearly indicates that there is a LL-G fault. Sub figure (iii) shows the load voltage and behaves as normal because of injecting DVR provides sufficient voltage during the fault time frame shown in sub figure (iv) of the main Fig. 5. Case-3: LLL-G Fault With DVR Figure 6 depicts a LLL-G fault with DVR. The sub figure (i) of Fig. 6 shows the three-phase system voltage. The three-phase fault occurs between the point 1 and 2 of time from 0.080 to 0.15 s. The sub figure (ii) shows the RMS voltage. Here, all the three-phase voltage red, blue and green phase magnitudes go on decreasing suddenly during the mentioned time frame. This clearly explains the three phase fault (LLL-G

Fig. 4 L-G fault with DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

36 S. K. Mishra et al.

Fig. 5 LL-G fault with DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

Performance Assessment of DVR in a Solar-Wind … 37

38

S. K. Mishra et al.

fault). The sub figure (iii) shows the load voltage and behaves as normal because DVR injects voltage during the fault time shown in sub figure (iv) and able to have normal load voltage. Case-4: L-G Fault Without DVR During the fault, the voltage magnitude of green phase goes on decreasing suddenly in Fig. 7. It happens because of the absence of DVR depicted from the sub figure (iv) of the main Fig. 7. Case-5: LL-G Fault Without DVR Figure 8 shows the LL-G fault without DVR. The sub figure (i) of the main Fig. 8 depicts the system voltage. Sub figure (ii) depicts the RMS voltage with the occurrence of double phase fault (LL-G fault) during the time frame from 0.08 to 0.15 s. During the fault, the two phase voltage magnitude, i.e. red and green phase goes on decreasing suddenly and magnitudes are out of our range. However, blue phase is slightly disturbed but remains as it is subsequently. Therefore, it is a case of double phase fault. If this fault sustain more time, it damages the equipment at the user end. The two-phase magnitudes are out of the range because of the absence of DVR. The absence of DVR is depicted from the sub figure (iv) of the main Fig. 8. Case-6: LLL-G Fault Without DVR Figure 9 shows the LLL-G fault without DVR. The sub figure (i) of the main Fig. 9 depicts the system votage. Sub figure (ii) depicts the RMS voltage with the occurrence of three-phase fault (LLL-G fault) during the time frame from 0.08 to 0.15 s. During these three-phase fault, all the phase voltage magnitude, i.e. red, blue and green phase goes on decreasing suddenly and magnitudes are out of our range except green. This fault is most dangerous, if it sustain more time then it may damage the equipment at the user end. Therefore, it is necessary to have DVR, which can inject the voltage during the three-phase fault which can clear the fault and save the equipment at the user end. The absence of DVR is depicted from the sub figure (iv) of the main Fig. 9. Case-7: ANN-Based fault detection model In this paper, feed forward back-propagation (FFBPNN) NN by the help of Lavenberg Marquardt approach is considered [20]. ANN approach has been validated in the presence of DVR. The ANN validation has been carried out by training and testing. Training has been carried out with the help of known targets and testing with unknown samples. The training portion has been made using inputs and matching targets. Input has been fed to ANN based FD (fault detection). The FD detector module is voltage signals in the presence of DVR. The targets are set at ANN detector are ‘0’ for normal supply operation and ‘1’ for fault. The optimal value of NN has been carefully chosen after training the network using hit and trial method. The final network has been chosen as a two-layered network and consists of ten neurons in the hidden layer using the transfer function tan-sig. The ultimate performance is found to be the mean square error set as 10−6 . The training accuracy is 100% in case of the FD module. Then NN is tested with samples, which have not been considered

Fig. 6 LLL-G fault with DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

Performance Assessment of DVR in a Solar-Wind … 39

Fig. 7 L-G fault without DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

40 S. K. Mishra et al.

Fig. 8 LL-G fault without DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

Performance Assessment of DVR in a Solar-Wind … 41

Fig. 9 LLL-G fault without DVR (i) three-phase system voltage (ii) sending end transmission line RMS voltage (iii) three-phase load voltage (iv) injective three-phase voltage

42 S. K. Mishra et al.

Performance Assessment of DVR in a Solar-Wind …

43

for training network. Figure 10 depicts the structure of ANN training-based fault detection assessment with the help of FFBPNN. Figure 11 depicts the L-G fault detection in the presence of DVR which has been considered at extreme end of solar irradiation at 1000 W/m2 and wind speed 15 m/sec, which can able to clear the fault at 150 ms time. It is observed that using ANN, the time taken to detect the fault is very fast as compared to conventional Simulink fault analysis. Therefore, ANN-based fault detection improves the performance of the proposed scheme.

Fig. 10 Structure of ANN training-based fault detection assessment

L-G fault including DVR 1 0.9 0.8

output of ANN

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

50

100

150

200

250

300

350

400

450

500

Time in millisec

Fig. 11 L-G fault at 10 km at wind speed (15 m/s) and irradiation of 1000 W/m2 in the presence of DVR

44

S. K. Mishra et al.

6 Conclusion Study of performance assessment is an important aspect in hybrid PV-WT grid connected system in the presence of DVR. In this paper, performance analysis in terms of various shunt fault (L-G fault, LL-G and LLL-G fault) have been simulated with and without DVR. It is observed that DVR compensates the voltage disturbances during the fault condition by injecting the reactive power through series transformer to the line. However, in the absence of DVR in a hybrid PV-WT grid connected system takes longer duration to mitigate the voltage disturbances as a result of which it may affect equipment at the user end. Further to validate the system reliability, an ANN based fault detection analysis has been performed at the maximum wind speed limit (15 m/s) of WT and irradiation of 1000 W/m2 in the presence of DVR, which can able to clear the fault at 150 ms time. Thus, the performance of hybrid PV-WT grid connected system in the presence of DVR using ANN takes less time to detect the fault and reduces the number of equipment trip at residential or commercial user /industry end. In the context of changing loading circumstances, this technique investigated the possible impact of DVR in a grid-tied hybrid solar PV-wind system. Three distinct case (L-G, LL-G and LLL-G) scenarios were created to evaluate the DVR efficacy in improving voltage regulation and hence system dependability. Each case scenario’s efficacy is assessed using both dynamic and transient reactions. The results revealed that in the presence of DVR, the voltage profile is successfully maintained, effectively counteracting the existence of excessive reactive power flow on the line and suppressing its negative consequences. As a result, DVR has been proven to be a requirement in the case of a hybrid system running in grid integrated mode in order to improve system performance.

References 1. A. Gowaid, A.S. Abdel-Khalik, A.M. Massoud, S. Ahmed, Ride-through capability of gridconnected brushless cascade DFIG wind turbines in faulty grid conditions—a comparative study. IEEE Trans. Sustain. Energy 4(4), 1002–1015 (2013) 2. O. Abdel-Baqi, A. Nasiri, Series voltage compensation for DFIG wind turbine low-voltage ride-through solution. IEEE Trans. Energy Convers. 26(1), 272–280 (2011) 3. R.A.J. Amalorpavaraj, P. Kaliannan, S. Padmanaban, U. Subramaniam, V.K. Ramachandaramurthy, Improved fault ride through capability in DFIG based wind turbines using dynamic voltage restorer with combined feed-forward and feed-back control. IEEE Access 5, 20494–20503 (2017) 4. K. Rajesh, A. Kulkarni, T. Anantha Padmanabha, ‘Modeling and simulation of solar PV and DFIG based wind hybrid system. Procedia Technol. 21, 667–675 (2015) 5. M.K. Dö¸so˘glu, A.B. Arsoy, Transient modeling and analysis of a DFIG based wind farm with super capacitor energy storage. Int. J. Elect. Power Energy Syst. 78, 414–421 (2016) 6. N.H. Woodley, L. Morgan, A. Sundaram, Experience with an inverter-based dynamic voltage restorer. IEEE Trans. Power Del. 14(3), 1181–1186 (1999) 7. A. Laudani, F. Riganti Fulginei, A. Salvini, Identification of the one-diode model for photovoltaic modules from datasheet values. Solar Energy 108, 432–446 (2014)

Performance Assessment of DVR in a Solar-Wind …

45

8. Y. Zhu, J. Yao, D. Wu, Comparative study of two stages and single stage topologies for grid-tie photovoltaic generation by PSCAD/EMTDC, in Proceedings of International Conference on Advanced Power System Automation and Protection, Beijing, China, pp. 1304–1309 (2011) 9. M. Shayestegan, Overview of grid-connected two-stage transformerless inverter design. J. Mod. Power Syst. Clean Energy 6(4), 642–655 (2018) 10. S. Talari, M. Shafie-Khah, G.J. Osório, J. Aghaei, J.P.S. Catalão, Stochastic modelling of renewable energy sources from operators’ point-of-view: a survey. Renew. Sustain. Energy Rev. 81(P2), 1953–1965 (2018) 11. S.K. Mishra, L.N. Tripathy, A novel relaying approach of combined discrete wavelet transform and artificial neural network-based relaying scheme in a unified power flow controller integrated wind fed transmission line. Int. J. Comput. Syst. Eng. Inder Sci. 5(5–6), 287–303 (2019) 12. S.K. Mishra, A Neuro-wavelet approach for the performance improvement in SVC integrated wind-fed transmission line. Ain Sham Eng. J. Elsevier 10(3), 599–611 (2019) 13. P.K. Dash, A.K. Pradhan, G. Panda, A novel fuzzy neural network based distance relaying scheme. IEEE Trans. Power Deliv. 15(3), 902–907 (2000) 14. H. Kim, B. Parkhideh, T.D. Bongers, H. Gao, Reconfigurable solar converter: a single-stage power conversion PV-battery system. IEEE Trans. Power Electron. 28(8), 3788–3797 (2013) 15. K. Perera, D. Wadduwage, S. Vasudevan, H. Lakshika, K. Boralessa, K.T.M.U. Hemapala, Reconfigurable solar photovoltaic systems: a review. Introducing Heliyon Chem. Eng. 6(11) (2020) 16. S. Marmouh, M. Boutoubat, L. Mokrani, MPPT fuzzy logic controller of a wind energy conversion system based on a PMSG, in 8th International Conference on Modelling, Identification and Control (ICMIC), pp. 296–302 (2016) 17. M. Benkahla, R. Taleb, Z. Boudjema, Comparative study of robust control strategies for a Dfig-based wind turbine. Int. J. Adv. Comp. Sci. Appl. (IJACSA) 7(2) (2016) 18. H. Benbouhenni, Comparative study between NSVM and FSVM strategy for a DFIG-based wind turbine system controlled by neuro-second order sliding mode. Majlesi J. Mechatronic Syst. 7(1), Feb 2018 19. C.H. Chong, A.R.H. Right, I. Ali, Wind turbine modelling and simulation using Matlab/SIMULINK, in IOP Conference Series: Material Science and Engineering (2021)

Development of IoT Middleware Broker Communication Architecture for Industrial Automation with Focus on Future Pandemic Possibilities: Industry 5.0 Sujit Deshpande

and Rashmi Jogdand

Abstract Internet of Things is becoming more prominent development domain globally. Developments in the Industrial IoT are necessary for consideration of future pandemic situations like COVID-19 pandemic. COVID-19 pandemic almost affected all domains throughout the world. Now it is necessary to think in a progressive manner and be ready to tackle such situations. One of the most affected sectors is the manufacturing sector, which failed to match the demand and supply chain due to frequent lockdowns. A development of recent Industry 5.0 is a result of a need for the industrial automation. Even though the challenge remains to lower the human– machine interfacing with the development of maximum IoT automation and we may call this as a building block of Industry 6.0 based on feedback from Industry 4.0. What if manufacturing industry is moulded like call centre support where workers can manage the production line by operating processes from anywhere? As an initial step, this paper presents the new middleware broker communication architecture design for MQTT and CoAP protocols, which can be a building block for Industry 6.0. This paper focuses on two key roles as the design of middleware broker communication architecture for MQTT and CoAP for industrial automation with consideration of common packet format parameters and development of deep learning algorithm “Multi-ProTest” for IIoT security analysis. This improves the overall system performance by reducing delay by 6.17% and jitter by 14.10%. Keywords IoT · IIoT · MQTT · COAP

S. Deshpande (B) · R. Jogdand Department of CSE, GIT, Belagavi, Karnataka, India e-mail: [email protected] R. Jogdand e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_4

47

48

S. Deshpande and R. Jogdand

1 Introduction The COVID-19 pandemic has produced critical concern in every area of life, supply chains (SC) especially. SCs encounter unrivaled weaknesses in lead times and so order levels, interruptions in networks components, and extreme demand variances [1]. The outbreak turned out to be a test for SCs concerning their potential to tolerate, versatility the potential to conform, as well as the potential to reestablish businesses and effectiveness after an interruption directing to the core role of strength in controlling the SCs in this unstable globe [2]. Right now, world-wide exploration is being carried out in the architecture design of smart as well as productive administration of the intelligent industry by way of pondering the IoT systems [3]. Industry 4.0 specifications have changed drastically the manufacturing arena by developing many solutions, just like Artificial Intelligence (AI), the Internet of Things (IoT), Cloud Computing, and Intellectual Processing [4]. Industry 5.0 is presently considered improving the exclusive resourcefulness of industry experts to team up with potent, intelligent, as well as legitimate equipment. Various technological visionaries acknowledge that Industry 5.0 will take back again the human being touch to the manufacturing industry. The cooperation amongst humans as well as machines strives to maximize production at a faster rate [5]. Even so, fresh concern rose because of COVID-19 outbreak circumstances that need remote access to production line to keep up the communal distance. The outbreak circumstances alert us to be prepared for future pandemic predicaments. As a core IoT development, a cross-layer-based clustering as well as routing algorithms are crafted by experts to lower network communication delay, latency as well as energy utilization [6]. Nevertheless, now there is a requirement of the middleware communication architecture design to satisfy Industrial IoT (IIoT) anticipations. Hence, this paper presents the new middleware broker communication architecture design and machine learning algorithm with training and testing for performance evaluation for IIoT.

2 Related Work IoT protocols allow applications to accumulate, save, and then process the data to resolve a range of challenges. IoT also strives to present secure and protect bidirectional interaction amongst adjoining equipment, including sensors, actuators, microcontrollers, etc. [7]. In [8], analysis conducted for the MQTT protocol and so it safeguards authentication mechanism, and so designs and execution for recognition of procedure determined by historic data nodes. The offered procedure can efficiently protect against numerous attacks. It is vital to present the MQ Telemetry Transport (MQTT) as well as Constrained Application Protocol (CoAP) protocol features relatively using the assortment of relevant simulator methods to test, evaluate, and validate the current strategies and enhance the effectiveness of prototypes [9]. In [10], StreamPipes connect is aim for domain analysts to balance and share time

Development of IoT Middleware Broker Communication …

49

series data as a component to the industry-proven open source IIoT analytics toolbox StreamPipes. Interacting with ongoing time series data from different equipment as well as detectors is an important process to allow data-driven decision planning in the Industrial Internet of Things (IIoT). In [11], module incorporates CoAP individualized messaging system and so it is equipped to offer end-to-end security with productivity as well as light-weight abilities. This can be applied in lightweight as well as productive communication predicament of Internet of Things (IoT) and Internet of Everything (IoE). In [12], author recommended a collection of qualitative and quantitative dimensions for benchmarking IoT middleware. Author utilized the publication—subscription of a substantial data set to evaluate two middleware platforms with FIWARE as well as oneM2M. In [13], the author proclaimed a new middleware communication architecture EMMA. The edge-enabled publish–subscribe middleware architecture processes the protocol message packets. It transparently migrates to MQTT clients and further brokers in close accessibility to boost QoS. In [14], a new load-balancing strategy presented by using a greedy algorithm which envisages the utilization of diverse MQTT messaging for every client that further minimizes inter-broker traffic. In [15], the communication latency of Open Platform Communication Unified Architecture (OPC UA) devices interchanging data through MQTT as a broker-based middleware is explored as well as compared to the client/server-based interaction. Study investigated the use case of one-to-many interaction for a crane model from the material handling area. In [16], a new alternative for data interoperability MQTT-CoAP Interconnector (MCI) is formulated at the application layer of IoT. It functions as a connection between the local message of MQTT and remote message of CoAP. In [17], research on discovering threats, as well as cyber-attacks is crucial facilities in the IIoT infrastructure is proclaimed. The research incorporates numerous machine learning algorithms to categorize the threat events incorporating different attacks as well as IIoT hardware failures. In [18], the influences of attacks are explored and so are progressively powerful with elevating precision of the regulation model. Also, several tradeoffs amongst control efficiency as well as security effectiveness of DRL-based IIoT controllers are assessed and many upcoming research recommendations are specified to obtain machine learning use in IIoT systems.

2.1 Industrial IoT Middleware Smart Industrial IoT (IIoT) [19, 20] is an enticing tool in the framework of the 4th industrial innovation, Industry 4.0. It is dependent on the communication amongst computer-integrated manufacturing as well as Artificial Intelligence (AI) solutions that can provoke to a lot of prospects for many smart industrial services [21]. Author regarded the EdgeX Foundry IIoT middleware system as an improvement engine amongst the field devices as well as business applications. Author also deemed security as an important component and so solutions to circumvent or reduce the likelihood of many security risks [22]. Heterogeneity is amongst the key challenges

50

S. Deshpande and R. Jogdand

for the application of the Industry 4.0. This is because of the assortment in the obtainable robots and the IIoT devices. Such machines work with several programming languages as well as communication protocols. To generate the integration of many of these machines convenient, author recommended TalkRoBots, a middleware that enables heterogeneous robots as well as IIoT devices to communicate collectively and shift data in a clear way [23]. In the next Sect. 3, the proposed middleware communication architecture and packet formats for CoAP and MQTT protocols are presented with proposed algorithm execution.

3 Proposed Architecture In this paper, we design middleware broker communication architecture for Industrial IoT communication model for the CoAP and MQTT protocol request/response for Industrial IoT. The key focus is on to lower the request processing time as well as the request routing paths. The proposed design is targeted for multiple protocol support where common gateway and device discovery can be used as a common entity. Figure 1 shows the proposed IIoT middleware architecture. As security is an important element for Industrial IoT, the new deep learning algorithm named “Multi-ProTest” is developed to analyse the threat data sets for Mirai attacks as Ecobee-Thermostate, Samsung-XCS7. As presented in Fig. 1, industrial plant devices and equipment sensors request can be discovered with device discovery reference gateways module where all available devices/sensors are pre-defined at the time of plant commissioning. The request handler forwards protocol priority-specific CoAP and MQTT requests to middleware broker panel. For CoAP requests, client data indexing panel plays an important role in registering the request and response time to identify QoS parameters. As per CoAP messaging formats, confirmable requests can acknowledge the process execution and for non-confirmable requests, “Fire and Forget” activates sensors from sensor panel, i.e., if the plant smoke detector/sensor sends the requests to smoke quencher sensor, then there is no need of acknowledgement and direct sensor action is expected. For MQTT priority requests, resource directory maintains the MQTT request/response queue in-line to publisher–subscriber execution with respect to sensor panel. The proposed design of middleware communication architecture can handle CoAP and MQTT protocol simultaneously with a single gateway which can be controlled remotely. By considering the common parameters of packet formats for CoAP and MQTT, the communication overhead can be lowered. The packet format for CoAP and MQTT protocol is shown in Figs. 2 and 3 simultaneously. For CoAP, packer header contains message ID and actual request/message is processed by payload. Similarly, for MQTT packet, two parameters can be used are Message Type (where Message ID can be assigned explicitly) and actual request/message is processed by payload.

Development of IoT Middleware Broker Communication …

51

Fig. 1 Proposed IoT middleware broker communication architecture for CoAP and MQTT

Fig. 2 CoAP packet format

Hence, by using the two common parameters, requests can be routed via middleware broker for efficient communication. In IIoT, the security enhancement can be done by message encryption. Further, for CoAP and MQTT protocols proposed Multi-ProTest algorithm can be used for security threat detection for large IIoT data sets.

52

S. Deshpande and R. Jogdand

Fig. 3 MQTT packet format

The proposed Multi-ProTest algorithm is executed for analysis of EcobeeThermostate, Samsung-XCS7 data sets using Python library and using proposed design of middleware communication architecture. Initially, data sets are used for device discovery, and CoAP and MQTT request parameters are already stored in data sets for analysis of mirai attacks. With proposed architecture, even more types of protocols can be processed. As shown in Fig. 3 of the MQTT packet format, the Message Type of incoming request can be assigned by message ID and can be uniformly processed with CoAP Message ID. The payload parameter is common for both MQTT and CoAP (refer Figs. 2 and 3) protocol which holds the request message. The pre-defined sensor/device priorities can be considered for processing higher priority request first. The benefit of processing common packet parameters is an easy encryption of messages, which may add more security for middleware broker communication architecture. Algorithm: Multi-ProTest Input

:

IoT device/sensor requests, CoAP message ID, CoAP payload, MQTT Message Type, and MQTT payload device discovery and attack data set: Ecobee-Thermostate, Samsung-XCS7

Output

:

Evaluation of request traffic for security analysis

Assumption

:

Only MQTT and CoAP protocol are considered here (continued)

Development of IoT Middleware Broker Communication …

53

(continued) Algorithm: Multi-ProTest 1

:

Parameter Initialization Array ProtocolType []; Array MessageType[]; Array MessageID[]; Payload[]; PublishRequest[]; SubscribeRequest[]; ResourceDirectoryQueue[]; CoAPDataIndex[]; Array RequestPriority []; Array HQueue []; //High priority queue Array MQueue [];//Medium priority queue Array LQueue [];//Low priority queue addRequest ();

2

:

for each ProtocolRequesti do if RequestPriority == ‘High’ then addRequest (‘High’) to HQueue []; else if RequestPriority == ‘Medium’ then addRequest (‘Medium’) to MQueue []; else addRequest (‘Low’) to LQueue [];

3

:

If ProtocolType == MQTT then redirect request to MQTT Request Handler for each MessageTypei assign MessageIDi && payload[x] send PublishRequest[] && get SubscribeRequest[] store data to ResourceDirectoryQueue[] else redirect request to CoAP Request Handler Save MessageIDi to CoAPDataIndex[] Record MessageIDi && payload[y]

4

:

if CoAPRequest = “Confirmable” send “Acknowledgement Status” to CoAPDataIndex[] else mark Non-confirmable status == “Fire and Forget”

5

:

for each MessageIDi in RequestPriority [] do MessageIDi + = PredefinedProtocolPriorityi && payload[x] + payload[y] end for

6

:

initiate training and testing

7

:

[MessageIDi if RequestPriority [] == null]

8

:

Go to step – 2

54

S. Deshpande and R. Jogdand

4 Result and Analysis This section discusses the performance parameters for proposed Multi-ProTest algorithm. As discussed earlier, the experimental scenario is executed for the array of devices, sensors, and actuators. The classification of threats for CoAP and MQTT protocols is done based on different types of attacks like Mirai attack. The common packet parameter execution is done without applying Multi-ProTest algorithm where each protocol request is just considered with priority level of sensor request (high, medium, and low) and tested for delay (Fig. 4) and jitter (Fig. 5). Delay is calculated for end-to-end request processing time while jitter is time taken for request processing between two packets. In other scenario, the module is tested with execution of MultiProTest algorithm with a simultaneous process of the payload for CoAP and MQTT request from devices/sensors. The training and testing of data set is carried out by means of various test scenarios as discussed in the next Sect. 4.1 of this paper.

4.1 Results and Analysis Multi-ProTest algorithm is benchmarked using format as given in Table 1. As per the benchmarking template given in Table 1, many test IDs executed to identify the performance of the proposed Multi-ProTest algorithm with proposed middleware Fig. 4 Performance analysis of delay for proposed middleware design with and without execution of Multi-ProTest

0.87 0.86 0.85 0.84 0.83 0.82 0.81 0.8 0.79

Delay With MultiProTest (ms) Delay Without Multi-ProTest(ms) Request 1 (Medium Priority)

Fig. 5 Performance analysis of jitter for proposed middleware design with and without execution of Multi-ProTest

Request 2 (High Priority))

Request 3 (Low Priority))

0.9 0.88 0.86 0.84 0.82 0.8 0.78 0.76 0.74 0.72

Jitter With MultiProTest (ms) Jitter Without MultiProTest(ms) Request 1 (Medium Priority)

Request 2 (High Priority))

Request 3 (Low Priority))

Development of IoT Middleware Broker Communication …

55

communication design for CoAP and MQTT protocol. Following graph represents the evaluation of delay for CoAP and MQTT requests with and without proposed Multi-ProTest algorithm. Figure 4 depicts that, without execution of Multi-ProTest the delay in request processing for multiple protocols (CoAP and MQTT) is more, whereas with execution of Multi-ProTest lowers the unnecessary hold-off time of broker because of serialized payload. The benefit of proposed middleware communication architecture design is that the common packet parameter processing lowers the response time. For MQTT requests, publish and subscribe routs through resource directory where Message ID works like indexing the resource activation and actual packets are queued in nature, whereas in case of CoAP requests, only confirmed request gets indexed and non-confirmable requests follows “Fire and Forget”. So, delay and jitter time lowers. This further increases the security of IIoT system as delay and/or jitter mainly occurs when there is a threat. So, identification of threat becomes simple and low cost.

5 Conclusion In this paper, we proposed a new design of middleware communication architecture and considered the two IoT protocols, one is CoAP and other is MQTT. We also discussed the differences between message packet formats to identify the common parameters which can be utilized for serialized request handling for multiple protocols. We proposed a new Multi-ProTest algorithm for middleware broker communication framework. As delay and jitter are commonly considered for security evaluation of IoT systems, proposed algorithm lowers the delay and jitter by considering the header parameters like message ID and payload for request processing. Also, algorithm redirects specific IoT protocol to specific protocol broker based on incoming protocol request type and priorities. The performance analysis shows that proposed middleware broker communication architecture with proposed algorithm lowers the delay by 6.17% and jitter by 14.10% for multiple protocols.

6 Future Scope In future, the Industry 6.0 can be initiated with more prominent development in the area of multi-platform, multiple protocols like XMPP, AMQP, DDS, and WebSocket to take IIoT to the next level of fully remote operations. Thus, focusing on the possibilities of future pandemic situations, manufacturing can be a seamless process.

56

S. Deshpande and R. Jogdand

Table 1 Proposed algorithm benchmarking for CoAP and MQTT protocol Test type

Test ID # 1

Aim

To identify performance of Multi-ProTest algorithm for CoAP and MQTT

Test category Performance Test type Load testing

Test description

Test scenario 1 • Fire requests via MQTT broker without Multi-ProTest algorithm • Fire requests via MQTT broker with Multi-ProTest algorithm • Fire requests via COAP broker without Multi-ProTest algorithm • Fire requests via CoAP broker with Multi-ProTest algorithm

Execution steps script

ensure that { Device discovery dataset ! = null Protocol priority table ! = null when { (.) CoAP and MQTT request fired at same time (!) MQTT message Type → MessageID; (!) CoAP MessageID (!) MQTT message payload (!) CoAP message payload (!) Serialized message payload corresponding to PRIORITY_SETTING; } then { (!) without Multi-ProTest_Activation: the entity send the messages and note the delay in ms; the entity send the messages and note the jitter in ms; (!) with Multi-ProTest_Activation: the entity send the messages and note the delay in ms; the entity send the messages and note the jitter in ms; }

Output: 1

Average CoAP and MQTT delay in (ms)

Measurements Overall delay without multi-protest

Overall Delay with Multi-ProTest

Remark

Recorded request time delay

0.86 ms

0.81 ms

Delay is reduced with Multi-ProTest

Output: 2

Average CoAP and MQTT jitter in (ms)

Measurements Overall jitter Overall jitter with Multi-ProTest without Multi-ProTest

Remark

Recorded request time delay

Jitter is reduced with Multi-ProTest

0.89 ms

0.78 ms

Development of IoT Middleware Broker Communication …

57

References 1. D. Ivanov, A. Dolgui, OR-methods for coping with the ripple effect in supply chains during COVID-19 pandemic: managerial insights and research implications. Int. J. Prod. Econ. 232, 107921 (2021) 2. M.D. Wood, E.M. Wells, G. Rice, I. Linkov, Quantifying and mapping resilience within large organizations. Omega 87, 117–126 (2019) 3. A. Rahman et al., SDN–IoT empowered intelligent framework for industry 4.0 applications during COVID-19 pandemic, in Cluster Computing (2021), pp. 1–18 4. Maddikunta, Praveen Kumar Reddy, et al. “Industry 5.0: a survey on enabling technologies and potential applications.“ Journal of Industrial Information Integration (2021): 100257. 5. Y.K. Leong et al., Significance of Industry 5.0, in The Prospect of Industry 5.0 in Biomanufacturing (CRC Press, 2021), pp. 95–114 6. H.B. Mahajan, A. Badarla, A.A. Junnarkar, CL-IoT: cross-layer Internet of Things protocol for intelligent manufacturing of smart farming. J. Ambient. Intell. Humaniz. Comput. 12(7), 7777–7791 (2021) 7. B. Mishra, A. Kertesz, The use of MQTT in M2M and IoT systems: a survey. IEEE Access 8, 201071–201086 (2020) 8. H. Yujia, H. Yongfeng, C. Fu,Research on node authentication of MQTT protocol, in 2020 IEEE 11th International Conference on Software Engineering and Service Science (ICSESS) (IEEE, 2020) 9. M. Bansal, Performance comparison of MQTT and CoAP protocols in different simulation environments, in Inventive Communication and Computational Technologies (Springer, Singapore, 2021), pp. 549–560 10. P. Zehnder et al., StreamPipes connect: semantics-based edge adapters for the IIoT, in European Semantic Web Conference (Springer, Cham, 2020) 11. A. Bhattacharjya et al., CoAP—application layer connection-less lightweight protocol for the Internet of Things (IoT) and CoAP-IPSEC Security with DTLS Supporting CoAP, in Digital Twin Technologies and Smart Cities (Springer, Cham, 2020), pp. 151–175 12. C. Pereira et al., Benchmarking Pub/Sub IoT middleware platforms for smart services. J. Reliable Intell. Environ. 4(1), 25–37 (2018) 13. T. Rausch, S. Nastic, S. Dustdar, Emma: distributed QOS-aware MQTT middleware for edge computing applications, in 2018 IEEE International Conference on Cloud Engineering (IC2E) (IEEE, 2018) 14. A. Detti, L. Funari, N. Blefari-Melazzi, Sub-linear scalability of MQTT clusters in topic-based publish-subscribe applications. IEEE Trans. Netw. Serv. Manage. 17(3), 1954–1968 (2020) 15. H. Raddatz et al.,Evaluation and extension of OPC UA publish/subscribe MQTT binding, in 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), vol. 1 (IEEE, 2020) 16. M. Dave, J. Doshi, H. Arolkar, MQTT-CoAP interconnector: IoT interoperability solution for application layer protocols, in 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) (IEEE, 2020) 17. G.E.I. Selim et al., Anomaly events classification and detection system in critical industrial internet of things infrastructure using machine learning algorithms. Multi. Tools Appl. 80(8), 12619–12640 (2021) 18. X. Liu et al., On deep reinforcement learning security for Industrial Internet of Things. Comp. Commun. 168, 20–32 (2021) 19. Y. Tian et al.,A blockchain-based machine learning framework for edge services in IIoT. IEEE Trans. Indus. Inf. (2021) 20. M. Wu et al., Multi-label active learning from crowds for secure IIoT. Ad Hoc Netw. 121, 102594 (2021) 21. F. Banaie, M. Hashemzadeh, Complementing IIoT services through AI: feasibility and suitability, in AI-Enabled Threat Detection and Security Analysis for Industrial IoT (Springer, Cham, 2021), pp. 7–19

58

S. Deshpande and R. Jogdand

22. J. John et al., DSLs and middleware platforms in a model-driven development approach for secure predictive maintenance systems in smart factories, in International Symposium on Leveraging Applications of Formal Methods (Springer, Cham, 2021) 23. D. Marcheras, et al.,A new middleware for managing heterogeneous robot in ubiquitous environments, in 2020 8th International Conference on Wireless Networks and Mobile Communications (WINCOM) (IEEE, 2020)

Graceful Labeling of Hexagonal and Octagonal Snakes Along a Path Lalitha Pattabiraman and Nitish Pathak

Abstract This paper focuses on rows of Hexagons linked to one another forming Hexagonal Snakes (HSn ). These Hexagonal Snakes are aligned as vertical columns by connecting them to every vertex of a Path. The vertices and edges of this graph are provided with numerical values using Graceful Labeling and satisfies the condition of Alpha Valuation. Keywords Hexagon · Edges · Vertices · Path · Octagon

1 Introduction Hexagons are highly compact together when as a unit and possess a number of unique features. These polygons have excellent geometrics, being the most convenient symmetrical figures which lie between circles and polygons. These six sided polygons can form a strongly bound design and are more adept in allowing close packing leading to zero wastage of the space and resulting in maximum utility [1, 2]. In this article, various models of Hexagonal Snakes are designed and labeled proving that these structures are Graceful Graphs. An extension of these structures is developed by constructing Octagonal Snakes and further generalized to C n snakes.

L. Pattabiraman (B) Department of Mathematics, SRM Institute of Science and Technology, Ramapuram, Tamilnadu, India e-mail: [email protected] N. Pathak Department of Information Technology, Bhagwan Parshuram Institute of Technology (BPIT), GGSIPU, New Delhi, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_5

59

60

L. Pattabiraman and N. Pathak

2 Definitions 2.1 Hexagonal Snake A Hexagonal Snake HSn can be obtained from a path u1 , u2 , u3 , …, un by joining ui to two new vertices vi,j and vi,j-1 and then joining them, respectively, with wi,j+3 and wi,j+1 and then bringing them together to the vertex vi,j-3 . Hence, every vertex of a path is connected by a C 6 Snake.

2.2 Octagonal Snake An Octagonal Snake OSn can be obtained from a path u1 , u2 , u3 , …, un by joining ui to two new vertices vi,j and vi,j-1 and then joining them, respectively, with two vertices wi,j+3 and wi,j+1 and these are in turn joined to two more corresponding vertices zi,j-3 and zi,j-2 , then bringing them together to the vertex vi,j-4 . Hence, every vertex of a path is connected by a C 8 Snake.

2.3 Cn Snake C n Snake C n S n can be obtained from a path u1 , u2 , u3 , …, un by joining every vertices of the Path to C n chains, where n ≥ 8 and n is even. The number of C n chains depends upon the number of vertices of the Path.

2.4 Graceful Labeling Graceful labeling of a graph which has m edges is done by assigning values to the nodes of the graph such that each vertex takes an unique label from 0 to m resulting in distinct numbering of edges of the graph which is done by finding out the absolute difference the values of the corresponding vertices which are on either side of that edge, showing that the magnitude of edges lies between 1 and m inclusive [3, 4].

2.5 Alpha Labeling An α-labeling is a Graceful labeling with satisfies the property that if there exists an integer λ so that for each edge vw either f (v) ≤ λ ≤ f (w). This λ must be smaller of

Graceful Labeling of Hexagonal and Octagonal Snakes Along a Path

61

the two vertex labels that yield the edge label as one. If a graph is bipartite which can show Graceful labeling then it is known as α-labeling of the graph.

2.6 Odd Even Graceful Labeling A Graph G is said to be Odd Even Graceful if the graph contains m edges then there exist an injection g from the vertex set of G to {1, 3, 5, …, 2m + 1} so that every edge is allocated with the number which is the difference between their end vertices. Theorem The Graph G1 = Pt (HS n ), where n ≥ 6 consisting of isomorphic Hexagonal snakes along a path is a graceful graph. Proof The Hexagonal snake consists of a set of Hexagonal chains, attached to one another through a vertex. These chains are tethered to every vertex of a path. Let ‘m’ be the number of vertices and n be the number of edges of this graph and let ‘t’ be number of vertices along the path (Pt ). The so formed graph Pt (HSn ) is a set of isomorphic Hexagonal Snakes connected to every vertex of the path (Pt ), thus leading to t number of snakes. Graceful labeling technique is applied to this graph so that the vertices and edges follow the necessary and sufficient condition of Graceful labeling [5, 6]. The labeling of vertices is as follows: f (ali ) = i;   where l = 1, s = 1, 2, 3, . . . , m−t . 2t f (ali ) = i + s;   where l = {2, 3, …, t}; i = 25 (t − 1); ∀t ≥ 2; s = 1, 2, . . . , m−t . 2t The labeling of the path is as follows: 



t f (xi ) = n−25i; ∀i = 0, 1, 2, . . . , − 1 2   t f (yi ) = 25i; ∀i = 0, 1, 2, . . . , − 1 2   f bl j = {(n − 1) − 25 j} − k;

where l = {1, 2, …, t}; j = {0, 1, 2, …, t – 1}; k = {0, 1, 3, 5, 6, 7, 8, …, 2t + 1}. The edges consists of labels between n and 1, inclusive satisfying the condition of Graceful Labeling.

62

L. Pattabiraman and N. Pathak

Hence, it is proved that the Hexagonal Snakes along a Path G1 = Pt (HSn ), satisfies the condition of Graceful Labeling. In this theorem, Graceful Labeling has been applied to a series of Hexagons joined one with the other through a vertex forming Hexagonal Snakes. These Hexagonal Snakes are attached along a Path. Graceful Labeling of vertices and edges are shown for these Hexagonal Snakes which are attached along the path. Also, it is found that this Graph admits Alpha Labeling. Thus, the Graph G1 = Pt (HSn ), where n ≥ 6 consisting of isomorphic Hexagonal snakes along a path is a graceful graph [7, 8]. Illustration The Graph G1 = P10 (HSn ), consisting of Isomorphic Hexagonal snakes along a path (Fig. 1). Construction of Isomorphic Hexagonal Snakes is as follows: The vertex set consists of V = {0, 1, …, 201} and the edge set consist of E = {1, 2, …, 201}. Therefore, we get p = 170 and q = 201. This graph consists of eight isomorphic snakes of hexagons connected on a path. The vertices are labeled in such a way that it follows graceful labeling. The labels appear in a systematic pattern. The vertex labels and the edge labels are found to be distinct thus proving that graceful labeling can be applied in double hexagonal snake. Also, it is found that this Graph admits Alpha Labeling. An αlabeling is a Graceful labeling with satisfies the property that if there exists an integer λ so that for each edge vw either f (v) ≤ λ ≤ f (w). This λ must be smaller of the two vertex labels that yield the edge label as one.

Fig. 1 Graceful labeling of isomorphic hexagonal snake along a path P10 (HSn )

Graceful Labeling of Hexagonal and Octagonal Snakes Along a Path

63

Theorem The Graph G2 = Pt (OS n ), where n ≥ 8 consisting of isomorphic Octagonal snakes along a path admits graceful labeling. Proof The Octagonal Snake is constructed by joining various Octagons together through a vertex. This snake is consecutively attached as chains to a path. Let ‘m’ be the number of Vertices and n be the number of Edges of this Octagon. Let ‘t’ be the number of snakes which are attached to the path Pt . The so formed graph Pt (OSn ) is a set of isomorphic Octagonal Snakes connected to every vertex of the path (Pt ), thus leading to t number of snakes. Graceful labeling technique is applied to this graph so that the vertices and edges follow the necessary and sufficient condition of Graceful labeling. The vertices are gracefully labeled as follows: f (ali ) = i + s;   . where l = 1, i = 0, s = 1, 2, 3, . . . , m+5t 2t   f bl j = {(n−32}−k; where l = {1, 2, …, t}; j = {0, 1, 2, …, t − 1}; k = {0, 1, 2, 3, …, 3t + 3}. f (ali ) = i + s;   where l = {2, 3, …, t}; i = 32(t − 1), ∀t ≥ 2; s = 1, 2, 3, . . . , m+5t . 2t The edges takes up the values between 0 and n, inclusive accepting the condition of  Graceful Labeling  f  (u) − f  (v), ∀u, v ∈ V (G). Hence, the Graph G2 = Pt (OSn ), where n ≥ 8 consisting of isomorphic Octagonal snakes (Fig. 2) along a path admits graceful labeling. Illustration Theorem The Graph G3 = Pt (CnSn), where n ≥ 2k, k = {1, 2, …} consisting of isomorphic Cycle Snakes admits Graceful Labeling. Proof The Cn Snake is constructed by joining numerous Cn cycles together connected through a vertex. This snake is consecutively attached as chains or snakes to a path. Let ‘m’ be the number of Vertices and n be the number of Edges of this Cn snake. Let ‘t’ be the number of snakes which are attached to the path Pt . The so formed graph Pt (C n S n ) is a set of isomorphic C n Snake connected to every vertex of the path (Pt ), thus leading to t number of snakes. Graceful labeling technique is applied to this graph so that the vertices and edges follow the necessary and sufficient condition of Graceful labeling. The vertices are labeled gracefully as follows: f (ali ) = i + s;

64

L. Pattabiraman and N. Pathak

Fig. 2 Graceful labeling of isomorphic octagonal snakes along a path

  where l = 1, i = 0; s = 1, 2, 3, . . . , m+5t . 2t   f bl j = {(n − m)/t}−k; where l = {1, 2, …, t}; j = {0, 1, 2, …, t − 1}; k = {0, 1, 2, 3, …, 3t + f }, ∀ f = {0, 1, 3, …}. f (ali ) = i + s;   . where l = {2, 3, …, t}; i = m/t (t − 1); ∀t ≥ 2; s = 1, 2, 3, . . . , m+5t 2t The edges takesup the values between n and 0, inclusive accepting the condition of  Graceful Labeling  f  (u) − f  (v), ∀u, v ∈ V (G). Hence, the Graph G3 = Pt (C n S n ), where n ≥ 2k, k = {1, 2, …} consisting of isomorphic Cycle Snakes admits Graceful Labeling. Theorem The Graph G4 = Pt (HS n ), where n ≥ 6 consisting of isomorphic Hexagonal snakes along a path admits odd even graceful labeling.

Graceful Labeling of Hexagonal and Octagonal Snakes Along a Path

65

Proof The Hexagonal snake consists of a set of Hexagonal chains, attached to one another through a vertex. These chains are tethered to every vertex of a path. Let ‘m’ be the number of vertices and n be the number of edges of this graph and let ‘t’ be number of vertices along the path (Pt ). The so formed graph Pt (HSn ) is a set of isomorphic Hexagonal Snakes connected to every vertex of the path (Pt ), thus leading to t number of snakes. Odd Even Graceful labeling technique is applied to this graph so that the vertices and edges follow the necessary and sufficient condition of this labeling. The vertices are numbered with Odd Even Graceful Labeling as follows: f (ali ) = i + s;   where l = 1, i = 0; s = 2, 3, . . . , n2 .   f bl j = {(n−1) − 50 j}−k; where l = {1, 2, …, t}; j = {0, 1, 2, …, t − 1}; k = {0, 2, 6, 10, …, 3t + 3}. f (ali ) = i + s;   where l = {2, 3, …, t}; i = 50(t − 1), ∀t ≥ 2; s = 2, 3, . . . , n2 . The edges takes up the values between 0 and n, inclusive accepting the condition of  Graceful Labeling  f  (u) − f  (v), ∀u, v ∈ V (G). Hence, the Graph G4 = Pt (HSn ), where n ≥ 6 consisting of isomorphic Hexagonal snakes along a path admits odd even graceful labeling (Fig. 3).

3 Conclusion Thus, in this research work, Graceful labeling of Isomorphic Hexagonal Snakes, Graceful labeling of Isomorphic Octagonal Snakes, Graceful labeling of Isomorphic Cn Snakes and Even Odd Graceful labeling of Hexagonal Snakes have been defined and designed. The generalized label values for the vertices and edges are formulated. Also, Alpha labeling is proved for Graceful labeling of Hexagonal Snakes along a path.

66

L. Pattabiraman and N. Pathak

Fig. 3 Odd even graceful labeling of hexagonal snakes

References 1. L. Pattabiraman, L. Tamilselvi, Graceful labeling of hexagonal snakes. AIP Conf. Proc. 2112, June 2019. ISSN 1551-7616 2. L. Pattabiraman, L. Tamilselvi, Analysis of psychology of students by graphical representation. IOP J. Phys. Conf. Ser. 1377, Nov 2019, ISSN 1742-6596 3. L. Pattabiraman, L. Tamilselvi, Hexagonal snakes with alpha labeling along a path. IOP J. Phys. Conf. Ser. 1504, May 2020. ISSN 1742-6596 4. G.H.J. Lanel, H.S.S.P. Jayawardena, A study on graph theory properties of on-line social networks. Int. J. Scien. Res. Publ. 10(3), 267, March 2020. ISSN 2250-3153 5. A. Gayathri, A. Muneera, T.N. Rao, T.S. Rao, Study of various dominations in graph theory and its applications. Int. J. Scien. Technol. Res. 9(2), Feb 2020. ISSN 2277-8616 6. P.L.K. Priyadarsini, A survey on some applications of graph theory in cryptography. J. Discr. Math. Sci. Cryptography, 2 Jun 2015 7. A. Chakraborty, T. Dutta, Application of graph theory in social media. Int. J. Comp. Sci. Eng. 6(10), 722–729, October 2018. https://doi.org/10.26438/ijcse/v6i10.722729 8. J. Fang, Unified graph theory-based modeling and control methodology of lattice converters, September 2021. Electronics 10(17), 2146. https://doi.org/10.3390/electronics10172146

Comparison and Analysis of Various Autoencoders Aziz Makandar and Kanchan Wangi

Abstract The autoencoder is family of deep neutral network which learns to reconstruct its input. It has three main parts encoder, code, and decoder. Autoencoders are effective unsupervised learning method which encode an input into a lower dimensional representation. This representation input consist of input as features are useful for image processing applications. The size of hidden representation is lesser then the original image that’s under complete autoencoder. If the size is greater than the hidden representation that is over complete autoencoder. This paper compares and evaluates many architectures of autoencoders model. Keywords Autoencoder · Simple autoencoder · Deep autoencoder · Convolution autoencoder

1 Introduction Autoencoders have basically three components encoder, code, and decoder [1, 2]. Its learns to reconstruct images by first transforming an input into a hidden representation of its features [3–5]. These representations can be used to compute semantic similarity in the images [6–8]. Encoder part which is learn to represent the features into an input as a vector in latent space [9, 10]. This is followed by code part, it is compact summary or compression of the input, and it is called as latent space representation [11]. Code calculate the k/n ratio where k is number of bytes, and n is number of noises in image. Figure 1 shows that encoder- decoder architecture. Autoencoder is presented the input image same as the output image which is reconstructed [12]. Decoder deconvolution is fully connected layer of the network which is reverse operation of encoder convolution in the architecture. A. Makandar · K. Wangi (B) Karnataka State Akkamahadevi Women’s University, Vijayapura, India e-mail: [email protected] A. Makandar e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_6

67

68

A. Makandar and K. Wangi

Fig. 1 Encoder–decoder architecture

2 Datasets For this work, we are using two datasets, 1. MNIST 2. CIFER 10. MNIST which is consisting of 28 * 28 Gy scale images and which is having 50,000 training images and 10,000 testing images. Totally, it is consists of 60,000 images of 0–9 Gy digits. Second dataset is CIFER 10 dataset which is subset of CIFER 100 dataset which is consisting of total 80 million tinny images of 32 * 32-pixel real-world color images. CIFER ten subset of dataset which is having total ten different and each category consists of training and testing images. For this work, we are using CIFER four datasets from original datasets.

Comparison and Analysis of Various Autoencoders

69

3 Evaluate Images For this research work, we are compares and analyzing three models of autoencoders architecture that are simple, deep, and convolutional autoencoders. IDE is used Jupyter Notebook, language python, and libraries are TensorFlow, Keras, NumPy, and scikit-learns.

3.1 MODEL 1: Simple Autoencoder Simple autoencoder is having one hidden layer with 50 epochs is having mean square error (MSE) value as 0.1133 (Fig. 2). One epoch means training entire dataset has a change to upgrade the internal model of its parameters. In Fig. 3, first row shows the original images of MNIST dataset and second row illustrates the results after performing the simple autoencoder on the datasets.

Fig. 2 Simple autoencoder epochs

Fig. 3 Results of simple autoencoder

70

A. Makandar and K. Wangi

Fig. 4 Deep autoencoder epochs

Fig. 5 Results of deep autoencoder

3.2 MODEL 2: Deep Autoencoder Deep autoencoder is composed of two symmetrical deep belief networks that are having three hidden layer which means deeper extracting the features of images [13–15]. After 50 epochs, MSE value for deep autoencoder is 0.1105 as shown in Fig. 4. Figure 5 represents, first row as the original images of MNIST datasets, and second row illustrates the results after performing the deep autoencoder on the datasets.

3.3 MODEL 3: Convolutional Autoencoder Convolutional autoencoder is variant of CNN that is used for unsupervised learning of convolution filters. It is applied in the task of image reconstruction to minimize errors by learning the optimal filters [16]. For convolutional autoencoder, we have applied three convolutional pool blocks to reduce the error while reconstructing

Comparison and Analysis of Various Autoencoders

71

Fig. 6 Convolutional autoencoder epochs

Fig. 7 Results of convolutional autoencoder

output images. As shown in Fig. 6, after 50 epochs, the MSE value of MNIST datasets is 0.1048. Figure 7 presented, first row as original images of MNIST dataset, and second row illustrates the results after performing the convolutional autoencoder on the datasets. For CIFER datasets, it is more complexity when it compared to the MNIST training datasets because here in CIFER dataset, it is dealing with real-world color images. It required additional convolutional layers to achieve for better results. CIFER datasets have images with various degrees of scale, rotation, translation, and occlusion. This demands futuristic model with the ability of extracting features with effectively. The proposed architecture (Fig. 8) is consisting of four convolutional blocks. Each block having two convolutional layers with 3 * 3 filters which is followed by a max pooling layers with 2 * 2 filters. The dropout layers are placing after each convolutional block to regularize the model. The convolutional layers have stride of one and padded to preserve initial size (Fig. 8). At the end the fourth convolutional block, we flatten out 2048 features which are extracted from the input images and feed it into three fully connected layers in which it is progressively reduce the size of hidden layer representation to 128 or 64. This

72

A. Makandar and K. Wangi

Fig. 8 Encoder part of the network for CIFER images

encoded vector can be used in future for image retrieval based on the query. And, the reverse operation performs at the decoder end. After completing the training approach for 50 epochs over the 20 k training images from the CIFER4 dataset, MSE loss is 0.3614 and accuracy of image is 91.67%. (Fig. 10). Results of convolutional autoencoder for CIFER dataset are shown in Fig. 11.

4 Experimental Results Autoencoder of adequate complexity that is capable of transforming images to encoded vectors. Model 1 (simple autoencoder) is consist only one hidden layer

Comparison and Analysis of Various Autoencoders Fig. 9 Training approach model for end-to-end layers

Fig. 10 Convolutional autoencoder for training approach for CIFER images

73

74

A. Makandar and K. Wangi

Fig. 11 Results of convolutional autoencoder for CIFER dataset

for per encoder and decoder. This has been trained with Adadelta optimizer for 50 epochs over 50 k MNIST images. Model 2 (deep autoencoder) is consist of three hidden layers for per encoder and decoder. This model was performed best when trained with Adadelta optimizer for 50 epochs over the same train set. Model 3 (convolutional autoencoder) is consist of three convolutional layer blocks with max pooling layers for per encoder and decoder. It is trained with Adam optimizer for 50 epochs over the same train set (Table 1). From MNIST dataset, it is clear that convolutional autoencoder is gives the best performance when the results are compared to the other models. Here, we taken only convolutional autoencoder for CIFER image dataset which has given MSE value of 0.3614, trained with Adam optimizer for 50 epochs over train set of 20 k images. Figure 12 represents original image, reconstruction from model A, reconstruction from model B, and reconstruction from Model C. The convolutional layers are trained with classification task on the CIFER dataset. This classification model is consisting of the four convolution layer blocks and with Table 1 Comparison of reconstruction errors

Model

Layers per encoder

MSE error

Model 1

(1 hidden layer)

0.1127

Model 2

(3 hidden layers)

0.1105

Model 3

(3 Conv-pool blocks)

0.1015

Fig. 12 Row 1—original image; Row 2—reconstruction from model A; Row 3—reconstruction from model B; Row 4—reconstruction from model C

Comparison and Analysis of Various Autoencoders

75

Fig. 13 Loss versus epochs training classification model

Fig. 14 Accuracy versus epochs training classification mode

three fully connected layers with a softmax loss function which can observed from Fig. 8. The model trained to reach revolutionary accuracy of 91.67%. The fully connected layers have 4096:4096:4 hidden layer units. The weights are trained using Adam optimizer with a learning rate of 1e−4 and decay rate is 1e−6. GPU hardware permits a batch size of 128 images per batch, with training time of 80 s per epoch. Dropout is set to 0.25 for two convolution blocks, while the last two blocks had dropout of 0.5. Figures 13 and 14 represent loss versus epochs on training classification model and accuracy versus epochs on training classification model. It should be esteemed that model was trained to its complete capacity and compares with revolutionary accuracy of model similar in architecture.

5 Conclusion It is clear from these experimental results that the model 3 with convolutional layers performs best. Model 2 deep autoencoder performance is good when it compares with model 1. Model 1 simple autoencoder performance is satisfactory. For the CIFER dataset, the proposed architecture for encoder part of network gets the encoded vector as an output. Reverse of that operation is performed in decoder part as presented in the training approach for end to end layers. The model was trained to reach the

76

A. Makandar and K. Wangi

accuracy of 91.67% on CIFER4 dataset. Overall conclusion is an autoencoder with convolutional layers is the best for subsequent task of image retrieval in future works.

References 1. A. Krizhevsky, G.E. Hinton, Using very deep autoencoders for content-based image retrieval, in ESANN 2011 2. V. Rupapara, M. Narra, N.K. Gonda, Auto-encoders for content-based image retrieval with its implementation using handwritten dataset, in 5th International Conference on Communication and Electronics Systems (ICCES) (IEEE, 2020) 3. A. Makandar, K. Wangi, Analysis and techniques of content based image retrieval using deep learning. J. Inf. Comput. Sci. 10(2), 932–939 (2020) 4. A. Prakash, Autoencoders for image retrieval, in IEEE Explore, December 8, 2017 5. T. Linderberg, Feature detection with automatic scale selection. Int. J. Comput. Vision 30, 79–116 (1998) 6. A. Babenko, A. Slesarev, A. Chigorin, V. Lempitsky, Neural codes for image retrieval, in European Conference on Computer Vision (ECCV), Zurich, 2014 7. K. Lin, H.-F. Yang, J.-H. Hsiao, C.-S. Chen, Deep learning of binary hash codes for fast image retrieval, in CVPR Workshop 2015 8. E. Karami, S. Parad, M. Shehata, Image matching using SIFT, SURF, BRIEF and ORB: performance comparison for distorted images, in Proceedings of the 2015 Newfoundland Electrical and Computer Engineering Conference, Canada, 2017 9. A. Makandar, K. Wangi, Content based image retrieval using image preprocessing techniques. Strad Res. 7(12), 413–419 (2020) 10. S.N. Raj, Comparison study of algorithms used for feature extraction in facial recognition. Int. J. Comp. Sci. Inf. Technol. (IJCSIT) 8(2), 163–166 (2017) 11. A. Makandar, K. Karibasappa, Wavelet based medical image compression using SPHIT. J. Comp. Sci. Math. Sci. 1, 769–775 (2010) 12. A. Ferreyra-Ramirez, E. Rodriguez-Martinez, C. Aviles, F. Lopez, Image retrieval system based on a binary auto-encoder and a convolutional neural network. IEEE Latin Am. Trans. 18(11), November 2020 13. W. Liu, Hashing by “deep learning”, IBM T. J. Watson Research Center 14. H. Stokman, T. Gevers, Selection and fusion of color models for image feature detection. IEEE Trans. Patt. Anal. Mach. Intell. 29(3), 371–381 (2007) 15. T. Kadir, M. Bardy, Scale, Saliency and image description. Int. J. Comput. Vision 45, 83–105 (2001) 16. S. Dutta, S. Ghatak, A.K. Das, M. Gupta, S. Dasgupta, Feature selection-based clustering on micro-blogging data, in Computational Intelligence in Data Mining. Advances in Intelligent Systems and Computing, vol. 711, ed. by H. Behera, J. Nayak, B. Naik, A. Abraham (Springer, Singapore, 2019). https://doi.org/10.1007/978-981-10-8055-5_78

A New Delta (δ)-Doped Partly Insulated SOI MOSFET for Analogue/RF Applications Jay Prakash Narayan Verma and Prashant Mani

Abstract The scalability, thermal efficiency, and analogue/RF performance of single-gate delta-doped completely depleted silicon on insulator MOSFETs are investigated in this paper. To lower the self-heating effects and enhance the highfrequency operation of D-Pi-SOI MOSFET, a new p-type delta-doped fully depleted SOI MOSFET (D-Pi-SOI MOSFET) is presented. The developed analogue/RF figure of merit, viz. the transconductance generation factor, transconductance, maximum oscillation frequency, unity gain frequency, and D-Pi-SOI MOSFET, demonstrate the device’s potential for analogue/RF applications. This device is also extremely scalable because of decreased subthreshold swing, drain-induced barrier lowering, and parasitic capacitances. The findings of this implementation demonstrate that the D-Pi-SOI MOSFET has a high gain-bandwidth product. The comprehensive manufacturing flow of a DSSB Pi-OX-δ-MOSFET is proposed, and demonstrating DSSB SOI MOSFET’s performance is notably enhanced with bare minimum steps. Keywords Metal oxide semiconductor · Dual material gate · Technology computer-aided design · Subthreshold slope

1 Introduction Electrostatic coupling involving the source/drain (S/D) and channel has a critical difficulty in scaling completely depleted (FD) SOI MOSFETs [1]. Many structures have been proposed to solve this problem, including ground-plane FDSOI, halo-doped FDSOI, graded channel FDSOI, thin-film FDSOI, and multi-gate designs [2]. Thinfilm FDSOI is one of these structures that have piqued the interest of researchers due to its planar design and decreased short-channel effect [3]. Moreover, as the product J. P. N. Verma (B) · P. Mani SRM-IST, NCR Campus, Modinagar, Ghaziabad 201204, India e-mail: [email protected] P. Mani e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_7

77

78

J. P. N. Verma and P. Mani

of junction depth and S/D doping density is inversely proportionate to the S/D series resistance (RSD ), RSD rises with decreasing Si film thickness (T si ). To address this issue, either the area of the S/D or the doping density in the S/D should be increased. Because increase in the doping density of S/D is complex due to technological restrictions, expanding S/D area is a viable option. However, for analogue/RF applications, this device presents several difficulties, including Miller capacitance involving gate as well as increased S/D, channel mobility deterioration, inherent parametric quantity fluctuations, and self-heating problem [4–6]. The existence of the Schottky barrier being at the metal–semiconductor (M–S) junction developed among metal S/D as well as channel, on the other hand, enhances the device’s subthreshold swing and ambipolar conduction [7, 8]. It is recently demonstrated that placing a heavily doped n-type Si layer on the interface reduces the tunneling barrier width at this junction, allowing electrons coming from the source to be injected inside the channel and thereby increasing the driving current of subjected device [9]. Many groups have made a great deal of effort through experiments [10–13] and simulations [14–17] to examine this problem and enhance performance of device. The self-heating effect is another difficulty with SOI MOSFETs. It is mainly owing to the buried oxide’s (BOX) poor thermal conductivity [18]. In order to overcome this issue, several design and BOX materials for FDSOI MOSFETs are proposed. The materials used are buried alumina [19], diamond, SiC, aluminum nitride [20], and air [21], and the designs include quasi-SOI MOSFET, selective BOX under the source/drain with groundplane [22], localized-SOI MOSFET and double-steps BOX SOI MOSFET. Owing to the role of a thermal conducting window underneath channel and the decrease in floating body effect, partial BOX solely under S/D MOSFET is predominantly promising of these approaches [8]. This device has received significant attention for a wide range of applications, including one-transistor DRAM cells [10] and performance and power-optimized integrated circuits [12]. This study proposes p-typedoped completely depleted Schottky barrier SOI MOSFETs. This device is helpful for thermally efficient nanoscale circuits as it lowers CMOS compatibility, thermal budget, and partly insulated S/D MOSFET (floating body effects and reduced selfheating). Furthermore, the p-type-delta layer underneath the channel enhances the device’s high-frequency performance. The main idea behind this device is that heat must be dissipated into the substrate due to the lack of a BOX under the channel. Moreover, the involvement of p-typedelta layer underneath epitaxial channel blocks route of multiple fringing field lines coming from the drain and decreases threshold voltage variations caused by random dopants [20]. As a result of the reduced self-heating, SCE (primarily drain-induced barrier reduction, DIBL), and threshold voltage fluctuation, this device is appropriate for subsequent scaling. The rest of the paper has been organized as below. Section 2 discusses the device structures as well as the simulation technique. Section 3 compares the dc performance, analogue/RF figures of merit of the proposed device, and Sect. 4 provides the conclusion.

A New Delta (δ)-Doped Partly Insulated SOI MOSFET …

79

2 Device Design and Structure The cross-sectional layout of a D-Pi-SOI MOSFET device is presented in Fig. 1a, and experimental characteristics are shown in Fig. 1b. The device’s Si layer of thickness 20 nm is fully developed, underneath a BOX layer of thickness 50 nm. The transistor’s source and drain are doped with n-type material (1 × 1020 /cm3 ), while the channel with a length of 100 nm is doped with p-type material (1 × 1018 /cm3 ). The simulations and model parametric quantity of the developed device are similar to those of SOI MOSFET [16] for calibration purposes. For generating and recombining the charge carriers over semiconductor–insulator interfaces and semiconductor–semiconductor interfaces, the Shockley–Read–Hall (SRH) model is employed [17]. The simulation also takes into account the field-interdependent mobility model, the Auger model for the minority carriers. Moreover, since there is no BOX underneath the channel, heat may be quickly drained into the substrate, decreasing the self-heating effect. Other advantages of this

Fig. 1 a Schematic plot of delta-doped DMG-FD-SOI MOSFET b experimented drain outcomes

80 Table 1 Several design values of delta-doped DMG-FD-SOI MOSFET

J. P. N. Verma and P. Mani S. No.

Design parameters

Values

1

Gate length

65 nm

2

Gate work function (ØM1 )

4.4

3

Box thickness (T box )

50 nm

4

Si surface layer thickness (T si )

20 nm

5

SiO2 layer thickness

3 nm

6

Source/drain doping concentration

n-type 1 × 1019 /cm3

7

Channel doping

p-type 1 × 1016 /cm3

8

Doping in delta layer

1 × 1020 /cm3

construction include high drive currents and flexible threshold voltage adjustment. Furthermore, due to the lack of an insulator underneath the channel, it is noted that substrate coupling might cause the fringing field lines originating from the drain to the channel to increase. Resulting, short-channel effects become more pronounced, and the driving current decreases. A high doped p-type-delta layer is inserted beneath the channel to block the passage of such field lines, as illustrated in Fig. 1a. Table 1 shows several design values of delta-doped dmg-fd-soi mosfet.

3 Simulation Results Analysis and Discussion A. Self-heating effect The output characteristic curve of D-Pi-SOI MOSFET plotted versus V gs values. Drift–diffusion simulations utilizing lattice heat equation is used to obtain the characteristics at different drain and gate biases as shown in Fig. 4. In Fig. 4a, it is observed that the output characteristics are improved which is attributed to the decrement in channel mobility occurred because of thermal effects. This further concludes that in the present device setup, degradation of channel mobility because of thermal effect is insignificant, further proving the high-temperature efficiency of the proposed device (Figs. 2 and 3). B. Extraction of parasitic capacitance Parasitic capacitance, being an important parameter in analogue/RF applications, notably gate-to-drain capacitance (C gd ), and gate-to-source capacitance (C gs ) as shown in Fig. 3, it is mandatory to obtain these components for the proposed structure. The energy band diagram of proposed structure in OFF condition is depicted in Fig. 2. In Fig. 3, the fluctuation of capacitance is plotted versus V gs for the proposed device. From the figure, it is observed that in D-Pi-SOI MOSFET both C gs and C gd

A New Delta (δ)-Doped Partly Insulated SOI MOSFET …

81

Band Energy (eV)

0.0

-0.5

-1.0

-1.5

-2.0

0.150

Valance Band tsi=10 nm Conduction Band tsi=10 nm Valance Band tsi=20 nm Conduction Band tsi=20 nm Valance Band tsi=30 nm Conduction Band tsi=30 nm Valance Band tsi=40 nm Conduction Band tsi=40 nm 0.175

0.200

0.225

0.250

0.275

0.300

0.325

0.35

Horizontal Axis (µm) Fig. 2 Energy band diagram in OFF condition of delta-doped DMG-FD-SOI MOSFET

are significantly low in comparison with other structures. This observed difference of parasitic capacitances can be attributed to the structural difference arising in the device. From the literature, the Miller capacitance arising in raised S/D and the gate is the reason behind high values of C gs and C gd [6]. This shows that the parasitic capacitances are not altered by the proposed modification in D-Pi-SOI MOSFET. C. DC performance Figure 4 plots drain current versus V gs for D-Pi-SOI MOSFET on a log scale. From Fig. 4, it is observed that the drive current of D-Pi-SOI MOSFET is higher than the normal soi mosfet structure. This is because of the increased current driving potential of the proposed device presence due to p-type-delta layer present beneath the channel. Furthermore, it also obstructs and isolates fringing field line’s path originating in the drain, thereby reducing leakage current of the device as portrayed in Fig. 4. The DC performance of short-channel SOI MOSFET is affected primarily by DIBL and subthreshold swing (S). The variation of DIBL and SS is plotted vs the physical gate length for D-Pi-SOI MOSFET. From the curve, it is found that DIBL and S in D-Pi-SOI MOSFET are critically low. This is because of the p-type-delta layer present beneath the channel, thereby reducing the potential couple effects that can occur from substrate and also cutting off the fringe field lines coming via the drain. D. Analogue performance Transconductance (gm ), Transconductance generation factor (gm /I d ), and resulting output conductance (gd ) are the important characteristics for analogue circuit applications. [12]. Since transconductance gives the amplification received from this device

J. P. N. Verma and P. Mani

Gate-Source Capacitance Cgs (F/cm 2)

82 9x10-16

(a)

8x10-16 7x10-16 6x10-16 5x10-16 4x10-16 3x10-16

tsi=10nm tsi=20nm tsi=30nm tsi=40nm

2x10-16 1x10-16 0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Gate Voltage Vgs (V)

Gate-Drain Capacitance Cgd (F/cm 2)

6x10-16

(b)

5x10-16

4x10-16

3x10-16

tsi=10nm tsi=20nm tsi=30nm tsi=40nm

2x10-16 0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Fig. 3 I d –V gs characteristics of the delta-doped D-Pi-SOI MOSFET for variable T si . Impact of the Si thickness on a C gs b C gd

as well as I–D shows the power dissipation in order to obtain the said amplification, the capacitances (C gs and C gd ) are observed in Fig. 3a, b and 6a, b can be taken as a quality parameter in analogue applications scenarios (Figs. 5 and 6). With higher transconductance generation factor, the efficiency of device for analogue application increases. Variation of gm with respect to gate length for D-PiSOI MOSFET (L g = 65 nm) is shown in Fig. 5. From Fig. 5a, it is observed that for a low inversion region, the transconductance gm in D-Pi-SOI MOSFET is highest

A New Delta (δ)-Doped Partly Insulated SOI MOSFET …

83

10-4

Drain Current Ids (A/µm)

10-5 10-6 10-7 10-8 10-9 10-10 10-11

Source doping Conc. 1e17 Source doping Conc. 1e18 Source doping Conc. 1e19 Source doping Conc. 1e20 Source doping Conc. 1e21

10-12 10-13 10-14 10-15 0.0

0.2

0.4

0.6

0.8

1.0

Gate to Source Votage Vgs (V) Fig. 4 I d –V gs characteristics of delta-doped D-Pi-SOI MOSFET for various source doping

in all the studied devices which are because of the reduction of subthreshold swing. In Fig. 5, the variation of gm and gd with respect to gate bias D-Pi-SOI MOSFET (L g = 65 nm) is plotted. In this figure, Y-parameter matrices generated from small signal AC analysis are used to extract the gd and gm values of all the devices. From this figure, we can observe that, D-Pi-SOI MOSFET gm is significantly reduced at a lower gate bias. Here in D-Pi-SOI MOSFET, a high S/D-to-channel link resistance diminishes the gm of the proposed device, while p-type-delta layer present below channel of D-Pi-SOI MOSFET results in improved gm value and better gate control. Also, at a much higher value of gate bias and despite of increased gd is shown in Fig. 5b in D-Pi-SOI MOSFET compared to the conventional SOI MOSFET, the significant enhancement in gm of the proposed device dominates the effect. Figure 6 plots variation of a C gs and b C gd capacitances of delta-doped D-Pi-SOI MOSFET with variation in source doping concentration. E. RF performance The RF performance of a MOSFET is delineated by the maximum oscillation frequency (F Max ) and unity gain frequency (Ft ). The Ft can be coined as the frequency at which the intrinsic current gain’s magnitude of short circuit is reduced to unity. In Fig. 7, the variation of Ft is plotted versus V gs for D-Pi-SOI MOSFET for L g = 65 nm. Examining the resulted curve, it is inferred that Ft in proposed are higher compared to conventional SOI MOSFETs. It is because of reduction in C gs and C gd values and improvement in gm of the proposed device. Also, despite having lower gm in D-Pi-SOI MOSFET, F t values of the proposed device in comparison with

84

J. P. N. Verma and P. Mani

Transconductance gm (µS/µm)

570 565 560 555 550 545 540 535 0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.05

0.06

Gate Length (µm)

Transconductance gd (µS/µm)

110 100 90 80 70 60 50 40 30 0.00

0.01

0.02

0.03

0.04

Gate Length (µm) Fig. 5 Influence of change in a gm and b gd of delta-doped D-Pi-SOI MOSFET at a fixed channel length of 65 nm

conventional SOI MOSFETs are typically higher. The reason for this is the planar structure of the device, leading to a significant reduction in parasitic capacitances.

A New Delta (δ)-Doped Partly Insulated SOI MOSFET …

85

6.0x10-16

1e17 1e18 1e19 1e20

5.0x10-16

(a)

Cgs (F)

4.0x10-16

3.0x10-16

2.0x10-16

1.0x10-16 0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Gate to Source Voltage Vgs (V) 3.5x10-16 3.0x10

2.5x10-16

Cgd (F)

(b)

1e18 1e17 1e19 1e20

-16

2.0x10-16 1.5x10-16 1.0x10-16 5.0x10-17 0.0 0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Gate to Source Voltage Vgs (V)

Fig. 6 Variation of a C gs and b C gd capacitances of delta-doped D-Pi-SOI MOSFET with variation in source doping concentration

4 Conclusion In the present paper, a 2D analytical model of delta-doped DMG-FD-SOI MOSFET is developed for surface potential compared with SILVACO TCAD simulation for authentication. The proposed device is also studied by varying thickness of Si layer, doping of the source region, and work function. The proposed structure is designed for enhanced analogue/RF performance, and the impact of transconductance (gm )

86

J. P. N. Verma and P. Mani 450

Unity Gain Frequency Ft (GHz)

400 350 300 250 200 150 100 50 0 0.0

0.2

0.4

0.6

0.8

1.0

Gate to Source Voltage Vgs (V) Fig. 7 Variation of Ft with V gs for D-Pi-SOI MOSFET for L g = 65 nm

and voltage gain ( A V ) is explored to examine several structure characteristics. The increased Si thickness and channel doping (1 × 1017 /cm3 ) enhanced the device threshold voltage from standardized 0.25 V value (over thickness = 10 nm) and I ON to I OFF , whereas the capacitances (C gs and C gd ) were found reduced. Strikingly, the proposed device has a minimal parasitic capacitance effect, making our proposed device a suitable candidate for high-speed application. The proposed simulation flow of delta-doped DMG SOI MOSFET shows that when p-type delta layer is combined under channel, they reduce the effect of self-heating and also at the same time lead to improvement of high-frequency performance of the proposed model in comparison with single material gate structure.

References 1. D.H. Morris, U.E. Avci, I.A. Young, Intel Corp, Tunnel field-effect transistor (tfet) based high-density and low-power sequential. U.S. Patent Application 15/992,080 (2019) 2. P.C. Yang, S.S. Li, Analysis of current-voltage characteristics of fully depleted SOI MOSFETs. Solid-State Electron. 36, 685–692 (1993) 3. R.H. Dennard, F.H. Gaensslen, H.-N. Yu, V.L. Rideout, E. Bassous, A.R. Leblanc, Design of ion-implanted MOSFET’s with very small physical dimensions. IEEE J. Solid-State Circ. SC-9, 256–268, May 1974 4. G. Wadhwa, B. Raj, Design, simulation and performance analysis of JLTFET biosensor for high sensitivity. IEEE Trans. Nanotechnol. 18, 567–574 (2019).https://doi.org/10.1109/TNANO. 2019.2918192

A New Delta (δ)-Doped Partly Insulated SOI MOSFET …

87

5. G. Wadhwa, B. Raj, Label free detection of biomolecules using charge-plasma-based gate underlap dielectric modulated junctionless TFET. J. Electron. Mater. 47(8), 4683–4693 (2018) 6. S. Voldman, D. Hui, L. Warriner, D. Young, R. Williams, J. Howard, V. Gross, W. Rausch, E. Leobangdung, M. Sherony, N. Rohrer, Electrostatic discharge protection in silicon-on-insulator technology, in 1999 IEEE International SOI Conference. Proceedings (Cat. No. 99CH36345) (IEEE, October, 1999), pp. 68–71 7. W. Long, O. Haijiang, J.-M. Kuo, K.K. Chin, Dual-material gate (DMG) field effect transistor. IEEE Trans. Electron Devices 46, 865–870 (1999) 8. A. Chaudhry, M.J. Kumar, Controlling short-channel effects in deep-submicron SOI MOSFETs for improved reliability: a review. IEEE Trans. Dev. Mater. Reliab. 4(1), 99–109 (2004) 9. S. Rewari, V. Nath, S. Haldar, S.S. Deswal, R.S. Gupta, Gate-induced drain leakage reduction in cylindrical dual-metal hetero-dielectric gate all around MOSFET. IEEE Trans. Electron. Devices 65(1), 3–10 (2017) 10. P. Mani, N. Srivastava, P. Singh, Analysis of electrical properties of narrow channel SOI MOSFETs. IJERT 12(12), 2312–2316 (2019) 11. V.P. Trivedi, J.G. Fossum, Nanoscale FD/SOI CMOS: thick or thin box. IEEE Electron Device Lett. 26(1), 26–28 (2004) 12. L. Grenouillet, M. Vinet, J. Gimbert, UTBB FDSOI transistors with dual STI and shrinked back gate architecture for a multi-VT strategy at 20 nm node and below, in Technical Digest of International Electron Devices Meeting, San Francisco, pp. 64–66, 2012 13. F. Balestra, S. Cristoloveanu, M. Benachir, J. Brini, T. Elewa, Double-gate silicon-on insulator transistor with volume inversion: a new device with greatly enhaced performance. IEEE. Electron. Device. Lett. EDL. 8(9), 410–412 (1987) 14. X. Zhou, W. Long, A novel hetero-material gate (HMG) MOSFET for deep-submicron ULSI technology. IEEE Trans. Electron Devices 45, 2546–2548 (1998) 15. O. Faynot, F. Andrieu, O. Weber, C. Fenouillet-Béranger, P. Perreau, J. Mazurier, T. Benoist, O. Rozeau, T. Poiroux, M. Vinet, L. Grenouillet, Planar fully depleted SOI technology: a powerful architecture for the 20 nm node and beyond, in 2010 International Electron Devices Meeting (IEEE, 2010), pp. 3–2 16. T. Ishigaki, R. Tsuchiya, Y. Morita, H. Yoshimoto, N. Sugii, T. Iwamatsu, H. Oda, Y. Inoue, T. Ohtou, T. Hiramoto, S. Kimura, Silicon on thin BOX (SOTB) CMOS for ultralow standby power with forward-biasing performance booster. Solid-State Electron. 53(7), 717–722 (2009) 17. M.J. Kumar, A. Chaudhry, Two-dimensional analytical modeling of fully depleted dual-material gate (DMG) SOI MOSFET and evidence for diminished short-channel effects. IEEE Trans. Electron Devices 15, 569–574 (2004) 18. A. Goel, S. Rewari, S. Verma, R.S. Gupta, Shallow extension engineered dual material surrounding gate (SEE-DM-SG) MOSFET for improved gate leakages, analysis of circuit and noise performance. AEU-Int. J. Electron. Commun. 111, 152924 (2019) 19. W. Long, H. Ou, J.-M. Kuo, K.K. Chin, Dual material gate (DMG) field effect transistor. IEEE Trans. Electron Devices 46, 865–870 (1999) 20. K.K. Young, Short-channel effect in fully depleted SOI MOSFETs. IEEE Trans. Electron Devices 36, 399–402 (1989) 21. X. Zhou, Exploring the novel characteristics of hetero-material gate field-effect transistors (HMGFETs) with gate-material engineering. IEEE Trans. Electron Devices 47, 113–120 (2000) 22. X. Zhou, W. Long, A novel hetero-material gate (HMG) MOSFET for deep-submicrometer ULSI technology. IEEE Trans. Electron Devices 45, 2546–2548 (1998)

Energy Monitoring with Trend Analysis and Power Signature Interpretation S. Divya, Akash Murthy, Siva Surya Babu, Syed Irfan Ahmed, and Sudeepa Roy Dey

Abstract Energy consumption and the science behind the patterns in electricity expenditure have been rapidly growing fields in technology for decades owing to the vital nature of fuel. This research explores room for advancement in exploiting consumption data and generating smart results powered by state-of-the-art algorithms and sensors. The line of experimentation for this problem statement is focused on enabling users to interpret their power consumption efficiently and interacts with live visual trends and limits their consumption to a customized estimate. Prediction models for forecasting consumption, analysing power signature of high-load appliances, and monitoring of appliance power state have been built based on parameters such as instantaneous and cumulative energy, power, and solar energy. Although the scope of this setup is limited to individual accommodations, the purpose of this research is to extrapolate the functionalities to an industrial scale, where the importance of restricted consumption, and protection against casualties is truly evident. The proposed model improves the accuracy of time series forecasting of energy consumption as compared to existing forecasting models. Keywords Time series forecasting · Statistical visualization · IoT · Electricity conservation · TBATS · Multiple seasonality · Quadratic regression

1 Introduction In a time, where ample resources are continually invested in finding alternative fuels and renewable energy resources, optimized and adequate electricity consumption is critical for a highly populous nation like India. However, the average citizen is inadequately informed of the specifics of their monthly electricity bill charges. Due S. Divya · A. Murthy · S. S. Babu (B) · S. I. Ahmed · S. R. Dey PES University Electronic City Campus, Bangalore, Karnataka, India e-mail: [email protected] S. R. Dey e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_8

89

90

S. Divya et al.

to the lack of awareness, individuals and corporations alike consistently expend excessive energy, causing significant wastage nationwide. This paper describes a product involving hardware and software components spanning over the domains of IoT in the form of a Wi-Fi capable sensor, cloud computing for storage and visualization of live streaming data, machine learning for the prediction models, Android application development for the app interfacing the consumer, and the data served by the hardware setup. Electricity consumption can be predicted accurately by taking into consideration past trends with exponentially smoothed weights. The ability to visualize live energy consumption shall enable a consumer to interact with their appliances more efficiently, leading to reduced wastage. Provisioning consumers to create customized alerts based on triggers shall enable a consumer to track consumption and make educated decisions on expenditure. This research paper indicates further advancements in the application in the form of intelligent cost-cutting suggestions that take into consideration the health and efficiency of an appliance and the generally accepted correct usage of an appliance. The data used for this research was collected from the residence of each of the authors over a span of a year. Due to the occurrence of multiple seasonality in the trends for cumulative energy and instantaneous voltage, the exponential smoothing state space model with Box-Cox transformation, ARMA errors, and Trend and Seasonal components (TBATS) was chosen. For triggers associated with heavy load appliances like air conditioners and geysers, quadratic regression was used to establish a baseline, and a 0.95 error was deemed admissible. This setup was tuned by trial and error. For the prediction and correlation between weather and solar energy, a fuzzy decision tree model was used.

2 Literature Review The authors in [1] discuss the various criteria to be considered to tune electricity consumption to perfection such as cost, temperature, and humidity. It investigated three basic prediction strategies—regression, artificial neural network (ANN), and the Kalman filter adaptation technique—and determined that the Kalman technique was the most efficient way for estimating utilization data. The authors of [2] emphasize the importance of appliance load monitoring in electricity analysis. It goes over numerous methods for acquiring appliance specific power usage information, which may then be applied to achieve load scheduling algorithms for optimal energy use. The running schedule of electrical appliances in a target system can be determined using non-intrusive load monitoring (NILM). NILM is an effective platform for obtaining meaningful information from any configuration that employs electromechanical devices. This method may also be used to assess the functionality of the power system. The paper thoroughly explores non-intrusive load monitoring for this purpose including data acquisition, feature extraction, load

Energy Monitoring with Trend Analysis and Power Signature Interpretation

91

identification, and system training. This research also provides a concise comparison of the steady-state and transient methods for load monitoring. Conversely, [3] acknowledged the use of appropriate prediction and forecasting models for electricity sector development as a technique to achieve efficient power consumption in appliance usage. As forecasting has progressed as a strategy for maximizing energy resources, the research revealed that the adoption of new advancements in the analysis of energy models would reduce economic loss. Furthermore, precisely projecting power use would allow for the effective distribution of available resources in the energy system, as well as encourage responsible appliance use. According to the study, larger energy savings could be achieved if appliance specific future electricity usage is calculated. The authors [4] conducted a thorough investigation into understanding and influencing home energy consumption behaviour in order to increase electricity efficiency and promote energy savings. Traditional energy systems are becoming digitized as communication technologies become more prevalent in the energy industry. This research examines household energy consumption behaviour in 3 components: time, user, and spatial dimensions. This study examines the difficulties of short and long-term projections of solar power generation based on data obtained from a solar cell, as well as their interaction with atmospheric observations. They discover that using fuzzy decision trees to anticipate reduce energy error by 22% when compared to a fixed projection equal to the mean of the relevant period. As a result, using fuzzy classification produced a significant improvement over the baseline. Nugaliyadde et al. [5] present two approaches for predicting electricity consumption in their paper—one uses a recurrent neural network (RNN) to forecast future energy use, while the other uses a long short-term memory (LSTM) network that only takes into account historic energy usage to predict future consumption of energy. These models were evaluated using a dataset from London’s smart energy metres that is freely accessible to the public. To determine its relevance, the RNN and LSTM networks were used to estimate electricity use for an urban household and a block of residences over a defined time period. Daily, trimester, and 13-month projections were made, covering short, mid, and long-term forecasting. For both the RNN and the LSTM networks, the average root mean square error was 0.1. The disadvantage with the ideas presented in this paper is that it is not viable to feed live data into a machine learning model such as an LSTM model and send the results to an API that will then serve results on the mobile app. This middleman of a neural network was replaced by utilizing a more static form of forecasting that did not require training a complex model. Hlaing et al. [6] in their paper describe a low-cost energy monitoring system that is easy to implement and aids in managing daily electricity consumption. The authors’ primary focus in this research is on IoT power management so as to combat human errors and catalyse cost reduction in electricity consumption with improved efficiency of the power regulation system. The proposed strategy intends to provide an affordable wireless sensor system and framework for smart energy, as well as a software platform capable of independently collecting and reporting information

92

S. Divya et al.

to advanced users, allowing them to monitor their existing smart metre. Users will have access to their electricity usage as a result of utilizing this technology, allowing them to reduce power waste and consumption costs. The system consists of a digital electricity metres, an Espressif Wi-Fi-based microcontroller, and administrative web services. The ESP8266 microcontroller capable of wireless communication is integrated in the smart metre and will utilize the common TCP/IP protocol to link the metre to the web-based application. The testing findings demonstrate that the proposed system works admirably in terms of efficiency and that it is suitable for use in practical uses for value automated energy metre readings. Marvugliaa and Messineo [7] discuss how energy consumption in industrialized nations’ civil sectors has increased dramatically in recent years, notably during the summer season. The significant growth in sales of air conditioning single and multisplit systems is one of the drivers of this surge. Determining the link between power consumption and the use of electric appliances is crucial in this context. This study proposes a model based on an Elman artificial neural network for estimating residential electric demand in a suburban region of Palermo’s surroundings for a short period of time, about one hour in advance. One of the study’s goals is to investigate the effect of air conditioning equipment on electricity usage. The authors of [8] discuss about how economic and demographic factors affect yearly power usage in Italy. These were looked into with the goal of creating a long-term consumption forecasting model. The historical data is based on a period of time from 1970 to 2007. Using historical power usage, gross domestic product (GDP), GDP per capita, and population, and various regression models were built. The first section of the research examines the estimation of domestic and nondomestic electricity consumption GDP, price, and GDP per capita elasticities. Domestic and non-domestic short run price elasticities are both found to be sufficient to −0.06, whereas end-of-day elasticities are shown to be adequate to −0.2 and −0.09, respectively. The elasticities of GDP and GDP per capita, on the other hand, are larger. Different regression models, backed by co-integrated or stationary data, are provided in the second portion of the research. To determine the validity of the presented models, several statistical tests are used. The created regressions are congruent with the official estimates, with divergence of 1% for the simplest scenario and 11% for the worst case, according to a comparison with national forecasts backed by advanced econometric models like Markal-Time. In the context of the historical period under discussion, these variations are to be regarded acceptable. In this study, Erdogdu [9] go into detail on the Turkish electrical market. In the early 2000s, the Republic of Turkey embarked on an ambitious electrical market reform programme that included privatization, liberalization, and drastic restructuring. The most contentious reason for recent changes has been the rapid increase in electricity usage; in other words, the whole reform process has been part of efforts to avoid an energy crisis. The research focuses on this problem by offering an electrical demand assessment and prediction, as well as comparing the results to government

Energy Monitoring with Trend Analysis and Power Signature Interpretation

93

estimates, using co-integration analysis and ARIMA modelling. The report found that consumers’ feedbacks to cost and income changes are limited, requiring economic compliance within the Turkish electricity market; and that current official electric load projections vastly overestimates demand, putting both a rational energy policy and a healthy electricity market in jeopardy. The authors of [10] investigate the dynamic modelling of district heating and cooling networks, which is becoming increasingly important in the shift to renewable energy sources and lower temperature district heating grids, as both temporal and spatial behaviour must be considered. They contend that despite much study and development in the subject, there are significant hazards and barriers to dynamic district heating and cooling simulation for everyday usage. The experiences obtained while constructing and working with a city-scale simulation of a regional heating system in Lulea, Sweden, are presented in detail. In the case study, the grid model may be a white box model, whereas consumer models are either black or grey box models. The system and operator models simulate the human and automatic operation of the combined heat and power station. A co-simulation environment combines all the components using the functional mock-up interface standard. In addition, the simulator’s validation is addressed. The paper’s lessons are offered, along with future research directions that correlate to the gaps and issues found.

3 Methodology The analytics problem statements taken up as part of this research paper include various tasks such as forecasting, prediction, trigger notifications, interactive visualizations, live data display, and statistical figures. For each of these tasks, prospective methods were researched and compared to elect the best approach. Our paper investigates the performance of the SARIMAX model, the TBATS model, quadratic regression, NILM approach, and fuzzy decision tree approach. These experiments have been documented in this section, and the results of various approaches have also been tabulated.

3.1 Dataset Used The ESP32 Wi-Fi microcontroller paired with a PZEM-016 sensor and MAX485 Modbus to TTL converter was used to collect data from the residence of the authors. Each house has a different setting with varying appliances and helps expand our access to data with different formats. 1. Independent housing with three-phase power and on-grid solar power generation 2. Apartment with single-phase power 3. Apartment with three-phase power.

94

S. Divya et al.

Fig. 1 Data acquisition using ESP32 sensor

Figure 1 represents a snapshot of the data collected using the ESP32. The _value column indicates the magnitude of the corresponding _field parameter. The InfluxDB dashboard has been utilized to visualize the electrical consumption in terms of various factors such as voltage, energy, power, and current. Additional factors such as power factor are also recorded. Furthermore, the tool provides filter options and the ability to view data collected over select spans of time. In Fig. 2, graph (a) depicts the solar readings recorded in a day. The orange trend line represents power, purple representing energy, blue being current, and cyan representing voltage. Since energy increments in units of one, it is fairly constant. Graph (b) indicates power consumed over a day in each of the three phases. The spikes in purple represent a high-load appliance being switched on. Graph (c) shows the recorded voltage in each phase during a day.

3.2 Time Series Modelling Methodology 3.2.1

ARIMA Model

The autoregressive integrated moving average (ARIMA) is a class of models that captures a suite of different standard temporal structures in time series data. This algorithm is a starting point in building an appropriate model to generate the most accurate forecast for our dataset.

3.2.2

SARIMAX Model

The seasonal autoregressive integrated moving average with exogenous factors (SARIMAX) is an updated version of the ARIMA model. This model has been chosen to forecast instantaneous energy for calculating electricity consumption. Due to the presence of seasonality in the data, SARIMA has been chosen. The variables in this model were thoroughly studied and given appropriate values to forecast accurately.

Energy Monitoring with Trend Analysis and Power Signature Interpretation

(a) Solar Power Readings

(b) Household Power Readings

(c) Three Phase Instantaneous Voltage Fig. 2 Readings on InfluxDB

95

96

S. Divya et al.

Fig. 3 Power on transition graph for air conditioner

3.2.3

TBATS Model

TBATS is an acronym for key features of the model: Trigonometric seasonality, Box-Cox transformation, ARIMA errors, Trend. This model tackles the multiple seasonality in the energy time series data. The data is partitioned into two parts—train and test. The trained model is evaluated using the test data. The energy consumption has been forecast for the next year based on existing data.

3.3 Trigger Modelling Methodology 3.3.1

Quadratic Regression Method for Curve Mapping

For heavy load appliances such as air conditioners and heaters, triggers have been set using the pattern of power states detected for the specific appliance. The pattern is then used to generate a quadratic regression equation. The incoming data has been tested against this equation over a window of 100 data points, with an error limit of 0.95 (Fig. 3). The model logarithmic curve used for the quadratic regression is the average of several AC power on state curves. The resulting equation is of the form A + Bx + C x 2 . These coefficients are used in the trigger module. The allowed error of 0.95 has been assigned after running a trial and error test.

3.3.2

Other Regression Techniques for Curve Mapping

Regression techniques such as Lasso regression and Ridge regression were also tested. Quadratic regression outperforms the counterparts in terms of accuracy, computation speed, and API request-response speed.

Energy Monitoring with Trend Analysis and Power Signature Interpretation

97

Fig. 4 Power on transition graph for geyser

Fig. 5 Trend of instantaneous power during a brown-out

3.3.3

Power State Detection with Threshold Mapping

A shown in Fig. 4, certain appliances have been found to reach a specific threshold voltage while powering on. Using these threshold values as triggers, the power state of the appliances can be tracked and used to gain insight on their power draw. This method has been used to monitor the geysers. Crucial states of power supply to an establishment such as brown-outs and power surges (see Figs. 5 and 6 respectively) can be detected using the available data, with the help of threshold mapping. The authors detected a two minutes’ window where a brown-out can be identified by reading the power magnitude. These thresholds have been recorded and may be used to generate automated warnings to consumers. Users can then act on these warnings to save their appliances. Figure 7 describes the pipeline through which data is collected. Starting from the data collection point via sensors to data ingestion and processing on the cloud platform.

98

S. Divya et al.

Fig. 6 Trend of instantaneous power during a power surge

Fig. 7 System architecture

4 Results and Discussion The ideas explored in this paper are aimed at gaining an edge over their counterparts in terms of the insightful analysis and customization. Thus, the data used to arrive at integral results has proven to be very crucial. On researching the datasets available on the Internet, the authors resorted to generating custom data with the use of the PZEM004T sensor. Other options like the PZEM-016 with an ESP8266 microcontroller have also been evaluated. The comparisons suggested that the PZEM-004T paired with an ESP32 Wi-Fi microcontroller were best suited in terms of expenses and requirements.

Energy Monitoring with Trend Analysis and Power Signature Interpretation

99

Fig. 8 Rolling window graph from prediction of instantaneous energy using SARIMAX model (predict for 1 month using previous 2 months’ data)

Fig. 9 Prediction of instantaneous energy using TBATS model (predict for 1 month using previous 2 months’ data)

Our choice of cloud database—Influx DB—was vital for the smooth conveyance of data through the various interacting platforms. This tool also provided powerful visualization and tools to write queries for testing methodologies. Several models were explored for each of the problem statements adopted in this paper. The cumulative energy has a linear trend without seasonality as tested with the Dickey—Fuller test. This forecast is also used to provide the final prediction. As discussed in the time series modelling section, the SARIMA, SARIMAX and TBATS models were tested and compared (Fig. 10). Figure 8 represents prediction using the SARIMAX model. The plot in orange depicts the predicted component. This model generated a result with 83% accuracy Figure 9 indicates the forecast generated by the TBATS model of a span of one month with a learning data of two months. The result gave 98% accuracy.

100

S. Divya et al.

Fig. 10 Prediction of energy using TBATS model (predict for 1 month using previous 2 months’ data) Fig. 11 Estimation of characteristic power on state of graph using quadratic regression

Table 1 Comparison of models tested

Model

Accuracy rate (%)

SARIMA SARIMAX TBATS

76 83 98

The TBATS model gave better results and proved to handle the seasonality better. The SARIMA, SARIMAX, and TBATS models were tested for time series forecasting. While the SARIMA and SARIMAX models gave moderate results with 76% and 83% accuracy, the TBATS model handled the existence of multiple seasonality and gave 98% accuracy. This model worked best for our dataset (Fig. 11; Table 1). The NILM approach, quadratic regression with triggers, Ridge and Lasso regressions were explored for appliance state detection. The NILM method proved to be more inclined to an electrical approach, whereas the regression approach was better

Energy Monitoring with Trend Analysis and Power Signature Interpretation

101

suited for our purpose. Moreover, quadratic regression outperformed its counterparts in terms of accuracy and speed of computation with almost perfect results. In conclusion, this paper involved the testing and comparison of several powerful analytical techniques, IoT modules, database provisions, and cloud resources. Thorough research and exploration enabled us to identify and employ the best alternatives at all stages, leading to a fully functional product.

5 Conclusion In our attempt to facilitate reduced energy consumption and efficiently use electricity for sustainability, we have successfully built an end-to-end product supported by a service that provides consumers the resources to educate themselves about the social and environmental implications of the commodities they purchase and use. This paper has enabled us to delve into the various aspects of electrical and computer science engineering. Being a data-driven research, we developed a sensor array to collect data in a way we deemed most relevant to our requirements, rather than resorting to pre-existing datasets. During the initial months, we acquired and analysed the raw data, to create inferences and establish a list of objectives to pursue as part of our research. In an effort to collaborate as a remote team and efficiently process data, a cloudbased service-oriented architecture was conceptualized. The premise of this thought process was to allow different services to access data securely and reliably. Readings polled from the sensor array are posted to the storage bucket instantaneously and ingested by the processing algorithms. One of the primary goals of the paper was to provide a forecast of electricity consumption based on existing time series data. This process began with researching and implementing the standard algorithms such as SARIMAX. This provided results with moderate accuracy, after which deeper analysis revealed the presence of multiple complex seasonality. The TBATS model was employed to handle this issue, which gave much higher accuracy. Another equally integral problem statement was to detect the power on and power off states of high-load appliances. The NILM model was thoroughly researched for this purpose, and during the implementation, we decided to utilize existing data in a more mindful manner and were able to solve this problem statement using specific triggers generated on the basis of existing data and keenly observing the trends therein. This involved employing the quadratic regression, and logarithmic extrapolation of incoming data, to trigger custom notifications to users. Several other alternatives such as Ridge–Lasso regressions were also tested. An interesting problem explored during the analytics phase was the correlation between weather and solar units generated. This problem tackled predicting the solar energy throughput based on the weather forecast. This was implemented using a fuzzy decision tree model.

102

S. Divya et al.

Finally, the end user is given access to the data collated by the sensor, to provide actionable insights. As next steps, we plan to make the collected data publicly available for analysts to build upon.

References 1. P. Ozoh, S. Abd-Rahman, J. Labadin, Predicting electricity consumption: a comparative analysis of the accuracy of various computational techniques, in 2015 9th International Conference on IT in Asia (CITA) (2015) 2. A. Zoha, A. Gluhak, M.A. Imran, S. Rajasegarar, Non-intrusive load monitoring approaches for disaggregated energy sensing: a survey (2012) 3. World Energy Outlook 2013 (International Energy Agency, 2013) 4. K. Zhou, S. Yang, Understanding household energy consumption behavior: the contribution of energy big data analytics (2016) 5. A. Nugaliyadde, U. Somaratne, K.W. Wong, Predicting electricity consumption using deep recurrent neural networks (2019) 6. W. Hlaing, S. Thepphaeng, V. Nontaboot, N. Tangsunantham, T. Sangsuwan, C. Pira, Implementation of WiFi-based single phase smart meter for Internet of Things (IoT), in 2017 International Electrical Engineering Congress (iEECON) (2017) 7. A. Marvuglia, A. Messineo. Using recurrent artificial neural networks to forecast household electricity consumption. Energy Procedia 14 (2012) 8. V. Bianco, O. Manca, S. Nardini, Electricity consumption forecasting in Italy using linear regression models. Energy (2009) 9. E. Erdogdu, Electricity demand analysis using cointegration and ARIMA modelling: a case study of Turkey. Turkey Energy Policy (2009) 10. P. Bacher, H. Madsen, H.A. Nielsen, B. Perers, Short-term heat load forecasting for single family houses (Industrial Electronics Society, 2013)

Analysis of NavIC Signal Data for Topological Precision Raj Gusain, Anurag Vidyarthi, Rishi Prakash, and A. K. Shukla

Abstract Ionospheric scintillations prevent the GPS receiver from locking on to the signal and make it difficult to compute a position. Small-scale composition in the ionospheric layer along the transmission source causes random fluctuations, which are created by the interaction of reflected light and reflected waves. Ionospheric scintillations and/or satellite unlock cause cycle-slip or total loss of GNSS signal. Low C/N o (carrier-to-noise ratio), ionospheric scintillation, receiver position (low latitude), low elevation angle, and cycle-slip error are the main issues affecting precisepoint positioning (PPP) accuracy. Since a lot of studies have been done using GPS signals, but our NavIC technology has yet to be explored for this research, it is a major challenge and still ongoing research in PPP. Continuous tracking of GNSS carrier phase signals provides high-precision PPP solutions. PPP requires ionospheric research, which geostationary satellites can help with. The NavIC system, which includes three geostationary satellites in orbit, can fulfill this need. Pre-processing of data before positioning and identification and correction of cycle-slip error are the two methods investigated in this study that provides more efficient PPP. Keywords Precise-point positioning · NavIC · Ionospheric scintillation · Pre-processing · Low latitude in navigation · Cycle slip

R. Gusain (B) · A. Vidyarthi · R. Prakash Department of Electronics and Communication, Graphic Era University, Dehradun, India e-mail: [email protected] A. Vidyarthi e-mail: [email protected] R. Prakash e-mail: [email protected] A. K. Shukla Space Applications Center, ISRO, Ahmedabad, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_9

103

104

R. Gusain et al.

1 Introduction Global navigation systems are used for high precise positioning in real time within a few meters. Precise point positioning (PPP) is required in civil and defense airborne applications, as well as geodetic applications such as glacier movement studies, tsunami alarming, seismic activity (i.e., in the case of an earthquake, there is an increase in total electron content of the ionosphere by which we can become aware of that activity) and space weather prediction [1–3]. These sensitive applications require highly accurate positioning. The factors affecting precise point positioning are low C/N o , ionospheric scintillation, receiver position (low latitude), low elevation angle, and cycle slip error [4, 5]. GNSS measurements are affected by noise and errors due to the propagation of signals through atmospheric layers. The different types of GNSS error sources are satellite and receiver clock errors, orbital errors, ionospheric abnormalities, errors due to earth’s magnetic field, tropospheric delays, receiver noise, and multipath. Ionospheric scintillation, defined as high ionospheric activities caused by ionospheric irregularities, is the most noticeable and significant problem in satellite navigation systems. Ionospheric irregularities are caused by fluctuations in the ionosphere’s electron density [6–8] and are the outcome of interfering refracted and/or diffracted waves which tend to alter radio signals and interfere with satellite communication and navigation systems, and degrades the positional accuracy. Ionospheric scintillation causes rapid phase and amplitude fluctuations, and satellite unlock causes cycle slip, in which the receiver loses lock on the signal and the signal cannot be tracked due to the lack of satellite measurements during the unlock period. As a result, the level of noise increases, degrading the signal strength. Loses of lock result in integer discontinuities in phase measurements, which are referred to as “cycle slips.” The occurrence of cycle slips is increased by high ionospheric activities (scintillations). The detection and correction of cycle slip in the case of ionospheric scintillation are critical not only for positioning and navigation but also for ionospheric scintillation research. Low SNR owing to poor ionospheric conditions and intense geomagnetic storms is the cause of cycle slips (scintillations) [9–14]. It is supposed that the ionosphere is uniform, but due to the presence of various irregularities like a solar flare, solar wind, and cosmic energetic particles uniformity is disturbed which causes ionospheric scintillations [3–5]. Many research has been conducted on the investigation of scintillation and its impact on GPS and other GNSS receivers using scintillation indices S4 and σ ϕ but recent studies show Rate of TEC index (ROTI) method is better for analysis of scintillation events and represents positioning errors [11, 15, 16]. The S4 scintillation index is used for measuring amplitude scintillation resulting in loss of received signal being tracked by satellite, whereas σ ϕ is the phase scintillation index resulting in cycle slips. The indian regional satellite navigation system (NavIC) provides accurate real-time positioning and positioning services over India and the region extending to 1500 km around India. Three geostationary satellites orbiting at 32.5° East, 83° East, and 131.5° East, and four geosynchronous satellites orbiting at 29° inclination. At 55°

Analysis of NavIC Signal Data for Topological Precision

105

E and 111.75° E, two pairs of GSO satellites cross the equator. IRNSS uses dualfrequency bands which is the main reason for its accuracy as compared to GPS using a single L-band frequency. The L5 band has frequencies between 1164.45 and 1188.45 MHz, while the S-band has frequencies between 2483.5 and 2500 MHz [10, 17, 18]. Scintillation is most intense in low-latitude areas and also strong in high latitudes during disturbed conditions. The ionospheric scintillations are more active in low-latitude regions due to an equatorial fountain. Ionospheric fluctuations have the greatest impact on low-elevation geostationary satellite signals, making this the most critical concern in NavIC positioning and navigation applications. As a result, NavIC satellites are better suited for ionospheric scintillation research than GPS satellites [10, 11, 15, 19]. In this paper, Sect. 1 discuss the relevance of precise positioning in many fields, as well as the factors that contribute to the deterioration of positional accuracy in the case of NavIC satellites. The experimental setup necessary to conduct this research is discussed in Sect. 2. The method for processing NavIC data to determine the total disturbed S1 band period is covered in Sect. 3. The results have been obtained based on the discussion in Sect. 4, and Sect. 5 contains concluding remarks based on the results.

2 Experimental Setup The basic setup of the IGS receiver hardware is depicted in Fig. 1, which acquires the NavIC, GPS, and GAGAN signals and is deployed at Graphic Era University, Dehradun, with latitude 30° N 16.045' and longitude 77° E 59.723' . In case of NavIC dual frequency signals L5 and S1 are obtained, whereas GPS and GAGAN only receive a single L-band frequency. In MATLAB, eight individual actual data files each of 3 hours are recovered and compiled to generate a 24 hours data file. The collected and assembled data in UTC format must be translated to IST format for signal processing. The data obtained consists of TOWC(s), azimuth and elevation angle (deg), pseudorange (m), doppler range (m), Iono-delay (m), C/N o (dB-Hz), and carrier delay (cycles) [10, 15]. The IGS receiver also provides other information such as satellite positions (sky plot), number of satellites, DOP (dilution of precision), and receiver velocity and elevation. The pseudorange measurements are highly unreliable, and TEC (total electron content) is obtained every second and processed for ambiguity, such as missing satellite measurements at one or both frequencies, and unexpected measured values (i.e. zero or very high), resulting in scintillations (severe fluctuations) in the determined TEC.

3 Methodology The IGS receiver collects the raw data, which is then pre-processed with MATLAB software. About 60% of 1st order error, or inaccuracy in pseudorange, caused by

106

R. Gusain et al.

Fig. 1 (a) Hardware setup (b) fixed antenna at rooftop for data collection

ionospheric abnormalities and/or satellite unlock is due to positional inaccuracy [20]. For both the frequency bands L5 and S1, the total disturbed period of a single satellite and multiple satellites is recorded. Frequent zero-value data from one or more satellites that can last for several minutes or hours is recorded for different parameters (C/N o , azimuth, elevation, pseudorange, and carrier range). The data was studied for one week, from December 18th, 2019 to December 23rd, 2019. A single satellite unlocking event lasts for a minimum of 1 second and a maximum of 18 hours, whereas multiple unlocking events last for a minimum of 1 second and a maximum of 2 hours. Multiple satellites unlock for a minimum of 1 second and a maximum of 2 minute in a single event, whereas multiple satellites unlock for a minimum of 1 minute and a maximum of 3 hours multiple times. As can be seen, positional accuracy is not maintained during satellite unlocking or scintillation events, hence modelling and forecasting of scintillation effects can be useful for improving positional accuracy.

4 Results and Discussions The carrier-to-noise ratio for different satellites at L5 and S1 band is plotted in Fig. 2a the C/N o (dB) at L5 shows less fluctuations as compared to S1 or dip in C/N o is frequent in S1 band Fig. 2b as compared to L5 band for different satellites for the highly disturbed day December 18, 2019. The strength of signal is weak for S1 band as compared to L5 band due to ionospheric scintillation and/or satellite unlock resulting in cycle slip error. Similar analysis of azimuth and elevation angle is plotted in Fig. 3a azimuth (degree) at L5 for all the six satellites does not show random variations as compared to S1 band Fig. 3b showing major fluctuations in the signal. In Fig. 3c elevation (degree) at L5 and Fig. 3d at S1 band shows random fluctuations. Analysis of pseudorange and carrier range at L5 and S1 band is shown in Fig. 4a–d depicting large variations. Furthermore, the estimated total disturbed

Analysis of NavIC Signal Data for Topological Precision

107

Fig. 2 Analysis of carrier to noise ratio performance (a) C/N o (dB) at L5 (b) C/N o (dB) at S1

period at S1 band for analyzing NavIC data (azimuth, elevation, pseudorange, and carrier range) for all the six satellites on December 18, 2019 is shown in Fig. 5a–d. A brief summary of NavIC data for the determination of the overall disturbed period for S1 band is given in Table 1.

Fig. 3 Analysis of azimuth and elevation (a) azimuth (degree) at L5 (b) azimuth (degree) at S1 (c) elevation (degree) at L5 (d) elevation (degree) at S1

108

R. Gusain et al.

Fig. 4 Analysis of pseudorange and carrier range (a) pseudorange (meters) at L5 (b) pseudorange (meters) at S1 (c) carrier range (cycles) at L5 (d) carrier range (cycles) at S1

5 Conclusion Data pre-processing and cycle slip error detection and removal techniques can solve the low C/N o problem that is obstructing the PPP due to satellite unlocking and Ionospheric scintillation. Satellite unlocking results in data with zero values being collected from one or more satellites due to the low C/N o . The duration can be several minutes long, with a total disturbance time of several hours. Cycle slip error is caused by ionospheric scintillation (low C/N o ) and/or satellite unlock, therefore, data must be pre-processed (zero value data can be pre-processed before positioning). The detection and elimination of cycle-slip errors are major challenge in PPP, and research is still ongoing. If cycle slips occur regularly, a better approach is to directly detect and rectify the cycle slips at the epoch. As a result, PPP may be used to model

Analysis of NavIC Signal Data for Topological Precision

109

Fig. 5 Analysis of NavIC data (azimuth, elevation, pseudorange and carrier range) for estimating total disturbed period at S1 band (a) azimuth (degree) at S1 (b) pseudorange (meters) at S1 (c) elevation (degree) at S1 (d) carrier range (cycles) at S1

Table 1 Analysis of the NavIC data for the determination of the overall disturbed S1 band period Date

Total disturbed period (second)

Single satellite duration (second)

More than one satellite duration (second)

Dec. 18, 2019

53,750

5320

3865

Dec. 19, 2019

67,710

8640

7895

Dec. 20, 2019

60,560

8340

7210

Dec. 21, 2019

48,860

7690

5385

Dec. 22, 2019

49,250

4430

3750

Dec. 23, 2019

53,770

5480

3640

the troposphere and ionosphere, with the other case benefiting from the removal of the second order error, which in PPP applications can be quite significant.

110

R. Gusain et al.

Acknowledgements The authors would like to express their gratitude to the Indian Space Research Organization (ISRO) space application center (SAC) in Ahmedabad for providing instruments and technical support for this study.

References 1. X. Luo, Z. Liu, Y. Lou, S. Gu, B. Chen, A study of multi-GNSS ionospheric scintillation and cycle-slip over Hong Kong region for moderate solar flux conditions. Adv. Space Res. 60(5), 1039–1053 (2017). https://doi.org/10.1016/j.asr.2017.05.038 2. Z. Liu, A new approach for cycle slip detection and fix using single GPS receiver’s single satellite dual frequency data containing arbitrarily large pseudo range errors. J. Glob. Position. Syst. 16(1) (2018). https://doi.org/10.1186/s41445-018-0013-8 3. C. Cai, Z. Liu, P. Xia, W. Dai, Cycle slip detection and repair for undifferenced GPS observations under high ionospheric activity. GPS Sol. 17(2), 247–260 (2013). https://doi.org/10.1007/s10 291-012-0275-7 4. S. Ji, W. Chen, D. Weng, Z. Wang, X. Ding, A study on cycle slip detection and correction in case of ionospheric scintillation. Adv. Space Res. 51(5), 742–753 (2013). https://doi.org/10. 1016/j.asr.2012.10.012 5. S. Banville, R. Sieradzki, M. Hoque, K. Wezka, T. Hadas, On the estimation of higher-order ionospheric effects in precise point positioning. GPS Solutions 21(4), 1817–1828 (2017). https://doi.org/10.1007/s10291-017-0655-0 6. S.N.V.S. Prasad, P.V.S. Rama Rao, D.S.V.V.D. Prasad, K. Venkatesh, K. Niranjan, On the variabilities of the total electron content (TEC) over the Indian low latitude sector. Adv. Space Res. 49(5), 898–913 (2012). https://doi.org/10.1016/j.asr.2011.12.020 7. T. Das, B. Roy, A. DasGupta, A. Paul, Impact of equatorial ionospheric irregularities on transionospheric satellite links observed from a low-latitude station during the minima of solar cycle 24. Indian J. Radio Space Phys. 41(2), 247–257 (2012) 8. M. Ulukavak, M. Yal, Analysis of ionospheric anomalies due to space weather conditions by using GPS-TEC variations (2018) 9. S.C. Bharadwaj, A. Vidyarthi, Process of detection, determination and correction cycle slip error: a review, in 2020 6th International Conference on Signal Processing and Communication (ICSC), pp. 331–336 (2020). https://doi.org/10.1109/ICSC48311.2020.9182734 10. S. Sinha, R. Mathur, S.C. Bharadwaj, A. Vidyarthi, B.S. Jassal, A.K. Shukla, Estimation and smoothing of TEC from NAVIC dual frequency data, in 2018 4th International Conference on Computing Communication and Automation, ICCCA 2018, pp. 1–5 (2018). https://doi.org/10. 1109/CCAA.2018.8777665 11. S.C. Bhardwaj, A. Vidyarthi, B.S. Jassal, A.K. Shukla, Study of temporal variation of vertical TEC using NavIC data, in 2017 International Conference on Emerging Trends in Computing and Communication Technologies, ICETCCT 2017, 2018-January, pp. 1–5 (2018). https://doi. org/10.1109/ICETCCT.2017.8280317 12. S.C. Bhardwaj, S. Shekhar, A. Vidyarthi, R. Prakash, Satellite navigation and sources of errors in positioning: a review, in Proceedings—2020 International Conference on Advances in Computing, Communication and Materials, ICACCM 2020, pp. 43–50 (2020). https://doi. org/10.1109/ICACCM50413.2020.9212941 13. S. Bhardwaj (n.d.), Evaluation of seasonal variability of first-order ionospheric delay correction at L5 and S1 frequencies using dual-frequency Navic system 14. S.C. Bhardwaj, A. Vidyarthi, B.S. Jassal, A.K. Shukla, Investigation of ionospheric vertical delay at S1 and L5 frequencies, based on thick-shell model using navic system, for mid latitude region of India. Prog. Electromagn. Res. M. 100(Nov 2020), 197–211 (2021). https://doi.org/ 10.2528/PIERM20112301

Analysis of NavIC Signal Data for Topological Precision

111

15. S. Sinha, S.C. Bhardwaj, A. Vidyarthi, B.S. Jassal, A.K. Shukla, Ionospheric scintillation analysis using ROT and ROTI for slip cycle detection, in 2019 4th International Conference on Information Systems and Computer Networks, ISCON 2019, pp. 461–466 (2019). https:// doi.org/10.1109/ISCON47742.2019.9036215 16. T.S. Sharan, S. Tripathi, S. Sharma, N. Sharma, Encoder modified U-net and feature pyramid network for multi-class segmentation of cardiac magnetic resonance images. IETE Tec. Rev. (2021). https://doi.org/10.1080/02564602.2021.1955760 17. S. Shekhar, R. Prakash, A. Vidyarthi, D.K. Pandey, Sensitivity analysis of navigation with Indian constellation (NavIC) derived multipath phase towards surface soil moisture over agricultural land, in 2020 6th International Conference on Signal Processing and Communication, ICSC 2020, pp. 138–142 (2020). https://doi.org/10.1109/ICSC48311.2020.9182714 18. V. Chamoli, R. Prakash, A. Vidyarthi, A. Ray, Capability of NavIC, an Indian GNSS constellation, for retrieval of surface soil moisture. 106(Nov), 255–270 (2020) 19. S. Dan, A. Santra, S. Mahato, A. Bose, NavIC performance over the service region: availability and solution quality. Sadhana—Acad. Proc. Eng. Sci. 45(1), 1–7 (2020). https://doi.org/10. 1007/s12046-020-01375-5 20. S.C. Bhardwaj, A. Vidyarthi, B.S. Jassal, A.K. Shukla, An assessment of ionospheric delay correction at L5 and S1 frequencies for NavIC Satellite System. 2020 Global Conference on Wireless and Optical Technologies, GCWOT 2020, 1, pp. 2–6 (2020). https://doi.org/10.1109/ GCWOT49901.2020.9391601

Advance Computing

Experimental Study on Resource Allocation for a Software-Defined Network-Based Virtualized Security Functions Platform S. D. L. S. Uwanpriya, W. H. Rankothge, N. D. U. Gamage, D. Jayasinghe, T. C. T. Gamage, and D. A. Amarasinghe Abstract With the advancements of technology, outsourcing organizations’ applications such as web servers, email servers, and security functions such as firewalls and intrusion detection systems. To cloud service providers (CSPs) has become a general practice within the business community. CSPs use their cloud infrastructure and provide these applications and virtualized security functions (VSFs) as services. Also, they take the advantage of software defined networks (SDN) approach to managing their network, which uses a software-based controller to maintain a programmable data plane. When providing VSFs as a service, resource management is one of the core aspects to consider as it drives the CSPs business toward profit, utilizing the resources to the maximum. In this paper, we have introduced a resource allocation approach with the objective to use existing non-used VMs with the minimal CPU and RAM variance, compared to the required CPU and RAM for a VSF as much as possible. According to our performance evaluation observations, the resource allocation decision is taken within few milliseconds. Keywords Software-defined networks (SDN) · Virtualized security functions (VSF) · Cloud service provider (CSP) · Resource allocation

1 Introduction Cloud computing has gained a significant attraction from the business community as it allows organizations to outsource their application servers and network functions to a cloud service provider (CSP) and run them inside the CSP’s infrastructure [1–6]. Software applications in the fields such as machine learning, health, and education can be implemented in the cloud infrastructure and offered as services [7, 8]. With the S. D. L. S. Uwanpriya · W. H. Rankothge (B) · N. D. U. Gamage · D. Jayasinghe · T. C. T. Gamage · D. A. Amarasinghe Sri Lanka Institute of Information Technology, Malabe, Sri Lanka e-mail: [email protected] N. D. U. Gamage e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_10

115

116

S. D. L. S. Uwanpriya et al.

advancements of technologies, now the CSPs have expanded their services to offer not only application servers and network functions but also different security functions as virtualized entities inside their cloud infrastructures. These security functions include firewalls, intrusion detection systems, intrusion prevention systems, etc., and they are known as virtualized security functions (VSFs). According to the work in [9], VSFs can be classified into four types of functions: Attack detection, prevention, deception, and mitigation. When offering VSFs as services, CSPs face several challenges, which are specific to security functions [9, 10]. VSFs are generally deployed as a set of functions, not a single function, and these set of functions require to be deployed in a sequence, which is known as a function chain. For an example, when a client request for a firewall instance and a deep packet inspection (DPI) instance, if CSP deploys the DPI instance first, and then the firewall instance, all the traffic flows will be deeply analyzed by the DPI, even though some of them will be dropped by the firewall later. Therefore, the sequence of deployment of the VSFs chain: firewall and DPI, is very important [11, 12]. Also, if these two security functions are deployed on two different servers, energy cost will be increased compared to deploying them in a single server. On the other hand, the use of a single server may lead to an overloaded server and reduced reliability. Deploying VSFs and allocating resources considering different aspects such as cost, energy consumption, load balancing, delay, network congestion, and network security requirements is complicated. Some of these factors are conflicting with each other, and therefore optimizing the placement of VSFs and resource allocation is a time-consuming optimization problem. In this research, we are exploring resource allocation approaches that simultaneously respond to the operational requirements of the network and do not compromise the security policies. We have assumed cloud platforms that uses a software defined network (SDN) based architecture to ensure centralize management platform and easy programmability of the network [13, 14]. We have introduced a resource allocation approach with the objective to use existing non-used VMs with the minimal CPU and RAM variance, compared to the required CPU and RAM for a VSF as much as possible. Only if none of the existing non-used VMs matching with the requirements, new VM will be created. According to our performance evaluation observations, the resource allocation decision is taken within few milliseconds. The resource allocation platform is integrated with two additional modules: network traffic monitoring and network traffic forecasting modules. After the initial resource allocation, according to the predicted traffic changes, CSP can scale in/out the resources dynamically in the future. The remainder of the paper is laid out as follows. Section 2 describes existing resources allocation methods in-depth. The proposed resource allocation approach is presented in Section 3. Our observations are presented in Section 4. The research conclusion and possible future work are presented in Section 5. Finally, references are listed.

Experimental Study on Resource Allocation for a Software …

117

2 Related Work Resource allocation in a cloud platform is a well explored area, especially when offering virtualized applications and virtualized network functions (VNFs) as services [15–17]. However, there are limited works for resource allocation for VSFs. In this section, we will discuss the existing work. In the work of [15], authors propose a resource allocation strategy based on a long short-term memory (LSTM) where the training operation is based on an asymmetric cost function minimization which can differently weighs the positive and negative prediction errors and the corresponding costs for over provisioning and under provisioning of resources. With their experiments, authors show that the proposed approach gains 30% cost saving. The authors of [18] propose SFC routing and cloud and bandwidth resource allocation (SRCBA) algorithm, which is based on segmented based routing. The algorithm follows two main steps: first, the n service function chains (SFC)s are sorted in decreasing bandwidth order, next it decides the placement in terms of point-of-presence (PoP) and network paths. The work of [19] proposes a path mapping strategy to solve the NFV resource allocation problem for decomposed network service chains, considering optimizing the average embedding cost, while providing an enhancement for the average execution time. When compared to the traditional approaches, the proposed method has reduced the execution time 39.58% and the operational cost significantly. The research work of [20] proposes a new multi-stratum resources integration (MSRI) architecture that is based on NFV in SDN data center with optical interconnections. Also, they have implemented a resource integrated mapping scheme for the specific architecture, that integrated the optimization of optical network and application resources. The authors of [21], propose resource allocation algorithms for network virtualization and resource allocation in wireless networks. The proposed algorithm follows two processes: virtualize the physical wireless network as multiple slices (separate virtual networks) and allocate physical resource within each virtual network. The authors use orthogonal frequency division multiplexing (OFDM) as to achieve efficient resource utilization. The work on [9] has done a comparative study on different approaches for optimal placement of VSFs. They have identified open research challenges VSFs placement and potential future work that can be carried out by interested researchers. The authors of [22] suggest an integer linear programming (ILP) based optimization model for VSFs placement in a network with the objective of minimizing server energy consumption.

3 Methodology The main objective of this research is to explore on a resource allocation algorithm that can optimize resource allocation process in a SDN based cloud platform that

118

S. D. L. S. Uwanpriya et al.

offers VSFs as a service. We have implemented the resource allocation platform, where the users can request for a set of VSFs, with their specific requirements, such as the required RAM size and number of CPUs . Also, they can specify the number of VSF instances of a specific type of VSF, they would like to request in the initial stage. Our resource allocation platform is integrated with two additional modules: network traffic monitoring and network traffic forecasting modules. Therefore, according to the predicted traffic changes, CSP can scale in/out the resources dynamically in the future without being sticked to the initial requirements given by the user. Scaling feature is offered to the users as a value-added service. The full system architecture, including all three modules: resource allocation, traffic monitoring, and traffic prediction, is given in Fig. 1. To process the clients resource allocation requirements, two different scripts are being used as follows: (1) script to create EC2 instance(s) to hosts the VSF(s) and (2) script to configure the EC2 instances with the required settings to host the VSF(s). The resource allocation platform process with the sequence of scripts execution is given in Fig. 2. A. Script to create EC2 instance(s) to hosts the VSF(s) When a customer requests a set of VSFs through the platform interface, the resource allocation process will be initiated through the script that creates EC2 instance(s) to hosts the VSF(s). First, the script will run the resource allocation algorithm to decide whether the user requirements can be satisfied with any of the existing virtual machines (VMs). All the information about existing non-used VMs is stored in a central database. To

Fig. 1 Resource allocation methodology

Experimental Study on Resource Allocation for a Software …

119

Fig. 2 Resource allocation platform process

decide the most suitable existing non-used VM, the algorithm considers two facts: RAM requirements and CPU requirements. The objective is to use the existing nonused VM with the minimal CPU and RAM variance, compared to the required CPU and RAM. Equations 1 and 2 are used by the resource allocation algorithm to decide the most suitable existing non-used VM. CPU Variance = RAM Variance =

%(CPU of existing VM − Required CPU) Required CPU ∗ 100

(1)

%(RAM of existing VM − Required RAM) Required RAM ∗ 100

(2)

The algorithm calculates CPU variance and RAM variance for each existing nonused VM and selects the existing non-used VM that gives the minimum variance. In case if more than one non-used VM has same minimum variance, the algorithm selects a random VM, out of them. If none of the existing non-used VMs has the required requirements, the resource allocation algorithm will create a new EC2 instance. The algorithmic approach is given in Fig. 3. After creating the EC2 instance, the data related to the created instance will be saved in the database. The data includes RAM, CPU details, active or not, etc. This information can be used by the CSP in the future for purposes such as load balancing between VSFs instances.

120

S. D. L. S. Uwanpriya et al.

Fig. 3 Resource allocation algorithm

B. Script to configure the EC2 instances with the required settings to host the VSF(s) Once the most suitable existing non-used VM is selected, or a new VM is created, as the second step, the script to configure the instances with the required settings will be executed. This script includes commands to configure resource parameters: CPU and RAM according to the customer’s requirements. In a situation where more than one VM instance is required as per the customer’s requirement, then script will include the resource parameters accordingly. After the script is executed, the VM is ready to be used for the VSF installation. C. Resource allocation for Scaling After the initial resource allocation: VSF instance creation and starting the VSF, the resource allocation platform monitors the network traffic for traffic pattern changes. In a situation where the network traffic is increasing and go beyond a pre-defined threshold, the resource allocation platform decides to add resources appropriately. First, it follows the vertical scaling (scaling up), where more CPU and RAM is added to the existing VNF instance. If the server where the existing VNF is residing already overloaded, then the resource allocation platform follows horizontal scaling (scaling out) and creates a new VNF instance. When creating a new VNF instance, the resource allocation platform follows the same procedure that was discussed in previous sub-section, with the objective to use the existing non-used VM with the minimal CPU and RAM variance, compared to the required CPU and RAM.

Experimental Study on Resource Allocation for a Software …

121

Fig. 4 After the VNF instances are created

4 Results and Discussion In this section, the performance evaluation of the resource allocation platform will be presented. To evaluate the resource allocation platform in a SDN cloud-based platform that offers VSFs as services, we have used AWS as the cloud platform. We have connected a database to store information of VSFs instances: instance id, the status of the instance, name of the instance, private IP address, and public IP address. We have assumed client requests for virtual firewalls and simulated the process of resource allocation platform. According to our observations, the resource allocation platform takes the resource allocation decision within few milli-seconds, achieving the optimization objectives. Once the resource allocation decision is taken, the VNF instance in the AWS cloud platform is created within few minutes according to the CPU and RAM requirements given. Figure 4 shows the output after the VNF instance is created.

5 Conclusion In this research, we have explored a resource allocation approach for a cloud platform that uses a SDN-based architecture and offers VSFs as services. The main objective with the resource allocation decision is to use existing non-used VMs with the minimal CPU and RAM variance, compared to the required CPU and RAM for a VSF as much as possible. Only if none of the existing non-used VMs matching with the requirements, new VM will be created. According to our performance evaluation observations, the resource allocation decision is taken within few milliseconds. The resource allocation platform is integrated with two additional modules: network traffic monitoring and network traffic forecasting modules. After the initial resource allocation, according to the predicted traffic changes, CSP can scale in/out the resources dynamically in the future. As the future work, we are planning to explore more existing resource allocation approaches and compare the performances for a SDN based cloud platform that offer VSFs as services cost-effective solution to overcome present high-cost challenges, the implemented solution will be an open-source solution. Since it is an open-source solution customer can customize this based on their preferences and that will be a benefit to the customers.

122

S. D. L. S. Uwanpriya et al.

References 1. W. Rankothge, J. Ma, F. Le, A. Russo, J. Lobo, Towards making network function virtualization a cloud computing service, in IEEE IM 2015 2. W. Rankothge, F. Le, A. Russo, J. Lobo, Experimental results on the use of genetic algorithms for scaling virtualized network functions, in IEEE SDN/NFV 2015 3. W. Rankothge, F. Le, A. Russo, J. Lobo, Optimizing resources allocation for virtualized network functions in a cloud center using genetic algorithms, in IEEE Transaction of Network and Services Management 2017 4. W. Rankothge, Past before future: a comprehensive review on software defined networks road map. Glob. J. Comp. Sci. Technol.: C Softw. Data Eng. April 2019 5. W. Rankothge, H. Ramalhinho, J. Lobo, On the scaling of virtualized network functions, in IEEE IM 2019 6. J. Ma, W. Rankothge et al., An overview of a load balancer architecture for VNF chains ho horizontal scaling, in IEEE CNSM 2019 7. W.H. Rankothge, D.M.S.B. Dissanayake, U.V.K.T. Gunathilaka, S.A.C.M. Gunarathna, C.M. Mudalige, R.P. Thilakumara, Plant recognition system based on neural networks, in ICATE 2013 8. W.H. Rankothge, S.V. Sendanayake, R.G.P. Sudarshana, B.G.G.H. Balasooriya, D.R. Alahapperuma, Y. Mallawarachchi, Technology assisted tool for learning skills development in early childhood, in ICTER 2011 9. S. Demirci, M. Demirci, S. Sagirogl, Virtual security functions and their placement in software defined networks: a survey. Gazi Univ. J. Sci. 2019 10. J. Carapinha, J. Jiménez, Network virtualization: a view from the bottom, in VISA’09: Proceedings of the 1st ACM Workshop on Virtualized Infrastructure Systems and Architectures, 2009 11. H. Hu, G.-J. Ahn, Virtualizing and utilizing network security functions for securing software defined infrastructure, in NSF Workshop on Software Defined Infrastructures and Software Defined Exchanges, Washington, D.C., USA, pp. 70 (2016) 12. R. Sherwood, G. Gibb, K.-K. Yap, G. Appenzeller, M. Casado, N. McKeown, G. Parulkar, A Network Virtualization Layer (2009) 13. Software-defined networking (SDN). VMware [Online]. Available: https://www.vmware.com/ topics/glossary/content/software-defined-networking 14. What is software-defined networking (SDN)?. blueplanet [Online]. Available: https://www.blu eplanet.com/resources/What-is-SDN.html 15. V. Eramo, Proposal and investigation of an artificial intelligence (AI)-based cloud resource allocation algorithm in network function virtualization architectures. Future Internet 12(Nov 2020) 16. S. Papavassiliou, Software defined networking (SDN) and network function virtualization (NFV). Future Internet 12(Jan 2020) 17. A. Haider, R. Potter, A. Nakao, Challenges in resource allocation in network virtualization, in 20th ITC Specialist Seminar, Hoi An, 2009 18. V. Eramo, F.G. Lavacca, T. Catena, M. Polverini, A. Cianfrani, Effectiveness of segment routing technology in reducing the bandwidth and cloud resources provisioning times in network function virtualization architectures. Future Internet 12(March 2019) 19. B. Raddwan , K. AL-Wagih, Path mapping approach for network function virtualization resource allocation with network function decomposition support. Symmetry 11(9), 16 Sept 2019 20. H. Yang, J. Zhang, Y. Ji, R. Tian, J. Han, Y. Lee, Performance evaluation of multi-stratum resources integration based on network function virtualization in software defined elastic data center optical interconnect 21. X. Lu, K. Yang, Y. Liu, D. Zhou, S. Liu, An elastic resource allocation algorithm enabling wireless network virtualization, in Wireless Communications and Mobile Computing, pp. 295– 308, 28 Dec 2012

Experimental Study on Resource Allocation for a Software …

123

22. S. Demirci, M. Demirci, S. Sagirogl, Optimal placement of virtual security functions to minimize energy consumption, in 2018 International Symposium on Networks, Computers and Communications (ISNCC)

Smart Intelligent Drone for Painting Using IoT: An Automated Approach for Efficient Painting P. Vidyullatha, S. Hrushikesava Raju, N. Arun Vignesh, P. Haran Babu, and Kotakonda Madhubabu

Abstract There are many occasions, a few people may die because of lack of efficient resources, lack of technology, and dependent more on human efforts when painting for high buildings and high temple arches. Whenever the high buildings and temples arches need to be coloring, they consume more time and a greater number of labor to complete it. The finishing of the given task depends on many external factors. To be independent of many external factors, a smart drone with loaded colors is assigned and loads the texture that is expected as output. As the intelligence is loaded to the drone as well as the specific sensors are embedded to it in order to notify the information about the color’s deficiency is detected, any resource to be required to complete the given task according to the given texture, any accidental collisions also notified and be a safeguard, and etc. The output of this study is an array that allows analysis on how many are identical textures, coloring were processed done using SSIM measure. The major advantages are manpower is becoming almost NIL while been painting; the time consumed for the task given is to be completed in less time. The accuracies and their performances are measured and are depicted in the results when comparing this approach against the traditional and semi-automated approaches. Keywords IoT · Smart drone · SSIM · Customized colors · Texture · Intelligence · Alerting · Performance · And Accuracy

P. Vidyullatha · S. H. Raju (B) · P. H. Babu · K. Madhubabu Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Green Fields, Vaddeswaram, Guntur 522502, India e-mail: [email protected] N. A. Vignesh Department of ECE, Gokaraju Rangaraju Institute of Engineering and Technology, Hyderabad, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_11

125

126

P. Vidyullatha et al.

1 Introduction Generally, the painting for temples as well as high buildings is a challenging task. This task is to be simplified using IoT and image acquisition and color picking approaches. In conventional approach, the man power is required as well as skilled labor is mandatory to get the design as we expect as outlook. In this context, the series of steps such as discussion with owner requirements from a variety of textures, designs, and etc., once confirmed the layout style, mix the appropriate colors, follow the color layout design, need to start from one side and try to move to other sides, need to mix the colors according to the design at color styles, need to mix again the colors when it became shortage for filling or become wastage when more quantity is mixed for coloring, and etc. All these activities may become positives and other time may become disadvantages particular to the scenario. For a small house or tiny hut, the scenario is somehow takes half day or one day but for a temple arches and high buildings consume more time even for a month or even for more time. In the process, the labor may die accidentally or mistakenly because of their negligence; cost for labor cost is more and may miss painting at some sides and difficult portions outside. Although by giving the salaries or the daily wages that would benefit the labor to survive their families, but results in time consuming and is not effective. Hence, the trend is expecting to automate the painting process and quality of painting is improved using the drones. There were certain paintings from specific states that are to be circulated to the public to get the revenue generation, and these are available in readymade forms which may be sticked to the walls. India is famous for certain painting types such Gond, Miniature, Madhubani, and Warli. These are also translated to digital form which could be painted permanently on the walls using our approach for user satisfaction. The proposed approach involves the following activities to complete the goal. (1) Make a computer-based image with customized colors on the each portion of original setup. (2) Would get the amount of colors needed based on partitions of the whole set-up. (3) Set-up the drone with customized color load based on the portions and provided whole image as well as partitioned regions. (4) Embedded few appropriate sensors for alerting certain issues and monitor the coloring taking place correctly or not. (5) If any color to become shortage based on texture given and scenario when painting really, would alert such scenario to fulfill the desired color design. (6) Drone set-up with palette as well as brushes in order to paint the required regions, start painting from one region till last region is completed. (7) Estimate the time as well as the accuracy, generate the graphs over such parameters.

Smart Intelligent Drone for Painting Using …

127

2 Literature Review Many studies are available but they are deal with different scenarios in which traditional approach only would be used and demonstrated. As per demonstration of [1], it describes various mobile robots that would walk, fly, and etc. They are going to handle many services like network communication, coordination, implementing AI, self driving, emotion expression, and etc. From the perspective of [2], it demonstrates the automatic robotic spraying in the automobiles field and their advantages such as reduced labor, improved quality, and cost effectiveness. It depends on control parameters such as fir shaping and painting flow. The estimation of dry film thickness (DFT) is computing using prediction model called multiple regressions. In regard to [3], it proposes trajectory planning based on point cloud slicing technology. The point cloud work piece is obtained using laser scanning. The section polysemy is computed on average, and normal points are estimated for smooth painting. Finally, interpolation algorithm is applied for space trajectory of spraying robot. It is proposed for automobiles painting. From the description of [4], it deals with many approaches that are used to paint the required freeform surface in which few are automated and two are traditional of kind. Among the models, electro-static technique became more popular but facing a fire hazard issue. In the view of [5], it deals with intelligent paint system that paints complex and any contour shape with benefits no wastage of paint and operation efficiency. It may face electrical hazard during the operation. In the view of [6], it captures the image using pi camera and it consists of DC motor, actuator, and raspberry pi for spraying the paint intelligently w.r.t. the input image. From the point of [7], it provides smart robot to paint based on three axis gantry approach that paints on the position of the parts and dry those parts. It increases production rate and operation efficiency is observed. With regard to [8], it designed a specific spray gun based on PLC micro-controller, which would automate the painting in most of industrial areas. Using gun and conveyor belt, the benefits such as cost reduced, productivity increased and reduces human efforts. As per [9], it deals with developing a spatial painting with exponential mean Bezier curves. It leads to frame a trajectory optimization for complex curves with satisfactory performance. From the discussion from [10], it proposes robotic spraying that consist of scissor lift for vertical moving and four wheeled base for horizontal moving, and IoT, sensors for distance measurement for painting. As per aim of [11], it proposes a Taguchi orthogonal array and gray relational analysis with parameters such thickness, surface roughness and film adhesion. The result is better when compared with other existing methods. From the orientation of [12], it proposes robot spraying of paint using path optimization approach. It discusses paint disposition rate as well as film thickness on the surface to get better performance. With respect to the description of [13], it also provides a variety of methodologies for painting. Among them, few are traditional; few are automated and discussed with pitfalls as well as benefits. From the source [14], it designs robot spraying that computes trajectory using geometric primitives and paints the surfaces. The laser scanner would read the surface and gun would spray the paint by following the trajectory. In regard to [15], it deals with drawing the art

128

P. Vidyullatha et al.

digitally using supporting painting applications such as 3D paint, adobe and etc. The applications consist of tool kit as well as environment to draw. As per [16], it deals with software with more effective resources that support the user to draw the image in digital form. As per direction of [17], it provides variety of products that spray based on the complexity of the surface, and it consist of motor, gun, and bucket of colors instantly mix for design of the texture. In regard of [18] and [19], the former deals with the IoT and respective sensors are used to detect the force of the pressure and simulate the cyclone there, and latter deals with detection of trained objects in the environment and generates a report on the objects. As per demonstration of [20] and [21], the former denotes IoT and its sensors that interact with remote doctor that programmed to detect the eye-sight and order the spectacle by the neatest shop tracked by the GPS, and latter provides a set-up of digital mask with enabled sensors and IoT for detecting infected COVID footprints in the environment. In the perspective of [22] and [23], the former deals with embedding of IoT and its sensors for automatic detection and payment of currency in the native form, and latter deals with detecting the weighted object, catches it using appropriate sensors and IoT. Regarding of [24] and [25], IoT and sensors are useful for detecting the ingredients in the food, also analyze the health bulletin on the user and estimate health in the future days in the former case, and latter case deal with GPS, IoT and sensors that deal with visited users reviews and recommend the places when user would plan to make trip on a specific place. As per [26] and [27], the former deal with app where mobiles could communicate for charging instantly using IoT and latter deals with detecting the fire leakages in the pipe and alert to the authority in advance using appropriate sensors and IoT. In regard of [28], the detection of fire in the forests using IoT and sensors and avoids by alerting. In regard to [29], it provides how to use SSIM measure efficiently so that computational burden is used. It leads to define multi-scale SSIM (MS-SSIM) with accuracy and performance as expected. Every study mentioned here would like to demonstrate the usage of IoT in their operation directly or indirectly in addition to the sensors as well as with available resources.

3 Proposed Work In this, there are three modules identified which are such as set-up resources, smart intelligent drone, and reporting the status. These modules would interact with each other for further progress of the intended objective (Fig. 1). The architecture of smart drone painting system is demonstrated using entities and attributes in the Fig. 2. (A) Set-up the resources: It requires the needed resources to start the painting. The inputs supplied such as digital form of picture, the area which needs painting, feed these two to the smart intelligent drone. It provides entities like bucket with allowed permitted weight, multiple colors are provided which could be

Smart Intelligent Drone for Painting Using …

Fig. 1 Flowchart of modules in the smart drone painting

Fig. 2 Smart drone painting architecture

129

130

P. Vidyullatha et al.

mixed based on the input digital picture, and specific sensors are equipped with the drone for alerting when the obstacle is found, alerting when the operation is completed, alerting when required to load colors required and etc. Pseudo_Procedure Setup_Resources (Colors[][], Digitial_Pic, Area): Input: Digital_Pic which should be whole image with sub-regions in case of large project or simple image in case of small project Output: Making ready to start painting Step 1: Design a digital image using available software where adobe is preferred Step 2: If Handling large project: Decompose that image into sub-regions, each of which is demonstrated with further elaborated digital pictures else Single digital image that would guide drone Step 3:Provides allowed weight for carrying the predefined-color bucket for instant mixing and painting Step 4:Provides sensors for taking appropriate action when specific behavior is identified Step 5: Based on time to finish, the drones count to be increased. (B) Smart Intelligent Drone: It is powered by charging, it may be constraint which should be overcame by instant charging from remote device when such low battery is found. The painting portion whatever is did is to be saved and continue with remaining portion in the sub-regions in case of large projects or rest in the small project when revoked the process. Hence, it is characterized by battery power, continue from rest when revoked from its previous work state, alerting when specific activity is identified, dissipate the information among the sensors and to the authorized center, scan the portion before painting, apply the knowledge of area and texture color from the supplied digital picture, and compare each sub-region against painting completed sub-region in case of large project and compare the specific portion against completed painted same portion. Pseudo_Procedure Smart_Intelligent_Drone(Sensors[],power, saving_painted_portions, specific_events): Input: Power, drone Output: Comparing the completed painting region against existing same region using structural similarity index (SSIM) Step 1: Verifies which side to start when using single drone, Provides number of drones, Assign decomposed digital images of the whole image differently to each drone. Step 2: Each sub-region when completed, report the time stamp for easy generation of the report.

Smart Intelligent Drone for Painting Using …

131

Step 3: For each sub-region, provide color bucket the drone carry for painting needs to be checked, fill the bucket based on estimated colors % required to complete, loop this until all assigned sub-regions are completed. Step 4: In the loop process of painting, any obstacle is found during its painting, alerting and expecting input from the user. Step 5: If the colors shortage is found, alert the specific color or combination of colors. It uses level reading sensor. Step 6: If the specific surfaces are unable to paint, alert the location of the spot to the user for further use action. It uses fixing sensor that return the co-ordinates and its spot in a digital form of supplied picture. Step 7: When the iteration reaches nth sub-region, and found its successfully completed, Alert the user operation is completed Step 8: Compare operation is applied in each sub-region against the painted and proposed, return 0 in each identical case using SSIM measure. (C) Reporting Status: It make a note on every abnormal activity found and generates a report that consists of duration of time taken to finish each sub-region in case of large project or the specific portion in case of small project. Pseudo_Procedure Report_Status_Intimation(Sub_regions[]): Input: Painted Sub-regions Output: Array of results after comparison Step 1: Loop from sub-region1 to sub-regionN Result[i] = SSIM(painted_sub_region[i], sub_region[i]) Save time stamp when success Save also time stamp when obstacle is identified, and expecting input from the user Step 2: Save all the components, generate a report Step 3: Based on painting result, further analysis is carried for improvement

4 Results The design of the smart drone intelligent painting system is demonstrated in Fig. 3. The order of activities from taking the input to processing, then to the output is stated in Fig. 4. The illustrative case study for this proposed system is demonstrated in Fig. 5. The accuracy and performance of the proposed method are demonstrated in Fig. 6.

132

P. Vidyullatha et al.

Fig. 3 Smart drone intelligent system with equipped resources

Fig. 4 Order of activities occurrence in the smart drone intelligent system

In Fig. 3, the integration of many resources would cause to form a prototyped form that would originate the intelligence of how to paint using spray gun and electrostatic magnetic approach as default. It leads to no wastage of paint while doing on proposed building or its interior or its exterior. For complex regions to paint although the digital form of the image is supplied, the intended approach plays a key role. For illustrating the temple type scenario, the

Smart Intelligent Drone for Painting Using …

133

Fig. 5 Illustrative scenario of temple gopurams (domes) painting

digital image is supplied first. It is sliced into sub-regions based on complexity, then automates the paint based on partitions assigned. The data represented in Fig. 6 for the proposed system against other existing approaches for evaluating the performance and accuracy. The performance is computed by taking the method in X-axis and time (minutes) in Y-axis and concludes that intended system would complete in quick time than other approaches. The accuracy is computed by taking the method in X-axis and % of accuracy in the Y-axis and concludes that intended system would work more effectively than other approaches.

134

P. Vidyullatha et al.

Fig. 6 Performances and accuracies of proposed against other approaches

5 Conclusion In this, the designed system would take digital form of the picture, which then broken into the sub-regions based on complexity. The painting process is initiated by scanning the every texture from the environment before painting in the order of sub-regions, start painting one after the other. Once all sides of a given task is completed, the array that obtained would consist of how accurate the identical portions are extracted using SSIM measure. During this process, any obstacle is found; user is alerted on it for further input. It also has capability to revoke from where it stopped last time, also able to notify when shortage of color composition is found, and alerted when certain surfaces are missed because of complex orientation, not able to track such portions. The performance and accuracy parameters judges the intended approach is worthy when compared against other approaches. The time consumed by this approach is very less and is not dependent on skill of the painter. The accuracy is computed based on number of identical painted regions against partitioned digital image.

References 1. F. Rubio, F. Valero, C. Llopis-Albert, A review of mobile robots: concepts, methods, theoretical framework, and applications. 16(2), April 16, 2019. https://doi.org/10.1177/172988141983 9596 2. K.V. Chidhambara, B.L. Shankar, Vijaykumar, Optimization of robotic spray painting process parameters using Taguchi Method. In: IOP Conference Series: Materials Science and Engineering, Volume 310, International Conference on Advances in Materials and Manufacturing Applications (IConAMMA-2017), August 2017

Smart Intelligent Drone for Painting Using …

135

3. X. Wei Chen, H.G. Li, L. Wang, Y. Zhang, Trajectory planning for spray painting robot based on point cloud slicing technique. Electronics 9(6), 908 (2020). https://doi.org/10.3390/electr onics9060908 4. The different types of paint application methods (2021). https://www.spraysystems.com/thedifferent-types-of-paint-application-methods/ 5. S.A. Shah, S. Vaidya, Spray-painting robot for painting irregular workpieces. Int. Res. J. Eng. Technol. (IRJET) 6(6 June 2019). e-ISSN: 2395-0056. https://www.irjet.net/archives/V6/i6/ IRJET-V6I6143.pdf 6. C.G. Nayana, M. Nandini, Development of autonomous wall painting robot using Raspberry Pi and image processing. IETE—2020 8(11). https://www.ijert.org/development-of-autonomouswall-painting-robot-using-raspberry-pi-and-image-processing 7. A. Saeed, H.M. Mithaiwala, A.I. Hussain, T. Kumar, Development of smart painting machine using image processing, 3C Tecnología. Glosas de innovación aplicadas a la pyme, 9(4), 95–119 (2020). https://doi.org/10.17993/3ctecno/2020.v9n4e36.95-119 8. P.S. Adne, S. Bansod, Automatic spray painting machine using PLC operated pneumatic spray gun, in IRJET Oct 4, 2019. https://www.irjet.net/archives/V6/i7/IRJET-V6I7158.pdf 9. W. Chen, J. Liu, Y. Tang, J. Huan, H. Liu, Trajectory optimization of spray painting robot for complex curved surface based on exponential mean Bézier method, November, 2017. https:// www.hindawi.com/journals/mpe/2017/4259869/ 10. E.A.E.I. Sulaiman, A.A.M. Ali et al., Design of an interior wall painting robot, October, 2017. Sudan University of Science and Technology College of Engineering 11. R. Bhalamurugan, S. Prabhu, Performance characteristic analysis of automated robot spray painting using Taguchi method and gray relational analysis. Arab. J. Sci. Eng. 40, 1657–1667 (2015). https://doi.org/10.1007/s13369-015-1628-7 12. W. Chen, D. Zhao, Path planning for spray painting robot of workpiece surfaces. Math. Prob. Eng. 2013, Article ID 659457, 6 pages (2013). https://doi.org/10.1155/2013/659457 13. Paint application methods, September, 2011. https://www.pfonline.com/articles/todays-paintapplication-methods 14. A. Pichler, M. Vincze, H. Andersen, K. Hausler, A method for automatic spray painting of unknown parts, IEEE Xplore, February 2002. https://doi.org/10.1109/ROBOT.2002.1013400 15. G.Y. Annum, Digital painting evolution: a multimedia technological platform for expressivity in fine art painting. J. Fine Studio Art 4(1), 1–8 (2014). https://doi.org/10.5897/JFSA2014. 0025 16. J. Cui, Research on digital painting art and its diversified performance, in 3rd International Conference on Economics, Social Science, Arts, Education and Management Engineering (ESSAEME 2017) 17. Automatic spray paint machine (2015). http://roboticpaint.com/ 18. G.S. Rao, Dr. S.H. Raju, Dr. L.R. Burra, V.N. Mandhala, P.S.R. Krishna, Smart cyclones: creating artificial cyclones with specific intensity in the dearth situations using IoT, February, 18–19th, SMART DSC-2021, Springer Series 19. V.L. Lalitha, S.H. Raju, V.K. Sonti, V.M. Mohan, Customized smart object detection: statistics of detected objects using IoT, in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 1397–1405.https://doi.org/10.1109/ICAIS50930.2021.939 5913 20. S.H. Raju, L.R. Burra, S.F. Waris, S. Kavitha, S. Dorababu, Smart eye testing, in Advances in Intelligent Systems and Computing, 2021, ISCDA 2020, 1312 AISC, pp. 173–181. https://doi. org/10.1007/978-981-33-6176-8_19 21. Dr. P. Tumuluru, Dr. S.H. Raju, CH.M.H. Sai Baba, S. Dorababu, B. Venkateswarlu, ECO friendly mask guide for corona prevention. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi. org/10.1088/1757-899X/981/2/022047 22. CH.M.H. Sai baba, Dr. S.H. Raju, M.V.B.T. Santhi, S. Dorababu, S.F. Waris, International currency translator using IoT for shopping. IOP Conf. Ser. Mater. Sci. Eng. 981(4). https://doi. org/10.1088/1757-899X/981/4/042014

136

P. Vidyullatha et al.

23. R. Mothukuri, Dr. S.H. Raju, S. Dorababu, S.F. Waris, Smart catcher of weighted objects. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757-899X/981/2/022002 24. Dr. S.H. Raju, Dr. L.R. Burra, S.F. Waris, S. Kavitha, IoT as a health guide tool. IOP Conf. Ser. Mater. Sci. Eng. 981(4). https://doi.org/10.1088/1757-899X/981/4/042015 25. Dr. S.H. Raju, Dr. L.R. Burra, Dr. A. Koujalagi, S.F. Waris, Tourism enhancer app: userfriendliness of a map with relevant features. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https:// doi.org/10.1088/1757-899X/981/2/022067 26. N. Sunanda, S.H. Raju, S.F. Waris, A. Koulagaji, Smart instant charging of power banks. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757-899X/981/2/022066 27. M. Kavitha, Dr. S.H. Raju, S.F. Waris, Dr. A. Koulagaji, Smart gas monitoring system for home and industries. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757899X/981/2/022003 28. V.R. Karumanchi, S.H. Raju, S. Kavitha, V.L. Lalitha, S.V. Krishna, Fully smart fire detection and prevention in the authorized forests,in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 573–579.https://doi.org/10.1109/ICAIS50930. 2021.9395969 29. A.K. Venkataramanan, C. Wu, A.C. Bovik, I. Katsavounidis, Z. Shahid, A hitchhiker’s guide to structural similarity, Date of latest submission January 30, 2021. https://arxiv.org/pdf/2101. 06354.pdf

Auto-Guide for Caroms Playing Using IoT CH. M. H. Saibaba, S. Hrushikesava Raju, P. Venkateswara Rao, S. Adinarayna, and M. Merrin Prasanna

Abstract The playing of the caroms manually or by computer environment would not be sent percentage accurately. In order to get sent out of sent results, IoT simulator is created over the carom board with audio as an aid. This proposed system integrates manual approach with simulated approach in order to achieve the 100% accuracies. There are certain situations where this system gives free hand to the player when there is no guaranty of result. The proposed system has many scenarios which are trained with accurate results; such successful scenarios would yield to accurate results. In a total of player turns, this system completes the rounds in less time with the support of the players co-ordination compared to manual approach. The experience of this proposed system is extraordinary and result in self-satisfaction. The predefined recorded set of successful scenarios are helpful in giving the guidance to the player, that would help to complete the board with efficient expected results than the results of manual or other existing approaches. The player’s sideline border is fixed with each programmed sensor, which directs which coin if hits, would be successfully fallen. The factors considered here are performance and accuracy, which makes the system more effective using IoT and specific machine learning technique. Keywords IoT · Machine learning · Sensors · Accurate · Caroms · Playing · Performance · Audio guide

CH. M. H. Saibaba · S. H. Raju (B) · P. V. Rao Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Green Fields, Vaddeswaram, Guntur 522502, India e-mail: [email protected] CH. M. H. Saibaba e-mail: [email protected] S. Adinarayna Department of CSE, Raghu Institute of Technology, Visakhapatnam, India M. M. Prasanna Department of ECE, Sri Venkateswara Institute of Science and Technology, Kadapa 516001, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_12

137

138

CH. M. H. Saibaba et al.

1 Introduction There are two kinds in playing the caroms are existed in which first is manual approach and second is computer-based automated approach. But when second approach is preferred, the user satisfaction may not guarantee. In manual caroms playing, player may lose the game at the critical timing at the crucial times to hit the coins. Human is emotional and may choose wrong coins, sometimes also behave with over-confidence. On board of players, players might choose wrong coins with over-confidence and emotions, which leads to lose the game rounds. To support the players who prefer the manual game, machine learning approach is preferred that guide predicts which coin to play and to what direction to hit, and pressure to keep. This help support may lead to the player to choose right selection of coins and direction. In the first approach, many factors to be simulated to guide at crucial times in playing. Such factors considered are of which coin to choose, toward which direction to hit, how much pressure to put for hitting, next coin to play by spotting, etc. In the second approach, the logic and selection of coins is done by the inherent approach and is also processed by the prediction logic incorporated based on the dimension of the board, color of the coins, and rules are framed using if, if else blocks, etc. In this computer approach, the board is of standard size but the boards in real time are of varying size. It requires the customized framework with many predefined libraries, which alert the user by audio and auto-spotter on the coins to play at present situation as well as future situations. The expected environment is to be simulated over the board and is to be dynamic and would change the guiding option based on player position. From those ways, playing the caroms is to be done. Among these, user expectations and to support to the manual approach, the virtual environment is created and fixed over the carom board which consists of many predefined tasks such as from starting of the game, recorded rules of the game, player turns to take in each round, points updation, the number of rounds fixed to decide the game-winner, etc. These embedded activities require support of specific sensors and built-in libraries for implementation of game playing. Each step while playing is to be guided by the virtual environment feed over the top of the board with space between the board and virtual environment. This kind of approach helps to the users to have a comfortable space to play but recommending the steps through the audio. When the player turn is over, automatically next player picks the turn to play and leads to dynamic outputoriented environment. It is customized in setting up of number rounds, number of teams, processing each round using the built-in rules, and accumulating players score after each step, etc. Hence, the simulated environment consists of the following order of steps to output the winner: (1) (2) (3) (4)

A light board with specific sensors is to be embedded. Program each sensor that extracts data from manual board and player. Impose rules and policies. Start from the first team till last team for their turns in each round. If all coins are cleared at any round, the round is finished.

Auto-Guide for Caroms Playing Using IoT

139

(5) Run this step 4 till the last round, and meantime, compute the coins total for each time. (6) After completion of specific rounds, the player who has more total is announced as winner.

2 Literature Review In this, the approaches that were worked on the simulation of games were taken as study and their objective is to be demonstrated in this section and also overview the significant studies that earmarked the evolution in this domain. As per overview of [1], the design of simulation based on the training and teaching approach is preferred for the development of games in terms of phases like applying and evaluation also. From the perspective of [2], it enhances the teaching learning by conducting post reviews over the simulation of game. This approach helps to produce the best teaching module by adding some touch-ups required. In the view from [3], it provides evaluation over pre-game, in-game, and post-game through the defined phases such as beginning, middle, and final stages. This leads making learning effect that optimizes the elements in the game and in more possible manner. In the respect of [4], the approaches such as exponential or case-based were used to evaluate the teaching learning over the simulation game, and draft the guidelines to make the best possible outcome. With the aspect of [5], the design of the board is elevated using C language and obtained from graphics header file and its in-built methods for getting floor in which player has to play. In the view from [6], the bot is designed that makes the average of the player and selects next shot to play. This kind of intelligence is provided by horizontal fixer with a stepper motor that would implement artificial intelligence. As per discussion from [7], the VR technology is useful for the development of many games where caroms are one in which players role are simulated. The attributes such as latency rate, the frame rate, and threshold affect the experience of the game. With regard to [8], the C programming language is used to build up the design of the carom board and simulation of full board. With the view of [9], the simulation of game is done using AI opponent and VR approach. The play of players is provided using artificial intelligence for outputting winner. As per from [6], design the tutor that helps to highlight the playing of the game. The factors are also shown the values and are adjusted in making the board principles to know. In the perspective from [10], the design of the board with colors, lines, and many required entities is implemented using C program language. Many in-built methods are used in getting appropriate design. From the source [11], many games are to be built using temporal difference learning and beam mini-max algorithm. The logic is to be implemented and is automated using the mentioned principles. From the overview from [12], the specific evaluation and active learning helps to report empirical work, and suitable frameworks might extract the relationship from the design. It helps to classify the games based on relationship among entities involved. In the aspect of [13], the player has to start collecting the items, and clearing the levels, as the game

140

CH. M. H. Saibaba et al.

progresses. There few games are explored which fall under the “clear world.” From the perspective of [14], the significant development methods are discussed in which big bang, and agile models are preferred in order to finish the game development in less time. With the review of [15], it focuses on pre-production, production, and postproduction activities in which postproduction to be paved more attention by the researchers in the future. It describes many useful resources and explores key utilities. It includes many factors like art, intelligence and coordinating makes it different from traditional methods. From the mentioned sources from [16–26], every study involves integration of appropriate sensors that extract the information from one module and give to other module in the process. This leads to automation of work and minimizes the human efforts. They provide detection of suitable spot and generates artificial cyclones, detection of objects using a smart camera and generates the report over the identified entities, remote application of virtual doctor and detecting eyesight, detection of infected entities in the surroundings for the corona pandemic, detection, and translation of currency automatically when shopping in the international shops, detecting the weighted objects and safely catch and drop at the floor, detection of health status in the future weeks based on past food habits, detection of ranked places and suggest to the user for travel guide using GPS, detection of portable gadget and supplies battery power from once device to required through an designated application, detection of gas leakages in the installed pipes and alert in advance to the client about the damage, and detection of fires and provide ways to avoid fire incidents in the forests in order to save wild animals respectively. The many studies specified here are classified based on the type of the process, implementation, and information of it. Hence, the proposed approach aims to focus on simulation of virtual environment that guides which coin to play and updates the scores after the round and announces the winner accurately using IoT where sensors are incorporated and intelligence is embedded. The summaries were depicted in Table 1.

3 Proposed Work This system explores the architecture of the intended system, the flowchart of the system, and the pseudo-procedures of the modules were depicted. For making the implementation to be simple, the theme is divided into three modules which are mentioned as below: (A) Scanning and Criteria for Playing: The objective of this is provided by the lighting module in a fixed aerial manner. It scans the board at the time of starting of the game. It is flexible for board dimension means that supports any type of the board. It calculates the corners in the pixel form, also loads the criteria such as striker to be between the think horizontal bar at the player size including circles and should not hit the coins over the diagonal arrow arcs at the player side, updates the scoreboard of a team whenever the player hit the coin over the corner, counts when the striker falls into the corner, hitting the coin directly

Auto-Guide for Caroms Playing Using IoT

141

Table 1 Demonstration of specific approaches in the literature review Type of method

Theme

Remarks

Teaching evaluation method

It makes learning concept more effective and would pass through certain phases to make quality resource

Discussed on learning and teaching for doing simulation of games but not involve real practice

Board design using C language

It uses built-in libraries that Logic not implemented for helpful in making the board how the play act with lines, arrows, corners, etc.

Carom bot

It is a device that chooses next coins and incorporates the stepper motor

It is computer automated game

Caroms using AI

Intelligence of playing and incorporation of principles are provided for completion of the game

Need to provide better accuracy

Game software development reviews

It describes development life cycle as well as required entities that make a difference against other methods

Need to choose a specific SDLC and complete in less time as per clients trend

Simulation of IoT

Different applications were built and are related to various real-time applications

Automation is to be excelled and required to enhance the theme wherever possibility is there

when only red coin was present, If any violation of rules like touching the coins at diagonal arcs, hitting directly the coin when red is alone was there, etc. (B) Incorporation of Intelligence: It processes the scores for each sub-round, switching players when hit is unsuccessful and there are coins of their color, updates score for each round, and computes total scores after specific rounds were over. It uses “dynamic coin_to_corner” as user-defined method in which prediction of which coin to push to the corner is suggested and that instruction is delivered by an audio. It is in communication with first module in order to give negative points when violates the rules of the board. (C) Instant Reporting: It takes coins into the account, each sub-round score w.r.t. player-wise is observed that leads to the round score after a specific team colored is over. It monitors the other rounds till the last round from the specific rounds as given as input. Also, each round number of negative scores is to be observed which would be later analyzed for better player involvement. It computes the accuracy achieved based on the output of the intended objective. Also, it floats the review form in which player has to fill details and rate it so that the satisfaction is judged. The following flow of interaction among the modules that were identified in this approach is depicted in Fig. 1. ER Architecture of semi-virtual approach is represented in Fig. 2.

142

Fig. 1 Flow of interaction of modules in the intended system

Fig. 2 ER Architecture of semi-virtual approach

CH. M. H. Saibaba et al.

Auto-Guide for Caroms Playing Using IoT

143

(A) Input: Dimensions of the board Output: Able to process and produce the valid or invalid coin selected Step 1: Fix (c1, c2), (c2, c3), (c3, c4), (c4, c1) as corners Step 2: Fix the space between (HC1, HC2), (HC2, HC3), (HC3, HC4), and (HC4, HC1) Step 3: Fix Diagonal Arcs such (DA1, DA2), (DA2, DA3), (DA3, DA4), and (DA4, DA1) Step 4: Read Rounds, score = 0, Teams, setup coins, load the starting coins value Step 5: Declare SuccessF_T1, FailF_T1, SuccessF_T2, FailF_T2, Negative_T1, Negative_T2, Negative_T1_count, Negative_T2_count Step 6: for sub_round to rounds if SuccessF_T1: play again till FailF_T1, then turn on T2 T1_Count+ = Coin_Value if SuccessF_T2: play again till FailF_T2, turn on T1 T2_Count+ = Coin_Value if coins_value_T1 are cleared: sub_round is completed; Count = count + 1; T1_round_count+ = 1 if coins_value_T2 are cleared: sub_round is completed; Count = count + 1; T2_round_count+ = 1 Step 7: If count == Rounds: Game is over if T1_round_count > T2_round_count: T1 Win else T2 Win (B) Pseudo_Procedure Dynamic_Coin_to_Corner (c1, c2, c3, c4) Input: corner pixels c1, c2, c3, c4 Output: Round count to be produced Step 1: If T1 is on: If coin == White T1_count = T1_count + 10 else if coin == Red and coin == White T1_count = T1_count + 25 else T2_count = T2_count + 5

144

CH. M. H. Saibaba et al.

If T2 is on: If coin == Black T2_count = T2_count + 5 else if coin == Red and coin == Black T1_count = T1_count + 25 else T1_count = T1_count + 10 Step 2: If T1 violates (HC1, HC2) or (HC3, HC4) or striker == (c1, c2) or (c2, c3) or (c3, c4) or (c4, c1) Negative_T1 = T1_count - 10 Negative_T1_count+ = 1 If T1 violates (HC2, HC3) or (HC4, HC1) or striker == (c1, c2) or (c2, c3) or (c3, c4) or (c4, c1) Negative_T2 = T1_count - 5 Step 3: If T1 violates (DA1, DA2) or (DA3, DA4) Negative_T1 = T1_ count - 10 Negative_T1_count+ = 1 Step 4: If T2 violates (DA2, DA3) or (DA4, DA1) Negative_T2 = T2_ count - 2 Negative_T2_count+ = 1 Step 5: Send these elements to instant reporting module (C) Pseudo_Procedure Instant_Charging (Rounds, Teams) Input: Rounds, Teams Output: Generation of the statistics Step 1: Each sub-round, T1 or T2 win is to be recorded based on clearing board Step 2: if number of times T1 > number of times T2: T1 win else T2 win Step 3: T1 and T2 how many times they involved in negative scenarios, to be recorded so that further analysis is to be judged.

4 Results In this, the snapshots of the proposed system and its working environment are demonstrated with descriptions.

Auto-Guide for Caroms Playing Using IoT

145

Fig. 3 Simulation of caroms playing using intended approach

The order of demonstration of output screens are windows encountered during the game process where first is basic carom board, second is feeding of inputs, third is fitting of lighting board with sensors as intelligence, fourth is round-1 details were specified in Fig. 3, and from Fig. 4, the details such as Round-2, Round-3, and final statistical report are produced. There are two ways to address the results of the intended system where the performance is viewed in a graph, and the accuracy is depicted in another graph. The list of approaches that are used to run the code and their execution compared to other approach is provided as below (Table 2). The above values w.r.to the approaches classified are depicted in Fig. 5. The list of approaches that are used to showcase their accuracy w.r.to other approaches is provided as below (Table 3). The accuracy is defined as number of correctly predicted samples by the number of total samples. Based on this, the values are mentioned based on the considered approaches and are depicted in Fig. 6. This intended system is getting popularized based on user satisfaction and is evaluated based on reviews collected from the clients. The list of approaches that are used to showcase their user satisfaction w.r.to other approaches is provided as below (Table 4).

146

CH. M. H. Saibaba et al.

Fig. 4 Information updating in the proposed system

Table 2 Performance observed for the mentioned approaches Name of method

Performance

Remarks

Manual approach

50

Emotions and over-confidence may cause losing of the game

Semi-automated approach

75

Trying to satisfy the user (player) and is brought based on user satisfaction

Full automated approach

100

User satisfaction is not guaranteed

The approaches considered for getting reviews and evaluation are carried based on these reviews. Figure 7 displays the performance of semi-virtual against other approaches. Figure 5 represents the performance of semi-virtual against other approaches.

5 Conclusion Based on the user satisfaction IS only the parameter, the intended system is taken up and should cater the carom players who are as beginning and intermediate level. The design is divided into three modules such as loading the criteria and scanning, incorporation of required intelligence, and instant reporting. These modules are to be taken in order, and they would communicate with each other to complete the goal.

Auto-Guide for Caroms Playing Using IoT

147

Fig. 5 Performance of semi-virtual against other approaches Table 3 Accuracy observed for the mentioned approaches Name of method

Accuracy (user satisfaction)

Remarks

Manual approach

50–60

Error rate is more

Semi-automated approach

70–90

Error rate is very less and should be focused w.r.to manual

Full automated approach

100

Error rate is NIL

Fig. 6 Accuracy of semi-virtual against other approaches

148

CH. M. H. Saibaba et al.

Table 4 User-self-satisfaction observed for the mentioned approaches Name of method

Accuracy (user satisfaction)

Remarks

Manual approach

30–50

User would play but emotions may degrade because of lack of guidance

Semi-automated approach

90–100

User should get on card because of good guidance

Full automated approach

0

It is automated but player is not active

Fig. 7 Performance of semi-virtual against other approaches

In this, the logic is processed by the middle module intelligence implementation there negative points, round-wise score are tracked and maintained. These details would be propagated to instant reporting where each player score, each sub-round score, and negative scores are displayed. Based on this, few other parameters such as efficiency and accuracy are also be evaluated, and they are differentiated in each separate visual graph.

References 1. J. McCormick, Simulation and gaming as a teaching method, July, 2006.https://doi.org/10. 1080/1355800720090405 2. C. O¸toiu, G. O¸toiu, Testing a simulation game as a potential teaching method for a masters course in human resources management. Procedia—Soc. Behav. Sci. 33, 845–849 (2012). ISSN 1877-0428, https://doi.org/10.1016/j.sbspro.2012.01.241 3. N.D. Faizan, A. Löffler, R. Heininger, H. Krcmar, Evaluation methods for the effective assessment of simulation games: a literature review, January, 2019, vol. 916. Springer.https://doi.org/ 10.1007/978-3-030-11932-4_59

Auto-Guide for Caroms Playing Using IoT

149

4. A. Löffler, D. Jacoby, N. Faizan, M. Utesch, H. Kienegger, H. Krcmar, Teaching methods for simulation games: the example of learning business process change, in 2019 IEEE Global Engineering Education Conference (EDUCON), 2019, pp. 1336–1344.https://doi.org/10.1109/ EDUCON.2019.8725263 5. C program to design a carrom board using graphics, April, 2021. https://www.geeksforgeeks. org/c-program-to-design-a-carrom-board-using-graphics/ 6. A. Doshi, S. Ranade, H. Painter, M. Kanojiya, Automated carrom bot. 7(6, June 2020). https:// www.irjet.net/archives/V7/i6/IRJET-V7I6782.pdf 7. M.Z. Khan, F. Hassan, M. Usman, U. Ansari, S. Noor, Virtual reality in multiplayer carrom game with artificial intelligence, in 2018 12th International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS), 2018, pp. 1–5. https://doi.org/10.1109/ MACS.2018.8628394 8. C program to design carrom board game using graphics programming full project for beginners, June, 2021. https://codingshiksha.com/c/c-program-to-design-carrom-board-game-using-gra phics-programming-full-project-for-beginners/ 9. P. Nanaware, S. Huisan, A. Bhalerao, T. Rathod, S. Raskar, Multiplayer carrom board game with an AI opponent. 7(5, May 2020). https://www.irjet.net/archives/V7/i5/IRJET-V7I51162. pdf 10. A. Prajapati, Computer graphics in C for CARROM design, October, 2012. https://blog.clecot ech.com/2012/10/program-for-carrom-design.html 11. Building an AI that can beat you at your own game, March, 2018. https://towardsdatascience. com/pathwayz-ai-3c338fb7b33 12. H.F. O’Neil, R. Wainess, E.L. Baker, Classification of learning outcomes: evidence from the computer games literature, December, 2005.https://doi.org/10.1080/09585170500384529 13. A.F.S. Barbosa, P.N.M. Pereira, J.A.F.F. Dias, F.G.M. Silva, A new methodology of design and development of serious games, June, 2014, vol. 2014, Article ID 817167. https://doi.org/10. 1155/2014/817167 14. Game Development Lifecycle Models. https://www.studytonight.com/3d-game-engineeringwith-unity/game-development-models 15. S. Aleema, L.F. Capretza, F. Ahmed, Game development software engineering process life cycle: a systematic review. J. Softw. Eng. Res. Dev. 4(6), 1–30 (2016). https://doi.org/10.1186/ s40411-016-0032-7 16. G.S. Rao, Dr. S.H. Raju, Dr. L.R. Burra, V.N. Mandhala, P.S.R. Krishna, Smart cyclones: creating artificial cyclones with specific intensity in the dearth situations using IoT, February, 18–19th, SMART DSC-2021, Springer Series 17. V.L. Lalitha, S.H. Raju, V.K. Sonti, V.M. Mohan, Customized smart object detection: statistics of detected objects using IoT, in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 1397–1405.https://doi.org/10.1109/ICAIS50930.2021.939 5913 18. S.H. Raju, L.R. Burra, S.F. Waris, S. Kavitha, S. Dorababu, Smart eye testing, in Advances in Intelligent Systems and Computing, 2021, ISCDA 2020, 1312 AISC, pp. 173–181. https://doi. org/10.1007/978-981-33-6176-8_19 19. Dr. P. Tumuluru, Dr. S.H. Raju, CH.M.H. Sai Baba, S. Dorababu, B. Venkateswarlu, ECO friendly mask guide for corona prevention. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi. org/10.1088/1757-899X/981/2/022047 20. CH.M.H. Sai baba, Dr. S.H. Raju, M.V.B.T. Santhi, S. Dorababu, Er. S. F. Waris, International currency translator using IoT for shopping. IOP Conf. Ser. Mater. Sci. Eng. 981(4). https://doi. org/10.1088/1757-899X/981/4/042014 21. R. Mothukuri, Dr. S.H. Raju, S. Dorababu, S.F. Waris, Smart catcher of weighted objects. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757-899X/981/2/022002 22. Dr. S.H. Raju, Dr. L.R. Burra, S.F. Waris, S. Kavitha, IoT as a health guide tool. IOP Conf. Ser. Mater. Sci. Eng. 981(4). https://doi.org/10.1088/1757-899X/981/4/042015 23. Dr. S.H. Raju, Dr. L.R. Burra, Dr. A. Koujalagi, S.F. Waris, Tourism enhancer app: userfriendliness of a map with relevant features. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https:// doi.org/10.1088/1757-899X/981/2/022067

150

CH. M. H. Saibaba et al.

24. N. Sunanda, S.H. Raju, S.F. Waris, A. Koulagaji, Smart instant charging of power banks. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757-899X/981/2/022066 25. M. Kavitha, Dr. S.H. Raju, S.F. Waris, Dr. A. Koulagaji, Smart gas monitoring system for home and industries. IOP Conf. Ser. Mater. Sci. Eng. 981(2). https://doi.org/10.1088/1757899X/981/2/022003 26. V.R. Karumanchi, S.H. Raju, S. Kavitha, V.L. Lalitha, S.V. Krishna, Fully smart fire detection and prevention in the authorized forests, in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 2021, pp. 573–579.https://doi.org/10.1109/ICAIS50930. 2021.9395969

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage Anupam Tiwari

and Usha Batra

Abstract The world today is witnessing a paradigm shift attributed to expedited penetration of new technologies in the ecosystem of routine human lives. One such domain, soon to be impacted is the way traffic and transportation system is organized today. The futurity of transport will be completely different from what we have today. It is going to be based on Internet-of-Vehicles. Still in evolving times though, the envisaged growth of the Internet-of-Vehicles has thrown multiple challenges of ensuring seamless, secure, robust exchange of information between devices. These challenges majorly include large data sets storage, real-time intelligent management and information security for the entire ecosystem of these connected vehicles. The Internet-of-Vehicles being proliferated further by autonomous vehicles in the not so distant future will add to the existing challenges. Blockchain technology, majorly associated with cryptocurrencies, has recently envisaged a range of benefits vide implementation across multiple domains, by offering its inbuilt characteristics including decentralized architecture, enhanced security, immutable and transparency, etc. This paper explores the advantageous connect between smart contracts and Internet-of-Vehicles and need of distributed storage to redefine trust enhancement, security and high storage generated among devices. It proposes architecture based on smart contracts while also simulating a limited part with Internet-of-Vehicles concluding with discussion on challenges ahead. Keywords Blockchain · Internet-of-Vehicles · Internet of things · Smart contracts · Distributed storage

1 Introduction “Internet-of-Vehicles” (IoV) relates to a distributed network which primarily constitutes of data building from a multitude of connected vehicles on the vehicular ad hoc networks (VANETs). Secure and infallible communication between these connected A. Tiwari (B) · U. Batra Department of CSE & IT, G D Goenka University, Gurgaon, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_13

151

152

A. Tiwari and U. Batra

vehicles in real time with the pedestrians, drivers, fleet management systems, etc., thus is critical to the overall ecosystem of IoV performance. The IoV ecosystem would deem to have connected to a myriad of communication networks [1] from which it is expected to derive useful real-time analytics for smooth operations. These would primarily include the following: • • • • •

Pedestrians and vehicles network Infrastructure en route and vehicles network Vehicles and vehicles network Dashboard diagnostics and vehicle network Vehicle to anything network.

IoVs are evolving finally from concept to realization today and thus is foreseen the plethora of challenges they come up with. At the onset, these challenges to IoVs would primarily include handling real-time traffic load data sets in a secure manner and ensuring no malicious intervention can influence the data and analytics generated.

1.1 Cyber Risks and IoV When any idea originates to achieve a functional benefit and has the power to make a paradigm shift in a society, the focus inherently remains to bring it to realization and subsequently implement it. So has been the road map of IoVs till date, where the focus has been primarily to bring it to ground realization without focusing at the onset on security and cyber risks deemed. Vide Fig. 1, it is seen that every major part in a vehicle as part of the IoV ecosystem, has an associated IT element [2] which may be an afflictive vulnerable security hole. In an IoV ecosystem context, the factors like firmware’s of each module, vulnerable APIs, interoperability issues and bugs, spoofing attacks, tampering of modules, repudiation, privacy attacks, distributed denial of service and escalation of privileges, etc., can be interpreted as possible vulnerabilities [1] into risking the IoV vehicles. The primary purpose of constituting an ecosystem of IoV is better traffic safety and efficiency in real time which could only be possible if the data sets are shared at the right time with the right node in the deemed design. As bought out above, there would be many orchestrated networks to attain this purpose, but at the same would also be multiple times vulnerable to cyber security violations, lest adequately catered for. Vide Fig. 1, the major vehicle subsystems as seen, if either of them are penetrated by a malicious intervention, could end up with a critically grievous repercussion. The problem escalates when multiple vendors of different vehicles with different sub vendors orchestrate together to run the network with multitude of unknown attack vectors. Thus, it is critical in an IoV ecosystem to assure authenticity of messages, authentication and non-repudiation of participating nodes. The fine tuning between privacy and security parameters will be of key importance to affect the security cordon architecture of IoV ecosystem which can be architecture for provisional privacy.

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

153

Fig. 1 Risk vectors: subsystem in a IoV node

Blockchain, by virtue of its inherent characteristics, can be of huge advantage to facilitate the exchange of data sets in a secure manner without any malicious interventions and attempts. The characteristics including decentralized nature, immutability, smart contracts, transparency, consensus, etc., will be of critical significance to realize the benefits in an ecosystem of IoVs [3]. While we infer the need and significance of exploiting these blockchain characteristics, it will also be significant to understand at the same time need for distributed storage.

2 Distributed Storage Decentralization is core to the blockchain technology and while we associate any domain like IoV with it, it will be also important to mention that blockchain platforms are not advised for storage of huge data sets owing to limitation of its cardinal parameters and metrics. Blockchains for maximum benefits can be stored with hashes and small bytes but not huge data sets in sizes of GBs and above which invariably will be the case of IoT. It is estimated that by the year 2025, IoT devices will generate 79.4ZB of data or more. Maximum works associating any domain with blockchain invariably take cloud storage as the data storage solution which negates the overall purpose of exploiting blockchain decentralization characteristics. It is

154

A. Tiwari and U. Batra

here that distributed storage protocols can extend the decentralization concept to storage. Evolving distributed storage protocols include InterPlanetary File System (IPFS), Storj, Swarm, Sia to mention a few.

3 Current Works A lot of work is being done to connect the blockchain and IoV domain of which few recent and major works are identified in brief below: • DistBlockNet model [4] for Internet of things (IoT) architecture coalesces the advantages of blockchain and software-defined network (SDN). DistBlockNet models accomplish this by employing the flow table in OpenFlow switches function and putting in the rules generated by the controllers and addressing the packet flows in SDN on blockchain. Notwithstanding, SDN also has its vulnerable side of malicious attacks [5]. • CONNECT [6] leverages the current publically available information received from each single blockchain to have a better knowledge of the surrounding environment. The study focuses to resolve the heterogeneous compatibility and interoperability of different blockchains. • RealTime blockchain authors [7] advise a RealTime blockchain for the IoVs for ascertaining authentication and asserting secure communication among the participating vehicles while also exploiting smart contracts for various services. • Fair blind signature scheme (FBSS)-based secure data sharing system on blockchain [8] addresses the IoV challenges by allowing participating trusted vehicles to get rewarded with some cryptocurrency on broadcasting messages among peer vehicle nodes to index transactions to next block. • In [9] this paper, the authors explore connecting the IoVs in consideration with big data, distributed and secure storage with proposing a model of the outward transmission of vehicle blockchain data supported with theoretical analytics and mathematical ensues. • In [10], the paper suggests a wide area network solution in form of a multi-layer IoT framework established on blockchain, wherein the IoT schema is fractioned into blockchain-based multilevel decentralized networks. • A blockchain architecture set up on data usage audit [11] to protect privacy by ascertaining accessibility and accountability on a hierarchical identity-based encryption mechanism. The architecture depends upon the auditable contracts deployed in the blockchain framework to render with anti-tamper evidence against data usage in complaisance. • CUBE [12] employs deep learning, blockchain, quantum hash encryption, cloudbased intelligence and endpoint protection to ascertain a secure automotive network. Beside above works, [13–17] also present specific works on IoV connect with blockchain architecture. The common connect in all these works predominantly

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

155

moves around smart contracts [18] and different variants of blockchain implementations. All these works have proposed multiple architectures for association with blockchain but storage of IoV data has been left to the realms of cloud or other centralized options.

4 Proposed System The network schema proposed vide Fig. 2, has the following main elements: • Cars (C1–C6), numbered in the schematic, representing six car vehicles equipped with modules as seen in Fig. 2. • Road site units (RSU) are special wireless dedicated short-range communication units placed near to the roads and function as a gateway between on-board units in the cars and the communications equipment. These are majorly responsible for the following:

Fig. 2 Proposed schematic of blockchain-enabled IoV ecosystem

156

– – – – – – –

A. Tiwari and U. Batra

Communicating safety warnings Traffic information Provision connectivity Assistance back up to moving vehicles in vicinity Encoding and decoding Encryption–decryption Communication on Internet

• Trusted centre (TC) mainly contains key distribution centre (KDC), certification authority (CA), simple public key infrastructure (SPKI), authentication centre (AC) and pseudonymous authentication centre (PAC) • Traffic management authority (TMA): Will be overall responsible to co-ordinate the activities and scheduled checks-tasks between different participating agencies and would additionally ascertain to bind the public keys with vehicles in the IoV ecosystem. • Tracers: The tracers are selected by TMA and are responsible to identify the true identity of fake-malicious message generators

4.1 Basic Schema Process Figure 2 shows a schematic of IoV, wherein an ecosystem of six IoVs has been considered for a sample which will be further simulated on Ganache and Truffle Suite. The vehicles part of this ecosystem are further part of the networks defined above and depicted in the schematic system in Fig. 2. Thus, the data hashes generated vide the different communication networks are collated via IPFS storage and published to the blockchain subsequently. The basic process and sequence of the activities go in the following sequence: The vehicles C1–C6 are registered at the TC and get validated and signed for once. • C1–C6 cars apply for pseudonymous certificate (PC) from PAC and get verified before pseudonym is issued. • Vehicles C1–C6 securely inter-communicate among each other and RSUs vide PC. • TC uses confidential algorithm to track the pseudonym of the pertinent malicious vehicle node. • TC convokes nodes of the tracking group to reveal the true identity of the malicious vehicle, and revoke issued PC and penalize as deemed.

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

157

4.2 Hardware and Software Setup 4.2.1

Hardware

• Specifications: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 02), Intel (R) Core (TM) i7-7200U CPU @ 2.50 GHz, Architecture x86_64, Network controller Intel Corporation Wireless 3165 (rev 79), 6xCPUs, 5.0.0–29-generic #31 ~ 18.04.1-LTS GNU/Linux, 16 GB DDR4 RAM. 4.2.2 • • • • •

Software and Applications

Ganache-core to version 2.13.1 [19] for working on a blockchain Truffle Suite version 5.1.52 [19] Metamask version 8.1.3 [20] Solidity v0.7.4 for smart contracts [21] IPFS protocol v0.10.0 [22]

5 Simulations and Results Multiple experiments were conducted to demonstrate the feasibility of the proposed schema in context of implementation of the IoV ecosystem on blockchain. In an envisaged setting, all the cars in the IoV ecosystem (six in given setting) are issued with public–private keys as seen in Table 1: IoV ecosystem on a blockchain has been proposed with smart contracts and exchange of tokens (ethers for simulation in proposed system). Thus for the experiment, five of the token penalties have been put in place in an IoV ecosystem while the move happens from point A to point B. 50 Ethers have been issued by default to each of the participating IoV vehicles. Once on road, these default values will get into credit–debit of ether tokens based on pre-rules set, few of which are defined in Table 2. These envisaged assumptions are just few to mention viz-a-viz an on ground scenario which will have hundreds of traffic violation types. In above, the ten gas price (GWEI) = 21,000, gas limit has been taken into consideration for purpose of calculations while the ether tokens are exchanged. Also the nominal transaction fees applied on the transactions, per se, as per the processing speed and indexing into the blocks of the blockchain are seen in Table 3. In the simulation and experiments intra vehicles in the IoV ecosystem, the following readings and transactions were observed along with block details as seen in Table 4. While the simulations which started with default values of 50 Ethers allotted to each of the participating C1 to C6 cars, the final transaction states of ether balance

f399ae6ed4c92851a28 f179ac9bc7140ceb4f03b6d 30635af5bdfc0701e7876c 5da19b4ddf3775714826a4eb 28f01e8cf5e578587511692 8bb4f4811c385bb6b c7b8656b422aace74586da 37d7372431fe14dcbc67030 047a64b239b7c0835f9

90d93f804064e8b877c090f6 acf7acb32e522e95bdf2e9807 17f98b33395a514 28a66f08f7aede7a78b1107db6 c18a9785193503b9be9b5531 acfd8618a2232e

0xB0908B6e032 fF8F79524292E9 B0175bD713F6aeD

0x25E1DED38B2ec 0839ccE6787225e8 Cc41bE8Bb97

0x784b5cbA80069059 EDE9cCF5a7d4c0F90 01D0aAF

0x1a702009E32F7435d5cc 95dD172a4F54DEe1dcC7

0xC9eC5bEc83028Ad3BFcb F4767Ad1d831a6011749

0xe9d7e06Ef1Be5F1336090 550909729A64dA74599

Car 1

Car 2

Car 3

Car 4

Car 5

Car 6

059b3c037d9db76f7a46f166c32b 48612b6ca6758cf73ffdd9 a847f2eacc4ea6

Private key

Public key

Car node

Table 1 IoV nodes credentials for experiment

158 A. Tiwari and U. Batra

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

159

Table 2 Token exchange set parameters Over speeding

Traffic light violation

Car lights not working

Driving inebriated

Driving discipline

Traffic rule adherence

−1.75

−1.5

−2.25

−2.0

+2.5

+2.0

Table 3 Transaction mining fees set

Transaction mining fees Slow transaction

Average transaction

Fast transaction

0.00017

0.00031

0.00042

was observed as seen in Table 5. These final figures of tokens per vehicle can be regularized further for compensation or penalties against the vehicle owners in periodic maintenance times of effected vehicles at service centres billings.

5.1 Smart Contracts Deployment Three sample smart contracts have been deployed on the configured Ganache blockchain at addresses as seen in Fig. 3: • Brakes (IoV_brakes.sol): Gets activated when the brakes reach the defined parameters of wear-tear and communicate to inform the respective owner, repair agency and supply vendor. • Route (IoV_route.sol): Gets activated when the intended route demands a diversion and effected nodes like battery charging points, fuel refills and TMAs are informed. • Speed (IoV_speed.sol): Gets activated when the defined parameters on speed as per road definitions are exceeded and effected agencies need to be informed. Thus, these smart contracts get actuated and deployed on the criteria designed related to application of brakes or route diversion attributed to any kind of jam foreseen en route or over speeding. The contract compilations of these solidity codes are observed as seen in Fig. 3. Figure 4 shows the smart contract deployment addresses and also shows one IoV_speed contract deployed on C1, i.e. Car 1 address, i.e. 0xB0908B6e032fF8F79524292E9B0175bD713F6aeD. The smart contract deployment details including the transaction hashes and effected blocks as observed in the experiment are seen in Table 6. A sample data set “Inertial sensor data on the right side of the vehicle, combined with GPS data” from [23] was uploaded to IPFS storage along with IoV logs and other data related to present work and is available at IPFS address hash as follows:

Credit vehicle address

0xe9d7e06Ef1Be5 F1336090550909729 A64dA74599

0x1a702009E32F 7435d5cc95dD172 a4F54DEe1dcC7

0x784b5cbA8006 9059EDE9cCF5a 7d4c0F9001D0aAF

0x25E1DED38B 2ec0839ccE6787225 e8Cc41bE8Bb97

Debit sender address

0x784b5cbA800 69059EDE9cCF5a 7d4c0F9001D0aAF

0x784b5cbA8006 9059EDE9cCF5a7 d4c0F9001D0aAF

0xe9d7e06Ef1Be5F 1336090550909729A64dA74599

0x1a702009E32F7435 d5cc95dD172a4F54DEe1dcC7

Transaction hash

0x329a13031d603e217 cd1d2f4229661ef3f4b3e74 bcc3ce49375c180d29a28651 0x1226356868b8e840 a15924073a0182f092effe0b5a924f040f0787eb23b2dafe

−2.25 ETH

0x2b07ccce694b73 e5e92b1b1228917625 a9fd1cb7b66f876e16f5f3323850fcer

0xb1c179d5c7e896 d37f54c26c6854ec9416 d669364f55c83a4935189767901634

+1.25 ETH

+1.50 ETH

+2.25 ETH

Token transfer

Table 4 Experiment data with block and transaction details (block 9 to 21) BLK no

9

10

20

21

160 A. Tiwari and U. Batra

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

161

Table 5 Ethers transaction details Car node

Public key

Initial ethers

Final ethers

Total blocks generated

Car 1

0xB0908B6e032fF8F 79524292E9B0175bD713F6aeD

50

57.73

134

Car 2

0x25E1DED38B2ec 0839ccE6787225e 8Cc41bE8Bb97

50

57.49

Car 3

0x784b5cbA80069 059EDE9cCF5a7d 4c0F9001D0aAF

50

51.74

Car 4

0x1a702009E32F7 435d5cc95dD172 a4F54DEe1dcC7

50

41.74

Car 5

0xC9eC5bEc83028 Ad3BFcbF4767Ad 1d831a6011749

50

36.99

Car 6

0xe9d7e06Ef1Be5 F1336090550909 729A64dA74599

50

54.25

Fig. 3 Smart contracts compilation

• QmYU1iuQSrrhWXxHmPxXpUXj3Cia7MB55r2CbFEsXxM9ai for file dataset_gps_mpu_left.csv • QmaPQZzjMEVQTcmGL57Bsmt9BMjUbpaRTcuzjX7uhf6HqM for file logsworks

162

A. Tiwari and U. Batra

Fig. 4 Smart contracts deployment details

• Qmd2q4gaoxGky46BVKrRq7kiXYzs19D4d6sUYkhhjhUsQ4 for screenshots of works directory

6 Results and Evaluation The experiments above depict a simulated view of the schematic suggested in Fig. 2, has though worked as a proof-of-concept vide the simulations and tools exploited but throws open a number a challenges [24] as will be discussed ahead. The proofof-concept vide the experiments establishes the certain connect of the near future IoV ecosystem and blockchain technology. The hashes in the blockchain for each of the data sets getting populated would be a major security hardening measure for validating and checking any attempts to existing data sets. None of the data sets, once populated in the blockchain would be subject to possible malicious modifications. The schematic is based on a private blockchain wherein all the participating nodes, i.e. vehicles and CS, TMA and TC, etc., are physically identified and digitally checked identities. Petersburg fork configurations in Ganache have been chosen over Constantinople as it offers advantage of creating smart contract transaction addresses without the need to deploy the contract and thus allowing users to transact off-chain. The transactions indexed in the blocks depending on the speed selected, i.e. either

Created contract address

0x5A680c162704 2e084eD04138f9 cD60D6373cF0fE

0x4f214aeA492A 27F8a42e8317E6 07DD11a35ABCC9

0xAcc267275b71d 7C28b33E432DF 2768fa70584978

0xac21313f1e0F 5C9a460BcB255d B49319dB322e66

Sender address

0xB0908B6e032 fF8F79524292E9 B0175bD713F6aeD

0xB0908B6e032f F8F79524292E9B 0175bD713F6aeD

0xB0908B6e032fF 8F79524292E9B0 175bD713F6aeD

0xB0908B6e032f F8F79524292E9B 0175bD713F6aeD 261,393

105,015

105,015

105,015

Gas used

Table 6 Contract addresses and transaction hashes obtained

0x2cc03081a6a42 363474f46caf2a1 cd33988eec13733 cb94b2daf914423056826

0xa79aadab4e0e 4baae6a07e45a1a 807209d4a9ba008 bde0567b6501283e98daeb

0xf8cd8b4dc13eef 08c38b2dd1ca33 2103e9c6299f299 9d55d4726418d2c6c93c1

0x60db0e330457 e1462c8a5649330 dc9d8ff0f473baf48 b27eaa29ec57612a7865

Transaction hash

1

3

5

7

Block

0x2Cc03081a6a4 2363474F46CAF2 A1Cd33988eEc13 733cb94B2DAf914423056826

0xAcc267275b71d 7C28b33E432DF2 768fa70584978

0xF8cd8B4dC13E EF08c38B2Dd1ca 332103e9c6299f29 99D55D4726418d2c6c93C1

0x60dB0E330457E 1462C8a5649330d C9d8Ff0F473baf48 b27eaA29Ec57612a7865

Creation transaction address

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage 163

164

A. Tiwari and U. Batra

slow, average or fast with 0.00017, 0.00031 and 0.00042 ethers have been set in the experiment which have repercussions to the effect of time and committing hash into the block. The experiment first focuses on the overview of actors participating in the proposed schematic with specific defined roles. Further to this, simulated data and exchange of tokens is shown as one of the ways blockchain can be exploited for a secure IoV ecosystem. The proposed use of FBSS offers unlinkability feature and identifiable anonymity to true users while debarring malicious users from misusing the traditional blind signatures feature. Few of the identified challenges vide the experiments scope and realization on ground are discussed below: • True Data sets: Blockchain by virtue of its characteristics offers immutable and unhackable mechanics in systems but certainly it would have to rely on other actors like TC, C1-C7, RSUs, CS, etc., for getting the right inputs which remains as one of the key challenges. • Smart contracts: Smart contracts are one of the most important blockchain features that cannot only automate a lot of things in the IoV ecosystem but also at the same time remove any hitherto traditional mediations. Though Solidity, as a language is evolving and bettering fast, but also at the same time seen an extra ordinary version speed release. The language has seen 64 fresh releases since its introduction with most of them being backward incompatible. • Scalability: While we envisage IoT connected devices to reach 64 billion by 2025, the sheer number of vehicles participating in the IoV ecosystem, as part of IoT will be a challenge to deal with. • Interoperability: All the devices and system working in synchronization with each other in the envisaged IoV ecosystem is expected to be interoperable with each other and communicate in real time. This would be a challenge for long since the current developments across the globe matures in isolation. • Unique Identification of devices and equipment in IoV ecosystem: All the participating vehicles will be identified by unique set of public–private keys and thus security by individual owners will be of paramount importance. A simple compromise by a smart malicious cyber-criminal or a careless owner can play havoc in a moving scenario of vehicles. A need to evolve and negate out such scenarios is deemed before going live and realizing a working IoV ecosystem. • Byzantine brokers: May be referred to as malicious elements who have the capability to abuse and play with decryption keys. Thus, they can effect into subscription corruption, publication delay, message reordering, or publication corruption [25]. With the architectural dependency on IT, more so on PKI and digital keys infrastructure, the threat of such byzantine brokers will loom large and needs additional efforts to negate any attempts. • Throughput: The IoV ecosystem enabled by blockchain will heavily bank on the speed of throughputs attained to ascertain quick committing of transaction hashes in block. This would be made possible with availability of redundant rugged networks with huge scalability and negligible downtimes with no lags ever.

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

165

• Distributed storage upload times: While in the current upload of data sets including inertial sensor data on the right side of the vehicle, combined with GPS data, the following upload times were observed: • 3m21.860s • 0m1.675s • 0m0.453s This is way to less for a real-time scenario implementation and would eventually need to be improved by efforts improving IPFS protocol parameters and configurations.

7 Conclusion The world of IoV has been lately assuming vast interest of researchers across the world owing to the kind of wide implementations envisaged in near future. The IoV will not just transfer data to the Internet employing V2R, V2P kind like communications but will be responsible to resolve diverse driving and traffic problems by exploiting sophisticated IT technology. Thus, the entire IoV ecosystem will be relying majorly on IT assets and technologies. With so much banking on IT infrastructure, security will be a major factor effecting into smooth functioning and drawing precise decisions in real time. It will be a step in the right direction that at the onset of IoV ecosystem setting in, right steps are made to harden the security indices too. This paper presents a way out for an IoV ecosystem enabled by blockchain technology on distributed storage protocol which will harden and make a rugged ecosystem. It is understandable that simply putting the ecosystem on blockchain would not guarantee an infallible IoV architecture since there will be third parties who have to ensure right data goes in at the right time. Blockchain is only ensuring immutability of data and authentication credentials once indexed, and this will be a major requisite for IoV ecosystem. Vide this paper, a concept proof-of-work has been presented with the aid of few blockchain platform applications and IPFS distributed storage protocol. Not only connecting the IoV and blockchain, but exploitation of smart contracts technology, in solidity language, in an IoV ecosystem has been demonstrated vide small envisaged scenarios. The architecture proposed has been demonstrated in a simulated partial environment which does not accomplish the actual ground states of affairs as would hold. Additionally, Ricardian contracts [26] may be a better option viz-a-viz smart contracts which would definitely play a significant role in removing third party mediations and expediting many IoV mundane tasks, but Ricardian with their lot of advantages also behold an important role in IoV ecosystem. Challenges bought out vide the paper remain a major hurdle in realization of blockchain and smart contracts in an IoV ecosystem. The need of standardization at global level and acceptance by major stake holders will be the key to realization of

166

A. Tiwari and U. Batra

smooth implementations. Hyperledger [27], CORDA R3 [28], Ethereum, etc., are few major emerging platforms that will contribute to the realization.

References 1. N. Sharma, N. Chauhan, N. Chand, in Security Challenges in Internet of Vehicles (IoV) Environment. 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), Jalandhar, India (2018), pp. 203–207. https://doi.org/10.1109/ICSCCC.2018.870 3272 2. F. Yang, S. Wang, J. Li, Z. Liu, Q. Sun, An overview of internet of vehicles. China Commun. 11(10), 1–15 (2014). https://doi.org/10.1109/CC.2014.6969789 3. F. Alkurdi, I. Elgendi, K.S. Munasinghe, D. Sharma, A. Jamalipour, in Blockchain in IoT Security: A Survey. 2018 28th International Telecommunication Networks and Applications Conference (ITNAC), Sydney, NSW (2018), pp. 1–4. https://doi.org/10.1109/ATNAC.2018. 8615409 4. P.K. Sharma, S. Singh, Y. Jeong, J.H. Park, DistBlockNet: a distributed blockchains-based secure SDN architecture for IoT networks. IEEE Commun. Mag. 55(9), 78–85 (2017). https:// doi.org/10.1109/MCOM.2017.1700041 5. W. Li, W. Meng, Z. Liu, M.H Au, Towards blockchain-based software-defined networking: security challenges and solutions. IEICE Trans. Inf. Syst. E103(D), 196–203 (2020). https:// doi.org/10.1587/transinf.2019INI0002, https://doi.org/10.1587/transinf.2019INI0002 6. V. Daza, R. Di Pietro, I. Klimek, M. Signorini, in CONNECT: CONtextual Name Discovery for Blockchain-Based Services in the IoT. 2017 IEEE International Conference on Communications (ICC), Paris (2017), pp. 1–6. https://doi.org/10.1109/ICC.2017.7996641 7. Y. Yuan, F. Wang, in Towards Blockchain-Based Intelligent Transportation Systems. 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro (2016), pp. 2663–2668. https://doi.org/10.1109/ITSC.2016.7795984 8. X. Wang, S. Li, S. Zhao, Z. Xia, A VANET privacy protection scheme based on fair blind signature and secret sharing algorithm. Automatika 58(3), 287–294 (2017). https://doi.org/10. 1080/00051144.2018.1426294 9. T. Jiang, H. Fang, H. Wang, Blockchain-based internet of vehicles: distributed network architecture and performance analysis. IEEE Internet Things J. 6(3), 4640–4649 (2019). https://doi. org/10.1109/JIOT.2018.2874398 10. C. Li, L.J. Zhang, in A Blockchain Based New Secure Multi-Layer Network Model for Internet of Things. Proceeding of 2017 IEEE International Congress on Internet of Things (ICIoT), Honolulu, HI (2017), pp. 33–41. https://doi.org/10.1109/IEEE.ICIOT.2017.34 11. N. Kaaniche, M. Laurent, in A Blockchain-Based Data Usage Auditing Architecture with Enhanced Privacy and Availability. Proceedings of 2017 IEEE 16th International Symposium on Network Computing and Applications (NCA), Cambridge, MA (2017), pp. 1–5. https://doi. org/10.1109/NCA.2017.8171384 12. Connected & Autonomous Vehicle (CAV) Security And Blockchain-Based Data Platform at https://cubeint.io/wp-content/uploads/2019/10/Cube-Whitepaper-Centered-v2-3.pdf. Accessed 20 Sept 2021 13. O. Nait Hamoud, T. Kenaza, Y. Challal, Security in device-to-device communications: a survey. IET Networks 7(1), 14–22 (2018). https://doi.org/10.1109/HICSS.2016.713 14. A. Chakravorty, C. Rong, in Ushare: User Controlled Social Media Based on Blockchain. Proceedings of 11th ACM International Conference on Ubiquitous Information Management and Communication (ACM IMCOM). Beppu, Japan January 05–07, 2017. https://doi.org/10. 1145/3022227.3022325 15. Z. Yang, K. Yang, L. Lei, K. Zheng, V.C.M. Leung, Blockchain based decentralized trust management in vehicular networks. IEEE Internet of Things J. (2018).https://doi.org/10.1109/ JIOT.2018.2836144

Blockchain-Enabled Internet-Of-Vehicles on Distributed Storage

167

16. C. Liu, K.K. Chai, X. Zhang, E.T. Lau, Y. Chen, Adaptive blockchain-based electric vehicle participation scheme in smart grid platform. IEEE Access (2018). https://doi.org/10.1109/ACC ESS.2018.2835309 17. F. Gao, L. Zhu, M. Shen, K. Sharif, Z. Wan, K. Ren, A blockchain based privacy-preserving payment mechanism for vehicle-to-grid networks. IEEE Network (2018). https://doi.org/10. 1109/mnet.2018.1700269 18. S. Wang, Y. Yuan, X. Wang, J. Li, R. Qin, F. Wang, in An Overview of Smart Contract: Architecture, Applications, and Future Trends. 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu (2018), pp. 108–113. https://doi.org/10.1109/IVS.2018.8500488 19. Truffle Suite at https://www.trufflesuite.com/ganache,accessed on 01 March 2021 20. M.K. Pawar, P. Patil, M. Sharma, M. Chalageri, Secure and scalable decentralized supply chain management using ethereum and IPFS platform. Int. Conf. Intell. Technol. (CONIT) 2021, 1–5 (2021). https://doi.org/10.1109/CONIT51480.2021.9498537 21. S. Rezaei, E. Khamespanah, M. Sirjani, A. Sedaghatbaf, S. Mohammadi, in Developing Safe Smart Contracts. 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC) (2020), pp. 1027–1035. https://doi.org/10.1109/COMPSAC48688.2020.0-137 22. M. Steichen, B. Fiz, R. Norvill, W. Shbair, R. State, in Blockchain-Based, Decentralized Access Control for IPFS. 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) (2018), pp. 1499–1506. https://doi. org/10.1109/Cybermatics_2018.2018.00253 23. J. Menegazzo, PVS—Passive Vehicular Sensors Datasets (2021) [online] Kaggle.com. Available at: tabulated (t), then reject H0 (Null Hypothesis), or accept alternative hypothesis (H1); if calculated (t) < tabulated (t), then accept H0 (Null hypothesis).

6.1 Hypothesis 1: “Phishing/Pharming the Most Common Means of Infections Used for Ransomware Attacks Have no Impact on the Cloud Adoption” Null Hypothesis (H0): Phishing/Pharming attacks have no impact on the cloud adoption. Alternative Hypothesis (H1): Phishing/Pharming attacks reduce the rate of cloud adoption.

6.2 Hypothesis 2: “Personal Data Security is the Prime Concern for all Business and it is the Biggest Security Challenge in the Cloud Adoption” Null Hypothesis (H0): Data in cloud is more secure than the security offered by on-premises environment.

Impact of Security Challenges Over Cloud Adoption—A Comparative…

203

Alternative Hypothesis (H1): Personal data security is the prime concern for business and it the biggest security challenge in the cloud adoption.

6.3 Hypothesis 3: “Identity Theft Crime has Noted the Highest Percentage Increase in 2020, Does it Slowdown the Acceptance Rate of Cloud Adoption” Null Hypothesis (H0): Identity theft crime has no impact on the acceptance of cloud adoption. Alternative Hypothesis (H1): Identity theft crime has slowdown the acceptance of cloud adoption. Following observations indicate that hypothesis 1, 2 and 3 accept the null hypothesis (H0). Since calculated r value significance level (0.05), and it also lies in confidence interval at (confidence coefficient 95%), it means that it accept the null hypothesis (H0).

6.4 Hypothesis 4: “Rising Extortion/Ransomware Attacks are Normally Acceptable and it Does Not Damage the Reputation of Cloud Adoption” Null Hypothesis (H0): Rising extortion/ransomware attacks is normally acceptable and it does not damage the reputation of cloud adoption. Alternative Hypothesis (H1): Rising extortion/ransomware attacks damage the reputation of cloud adoption, and it is not acceptable. Since calculated r (0.906) value > critical r (0.878) value, thus it indicates that the correlation is not significant. Calculated (t) or test statistics: 3.707 > tabulated (t), so Reject H0 (Null hypothesis). Probability P (two sided) lies beyond the confidence interval at (Confidence coefficient 95%). As per the Table 5, the calculated p-value is < significance level (0.05) and it doesn’t lie in confidence interval at (confidence coefficient 95%), it means that it reject the null hypothesis (H0) and accept the alternative hypothesis (H1).

2017

145.3

2016

118.7

2018

196.7

2019

242.6

2020 270

17,146 14,938 51,146 43,101

27,573 30,904 50,642 38,128

16,878 17,636 16,128 16,053

Extortion

Personal data breach

Identity theft

Source Author

81,029 84,079 65,116 61,832

Non-payment /non-delivery

43,330

45,330

76,741

0.63

0.694

0.906

108,869 0.208

0.878

Significance of observed correlation (calculated (t) or test statistics

3.707

Significant 1.4

Significant 1.67

Not significant

Significant 0.368

Significant 2.945

Coefficient Critical value Decision correlation of correlation ® coefficient(0.05 level of significance for two-tailed test)

Phising/Pharming 19,465 25,344 26,379 114,702 241,342 0.862

Crime type/revenue ($ billion)

3.182

Critical value of t (two-tailed test)

0.034 0.194 0.255

−0.486 to 0.978 −0.568 to 0.972

0.737

−0.826 to 0.921 0.118 to 0.994

0.06

−0.085 to 0.991

Accept H0

Accept H0

Reject H0

Accept H0

Accept H0

Confidence Probability Decision interval at P (two (confidence sided) coefficient 95%)

T-Statistics (two-tailed test) over IC3 complaint statistics 2020—top five crime type comparison of last five years and worldwide public cloud service revenue (billion USD)—year-wise achieved

Table 5 T-statistics (two-tailed test)

204 M. N. Islam et al.

Impact of Security Challenges Over Cloud Adoption—A Comparative…

205

7 Findings The data is collected from two sources to establish a relation between the cloud security and cloud adoption (Sect. 3). This research work analyses the contribution and findings the facts and relativity between the cloud security and cloud adoption. (i)

(ii)

(iii) (iv)

(v) (vi) (vii)

Cloud spend is continuously rising, and expected annual spend growth is nearly 17% or more, actual annual spend growth is more than expected growth for last five years. Ransomware attacks have increased rapidly, more than two third (68.5%) of the participating organizations are affected of at least one successful ransom attacks, and 71.6% of ransom payer have recovered their data in 2021. Extortion shows the diverse impact on the cloud adoption. Cloud security has gained trust over on-premises system, acceptance rate of public cloud is rising. Data in cloud is more secure than the security offered by on-premises environment. Identity theft crime has noted the highest percentage increase in 2020, nevertheless, it has barely any impact on the acceptance of cloud adoption. Phishing/Pharming perpetrated highest attack in 2020 hardly impacts on cloud adoption. Business E-mail Compromise (BEC) attack is costliest.

8 Conclusion Though the cloud architecture is highly beneficial and robust, still its issues and challenges are at extreme. Cyberattack and cyber-frauds are continuously increasing. Maximum data breach attacks involves malware infection. Phishing, credential abuse, Dos/DDoS and SQL injections are the other major security issues for the organization. The aim of this research is to show a significance correlation between the security challenges over the cloud adoption, and it is hypothetically tested. Hypotheses conclude that even the security is biggest challenge, it has no impact on the cloud adoption. Some facts of this research includes #Phishing/Pharming is the most common means of infections used for ransomware at tacks, #Rising extortion/ransomware attacks damage the reputation of cloud adoption and it is not acceptable and #Data in cloud is more secure than the security offered by on-premises environment. Acknowledgements This is to acknowledge that this research work has been supported by Visvesvaraya PhD Scheme, MeitY, Govt. of India. ‘MEITY-PHD-2214’.

206

M. N. Islam et al.

References 1. NIST Special Publication 500–291,Version 2 (2013) [online]. Available at: https://nvlpubs.nist. gov/nistpubs/SpecialPublications/NIST.SP.500-291r2.pdf 2. Gartner Forecasts Worldwide Public Cloud End-User Spending to Grow 23% in 2021 (2021). Retrieved 30 July 2021, from https://www.gartner.com/en/newsroom/press-releases/2021-0421gartner-forecasts-worldwide-public-cloud-end-user-spending-to-grow-23-percent-in-2021 3. 2021 Retrieved 30 July 2021, from https://www.oracle.com/a/ocom/docs/cloud/oracle-cloudthreat-report-2020.pdf 4. 2021, from https://www.coalfire.com/documents/whitepapers/sec urealities-cloud-securityreport 5. M. Ryan, Cloud computing security: the scientific challenge, and a survey of solutions. J. Syst. Softw. 86(9), 2263–2268 (2013) 6. Cloud Security Report | Check Point Software (2020). [online] Available at: 7. Cloud Security Report | Synopsys (2019). [online] Available at: 8. Cloud Security Report-Cybersecurity Insiders (2018). [online] Available at: 9. Cybersecurity Trends Report Cybersecurity Insiders (2017). [online] Available at: 10. C. Team, Cloud Malware: 5 Types of Attacks and 3 Security Measures (2021). From https:// cloud.netapp.com/blog/blg-cloud malware-5-types-of-attacks-and-3-security-measures 11. 2021 State of the Cloud Report | Flexera. [online] Available at: 12. Cloud Computing Attacks: A New Vector for CyberAttacks (2021). From: https://www.apr iorit.com/dev-blog/523-cloud-computing-cyber-attacks 13. OWASP Top Ten Web Application Security Risks | OWASP from https://owasp.org/www-pro ject-top-ten/ 14. Cyberthreat Defense Report 2021 | CyberEdge Group from https://cyber-edge.com/cdr/ 15. H. Tabrizchi, M. Kuchaki Rafsanjani, A survey on security challenges in cloud computing: issues, threats, and solutions. J. Supercomput. 76(12), 9493–9532 (2020). https://doi.org/10. 1007/s11227-020-03213-1 16. Top11 cloud security challenges and how to combat them. (2021). Retrieved 30 July 2021, from https://searchcloudsecurity.techtarget.com/tip/Top-cloud-security-challenges-andhow-to-combat-them 17. M. Nurul Islam, S. Quadri, Cloud security: needs, issues and challenges. TEST Eng. Manage. 83(March–April 2020), 12537–12552 (2020), ISSN: 0193-4120 18. 2021, from https://www.ic3.gov/Media/PDF/AnnualReport/2020_IC3Report.pdf

A Passive Infrared-Based Technique for Long-Term Preservation of Forensic Evidence Vijay A. Kanade

Abstract Accurate and precise measurement of evidence is fundamental to forensic investigations. Incorrect mapping of evidence to the surroundings can lead to the eventual exclusion of evidence or spark a debate over its significance. The paper proposes a technique to record the traces of thermal radiation at the crime scene to recreate and preserve the evidence for future forensic analysis. The research introduces a novel application of passive infrared radiation to preserve biological evidence for a longer duration. Keywords Infrared thermography · Thermal radiation · IR camera · 3D reconstruction · Computer vision · Forensic science · Biological evidence

1 Introduction Forensic science runs on the backbone of evidence. Traditionally, manual investigative tools including photography and sketching have been used to better understand and examine the crime scene environments. However, manual techniques are prone to errors and vulnerabilities. Citing these reasons, recent advances in technologies have been adopted extensively to handle crime investigations [1]. The need for accurate data gathering and error free evidence mapping of crime scenes has led to widespread adoption of laser scanning technology. It helps to capture and reconstruct the scene to the minutest of details. Such new systems and tools allow investigators to review the datasets without any expertise or complex instruments. Events such as identifying bullet trajectories, determining the weapons involved and bloodstain mapping are recognized with these new aged technologies. Such events are used by investigative teams to reconstruct complex scenes in 3D spaces [2, 3]. However, preserving biological bodies still poses a challenge. Although, medical fraternity employs embalming to preserve bodies for anatomical studies, it still is not a full-fledged solution [4]. Several problems including cost, infrastructural facilities, V. A. Kanade (B) Pune, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_17

207

208

V. A. Kanade

embalming process, storage and licensing from the government make human body preservation difficult. Additionally, the advanced 3D immersive technologies used for crime scene recreation are still in the development stage. Hence, the accuracy of the existing tools makes them highly unreliable. Forensic science does not demand body preservation at all times. However, high profile cases require thorough study and analysis of the deceased body to come to a definitive conclusion. Such investigations call for longer than expected preservation of biological bodies. Bodies are either cremated or buried after a while, depending on different religious beliefs. And this acts as a serious bottleneck for the forensic sciences. The detailed analysis of the biological evidence tends to reveal new and fresh information as the case progresses. Hence, long-term preservation of forensic evidence is an area of concern for most investigators. Looking at the current state of forensic sciences, there seems to be a long standing need to address the problem of long-term forensic evidence preservation. The research proposes a technological solution that addresses this issue.

2 IR-Based Technologies Infrared (IR) radiation is emitted by every object above absolute zero (0° Kelvin) temperature, which includes human beings. The wavelength of these radiation depends on the temperature, area and characteristics of the object. While the wavelength of IR is between 0.75 and 1000 microns, humans give off radiation of 12 micron wavelength. Some common medical devices such as handheld infrared ear or forehead thermometers are used to monitor body temperatures by detecting IR radiations. Similarly, IR cameras are used on airports, bus terminals or ports for faster screening of travelers. Such devices have also helped in containing the spread of pandemics (COVID-19), swine influenza and others. Additionally, various personal identification systems such as facial recognition systems, iris scanning systems, smart surveillance systems, palm vein authentication systems and others utilize IR-based technologies. It can be observed that in all these systems IR cameras play a critical role. In the forensic context, infrared thermography (IRT) and IR spectrophotometry have played a pivotal role in crime investigations [5]. IR photography of physical evidences supports re-examination for a long time [6]. IR imaging (passive and active) is used to detect contact and non-contact thermal radiations. IR spectrophotometry on the other hand has been used to study and analyze polymers and materials including fibers, adhesives, coatings, etc. Some of the key forensic applications include post-mortem interval detection, detection of tire prints, differentiation of gunshot residues and blood stains, explosive analysis, analysis of blunt force injuries, bite and tooth mark detection, DNA analysis, fingerprint detection, identification of tattoos and plenty others [7]. All the above

A Passive Infrared-Based Technique for Long-Term Preservation …

209

examples use IR-based techniques effectively and help in limiting crimes, further leading to criminal arrests in many cases.

3 Passive IR Imaging Passive IR imaging detects non-contact thermal radiations, without the requirement of any external energy source. Although these radiations are invisible to the human eye, they are observable on infrared camera. Thus, this method is employed to study, analyze and investigate the heat traces left behind by objects and humans. Human body temperature is generally above the normal environmental temperature. This results in humans leaving heat traces along the places they visit. Passive IR imaging is thereby useful in gaining insights into the presence of people or identifying objects they recently interacted with. Crime scene reconstruction and verification of criminal statements is very much possible with passive IR imaging.

3.1 Experiments and Results Figure 1 shows an image of a person holding a mug and its corresponding IR representation. The IR representation is shown here to highlight the thermal radiations in the vicinity of the mug.

Fig. 1 IR representation of a captured image

210

V. A. Kanade

Fig. 2 PIR sensor captured image

Figure 2 depicts a PIR sensor that we used to capture the image of the mug without the individual’s hand on it. The second part of the image shows the glimpses of heat traces left behind on the mug by the individual’s hand. The heat traces are distinctly observable here since the image is captured minutes after the person removes his hand from the mug. We observed that the heat on the mug due to the individual’s hand was visible only for about an hour. Later, as the heat dissipated into the surroundings, the redness on the mug eventually reduced. Thus, we were able to conclude that the heat traces depend on various factors such as temperature difference, the material characteristics, and the surrounding conditions. Nevertheless, the above two images provide qualitative information such as time since an object (i.e., mug in this case) was used. Such data draws insights necessary to propel the forensic processes. In addition to heat traces, passive IR imaging also helps in visualization of wet traces including blood stains.

4 Proposed Model for Long-Term Preservation of Biological Evidence In the proposed model, we reconstruct the 3D replica of the human body after capturing the heat traces of an individual [8]. This is achieved by using passive IR technology. The recreated body is thereby available to investigators for examination for a longer duration of time. The working of the model is as illustrated below (Fig. 3):

A Passive Infrared-Based Technique for Long-Term Preservation …

211

Fig. 3 IR camera with PIR sensors capturing thermal data of the deceased

1. On visiting the crime location, use a camera (i.e., Kinect device) equipped with PIR sensors to record and capture the image of the deceased [9]. 2. The PIR sensors capture the heat signature of the body to the minute details. For example, it has been identified that the heat pattern emitted by veins of a human hand are unique to each individual. Thus, the structure of the hand veins can be easily determined just by capturing their heat pattern. As a result, palm authentication systems are now being used at ATMs, airports and entrances of security apartments. Additionally, thermal imaging is also used in detection of skin lesions [10]. 3. On recording the body heat data, the camera stores it for a period of time. 4. The recorded temperature data is then fed into an IR projector that creates a 3D replica by projecting IR radiation corresponding to the data. 5. The projected 3D replica of the human represents the copy of the deceased individual (Fig. 4).

4.1 Technical Details The specifications of the thermal camera used to capture the image of the deceased individual are disclosed in the Table 1:

212

V. A. Kanade

Fig. 4 Reconstructed 3D replica by using IR projector

Table 1 Thermal camera specifications

Table 2 Hardware specifications

Sr. No.

Camera specifications

Value

1

IR resolution (min.)

640 × 512 px

2

Temperature range

16–390 C

3

Thermal sensitivity

50 mK

4

Focal length

30 mm

5

Accuracy

±5 °C (±9 °F)

Sr. No.

Hardware specifications

Value

1

F-stop

f/5.6

2

Exposure time

1/250 s

3

ISO-speed

4000

4

Max aperture

4.336

5

Brightness

3.404

To have a better quality image, some of the critical hardware specifications of the IR camera are specified in the Table 2:

5 Advantages The proposed model provides a unique solution to preserve biological bodies of deceased individuals for a long duration. This is beneficial for high profile or complicated cases wherein it is difficult to ascertain the reasons for someone’s death. As examination and investigation is a time intensive process, preserving bodies for long

A Passive Infrared-Based Technique for Long-Term Preservation …

213

is an essential aspect of the forensics. Fortunately, with the advancement of technology, passive IR technology presents us with a chance to do so. Thus, it can have large scale implications for forensic sciences.

6 Conclusion Infrared imaging plays a pivotal role in forensic cases. The paper presents a noninvasive, non-destructive technique that can provide insightful information of the crime scene which is invisible to a naked eye. It provides crucial leads for investigating teams and can serve as critical evidence in the court of law. The novel application presented in the paper for long-term preservation of forensic evidence by harnessing passive IR technique can only further strengthen the investigative analysis. The research proposal gives a first-hand account of the possibility of preserving deceased body for longer period just by exploiting the right kind of technology. Acknowledgements I would like to extend my sincere gratitude to Dr. A. S. Kanade for his relentless support during my research work. Conflict of Interest The authors declare that they have no conflict of interest.

References 1. Crime Scene Investigation, A Guide for Law Enforcement, September 2013 2. G. J. Edelman, Infrared imaging of the crime scene: possibilities and pitfalls, spectral analysis of blood stains at the crime scene. J. Forensic Sci. 2013 58(5), 1156–62 (2014) 3. D. Raneri, D. Raneri, Enhancing forensic investigation through the use of modern threedimensional (3D) imaging technologies for crime scene reconstruction. Aust. J. Forensic Sci. (2018). https://doi.org/10.1080/00450618.2018.1424245 4. L. Poole, World-First Replica 3D body Parts Teach Medical Students Anatomy, Aug 4th, 2016 5. E. Mistek, I. K. Lednev, FT-IR Spectroscopy for Identification of Biological Stains for Forensic Purposes, Special Issues-08–01–2018 33(8), 8–19 (2018) 6. K. Parmelee, Detective Somerset County Prosecutor’s Office Forensic Laboratory, New Jersey, Infrared Photography: A forgotten tool for investigators 7. A. Mahmut, Y. Hekimoglu, O. Gumus, Usage of Infrared-Based Technologies in Forensic Sciences (2016). https://doi.org/10.5772/62773 8. G. Clarkson, S. Luo, R. Fuentes, in Thermal 3D Modelling. Proceedings of the 34th ISARC. 34th International Symposium in Automation and Robotics in Construction, 28 Jun–01 Jul 2017, Taipei, Taiwan. IAARC (2017), pp. 493–499 (orcid.org/0000–0001–8617–7381) 9. An Introduction to the Kinect Sensor, July 15th, 2012 10. C. Herman, Three-Dimensional Thermal Imaging for the Detection of Skin Lesions and Other Natural and Abnormal Conditions (Johns Hopkins University, US20130116573A1, May 9th, 2013)

Smart Contract Assisted Public Key Infrastructure for Internet of Things Pinky Bai, Sushil Kumar, and Upasana Dohare

Abstract IoT devices use PKI to authenticate themselves in the network to manage the authenticity of network, data confidentially and data integrity. This paper proposes an approach to implement the smart contract-based Public Key Infrastructure (SCPKI) for the Internet of Things. In the proposed approach, Smart contracts are used for public key life cycle management starting from the generation of the public key, distributing public key to the revocation of the public key in the form of X.509 certificate. SC-PKI does not have any third-party centralized certification authorities (CAs) to issue, validate, and revoke the certificates. All these activities are executed by the smart contract on a dedicated blockchain network. The proposed approach enhances the network’s security, avoids single-point failure, and increases the certificate verification performance. Keywords Internet of things · Blockchain · PKI · Smart contract

1 Introduction The successful execution of the IoT system largely depends on the inbuilt security of IoT devices. Various IoT system elements such as IoT devices, operating platforms and systems, and communication, among “things” pose many security challenges like DDoS, eavesdropping, replay attack, phishing attack, data breach, etc. A study conducted by Hewlett Packard Enterprise shows that 70% of IoT devices are P. Bai (B) · S. Kumar Wireless Communication and Networking Research Lab, School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi 110067, India e-mail: [email protected] S. Kumar e-mail: [email protected] U. Dohare Deprtment of Computer Science and Engineering, IIMT College of Engineering, Greater Noida, Utter Pradesh, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_18

215

216

P. Bai et al.

susceptible to one or another kind of security breach. These security breaches put our physical infrastructure, food supply, water system, power network, public data, health, etc., at high risk [1–3]. The main three pillars of IoT security are confidentiality, Integrity, and Availability. Public Key Infrastructure (PKI) plays a vital role in ensuring security in the digital world. The Public Key Infrastructure is a significant approach to provide authenticity and integrity in IoT devices. However, the requirement of high computing power in PKI and associated high cost makes implementing these approaches less efficient for IoT devices [4]. The PKI manages the public key and certificate for the system. PKI depends on a trusted third-party, i.e., Certification Authority (CA), to manage the digital certificates which store the public key. Management of digital certificates includes signing, verifying, and revoking of the certificate [5]. The X.509 digital certificate is a standardized format for the public key certificate. IoT devices use PKI to provide user authentication, data confidentiality, data access control, and data integrity. PKI is the backbone for securing communication among IoT devices and other platforms [5, 6]. “When you are looking at authenticating devices, the only real standards at the moment that offer any real interoperability tend to be Public Key Infrastructure” [5]. Today, PKI used in IoT ecosystem to secure communications has several drawbacks arising from its centralized and non-transparent design. Here, we propose a decentralized PKI solution for IoT devices using blockchain. The smart contracts manage PKI functions, i.e., management of certificates, distribution, verification, and revocation in the proposed solution. The smart contracts run on the private blockchain. The X.509 certificate is also modified as per the requirement. Our Contribution: The proposed approach introduces the BT-Node to perform the PKI operations on behalf of constrained IoT device. The approach provides solution for single-point failure, security and high performance on verification time. The Study is structured as a description of IoT PKI, Literature survey, Proposed work, Analysis and Result, and Conclusion with Future Work.

2 IoT PKI 2.1 Gaps While Using CA-Based PKI The centralized nature of PKI is the root of the single-point failure problem. PKI needs a third-party CA to validate the certificates. The CA-based PKI or centralized PKI poses some other gaps, such as centralized control, which may not update the certificates in time, resulting in high security risks. If the secret key of the root CA or any CA in the chain gets compromised, then bogus certificates can be issued to untrusted parties, enabling impersonation of identity [7–9].

Smart Contract Assisted Public Key Infrastructure for Internet …

217

IoT systems have their own issues along with standard CA-based PKI. IoT device owners have problems managing certificates for their devices because of the constrained nature of IoT devices and device heterogeneity. IoT devices do not follow any standard protocol because there is no standard protocol to manage the IoT device certificates [10]. Due to difficulties in storage, retrieval, and revocation of certificates, the device manufacturers manage the devices’ keys instead of device owners. Thus risks associated with the manufacture’s server where keys are stored increase the attack surface for IoT devices, making IoT devices more vulnerable [11, 12].

2.2 Applicability of Blockchain We have already discussed the issues related to CA-based centralized PKI. Blockchain has some special characteristics such as decentralization, anonymity, and auditability makes blockchain suitable for decentralized PKI. Researchers [9–14] have proposed solutions in the same direction.

3 Literature Survey In this section, we discuss the already developed framework to implement the PKI for IoT devices. The result is divided into two parts; traditional PKI solutions for IoT and blockchain-based PKI solutions.

3.1 Traditional IoT PKI The protocols used over the Internet for secure communication, such as Transport Layer Security (TLS) and Secure Socket Layer (SSL), are not preferable for IoT. The main problem is the too large size of certificate for memory-constrained IoT devices. The large size of the X.509 certificate creates a problem for the resourceconstrained devices because communication with a large footprint consumes most of the computing resources. The cryptography protocol used in PKI also needs a large RAM and ROM size, making PKI not suitable for IoT devices. Multiple solutions are proposed for reducing the size of certificates for IoT [16–19]. We discuss some of the protocols next. Researchers proposed a raw public key in the Datagram Transport Layer Security (DTLS) handshake protocol to reduce the burden of storing and transmitting the X.509 certificate. The IoT devices are pre-loaded with the servers’ public key and the device’s public key. But the chain of trust and other trust chain does not exist in this approach because there is no need for CA to sign the certificate [16].

218

P. Bai et al.

In the same direction, Schukat et al. has suggested the preinstallation of a smaller footprint of X.509 certificate V3 with extensions on the device. This approach does not update or revoke certificates, which can lead to insecurity for the IoT Networks. Daniel et al. have used Concise Binary Object Representation (CBOR) to compress the X.509 certificate’s size. Certificate compression rate of up to 30% can be achieved using CBOR, a lightweight encoding scheme [15]. The primary focus of this approach is energy reduction using an encoding format. The approach is theoretical and not experimentally validated. While considering the IoT ecosystems’ standards and fully functional protocol for constrained IoT devices, Kothmayr et al. have presented standard-based security architecture. The two-way authentication is used to provide high security for the IoT. The solution is the first solution, which is fully implemented to provide secure communication using two-factor authentication. This work is in the direction of standardization for IoT security communication [16–18].

3.2 Blockchain in IoT PKI Blockchain technology provides decentralized PKI solutions. Many solutions are proposed in the direction of single-point failure prevention, scalability, CA misbehavior prevention, security, time efficiency, and many more. Table 1 presents some of the work in the same direction. From the literature survey, we found that PKI for constrained IoT devices depends on the interoperability, network size, and IoT device itself that how constrained it is. Digital certificate deployment on IoT devices is expensive because of computational power, energy, storage space, and sensing and actuating platforms. So we propose explicit certificate schemes to secure IoT applications. In the proposed schemes, the modified certificate stores at BT-Node instead of IoT device.

4 Proposed Work: SC-PKI This section presents a detailed description of our approach to design the PKI for the Internet of Things based on a smart contract: “Smart Contract assisted PKI (SC-PKI) for IoT.” This section discusses an overview of the approach, algorithms, architecture, and work process of PKI using the smart contract on blockchain for memory-constrained IoT devices.

Smart Contract Assisted Public Key Infrastructure for Internet …

219

Table 1 Blockchain solutions for IoT PKI References Approach

Targeted parameter

[8]

– Three different blockchain-based alternatives to Lightweight traditional CA-based PKI for certificate management – The light sync client by Ethereum gives a solution of storing a part of blockchain rather than a full blockchain

[12]

– Certificates are stored in the IoT device instead of the manufacturing server to prevent key leakage – Proof of work (PoW) is used to carry out PKI operations – PoW needs high energy and computation

Scalability, single-point failure

[13]

Smart contract-based PKI

Decentralization

[19]

– Smart contracts for detecting, publicizing, and automatically responding to CA misbehavior – Trace the misbehavior of CA in certificate revocation – This work does not cover all the functionalities of PKI except certificate revocation

CA misbehavior

[20]

– Automatic responses to CA misbehavior and incentives for those who help detect misbehavior – The software needed to test IKP is not yet publicly available

CA misbehavior

[21]

– Privacy-aware PKI in which public key does not Privacy, single-point of failure link with identity – Based on a web of trust model

[22]

Smart contract-based PKI on Ethereum blockchain Trustless

[23]

– Storage of key value pair parameters on blockchain – Access control and privacy policy parameters are not considered

Storage

4.1 Overview SC-PKI for IoT has three main components: IoT device, Blockchain Node (BTNode), and blockchain network. BT-Nodes contain a smart contract that dictates the system’s protocols and acts as an interface to the device and the blockchain to manage X.509 certificates for the IoT devices. BT-Node behave like an IoT gateway. IoT gateway maintains the application software for the IoT constrained devices. Figure 1 shows the architecture of the proposed approach. All the BT-Node make the closed private blockchain network, and one or more IoT devices are participating in the blockchain network via BT-Node.

220

P. Bai et al.

Fig. 1 SC-PKI architecture

Here, we assumed that the IoT device has minimum functionality, i.e., network interface, key generation module, and nonvolatile memory to store the key pair and Hash of certificate. The key generator follows the Elliptic Curve Cryptography (ECC) to generate the key pair (Public and Private Keys) [24]. Each IoT device is uniquely defined. The BT-Nodes have high computing power. The BT-Nodes register themselves on the blockchain network with their digital certificates. The BT-Nodes stores the blockchain/distributed ledger, and the ledger maintains the list of all registered IoT devices and the network transactions. The smart contract is deployed on all the BT-Nodes (IoT-gateway). Ledger used PBFT consensus to synchronize the Ledgers stored on each BT-Node [25]. The X.509 parameters also include the BT-Node ID with a standard parameter like version, serial number, key generation algorithm, and extensions. Figure 5 represents the X.509 certificate [26]. Figure 1 shows the basic architecture of the proposed framework. The IoT devices, the blockchain network, and the BT-Nodes and peer-to-peer connection are represented in Fig. 1.

Smart Contract Assisted Public Key Infrastructure for Internet …

221

4.2 The Work Process In this section, we discuss the work process of SC-PKI for IoT networks. We divide our approach into three main component: • Registration of Device • Verification and Authentication of certificates or Devices • Certification Revocation List (CRL) (a) Registration and Certificate Generation: In our proposed approach, the IoT device itself generates the key pair rather than the manufacturer. The IoT devices communicate to BT-nodes directly through a network interface such as Ethernet, Wi-Fi, Bluetooth or ZigBee. Figure 2 is a workflow diagram for IoT device registration on the blockchain network. Below are the eight steps for registration of an IoT device on SC-PKI: Step 1: IoT device generates the key pair Public Key (PB ) and Private Key (PR ) for certification. Step 2: IoT device sends Device ID and Public key to BT-Node for registration. Fig. 2 Registration

222

P. Bai et al.

ID(IoT device)− > ID(BT − node) : request = (ID, PB ) Step 3: BT-Node run the query to ledger to check if the device is already registered or not. Step 4: If the device is not registered, the BT-Node call function generates the x.509 certificate for the device with the requested public key. ID(BT − node)− > generate(IDX , PB , ID(BT − node)) Step 5: The function generates X.509 certificate (Fig. 2). Step 6: The BT-Node signs the X.509 certificate with its own private key and updates the ledger with the Device ID, signed X.509 Certificate, and validity of the certificate. Signed(CertA ) = sign(PR (BT − Node), CertA ) Step 7: BT-Node send the signed CertX to the IoT device. ID(BT − Node)− > send(signed(CertA ), ID(IoT device)) Step 8: IoT device stores the X.509 certificate. (b) Verification and Authentication: When a device IDA wants to establish a secure connection to another device IDB , the process will execute the following steps: Step 1: Device IDA sends its device ID and the signed CertA with the private key to Device IDB . Step2: Device IDB sends a request to BT-Node to authenticate the device IDA . Step 3: BT-Node call the “Authentication” function to authenticate the device. Authn (IDA , CertA ). Authn functions execute as follow: • Query to the ledger to extract details of IDA . Query(ID A ) • Ledger revert back registered device ID, the public key, the certificate, Validity of Certificate, and Status. • Decrypt the received CertA with PB key (extracted from the ledger) against the IDA . • If the decrypted received CertA from the device is matched with extracted from ledger CertA , go to step 4; otherwise, the message “Authentication Fail.” • If the device’s status is “Revoked,” the process is terminated, and with “Device status is revoked” message.

Smart Contract Assisted Public Key Infrastructure for Internet …

223

• If Validity (CertA ) > Current Date, then give the message “Authentication successful”; otherwise, abort the authentication by giving a message “Validity Expire.” • Otherwise, give the message “Authentication Failure.“ Step 4: Communicate the message to the Device (IDB). Figure 3 is the workflow diagram for a device’s authentication process when the device requests for connection. (c) Certification Revocation List (CRL) If an attacker steals the device, then in such a case, an attacker would be able to change the value of public and private keys on the ledger, and after changing the value of keys, the IoT device cannot be authenticated by others. Attackers can Fig. 3 Verification and authentication

224

P. Bai et al.

use stolen devices to harm the owner; for instance, an unauthorized user with a stolen smartwatch can unlock the owner’s car or house. To prevent unauthorized use of the stolen device or other misuses of the device, the BT-Node can revoke the ID of the device as follows: Step1: The BT-Node calls the revoke function. Revoke (IDX ) Step 2: Status on the ledger of ID is changed from Active to Revoked. Step 3: The revoked device ID and respective Public key and the certificate will reside in the ledger until its validity is not expired. Figure 4 shows the workflow of the above process. Fig. 4 Revocation of certificate

Smart Contract Assisted Public Key Infrastructure for Internet …

225

Fig. 5 X.509 certificate

5 Implementation The prototype is implemented with Hyperledger Fabric blockchain. At the initial, five BT-Nodes create the blockchain network, and each BT-Node is assigned ID to authenticate them on the blockchain. The “SC-PKI” smart contract uses “GO” language to program the PKI functions, i.e., “Generate ()”, “Authn()”, and “Revoke()”. The “SC-PKI” smart contract is deployed on each BT-Nodes. The IoT device’s requests are simulated using the command line, and no actual devices are connected with the system.

6 Analysis and Results The proposed SC-PKI approach provides secure PKI implementation for IoT devices. This approach provides security, offers no single-point failure, lightweight features, and enhanced battery life. The following are the outcomes of the SC-PKI approach.

6.1 Two-Layer of Security The proposed approach provides two-layer of security to the IoT ecosystem. First is, the BT-Nodes create the closed network. Only trusted BT-Nodes can participate

226

P. Bai et al.

in the network, and BT-Nodes authenticate and register themselves on the ledger before participate in the PKI process. The Hyperledger blockchain or other private blockchain can fulfill this requirement. The second layer of security is at the device level. The IoT device itself generates and stores key pairs in the proposed scheme, which makes it more secure than the traditional PKI approach. Since the key pair of IoT devices are stored with manufacturers, these devices are also vulnerable to attack the manufacturer’s server, and keys can be leaked by hacking the manufacturer’s server. However, we propose that these vulnerabilities can be addressed by storing the key on the IoT device itself.

6.2 Lightweight The blockchain implementation needs a large storage space, which limits blockchain deployment on memory-constrained IoT devices. But in our proposed work, the devices do not need to install the blockchain or smart contract. The BT-Nodes have smart contracts and a blockchain ledger. IoT devices need to connect with BT-Node only. Further, the use of ECC in key generation ensures less requirement of computation power to generate Public and Private Keys.

6.3 Better Battery Life The Computation and processing overhead at the end devices (IoT devices) is reduced with the help of lightweight ECC key generation for Public and private keys. And BT-Nodes perform the authentication and verification process. Thus, the battery of IoT devices are not much consumed in PKI process, and IoT device lives more.

6.4 Certificate Verification Time The performance of a blockchain system depends on many factors like block size, transaction size, network size, system hardware, software, etc. The PC uses for simulation has the following configuration: • Six Core CPU (i7 intel core 10th generation) • 32 GB memory • 512 GB SSD We measured the time taken for authentication and verification of the digital certificate. We run five rounds on with different BT-Nodes and take an average of all results. On average, the system takes 12 ms to verify and authenticate IoT devices

Smart Contract Assisted Public Key Infrastructure for Internet …

227

Verification Time

Time (ms)

Fig. 6 Certificate verification time

16 14 12 10 8 6 4 2 0

13

11

5

10

14

14.5

15

20

Numer of nodes

Verification Time Time (ms)

Fig. 7 Comparison with CA-based and emercoin

450 400 350 300 250 200 150 100 50 0

391

128 15

on the network. The time increases as the network size increases. The experimental results plot in Fig. 6. Figure 7 shows the comparison of SC-PKI with CA-based and Emercoin [12] approaches. The CA-based approach takes around 391 ms to verify the certificate against the specific user while our proposed approach takes less than 15 ms when network size is large and the time can be decreased when network is small.

7 Conclusion and Future Work Smart contract-based PKI is a viable alternative to the CA-based PKI to get desirable security. In this paper, we present an approach for decentralized lightweight PKI by using the smart contract. This approach is designed for memory-constrained devices. The decentralized nature of SC-PKI prevents single-point failure. The storage and verification of PKI certificate at BT-Node makes the approach lightweight and fast. For future work, privacy can be explored using various key exchange schemes in the SC-PKI approach.

228

P. Bai et al.

References 1. M. Frustaci, P. Pace, G. Aloi, G. Fortino, Evaluating critical security issues of the IoT world: present and future challenges. IEEE Internet of Things J. 5(4), 2483–2495 (2018) 2. K. Pankaj Kumar, S. Kumar, A. Jaiswal, M. Prasad, A. H. Gandomi, Towards precision agriculture: IoT-enabled intelligent irrigation systems using deep learning neural network. IEEE Sens. J. (2021) 3. W. Zhou, Y. Jia, A. Peng, Y. Zhang, P. Liu, The effect of IoT new features on security and privacy: new threats, existing solutions, and challenges yet to be solved. IEEE Internet of Things J. 6(2), 1606–1616 (2019) 4. M. El-hajj, A. Fadlallah, M. Chamoun, A. Serhrouchni, A survey of internet of things (IoT) authentication schemes. Sensors 19(5) (2019) 5. M. Schukat, P. Cortijo, in Public Key Infrastructures and Digital Certificates for the INTERNET OF THINGS. 2015 26th Irish Signals and Systems Conference (ISSC), Carlow (2015), pp. 1–5 6. Digicert, PKI—The Security Solution for the (2017) 7. Internet of Things. [White Paper]. https://www.digicert.com/internet-of-things/iot-pki-whitep aper/ 8. S. Yao, J. Chen, K. He, R. Du, T. Zhu, X. Chen, PBCert: privacy-preserving blockchain-based certificate status validation toward mass storage management. IEEE Access 7, 6117–6128 (2018) 9. H. Ning, A security framework for the internet of things based on public key infrastructure. Adv. Mater. Res. 671, 3223–3226. Trans Tech Publications Ltd (2013) 10. A. Singla, E. Bertino, in Blockchain-Based PKI Solutions for IoT. 2018 IEEE 4th International Conference on Collaboration and Internet Computing (CIC), Philadelphia, PA (2018), pp. 9–15 11. R. Rivera, J.G. Robledo, V.M. Larios, J.M. Avalos, in How Digital Identity on Blockchain Can Contribute in a Smart City Environment. 2017 International Smart Cities Conference (ISC2), Wuxi (2017), pp. 1–4 12. J. Won, A. Singla, E. Bertino, G. Bollella, in Decentralized Public Key Infrastructure for Internet-of-Things. MILCOM 2018- 2018 IEEE Military Communications Conference (MILCOM), Los Angeles, CA 2(018), pp. 907–913 13. M. Al-Bassam, in SCPKI: A Smart Contract-Based PKI and Identity System. Proceedings of the ACM Workshop on Blockchain, Cryptocurrencies and Contracts (2017), pp. 35–40 14. A. Yakubov, S. Wazen, W. Anders, S. David, in A Blockchain-Based PKI Management Framework. The First IEEE/IFIP International Workshop on Managing and Managed by Blockchain (Man2Block) colocated with IEEE/IFIP NOMS 2018, Tapei, Tawain 23–27 April 2018 15. K. Daniel, M. Hammoudeh, in Optimisation of the Public Key Encryption Infrastructure for the Internet of Things. Proceedings of the 2nd International Conference on Future Networks and Distributed Systems (2018) 16. S.L. Keoh, S.S. Kumar, H. Tschofenig, Securing the internet of things: a standardization perspective. IEEE Internet of Things J. 1(3), 265–275 (2014) 17. T. Kothmayr, C. Schmitt, M. Hu, M. Brünig, G. Carle, DTLS based security and two-way authentication for the Internet of Things. Ad Hoc Netw. 11(8), 2710–2723 (2013) 18. F. Forsby, F. Martin, P. Panos, S. Raza. Lightweight x. 509 digital certificates for the internet of things. in Interoperability, Safety and Security in IoT (Springer, Cham, 2017), pp. 123–133 19. H. Khemissa, D. Tandjaoui, in A Novel Lightweight Authentication Scheme for Heterogeneous Wireless Sensor Networks in the Context of Internet of Things. 2016 Wireless Telecommunications Symposium (WTS), London (2016), pp. 1–6 20. S. Matsumoto, R.M. Reischuk, IKP: Turning a PKI Around with Blockchains. IACR Cryptol. ePrint Arch. p. 1018 (2016) 21. L.M. Axon, M. Goldsmith, PB-PKI: A Privacy-Aware Blockchain-Based PKI (2016) 22. C. Patsonakis, K. Samari, A. Kiayiasy, M. Roussopoulos, in On the Practicality of a Smart Contract PKI. 2019 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPCON), Newark, CA, USA (2019), pp. 109–118

Smart Contract Assisted Public Key Infrastructure for Internet …

229

23. S. Yao, J. Chen, K. He, R. Du, T. Zhu, X. Chen, PBCert: privacy-preserving blockchain-based certificate status validation toward mass storage management. IEEE Access 7, 6117–6128 (2019). https://doi.org/10.1109/ACCESS.2018.28898 24. B. Pinky, in Energy Efficient Communication Protocol at Network Layer for Internet of Things. 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN). IEEE (2018) 25. V. Gayoso Martínez, L. Hernández Encinas, C. Sánchez Ávila, A Survey of the Elliptic Curve Integrated Encryption Scheme (2010) 26. H. Sukhwani, J. M. Martínez, X. Chang, K. S. Trivedi, A. Rindos, in Performance Modeling of PBFT Consensus Process for Permissioned Blockchain Network (Hyperledger Fabric). 2017 IEEE 36th Symposium on Reliable Distributed Systems (SRDS). IEEE (2011), pp. 253–255

IoTFEC-19: Internet of Things-based Framework for Early Detection of COVID-19 Suspected Cases Mahmood Hussain Mir, Sanjay Jamwal, Shahidul Islam, and Qamar Rayees Khan

Abstract World is battling with COVID-19 Pandemic, it has infected 236 million and taken over 4.83 million lives globally. In India alone, 33.89 million are infected and caused over 4.49 Lakh of fatalities. In Jammu and Kashmir, 3.3 Lakh are infected and caused over 4.4 thousand deaths. Researchers are trying their best to come up with solutions that can combat COVID-19. It is slated that the world has to battle with it and to follow SOP’s until and unless an effective vaccine will be developed. On the technological side, IoT is a new and promising area to combat with the COVID19 Pandemic. Nowadays, smartphones and wearables have various onboard sensors like temperature sensor, proximity sensor, an audio sensor, camera, inertial sensor, color sensor, etc. that can be used in getting the data of a person. The temperature sensor reading can tell us the temperature of a person. Based on that reading, a person can be sent for further clinical tests at an initial level. IoT technology can accurately manage patient information, and can be effective in proactive diagnosis with reduced cost. The IoT technology, a set of well-organized components, can work together as a part of an system to fight against and will lower the spread of the COVID-19 Virus. IoT has gained considerable attention from almost every field, such as industries and especially from healthcare. IoT technology is reshaping the traditional healthcare system by incorporating technology into it. In this article, a IoT layered architecture have been proposed with three different layers. The detailed working of all three layers is explained. A novel IoTFEC-19 framework has been proposed to detect COVID-19 suspects early with sensors and wearable devices. The Framework consists of three layers: Sensing Layer or Data Collection Layer, Data M. H. Mir (B) · S. Jamwal · S. Islam · Q. R. Khan Department of Computer Sciences, Baba Ghulam Shah Badshah University, Rajouri, India e-mail: [email protected] S. Jamwal e-mail: [email protected] S. Islam e-mail: [email protected] Q. R. Khan e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_19

231

232

M. H. Mir et al.

Analytics Layer, and Prediction Layer. In Sensing Layer, different sensors are used to collect the data and send it for analysis. In Analysis Layer, the level of symptoms is calculated from the data received from the Sensing Layer. The third layer is the Prediction Layer, in which prediction is made from the computed values in Analysis Layer, whether the suspected may be COVID-19 positive or negative. The last layer is the cloud layer used for storage services and data used for further analysis. The proposed IoTFEC framework aims to detect the COVID-19 suspected early, provide early treatment, and stop further spreading. Keywords IoT · Sensors · COVID-19 · Framework · Healthcare

1 Introduction Internet of Things (IoT) is a novel and promising ICT area that connects physical objects through communication technology such as BLE, Wi-Fi to the Internet [1]. The current objects embedded with sensing, communication, and computation capabilities can qualify as smart devices. Smart devices can capture data and share with other devices to achieve the required task [2]. There are lot of potential areas of IoT such as smart cities, smart transportation, smart homes, smart agriculture, and smart healthcare system. Smart healthcare system is most potential research area of IoT due to the advancement in sensors and medical devices technologies [3]. The increasing cost of healthcare facilities and existence of many new diseases around the globe needs urgent transformation of traditional healthcare system. The traditional healthcare system is hospital-centric which is to change to person-centric system [4]. IoT technology has potential to boom healthcare system. IoT can better utilize the available resources, with reduced cost and with minimal human interaction [5]. The use of IoT technology in healthcare for monitoring the COVID-19 pandemic. Vast number of people die due to lack or incorrect and untimely information about their health. The use of IoT technology can quickly notify individuals health parameters through the deployed or wearable sensors [6]. The IoT technology can watch and capture the routine activities of an individual and can make necessary alerts if there is any health issue [7]. In late 2019 a new disease severe acute respiratory syndrome coronavirus (SARSCoV) came into existence, named CoV-19 [8]. Firstly, by WHO it was declared as epidemic with its epicenter in Wuhan China, but later it was declared as pandemic.1 In September 2021 the number of COVID-19 confirmed cases were exceeded 236. 62 million with 3.7% mortality rate.2 COVID-19 has spread threat all over the world as more and more people were dying due to it. Researchers from all fields were investigating and continuing to set up varied solutions that combat the COVID-19 1

https://www.who.int/emergencies/diseases/novel-coronavirus-2019?gclid=CjwKCAjwtfqKBhB oEiwAZuesiPMFVE2PTOVcr2amZrDTBmrKniqULEXE1xUpljc13NR2FBm2OBI9_hoCHHcQ AvD_BwE. 2 https://covid19.who.int/

IoTFEC-19: Internet of Things-based Framework for Early …

233

[9]. As of now, the only way to handle the COVID-19 is to slow down the pace of spread by following the SoP’s like, social distancing, face masks and washing hands etc. On the other hand, use of IoT technology can also slow down the spread of COVID-19 through monitoring and early identification of suspects. In IoT large array of sensors are used to collect the data from different sources through networks [10]. IoT is potential technology that can deal with large amounts of data and lower the transmission time of critical information that can help provide timely response during COVID-19 [11]. COVID-19 previously known as 2019-nCoV comes under the family of Coronaviridae that causes illness from common cold to more severe diseases. In 2012, one coronavirus named MERS-CoV originated in Saudi Arabia with a 35% fatality rate [12]. SARS-CoV is another viral respiratory disease and comes under the family of Coronaviruses, that first reported in 2003 in Southern China. Later both these MERSCoV and SARS-CoV spread in many across the globe [13]. The 2019-nCoV named as SARS-CoV-2 later simply name COVID-19 came into existence in November 2019 [14]. It comes under the family of Coronaviruses, but with new strain varying from its previous ones in physical and chemical properties and was identified as novel enveloped RNA beta coronavirus. The virus is more vulnerable and transferable than its previous strains and infection factors are high. COVID-19 has infected 236.62 million people and taken over 48.32 million lives from the date it exists. The WHO on March 11 2020, has declared novel COVID-19 as pandemic. In order to stop the spread of COVID-19 most of the countries across the globe have shut down all the traffic including air, railways and markets. Many countries have also imposed restrictions or lock down the cities [15]. The COVID-19 pandemic has opened a vast area for academics, researchers, companies, and investors to provide solutions [16]. The traditional system of healthcare cannot cope up with the current pandemic situation [17]. Most of researchers around the globe are struggling to develop the technique that can cope with such challenges. Due to the advancement in mobile technology smart devices in healthcare have impacted the world [18]. Now a days smart phones have inbuilt onboard lot of sensors with computation capabilities. By these onboard sensors, it is not impossible to capture the real-time data of an individual [19]. Smart phones can act as input devices for sensing, storing large volume of data. The collected data can then be analyzed with the help of algorithms and applications to generate results [20]. By the use of technology, it is possible to detect the COVID-19 suspects in early stages to eliminate the spread of infection. Tracking and quarantining of COVID-19 positive and suspected cases can be tracked and monitored with the help of onboard mobile phone sensors [21]. Travel history and other parameters can be accessed of an individual through smart phones by public health agencies and other players [22]. IoT has revolutionized the world, which allows the integration of physical objects with inbuilt communication technologies to the Internet that provides real-time status of individuals anywhere, anytime [23]. Integration IoT with other technologies like Machine Learning (ML) and Artificial Intelligence (AI) can revolutionize each sector, mostly the healthcare sector [24]. Khan et al. [25] illustrates an example of HIV/AIDS drug regimen selection employing the fuzzy discrete event system (FDES). The future

234

M. H. Mir et al.

predictions can be made where and when the disease is likely to spread to which part, so that arrangements can be made accordingly. The IoT can act as data source, and data can be analyzed with ML algorithms for better insights [26]. IoT has made it easy for doctors and health professionals to give best treatment to patients. By IoT a centralized information system can be created where all activities are stored digitally [27]. The Novel Contribution of this work is as under: • An IoT layered architecture have been proposed with three different layers, Sensing Layer, Data Analytics Layer, and Prediction Layer. • A novel IoTFEC-19 framework has been proposed to detect COVID-19 suspects early with sensors and wearable devices. The proposed IoTFEC-19 framework is divided into three components: (1) Data collection using wearables and sensors. (2) Data analysis and prediction module that ML techniques. (3) Cloud infrastructure. The proposed framework aims to predict and detect the COVID-19 suspects early to reduce the disease severity and eliminate its spread. The rest of the article is organized as follows. Section 2 describes the latest relevant literature about the technologies, frameworks and techniques. Section 3 details out the proposed IoTFEC-19 framework. Section 3.1 elaborates the proposed layered architecture with different layers. Section 3.2 explains the proposed framework with its interrelated modules. This section also explains the flow of the tasks. Section 4 briefly introduces the emerging trends and opportunities. Section 5 concludes the article.

2 Background COVID-19 repugnant word across the globe since its outbreak in November 2019. But on the other hand, it becomes most searched word on Internet from a research point of view. The vast research is going on the impact of IoT technology to combat with COVID-19. Different kinds of research articles are available on different domains such IoT-based frameworks, IoT and ML, IoT and AI, and other mathematical models. Here, in this article latest research articles are reviewed and summarized systematically. The [28] have conducted a systematic literature review of IoT-based healthcare systems. The article also discussed the important challenges in delivering the health services integrated with IoT. Authors in [29] proposed a solution to urban area by providing a smart healthcare system using IoT devices. Most of the issues are taken into consideration like safety, security, and response are discussed. The proposed system was evaluated on NetSim and NS2. Darwish et al. [30] have proposed a cloud-IoT-based system, which integrates the two technologies to develop healthcare system. The article had also discussed the issues in integrating and trends in cloud-IoT-based healthcare system. Authors have classified these issues in 3 different categories viz, technology, communication, and intelligence. Alshraideh et al. [31]

IoTFEC-19: Internet of Things-based Framework for Early …

235

have proposed IoT-based detection system for cardiovascular diseases. The authors have used machine learning techniques in their article. Maghded et al. [19] proposed a system based on smartphone, which takes the data from onboard sensors like temperature sensor. The authors in [32] reviewed a lot of literature of AI technology used for COVID-19 research. The paper has also highlighted the integration of IoT with artificial intelligence. The article [33] have presented a detailed review of IoT and elaborated the challenges in same. The article gives insights of IoT concepts, technologies, applications, and research gaps. Bai et al. [34] have developed a model for early detection of COVID-19 suspects named as nCapp. Based on given input the system generates the diagnosis automatically and categorizes the patients into three viz, confirmed, suspected or suspicious. In [35] the authors gave detailed view on applications of IoT in Medical healthcare system. Siriwardhana et al. [36] a comprehensive review on latest technologies have been explored such 5G, AI, UAV, and IoT in mitigating COVID-19. Similarly same has been explored in [37]. In [38] various application areas have been discussed such as ambient living, IoT transportation, and smart cities. Verma et al. [39] have proposed cloud-centric IoT-based smart health framework. The framework calculates the severity of diseases in students based on temporal mining. The authors have used the data set of 182 entries and have applied various machine learning techniques. Moreover, the proposed system is generating alerts in emergency cases. Javaid and Khan [40] numerous literatures have been studied using search terms IoT in healthcare, IoT and COVID to identify the technologies. Authors have identified seven major technologies that seem too helpful in healthcare with respect to IoT. Authors have identified sixteen major applications of IoT in medical care and briefly described them. Kumar et al. [17] have reviewed a literature on COVID-19 pandemic, IoT for COVID-19, IoT In healthcare, monitoring techniques, detection techniques, IoT architectures. Authors have also proposed an architecture to avoid the spread of COVID-19. The proposed architecture based on cloud-IoT technology. Verma and Sood [41] in this article authors have developed an IoT-cloud-based disease diagnosis framework. The framework is divided into three different layers, User system, Cloud system, and Alert generation. The framework is more patient-centric that collects the data using sensors continuously [42]. Artificial Intelligence technology is full of potential that can tackle during pandemic. In this, authors have reviewed the AI-based techniques that can combat COVID-19. AI technology plays an important role in identifying the cluster of infection and to predict the future infection area by analyzing the previous data. Authors have identified seven potential applications of AI to combat COVID-19 Pandemic. Otoom et al. [43] have proposed an IoT-based real-time COVID-19 detection and monitoring system. The proposed framework uses IoT to collect or sense the real-time symptom data from suspected cases and to monitor the recovered case. The proposed system is also analyzing the collected data to study the nature of virus. The framework is comprised five components and is based on IoT-cloud model. Singh et al. [11] in this article authors have done systematic review using the search terms IoT, Internet of Things and COVID-19. During their review twelve application areas of IoT have been highlighted. A COVID-19 infected person can be monitored more sophisticatedly and symptoms are identified more correctly.

236

M. H. Mir et al.

3 Proposed IoTFEC-19 Framework 3.1 Proposed Architecture The IoT-based systems are mostly layered-based architectures. Our proposed architecture is divided into three different layers. The Fig. 1 gives the graphical representation of the proposed layered architecture. The proposed architecture is divided into four layers: Data Collection Layer, Data Analysis Layer, Prediction Layer, and Cloud Layer. Data Analysis Layer and Prediction Layer are two Sub-Layers in our proposed system. Data Collection or Sensing Layer is used to collect the data with the help of deployed sensors. Different sensors are used to collect the data such as a temperature sensor is used to collect of body temperature of person. Heart rate sensor is used to collect the heart beat of a person, BP sensor is used to collect blood pressure of a person and pulse sensor is used to collect the pulse of a person, etc. The collected data is sent to Data Analysis and Prediction Layer. The data is analyzed and preprocessed and results are generated. In the same layer, prediction is whether the person is COVID-19 suspected or not. If suspected, the required actions are taken if the sensor values are below or above the threshold or predefined values. In case of temperature, if temperature is above threshold value, then the person may be COVID19 suspected. In same way, if heart beat is above or below the predefined values then a person may be COVID-19 suspected. On the basis of computed values of different sensors, the prediction is whether a person is COVID-19 suspected or not. In last Layer that is the Cloud Layer which acts as storage and further insightful analysis of data received from layers below it. Cloud Layer is responsible for doing high end analytics, various machine learning algorithms are employed to get better results. The health-related collected attributes are compared with already stored medical measurements.

Fig. 1 Three layered architecture of proposed system

IoTFEC-19: Internet of Things-based Framework for Early …

237

3.2 Proposed Framework The proposed methodology is described in the Conceptual framework of IoT based COVID-19 early detection system. The proposed system can be called as Internet of Things-based Framework for Early Detection of COVID-19 Suspected Cases (IoTFEC-19). The graphical representation of the proposed IoTFEC-19 is given in Fig. 2. Data collection layer is responsible for collecting of relevant symptoms through deployed sensors and wearables. Data is collected ubiquitously from various sensors embedded on various places and from wearables through BSN (body sensor network). Other parameters such as travel history and contact history are collected from mobile phone location registries and other applications. The second layer is divided into two sub-modules, data analysis and prediction layer. In first module, the data received from sensing layer is analyzed and results are generated. In this layer, ML algorithms are running to process the incoming data. In second module, prediction is based on results generated by the analysis module to predict the potential suspect of COVID-19. The prediction is also made for the possible area or containment zone in the area. The calculated values are compared with already available values to predict the diseases. The data then send to cloud for storage and necessary actions. The third layer is the cloud layer which is responsible storing the data received from layer below it. The main aim of this layer is to extract the meaningful information from received data. The data stored is accessed by health officials to understand the disease better. This layer is connected to the internet, which maintains all health records and communicates recommendations for predicted results and health workkers recommendations.

Fig. 2 Conceptual view of proposed system

238

M. H. Mir et al.

Fig. 3 Flow diagram of proposed framework

3.3 Data Flow of Proposed Framework Figure 3 describes the flow of the proposed framework. (1) The symptoms are collected from various deployed sensors and wearables. These symptoms are fever, O2 saturation, pulse, shortness of breath, BP, heart rate, travel history, contact history, etc. (2) The symptoms sensed by the sensor layer are sent to the data analysis layer to identify the suspected persons. The analysis is done through algorithms deployed in the analysis layer. The prediction is also made by employing machine learning algorithms to predict whether the person is COVID-19 suspected or not. This module is also responsible for predicting the severity of disease and containment area. (3) The results are stored in the cloud to access it by health professionals to understand data better. Moreover, better predictions can be made by analyzing the stored data from the cloud databases.

4 Research Opportunities and Emerging Trends Deploying of IoT-based architecture never an easy task. Moreover, the implementation of IoT-based systems during COVID-19 faces many challenges. The main concern in deploying IoT systems is the security and privacy issues of data received during the COVID-19 pandemic. The “COVID-19” and “positive” are the most repugnant terms of the year 2020. No one wants to disclose the information about their COVID-19 results. Healthcare data is critical to ensuring the privacy and security of patients data is a challenge. Lot of devices are getting added in the array of IoT, so scalability is another challenge. The complex network of heterogeneous objects collecting the data, so heterogeneity is another challenge. The more devices more bandwidth, as the number of devices grows exponentially, the more the bandwidth is needed. During the COVID-19, the life of utmost importance. Errors and

IoTFEC-19: Internet of Things-based Framework for Early …

239

delays may cause loos human lives. Lot of research is going on in developing the security algorithms. IoT devices are tiny objects with less power, less memory, etc. lot of research is going to develop lightweight security algorithms. This is still an open issue lot of research is needed to be done. Blockchain is another field by which security and privacy issues can be solved. Lot of research is to be done to develop the IoT based healthcare that can combat during pandemic. COVID-19 has opened a lot of research opportunities for academicians, researchers, industries, etc.

5 Conclusion IoT is potential technology that can combat with COVID-19 pandemic. IoT is network of integrated objects in which objects are connected to the Internet. In case of a pandemic like situation it can automatically communicate with the system. This paper has proposed a layered architecture and IoTFEC framework for early detection of COVID-19 cases. The architecture has three layers: sensing layer, analysis and prediction layer, and cloud layer. Apart from architecture, The proposed IoTFEC framework has three different components: data collection layer, analysis and prediction layer, and cloud layer. The framework collects the symptoms and analyze them using various algorithms and prediction is made whether the suspected is COVID-19 suspected or not. The proposed framework could detect the suspects in early-stages, minimize the spread of infection, and monitor the COVID-19 positive patients. The framework stores the symptomatic data in cloud, the data can be used to analyze the disease further better. The use of technology in healthcare will eliminate the errors made by humans to improve the quality of lives.

References 1. N.M. Kumar, P.K. Mallick, The Internet of Things: insights into the building blocks, component interactions, and architecture layers. Procedia Comput. Sci. 132, 109–117 (2018). https://doi. org/10.1016/j.procs.2018.05.170 2. J. Gubbi, R. Buyya, S. Marusic, M. Palaniswami, Internet of Things (IoT): a vision, architectural elements, and future directions. Futur. Gener. Comput. Syst. 29(7), 1645–1660 (2013). https:// doi.org/10.1016/j.future.2013.01.010 3. H. Ahmadi, G. Arji, L. Shahmoradi, R. Safdari, M. Nilashi, M. Alizadeh, The Application of ˙ Internet of Things in Healthcare: A Systematic Literature Review and Classification, vol. 18, no. 4 (Springer, Berlin Heidelberg, 2019). https://doi.org/10.1007/s10209-018-0618-4 4. S. Tuli et al., Next generation technologies for smart healthcare: challenges, vision, model, trends and future directions. Internet Technol. Lett. 3(2), e145 (2020). https://doi.org/10.1002/ itl2.145 5. S. Vidya Priya Darcini, D.P. Isravel, S. Silas, in A Comprehensive Review on the Emerging IoT-Cloud based Technologies for Smart Healthcare. 2020 6th International Conference on Advanced Computer Communication System ICACCS 2020, pp. 606–611 (2020). https://doi. org/10.1109/ICACCS48705.2020.9074457

240

M. H. Mir et al.

6. S. Islam, S. Jamwal, M.H. Mir, Leveraging fog computing for smartınternet of thingscrop monitoring farming in Covid-19 era. Ann. R.S.C.B. 25(6), 10410–10420 (2021) 7. P.N. Mahalle, S.S. Sonawane, Internet of things in healthcare. SpringerBriefs Appl. Sci. Technol. 13–25 (2021). https://doi.org/10.1007/978-981-33-6460-8_2 8. N. Zhu et al., A novel coronavirus from patients with pneumonia in China, 2019. N. Engl. J. Med. 382(8), 727–733 (2020). https://doi.org/10.1056/nejmoa2001017 9. M. Nasajpour, S. Pouriyeh, R.M. Parizi, M. Dorodchi, M. Valero, H.R. Arabnia, Internet of Things for current COVID-19 and future pandemics: an exploratory study. J. Healthc. Inf. Res. 4(4), 325–364 (2020). https://doi.org/10.1007/s41666-020-00080-6 10. M. Kamal, A. Aljohani, E. Alanazi, IoT Meets COVID-19: Status, Challenges, and Opportunities (2020), [Online]. Available: http://arxiv.org/abs/2007.12268 11. R.P. Singh, M. Javaid, A. Haleem, R. Suman, Internet of things (IoT) applications to fight against COVID-19 pandemic. Diabetes Metab. Syndr. Clin. Res. Rev. 14(4), 521–524 (2020). https://doi.org/10.1016/j.dsx.2020.04.041 12. S.Y. Zhang et al., Clinical characteristics of different subtypes and risk factors for the severity of illness in patients with COVID-19 in Zhejiang, China. Infect. Dis. Poverty 9(1), 1–10 (2020). https://doi.org/10.1186/s40249-020-00710-6 13. N.C. Peeri et al., The SARS, MERS and novel coronavirus (COVID-19) epidemics, the newest and biggest global health threats: what lessons have we learned? Int. J. Epidemiol. 49(3), 717–726 (2021). https://doi.org/10.1093/IJE/DYAA033 14. W. Guan et al., Clinical characteristics of coronavirus disease 2019 in China. N. Engl. J. Med. 382(18), 1708–1720 (2020). https://doi.org/10.1056/nejmoa2002032 15. D. Cucinotta, M. Vanelli, WHO declares COVID-19 a pandemic. Acta Biomed. 91(1), 157–160 (2020). https://doi.org/10.23750/abm.v91i1.9397 16. A. Sufian, A. Ghosh, A. Safaa, F. Smarandache, Since January 2020 Elsevier has Created ˙ a COVID-19 Resource Centre with Free Information in English and Mandarin on the Novel Coronavirus COVID-19 (The COVID-19 resource centre is hosted on Elsevier Connect, the company’ s public news and information”, 2020) 17. K. Kumar, N. Kumar, R. Shah, Role of IoT to avoid spreading of COVID-19. Int. J. Intell. Netw. 1(May), 32–35 (2020). https://doi.org/10.1016/j.ijin.2020.05.002 18. A.S.R. Srinivasa Rao, J.A. Vazquez, Identification of COVID-19 can be quicker through artificial intelligence framework using a mobile phone-based survey when cities and towns are under quarantine. Infect. Control Hosp. Epidemiol. 41(7), 826–830 (2020). https://doi.org/10. 1017/ice.2020.61 19. H.S. Maghded, K.Z. Ghafoor, A.S. Sadiq, K. Curran, D.B. Rawat, K. Rabie, in A Novel AI-enabled Framework to Diagnose Coronavirus COVID-19 using Smartphone Embedded Sensors: Design Study. Proceedings 2020 IEEE 21st International Conference Infracture Reuse Integrated Data Science IRI 2020, pp. 180–187 (2020). https://doi.org/10.1109/IRI49571.2020. 00033 20. P. Verma, S.K. Sood, Fog assisted-IoT enabled patient health monitoring in smart homes. IEEE Internet Things J. 5(3), 1789–1796 (2018). https://doi.org/10.1109/JIOT.2018.2803201 21. M.U. Ashraf, A. Hannan, S.M. Cheema, Z. Ali, K.M. Jambi, A. Alofi, in Detection and Tracking Contagion using IoT-Edge Technologies: Confronting COVID-19 Pandemic. 2nd International Conference on Electrical Communication and Computer Engineering ICECCE 2020 (2020). https://doi.org/10.1109/ICECCE49384.2020.9179284 22. M. Ndiaye, S.S. Oyewobi, A.M. Abu-Mahfouz, G.P. Hancke, A.M. Kurien, K. Djouani, IoT in the wake of Covid-19: a survey on contributions, challenges and evolution. IEEE Access 8, 186821–186839 (2020). https://doi.org/10.1109/ACCESS.2020.3030090 23. T. Ladd, O. Groth, The internet of things. Econ. (United Kingdom) 411(8964) (2015) 24. A.A. Hussain, O. Bouachir, F. Al-Turjman, M. Aloqaily, AI Techniques for COVID-19. IEEE Access 8, 128776–128795 (2020). https://doi.org/10.1109/ACCESS.2020.3007939 25. N.J. Khan, G. Ahamad, M. Naseem, Q.R. in Khan, Fuzzy Discrete Event System (FDES): A Survey. Lecture Notes in Electrical Engineering, vol. 723 LNEE, pp. 531–544 (2021). https:// doi.org/10.1007/978-981-33-4080-0_51

IoTFEC-19: Internet of Things-based Framework for Early …

241

26. A.M.U.D. Khanday, S.T. Rabani, Q.R. Khan, N. Rouf, M. Mohi Ud Din, Machine learning based approaches for detecting COVID-19 using clinical text data. Int. J. Inf. Technol. 12(3), 731–739 (2020). https://doi.org/10.1007/s41870-020-00495-9 27. S. Tuli, S. Tuli, R. Tuli, S. Singh, Since January 2020 Elsevier has created a COVID-19 resource centre with free information in English and Mandarin on the novel coronavirus COVID-19. The COVID-19 resource centre is hosted on Elsevier Connect, the company ’ s public news and information (2020) 28. M. Usak, M. Kubiatko, M.S. Shabbir, O.V. Dudnik, K. Jermsittiparsert, L. Rajabion, Health care service delivery based on the Internet of things: a systematic and comprehensive study. Int. J. Commun. Syst. 33(2), 1–17 (2020). https://doi.org/10.1002/dac.4179 29. M. Rath, B. Pattanayak, Technological improvement in modern health care applications using Internet of Things (IoT) and proposal of novel health care approach. Int. J. Hum. Rights Healthc. 12(2), 148–162 (2019). https://doi.org/10.1108/IJHRH-01-2018-0007 30. A. Darwish, A.E. Hassanien, M. Elhoseny, A.K. Sangaiah, K. Muhammad, The impact of the hybrid platform of internet of things and cloud computing on healthcare systems: opportunities, challenges, and open problems. J. Ambient Intell. Humaniz. Comput. 10(10), 4151–4166 (2019). https://doi.org/10.1007/s12652-017-0659-1 31. H. Alshraideh, M. Otoom, A. Al-Araida, H. Bawaneh, J. Bravo, “A Web based cardiovascular disease detection system. J. Med. Syst. 39(10) (2015). https://doi.org/10.1007/s10916-0150290-7 32. T.T. Nguyen, Q.V.H. Nguyen, D.T. Nguyen, E.B. Hsu, S. Yang, P. Eklund, Artificial Intelligence in the Battle against Coronavirus (COVID-19): A Survey and Future Research Directions (2020), pp 1–14. [Online]. Available: http://arxiv.org/abs/2008.07343 33. D. Das, D. Misra, G. Das, Review on Internet of Things ( IoT ): Making the World Smart (Springer, 2018) 34. L. Bai et al., Chinese experts’ consensus on the Internet of Things-aided diagnosis and treatment of coronavirus disease 2019 (COVID-19). Clin. eHealth 3, 7–15 (2020). https://doi.org/10. 1016/j.ceh.2020.03.001 35. M.N. Mohammed, S.F. Desyansah, S. Al-Zubaidi, E. Yusuf, An internet of things-based smart homes and healthcare monitoring and management system: review. J. Phys. Conf. Ser. 1450(1) (2020). https://doi.org/10.1088/1742-6596/1450/1/012079 36. Y. Siriwardhana, C. De Alwis, G. Gur, M. Ylianttila, M. Liyanage, The fight against the COVID19 pandemic with 5G technologies. IEEE Eng. Manag. Rev. 48(3), 72–84 (2020). https://doi. org/10.1109/EMR.2020.3017451 37. V. Chamola, V. Hassija, V. Gupta, M. Guizani, A comprehensive review of the COVID-19 pandemic and the role of IoT, drones, AI, blockchain, and 5G in managing its impact. IEEE Access 8(April), 90225–90265 (2020). https://doi.org/10.1109/ACCESS.2020.2992341 38. S. Nizetic, P. Solic, D.L.-I. Gonzalez-de-Artaza, L. Patrono, Since January 2020 Elsevier has created a COVID-19 resource centre with free information in English and Mandarin on the novel coronavirus COVID-19. The COVID-19 resource centre is hosted on Elsevier Connect, the company’s public news and information (2020) 39. P. Verma, S.K. Sood, S. Kalra, Cloud-centric IoT based student healthcare monitoring framework. J. Ambient Intell. Humaniz. Comput. 9(5), 1293–1309 (2018). https://doi.org/10.1007/ s12652-017-0520-6 40. M. Javaid, I.H. Khan, Internet of Things (IoT) enabled healthcare helps to take the challenges of COVID-19 pandemic. J. Oral Biol. Craniofacial Res. 11(2), 209–214 (2021). https://doi.org/ 10.1016/j.jobcr.2021.01.015 41. P. Verma, S.K. Sood, Cloud-centric IoT based disease diagnosis healthcare framework. J. Parallel Distrib. Comput. 116(January), 27–38 (2018). https://doi.org/10.1016/j.jpdc.2017. 11.018 42. R. Vaishya, M. Javaid, I.H. Khan, A. Haleem, Artificial Intelligence (AI) applications for COVID-19 pandemic. Diabetes Metab. Syndr. Clin. Res. Rev. 14(4), 337–339 (2020). https:// doi.org/10.1016/j.dsx.2020.04.012 43. M. Otoom, N. Otoum, M.A. Alzubaidi, Y. Etoom, An IoT-Based Framework for Early ˙ Identification and Monitoring of COVID-19 Cases (2020)

IoT-Based ECG and PCG Monitoring Systems V. A. Velvizhi, M. Anbarasan, S. Gayathri, K. Jeyapiriya, and S. Rajesh

Abstract The utmost momentous problem for human race is the health care due to hasty surge in population and therapeutic expenditure. Word Health Organization reports that population aging is the major cause of the issue. It is essential to monitor regularly the well-being of aged people. This imposes a heavy burden with the prevailing medical scheme. Hence there is a need for initial diagnosis of the disease at low cost. An effective healthcare monitoring system can be designed using Image Processing. Internet of Things assisted electrocardiogram and phonocardiogram monitoring system was proposed for secured data transmission. This system aids in continuous monitoring of the heart. Keywords Electrocardiogram · Phonocardiogram · Preprocessing · Segmentation · Convolutional neural network

V. A. Velvizhi (B) · S. Gayathri · K. Jeyapiriya Department of Electronics and Communication Engineering, Sri SaiRam Engineering College, Chennai-44, India e-mail: [email protected] S. Gayathri e-mail: [email protected] K. Jeyapiriya e-mail: [email protected] M. Anbarasan Department of Mechanical Engineering, University College of Engineering, Tindivanam, India e-mail: [email protected] S. Rajesh Department of Mechanical Engineering, Kalasalingam University, Krishnankoil, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_20

243

244

V. A. Velvizhi et al.

1 Introduction World Health Organization has reported that CVD is the leading cause of global death. It takes almost 17.9 lives every year [1]. The electrocardiogram (ECG) has long been a popular tool for assessing and diagnosing cardiovascular problems (CVDs). In the literature, there were numerous ECG monitoring systems which is growing at an exponential rate. As a result, it is difficult for scholars and healthcare professionals to select, compare, and assess systems that meet their demands while also meeting the monitoring criteria. This emphasizes the need for a reliable reference to guide the design, sorting, and examination of ECG monitoring systems, which would benefit both researchers and practitioners in the field. Hence Image processing play a major role in health monitoring system. Image acquisition, storage, preprocessing, segmentation, representation, recognition, and interpretation are all performed by an image processor, which then displays or records the generated image. The basic procedure involved in an image processing system is depicted as a block diagram in Fig. 1. Image acquisition via an imaging sensor in conjunction with a digitizer to digitize the image is the first stage in the process as shown in the diagram. The image is then improved before being provided as an input to the subsequent operations in the preprocessing step. Typical preprocessing tasks include improving, eliminating noise, separating regions, and so on. A new method for IoT-based ECG monitoring system was proposed in this research [2, 3]. ECG data is collected via a wearable monitoring node and wirelessly transferred to the IoT cloud. The HTTP and MQTT protocols are used in the IoT cloud to offer users with visual and fast ECG data. The current methods for recognizing radar signal emitters are reliant on preexisting knowledge. Furthermore, modern emitter recognition must address the obstacles posed by low possibility of intercept skill. The other obscuration approaches are based on complex signal modulation, while also demonstrating a relatively good capacity to recover feeble signals at small SNR levels. Furthermore, even with low SNR values, EEMD is proven to extract important signal structures, demonstrating

Fig. 1 Block diagram of fundamental sequences of an image processing system

IoT-Based ECG and PCG Monitoring Systems

245

its capacity to suppress noise [4, 5]. Finally, it is shown that EEMD-GST has a significantly stronger time-frequency concentrating property than either the ordinary S-transform or the short-time Fourier transform. A completely integrated Analog Front—End (AFE), a marketable microcontroller unit (MCU), a secure digital (SD) card, and a Bluetooth module make up the wearable ECG sensor. Due to the AFE design, the entire sensor is relatively compact, measuring only 58 × 50 × 10 mm for wearable monitoring applications, and the overall power degeneracy in a complete cycle of ECG capture is only 12.5 mW. The experimental findings showed that the suggested method is capable of enhancing arrhythmia diagnosis accuracy and recognizing the utmost common aberrant ECG patterns in a variety of events. Finally, a wearable, accurate, and energyefficient system was proposed for long-term and context-aware ECG monitoring that does not require the use of a kinetic sensor and instead relies on the widely used smartphone. In this paper, an IoT assisted echocardiogram and phonocardiogram monitoring framework was proposed. The system is capable of transmitting data securely for continuously monitoring the cardiovascular activities.

2 Methodology 2.1 Need for Cardio Vascular Health Monitoring System With a tremendous expansion in social inhabitants and sanctuary custom, Medicare facilities have become the greatest critical concerns for people and the government. Elderly people’s health should be tested more frequently, as a more noticeable test of current medical frameworks. Disease in human should be diagnosed in a comfortable and exact manner at a reasonable price must be carefully considered. The diagnostics focused on the electrocardiogram (ECG) in both therapies and therapeutic research have been extensively related as a result of the expansion, experience, and competence developed for heart examination. The suggested IoT aided ECG Framework has the latent to increase the correctness, precision, and consistency of an investigative scheme by determining the acceptance of ECG and PCG data. This IoT framework enables a device to communicate a process with the help of the environment.

2.2 Elements of the Proposed System The proposed method comprises a biomedical digital signal processing unit that monitors the electrocardiogram (ECG) and the phonocardiogram (PCG) via image processing. Figure 2 represent the block diagram of the proposed mode. It includes

246

V. A. Velvizhi et al.

Fig. 2 Block diagram of the proposed system

preprocessing of the input image obtained from publicly available database. The next step is to segment the image in to blocks of equal size. The Convolutional Neural Network is applied to the segmented dataset to obtain the features relevant to the image. The main goal of this proposed work is to identify a patient’s health status by sending an alarm message using an Arduino and updating the information in a webpage using Nodemcu, for the classification of ECG and PCG signals [6–8]. This research makes use of ECG and PCG datasets. The ECG dataset (normal or pathological) is sent to MATLAB, and the image is processed. Then, PCG dataset (normal or abnormal) is processed in a similar way.

3 Module Description ECG and PCG health monitoring system consist of • • • • •

Input Image Preprocessing Segmentation Convolutional Neural Network Output.

3.1 Input Image An input image is read from the publicly available database in MATLAB. Using the read command, load an image into the workspace. It is described as the action of collecting an image from a source, usually a hardware-based source, for processing in image processing. It is the first stage in the workflow sequence because no processing can take place without an image. The image that is obtained has not been processed in any way. Preprocessing steps done in [9] for Involuntary Discovery of Lung Cancer

IoT-Based ECG and PCG Monitoring Systems

247

Fig. 3 Input image of ECG and PCG signal

Nodules in Computerized Tomography Images can be implemented. From [10] the method of edge map extraction of infrared breast images has been utilized. Figure 3 depicts the input image of electrocardiogram (ECG) and phonocardiogram (PCG).

3.2 Segmentation To detect and identify objects and margins (lines, curves, etc.) in an image, pixels in a region are comparable based on similarity norms such as tint, intensity, or texture. Figure 4 represent typical ECG and PCG signal. All the input images are resized into same dimensions. The image will be distorted if the aspect ratio of the image is not the same. Sudha [8] depicts the Modified Contourlet Transform-Based Effective Image Compression Context that can be implemented for compression process. Fig. 4 Segment of ECG and PCG signal

248

V. A. Velvizhi et al.

3.3 Convolutional Neural Network The CNN algorithm takes an input image and processes it via layers to find features and recognize the image before producing a classification result. Convolutional and pooling layers alternate in the design of the CNN, which is followed by a sequence of fully linked layers. The input of the next layer in the CNN is the output of each layer in the CNN.

3.4 K-Nearest Neighbor Algorithm KNN is a classification method that approaches the function locally and all calculation is postponed until after the function has been evaluated. Because this method relies on distance for classification, normalizing the training data can greatly increase its performance if the features represent various physical units or come in wildly different scales.

3.5 Hardware Tools The hardware part of the module consists of Arduino UNO, NODE MCU, GSM. Within Atmel’s megaAVR family, the ATmega328 is a single-chip microcontroller. The Arduino Uno uses a Harvard architecture with an 8-bit RISC processing core that has been tweaked. The ESP8266 GPIOs are accessible through the Node MCU Development Kit. The only thing to keep in mind is that the pins on the Node MCU Dev kit are numbered differently than the ESP8266’s internal GPIO notations, as seen in the image and table below. The D0 pin on the NodeMCU Dev kit, for example, is mapped to the ESP8266’s internal GPIO pin 16. Figure 5 shows complete module of the proposed system.

4 Results and Discussion This research involves biomedical digital signal processing with image processing for electrocardiogram (ECG) and phonocardiograph monitoring (PCG). The most essential approaches and methodologies utilized in this project are to detect the patient’s health conditions by sending an alert message using an Arduino and updating the information in a webpage using NODE MCU, and to classify ECG and PCG signals using Node MCU

IoT-Based ECG and PCG Monitoring Systems

249

Fig. 5 Complete module of the proposed system

Figure 6 shows the R peak localization of done through Wavelet Transform with automatic annotation in normal image. Similarly, Figure 7 represents the output of abnormal ECG image. This research makes use of ECG and PCG datasets. ECG dataset (normal or abnormal) is given and image is processed using MATLAB. After that, a PCG dataset (normal or abnormal) is provided, which is processed as well. Figure 8 represent the output for PCG signal specifying normal image. If the ECG and PCG datasets are normal, a message with the word “NORMAL” will be communicated to physician and the custodians, and the word “NORMAL” will also be updated on the homepage using IoT. Figure 9 shows the output of abnormal PCG. The ECG and PCG datasets are abnormal, a message with the word “ABNORMAL” will be communicated to the physician and custodians. Figure 10 depicts the message sent to the patient, physician„ and patient. Fig. 6 Output of normal ECG

250 Fig. 7 Output of abnormal ECG

Fig. 8 Output of normal PCG

Fig. 9 Output of abnormal PCG

Fig. 10 Message stating abnormal image

V. A. Velvizhi et al.

IoT-Based ECG and PCG Monitoring Systems

251

The abnormal message is also updated and stored in IoT website through NODE MCU.

5 Conclusion In the foreseeable future, much more research is required to address many more IoT concerns. To begin with, there are no well-defined standards to integrate the IoT’s many systems and interfaces. Last but not least, when building the IoT, security and privacy issues should be considered. The Internet of Things (IoT) is a new technology paradigm that allows omnipresent things or objects to interact with one another and connect to the Internet.

References 1. https://www.who.int/health-topics/cardiovascular-diseases#tab=tab_1 2. H. Kim, S. Kim, N. Van Helleputte, A. Artes, M. Konijnenburg, J. Huisken, R.F. Yazicioglu, A configurable and low-power mixed signal SoC for portable ECG monitoring applications. IEEE Trans. Biomed. Circ. Syst. 8(2), 257–267 (2013) 3. T.A M. Phan, J.K. Nurminen, M. Di Francesco, in Cloud Databases for Internet-ofThings Data. Internet of Things (iThings), 2014 IEEE International Conference on, and Green Computing and Communications (GreenCom), IEEE and Cyber, Physical and Social Computing (CPSCom), IEEE. IEEE (2014), pp. 117–124 4. L. Hou, S. Zhao, X. Xiong, K. Zheng, P. Chatzimisios, M.S. Hossain, W. Xiang, Internet of things cloud: architecture and implementation. IEEE Commun. Mag. 54(12), 32–39 (2016) 5. L. Lei, Y. Kuang, N. Cheng, X. Shen, Z. Zhong, C. Lin, Delay-optimal dynamic mode selection and resource allocation in device-to-device communications—part ii: practical algorithm. IEEE Trans. Veh. Technol. 65(5), 3491–3505 (2015) 6. F. Miao, Y. Cheng, Y. He, Q. He, Y. Li, A wearable context-aware ECG monitoring system integrated with built-in kinematic sensors of the smartphone. Sensors 15(5), 11465–11484 (2015) 7. G.V.H. Prasad, L.N. Thalluri, Enhanced Performance of PCG Signal using Effective Feature Extraction Method 8. G. Sudha, Modified contourlet transform based effective image compression framework for wireless sensor networks. J. Adv. Res. Dynam. Control Syst. 10(1), 216–227 9. D. Jose, A.N. Chithara, P.N. Kumar, H. Kareemulla, Automatic detection of lung cancer nodules in computerized tomography images. Nat. Acad. Sci. Lett. 40(3), 161–166 10. J. Thamil Selvi, G. Kavitha, C.M. Sujatha, Fourth order diffusion model based edge map extraction of infrared breast images. J. Comput. Methods Sci. Eng. 19(2), 499–506

Data Science and Data Analytics

Comparison Based Analysis and Prediction for Earlier Detection of Breast Cancer Using Different Supervised ML Approach Soumen Das, Siddhartha Chatterjee, Debasree Sarkar, and Soumi Dutta

Abstract Breast cancer is one of the most leading diseases in women which leads to an increase in the mortality rate unexpectedly. The tremendous growth of breast cells forms tumors that sense as lumps. Generally, two categories are most important Benign (non-cancerous) and Malignant (cancerous). In medical science, different treatment strategies are there to identify which kind of tumor it is (Benign or Malignant). Image tests (X-rays, CT scan), ultrasound, MRI, radiation therapy, chemotherapy, immune therapy, surgery, biopsy are the most common techniques that are used in hospitals depending upon different stages of lumps which is very important but expensive in breast cancer detection. As a solution in our paper, we have discussed a comparison-based analysis of different machine learning approaches on Wisconsin Breast Cancer (WBC) dataset for measuring the variance (deviation between training accuracy and test accuracy) for early detection of breast cancer. But the question is what happens when the dataset is noisy or binary classification (Benign or Malignant) is crucial. Support vector machine (SVM) is an improvement over accuracy measure with respect to others. Binary classification is not possible always for higher-dimensional data, sometimes data that is not separable in 2D may be separable in 3D or higher dimension using transformation known as Kernel trick. In our literature, we have computed variance against nine machine learning models, three out of them [Gaussian naive Baye’s (0.014330739682853), SVMpolynomial (0.019316054532322), SVM-sigmoid (0.000344725696839)] provide positive variances, whereas others are provided negative variances [Logistic regression (−0.046554384582553), K-nearest neighbor (−0.018483863554286), decision tree (−0.083916083916084), random forest (−0.009291178305352), SVM-linear S. Das (B) · D. Sarkar Indian Institute of Technology Kharagpur, Kharagpur, WB 721302, India e-mail: [email protected] S. Chatterjee Department of Computer Science and Engineering, College of Engineering and Management Kolaghat, Purba Medinipur, WB 721171, India S. Dutta Department of Computer Application, Institute of Engineering and Management, Kolkata, WB 700091, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_21

255

256

S. Das et al.

(−0.023227945763157), SVM-RBF (−0.018533110082405)]. Positive variance is an improvement of test accuracy, whereas negative variance is a loss of test accuracy. Our goal is to consider only positive cases among which give the best result (where the variance vanishes or approximately tends to zero). Keywords Bias · Variance · Underfit · Overfit · Supervised · Training accuracy · Test accuracy

1 Introduction Early detection and diagnosis of breast cancer [1–5] gradually increase the survival of patients which reduces the mortality rate among women. But the challenging task is correctly identifying the tumor (Benign or Malignant) for the correct prediction. As per World Health Organization (WHO) in 2020, 2.3 million women are diagnosed, 685,000 deaths globally. Global Breast Cancer Initiative (GBCI) is aimed to reduce the mortality rate by 2.5% per year so that 2.5 million deaths can happen because of breast cancer globally “between 2020 and 2040.” Mammography (breast X-ray) is the first initiative to identify the type of tumor (Benign or Malignant). When the tumor is normal called Benign (non-cancerous grows very slow and does not spread), and when abnormal called Malignant (cancerous grows very rapidly and destroys neighboring tissue). Among all the strategies, biopsy (taking the sample to identify the kind of tumor either it is Benign or Malignant) is more effective in medical science. In our literature, we have discussed training accuracy, test accuracy, variance depending upon nine machine learning approaches (decision tree [6], logistic regression, KNN [7–9], random forest [1, 2, 7], Gaussian naïve Baye’s [7], SVM-linear [7, 10], SVM-RBF [11, 12], SVM-sigmoid [13], SVM-polynomial [7, 14, 15]). Mainly 16 steps we have followed to get our outcome against each of the models. Initially reading of dataset [16] and see the description, secondly recognize the empty column then drop them, for noise reduction, thirdly encode them (Malignant as = > 1, Benign as => 0), fourthly split the dataset in standard ratio (training = 75% and test = 25%). Next step is most important “model selection or classifier selection” and finally results in analysis where some parameters are essential, namely precision [exactness of classifier, ratio between actual positive (TP) over (TP + FP (miss classified as positive))], recall or sensitivity [completeness of classifier, ratio between actual positive (TP) over (TP + FN (miss classified as negative))], F1-score [harmonic mean applied on precision and recall, best score = 1.0 (best precision, best recall), lowest score = 0 (either precision or recall is 0)], accuracy [observations that are predicted correctly (TP + TN) over total number of observations (TP + FP + TN + FN)], support (count the occurrence of each class inside the dataset). If sensitivity (increases (↑)) implies TP (increases (↑)) and FN (decrease (↓)), and if specificity (↑) implies TP (↑) and FN (↓).If accuracy score (↑) model accuracy (↑) and vice versa. If sample size (↑), precision (↑) but which does not affect the accuracy, so

Comparison Based Analysis and Prediction for Earlier Detection …

257

there may be a chance of either overfit or underfit, where bias–variance balancing (trade-off) comes under consideration for good model selection. The detailed flow of action we have mentioned later.

2 Bias–Variance Trade-off Trade-off between bias–variance is an important aspect in any machine learning model, where bias is the amount of error/inaccuracy hidden in a model; on the other hand, variance is the amount of imprecision. In other words, bias is the difference value between expected value and actual value whereas variance is the deviation between training accuracy and test accuracy. A model may be “accurate (bias↓) but not precise (variance↑)” or “precise (variance ↓) but not accurate (bias↑)” or “neither precise (variance↑) nor accurate (bias↑)” or “both precise (variance↓) and accurate (bias↓).” Trade-off means induced tension due to bias and variance. If we see very closely, we can find bias is inversely proportional to variance, increase of bias will decrease the variance and vice versa. Increase of bias will definitely decrease of variance, so that the model does not learn the training data so well leads cause an underfit, inversely low bias and high variance lead to an overfit, so the model learns the training data too much. But both these cases are not desirable to make a good model. If we notice carefully at some point of time bias and variance are very close to each other as they are inversely proportional. Our aim is to maintain a balance between both bias and variance to maintain the trade-off such that we can able to design a good model. But in worst-case analysis if bias and variance both are high, then it is simply impractical. In general, for linear model (linear regression [1], logistic regression [2], linear discriminate analysis [14]) bias is high, variance is low and for nonlinear model (K-nearest neighbor, decision tree, support vector machine) bias is low, variance is high.

3 Comparison- Based Literature Review In health care and medical imaging, automation is mandatory. AI-enabled models are very much useful in current trends. For that different machine learning models, deep learning models are used in medical diagnosis. We will discuss some useful models very shortly for better understanding.

NP

Both and supervised

Both and supervised

Both and supervised

Both and supervised

Decision tree (DT)

Logistic regression

KNN

Random forest (RF) NP

NP

P

Parametric (P)/Nonparametric (NP)

Used in classification/regression/both

Model name

ER

LR

ER

ER

Eager lean (ER)/Lazy leaner (LR)

High(↑) Bias and Low(↓) Variance K = 1,error zero K = odd (2-class model) K = 5 for standard model k↑ = > (Bias↑and variance↓) k↓ = > (Bias↓and variance↑)

Linear Regression + sigmoid/logistic function K = No of NN (prior-input) If k↑ => Train score↑ and Test score↓. If k↓ = > Train score↓&Test score↑

(continued)

RF + (#DT↑) => Overfit↓ variance never be zero but minimum as no of tree increases in RF

Low (↓)Bias High (↑) Variance, Overfit problem Prune tree => variance↓

Leaf node => Class labels Internal node = > Features Branch => Decision rule

RF is an ensemble (Bagging/Boosting) of decision trees, prediction based on majority voting

Bias-variance

Description

258 S. Das et al.

P

Both and supervised

Both and supervised

Both and supervised

Both and supervised

Both and supervised

Gaussian naive Baye’s

SVM-linear

SVM-RBF

SVM-sigmoid

SVM-polynomial

P

P

P

P

Parametric (P)/Nonparametric (NP)

Used in classification/regression/both

Model name

(continued)

ER

ER

ER

ER

ER

Eager lean (ER)/Lazy leaner (LR)

Bias↑ and Variance↓ => Underfit. Use Laplace Smoothing as a solution of “Zero Frequency” Add 1 to Numerator and add k to denominator such that Posterior Probability /= 0

Bias-variance

C => Tuning parameter and γ-controls bias–variance trade-off. If C↑= > Bias↑ and Variance↓ = > underfit. = > Misclassification ↑ If C↓ = > Bias↓and F(x, xj ) = tanh(αxay + Variance↑ c) = > overfit. = > For some α > 0 and c < 0 Miss-classification.When(C = 0) = > No Misclassification, F(x, xj ) = (x. xj + 1)d get maximum margin. When Less efficient,modified γ↑ = > C become negligible version of linear kernel, and γ↓ => C, affects the D => degree model like Linear model F(x, xj ) = exp(-γ *||x-xj ||^2) 0≤γ≤1 γ = 0.1 (most preferred)

K (x, xi ) = sum(x * xi )

Suffers from “Zero Frequency,” P(xk |Ci) = 0'' problem. Perform better with less Training data if Assumption of conditional independence is true

Likelihood∗Prior Evidence

Assume-features are Independent Useful when Data Size is Large Posterior =

Description

Comparison Based Analysis and Prediction for Earlier Detection … 259

260

S. Das et al.

4 Methodology

Steps for Evaluation and Prediction Load and Read Dataset (CSV file) using panda’s data

1.

Frame through command line arguments. Detail Descriptions about Datasets for first few rows.

2.

3.

Find the Dimension of Dataset # Row count = 569, # Column count=33

Count of all columns that contains empty (Nan, Nan, Na) values.

4.

5. 6.

Drop the Empty column/Useless column.

Find the Dimension of Dataset after dropping Empty column. Count for # Malignant (M) =357 , # Benign (B) =212

7.

& visualize Bar graph 8.

Recognize the data type of Features / Columns.

9.

Encode the Categorical data into numeric value Malignant as => 1, Benign as => 0

10. Create a Scatter plot / pair plot (where one variable In the same row matched with another variables value). 11. Find out the correlation among columns & visualize. 12. Splitting the Dataset into two parts Training (Independent) = >X (75%), Target (Dependent) =>Y (25%) Can vary X & Y 13. Perform Feature Scaling (To bring all the features to the same level of magnitude 0-100 or 0-1) 14. Most Importantly Model Selection 15. Training & Accuracy measure 16. Evaluate Confusion Matrix & measure Test accuracy of each model

Comparison Based Analysis and Prediction for Earlier Detection …

5 Output Results See the Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 and12.

Fig. 1 Description of dataset

Fig. 2 Count empty value, last column is empty

261

262

S. Das et al.

Fig. 3 Count benign = 357 and malignant = 212 cells and plot bar graph

Fig. 4 Encode M => 1 and B => 0

6 Result Analysis The bias–variance trade-off is the most important benchmark for introducing the good model to overcome “underfit” (bias↑) and “overfit” (variance↑). As they are inversely proportional, the increase of one decreases the other and vice versa; hence, either too much increases or decreases both lead to either in “underfit” or in “overfit.” As a solution, the balanced trade-off between both measures is an important aspect of a good model. It is not necessary that the highest training accuracy always will give the best test accuracy. From Fig. 8f, we get decision tree that provides the highest training accuracy (approx 100.00%) but decreases test accuracy (approx 91.60%) with decreased variance (−8.3916) and Rank is 9. Similarly, it is not also obvious highest test accuracy will give the least variance, random forest provides the highest test accuracy (approx 98.60%) and second-highest training accuracy (approx

Comparison Based Analysis and Prediction for Earlier Detection …

263

Fig. 5 Independent set (X) and dependent set (Y)

Fig. 6 Pair plots diagram (orange for Malignant tumors, M => 1, and blue for Benign B => 0)

99.53%) with decreased variance (−0.9291) and rank is 2. Most importantly SVMsigmoid classifier is scored Rank 1, although it provides training accuracy (95.07%) and test accuracy (95.10%) both are not highest with increased variance (0.0345). Hence, SVM-sigmoid kernel trick is an improvement with respect to variance over others such that variance vanishes (0.0345 = approx 0). In the future, we will use an artificial neural network as an optimal prediction method for better accuracy.

264

Fig. 7 Pair-wise features correlation analysis using HeatMap

Fig. 8 Training accuracy of each model

S. Das et al.

Comparison Based Analysis and Prediction for Earlier Detection …

[0] Logistic Regression

[3] Decision Tree

[6] SVM-Polynomial

[1] k-Nearest Neighbor

[4] Random Forest

[7] SVM-Sigmoid

265

[2] Gaussian Naïve Baye’s

[5] SVM-Linear

[8] SVM-RBF

Fig. 9 Test accuracy, precision, recall, F1-score, and support values of each model

Fig. 10 Model variance measure and rank finding

266

S. Das et al.

Fig. 11 Bar graph for variance measure

Fig. 12 Line graph for variance measure

References 1. M.A. Naji, S. El Filali, K. Aarika, E.L. Habib Benlahmar, R.A. Abdelouhahid, O. Debauche, Machine learning algorithms for breast cancer prediction and diagnosis. Procedia Comput. Sci. 191, 487–492 (2021), ISSN 1877-0509. https://doi.org/10.1016/j.procs.2021.07.062 2. N. Arya, S. Saha, Multi-modal classification for human breast cancer prognosis predicttion: proposal of deep-learning based stacked ensemble model. IEEE/ACM Trans. Comput. Biol. Bioinf. https://doi.org/10.1109/TCBB.2020.3018467

Comparison Based Analysis and Prediction for Earlier Detection …

267

3. K.K. Mümine, Breast cancer prediction and detection using data mining classification algorithms: a comparative study. Tehnicki Vjesnik. 26, 149–155 (2019). https://doi.org/10.17559/ TV-20180417102943 4. W. Yue, Z. Wang, H. Chen, A. Payne, X. Liu, Machine learning with applications in breast cancer diagnosis and prognosis. Designs 2, 13 (2018). https://doi.org/10.3390/designs2020013 5. P. Gupta, L. Shalini, Analysis of machine learning techniques for breast cancer prediction. Int. J. Eng. Comput. Sci. 7(05), 23891–23895 (2018). Retrieved from http://www.ijecs.in/index. php/ijecs/article/view/4071 6. Y.-Q. Liu, C. Wang, L. Zhang, in Decision Tree Based Predictive Models for Breast Cancer Survivability on Imbalanced Data. Proceedings of International Conference on Bioinformatics and Biomedical Engineering (2009). https://doi.org/10.1109/ICBBE.2009.5162571 7. Maheshwar, G. Kumar, in Breast Cancer Detection Using Decision Tree, Naïve Bayes, KNN and SVM Classifiers: A Comparative Study. 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT) (2019), pp. 683–686. 10.1109 /ICSSI T46314.2019.8987778 8. K. Odajima, A.P. Pawlovsky, in A Detailed Description of the Use of the kNN Method for Breast Cancer Diagnosis. 2014 7th International Conference on Biomedical Engineering and Informatics (BMEI), pp. 688–692 (2014) 9. J.R. Marsilin, G. Wiselin Jiji, An efficient CBIR approach for diagnosing the stages of breast cancer using KNN classifier. Bonfring Int. J. Adv. Image Process. 2(1), 1–5 (2012) 10. M. Islam, H. Iqbal, M. Haque, M.K. Hasan, Prediction of Breast Cancer Using Support Vector Machine and K-Nearest Neighbors (2017). https://doi.org/10.1109/R10-HTC.2017.8288944 11. V.K. Dubey, A.K. Saxena, in Hybrid Classification Model of Correlation-Based Feature Selection and Support Vector Machine. Proceedings in 2016 IEEE International Conference on Current Trends in Advanced Computing (ICCTAC), IEEE (2016) 12. M. Hussain et al., in A Comparison of SVM Kernel Functions for Breast Cancer Detection. 2011 Eighth International Conference Computer Graphics, Imaging and Visualization (2011), pp. 145–150 13. T.S. Somasundaram, Y. Rejani, Early detection of breast cancer using SVM classifier technique. Int. J. Comput. Sci. Eng. 1, 127–130 (2009) 14. D. Sarkar, S. Das, Automated Glaucoma Detection of Medical Image Using Bio Geography Based Optimization, vol. 194. (Springer, Singapore, 2017) , pp. 381–388, ISBN 978-981-103908-9. https://doi.org/10.1007/978-981-10-3908-9_46 15. A.F. Agarap, On Breast Cancer Detection: An Application of Machine Learning Algorithms on the Wisconsin Diagnostic Dataset (2017). https://doi.org/10.1145/3184066.3184080 16. UCI Machine Learning Repository: Breast Cancer Wisconsin (Original) Data Set. https://arc hive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Original%29

Smart Crop Prediction and Assistance Archana Gupta and Ayush Gupta

Abstract Agriculture is the backbone of the economy of our country. Agriculture and its allied work is undoubtly the largest livelihood provider to the vast rural people. Modern agronomy, plant breeding, pre-analysis of weather condition and soil conditions, agrochemicals such as pesticides and fertilizers and technological developments like using IoT devices for collecting data, usage of machine learning for the prediction, etc., have sharply increased crop yields. Climate conditions play very important role for the growth of any crop. Agrochemicals can enhance the crop growth when used in proper amount and right composition. Every year there are losses in many crops cultivation due to lack of climatic condition or soil nutrition. Internet of things (IoT) is the technology which helps to communicate with things like systems, machines, or static objects around in various ways. IoT technology can be used to interact with real-time facts and figures. Machine learning (ML) is an application which provides the capability to learn automatically to the system. It is a part of artificial intelligence (AI). ML helps in improvement from experience without being explicitly programmed/ modified. Machine learning in agriculture can be used to improve the productivity and quality of the crops for the betterment of the society. Keywords Arduino · IOT · Sensors · Machine learning · GSM

1 Introduction In present era, with the advancement of new technologies in the technological world, there is a need to bring advancement in the agricultural world. Various researchers have already been proposed many systems based on IoT or/and machine learning A. Gupta (B) KJSCE, SVU, Mumbai 400077, India e-mail: [email protected] A. Gupta Department of CSE, BITS, Pilani 333031, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_22

269

270

A. Gupta and A. Gupta

to improve and increase the crop yielding. IoT device/circuit is used to connect the ground module which includes the sensors. Data from the sensors can be collected and stored on the cloud for further processing. There are varieties of applications of IoT which includes reading QR code printed on book, TV/light/fan control, A/C control, traffic control, agriculture and many more. IoT has diverse application domain from health industry, energy sector, telecom industry, transportation field, production area, designing sector, etc. Connectivity, remote data management and security are the three important aspects that need to be consider while using IoT-based system. ML algorithms can be very efficiently used for the future prediction. Machine learningbased real-time analysis is performed to predict the future condition of the crops based on the current observed statistics. To improve/increase the crop productivity efficiently, it is required to monitor the environmental conditions in the field and in its surroundings. In agriculture-based applications, soil characteristics, weather conditions, moisture and temperature are some of the important parameters that have to be properly analysed/monitored to enhance the yield. Internet of things (IoT) is being used in several real-time applications. The introduction of IOT along with the sensor network in agriculture compliments the traditional way of farming which may results in good and healthy crop yielding. IoT-based devices may assist farmers in online crop monitoring using to stay connected to his field from anywhere and anytime. Various sensors are used to monitor and collect information about the field conditions. That collective information about the farm condition can be delivered to the farmer through GSM technology.

2 Literature Survey IOT IN AGRICULTURE With the decrease in cost of IoT devices, it has started to become increasingly practical to use such devices to aid farmers. Various approaches [1, 2] apply these technologies to monitor crop health and aid the farmer in decision making. Approaches like [2] use IoT devices to monitor the immediate environment for crops. Naveenbalaji et al. [2] use only temperature, soil moisture and humidity as indicators of the environment. These metrics may be stored for future analysis or monitored in real time. However, no analysis is proposed in this approach to predict crop health. ML IN AGRICULTURE There have been some approaches to improve crop yield through the use of IOT and ML [1]. The approach mentioned in [1] uses NPK and moisture sensors to measure and monitor various factors in the immediate environment of the crops. They also train a logistic regression model to predict the condition of the crops. Automated corrective action may also be taken in case some factors deviate too much from the ideal. Such systems are used only to monitor the conditions.

Smart Crop Prediction and Assistance

271

Some approaches have also used the power of machine learning to identify the diseases which can occur in crops and suggest the best pesticides [4], which used a supervised machine learning algorithm C4.5 [5]. SENSOR DATA ANALYSIS In some approaches, IoT devices are used to measure various environment metrics, and these are stored for future analysis. While some papers do not elaborate on the further analysis done [1], there have been certain attempts to analyse the correlation between the sensor data and the actual harvest [3] to determine the optimum level of various environment metrics for specific plants. When these optimum metrics are known, approaches like [2] can monitor the crop environment and take corrective action if required. CROP PREDICTION There have been some efforts towards the task of predicting which crops to plant in the given conditions [6]. Using the readings from NPK, humidity and pH sensors as inputs, [6] used algorithms like SVMs and logistic regression to try to predict which crops to plant. However, they did not consider the long term climatic conditions of the region.

3 Proposed System The proposed system will predict the suitable crop for the field based on (Fig. 1): 1. 2. 3. 4.

Soil nutrition (nitrogen, phosphorus and potassium) Rainfall prediction Temperature prediction Moisture content of soil.

Many researchers [1–6, 9] proposed many systems/methodologies for the suitable crop predictions based on real-time data. Radhika and Narendiran [6] have presented the system using ML algorithms. In this proposed system, sensors (DHT, FC and NPK) are used for the collection of real-time data. In the present scenario, it is not at all easy to predict the atmospheric conditions based on the historical data. The seasonal atmospheric conditions are varying every year due to various reasons like global warming, natural calamities, etc. Thus in this proposed system, time series analysis is used to predict the temperature and humidity of the atmosphere of the field more accurately. The following three outputs are expected from the proposed system: 1. Suitable crops prediction for the field 2. Fertilizers recommended for the specific crop 3. SMS alert to the farmer for irrigation when soil moisture is not adequate.

272

A. Gupta and A. Gupta

Fig. 1 Proposed system

Following methodologies are used in the proposed system for the crop prediction [7–10]: 1. ML algorithms for time series analysis of the rainfall, temperature and humidity of the historical data for the given specified period. 2. ML algorithms for time series analysis of the rainfall, temperature and humidity of the real-time collected data for the same specified period. 3. ML/DL algorithms for the prediction of the rainfall, temperature and humidity of the given region. 4. Collect data w.r.t. soil nutrition components like nitrogen, potassium and phosphorus from the sensor. 5. Dataset w.r.t. various Indian crops along with required climatic conditions and soil nutrient are used. 6. Classifier is proposed to predict the crop based on rainfall, temperature, humidity and soil constituents with some allowable deviation in the values.

Smart Crop Prediction and Assistance

273

7. It is very much possible that soil composition is not completely accurate for the predicted crop. In this system, a module is added for the fertilizer recommendation based on the predicted crop. Soil nutrient recommendation module will be implemented using data mining techniques. 8. Another dataset which consists of Indian crop name along with its soil nutrient requirement will be used for implementing soil nutrient recommendation module. 9. Finally, a module is incorporated to send SMS alert to the farmer whenever soil moisture goes below certain limit, i.e. whenever soil gets dry. This module will facilitate the farmer by giving the alert; thus, farmer need not to keep check on the field all the time.

4 Hardware and Software Used This section describes the set of hardware components with its specifications and software requirement for setting up a system for crop prediction and provides support. (A) Hardware components (a) Sensors: (i) DHT-11: DHT-11 sensor is available to read the temperature and humidity of the field. This sensor provides digital data as output. (ii) FC-28: FC–28 sensor is used to detect the moisture content in the soil. It consists of two electrodes which are placed inside the soil. This sensor provides both types of output, analog and digital. Analog-todigital converter is used if analog output is used, which then converts the signal into a form that is readable and can be processed and can be utilized as input to the ML model. (iii) NPK: NPK sensors are the accurate and widely used sensor to detect the nitrogen, phosphorous and potassium content in soil. The soil NPK sensor is having high precision, low cost, quick responsive and portable sensor that works with Modbus RS485. (b) Arduino Board: Arduino is a microcontroller which is a very popular open-source electronics platform. One can easily use hardware and software on Arduino. It can take inputs from various sources like sensors; data input from cloud, etc., process it and generates an output in variety of ways like initiate a motor, light a LED and many more. (c) Breadboard Breadboard is used to form wired connections among various electronic components and in between network of electronic components and microcontroller like Arduino/Raspberry Pi, etc. Connections are not soldiered

274

A. Gupta and A. Gupta

permanently in breadboard and cane be change as per requirement. It is a rectangular board having holes for vertical and horizontal connections. (d) Connecting wires Connecting wires are required to make connection among various electronic devices, sensors, microcontroller and breadboard. (e) Power supply DC 9 V power supply is needed to work on the proposed system. (B) Software: (a) Python: Python 3.9 will be used to implement the proposed system using machine learning models. (b) TensorFlow: An open-source machine learning library developed by the Google Brain team very efficiently used to implement predictive analysis models. (C) Cloud Infrastructure An Amazon Elastic Compute Cloud (Amazon EC2) services can be used to for storing the data, processing the data and use the data. It is a Web-based service available for secure, resizable computing capacity on the cloud.

5 Circuit Diagram The circuit diagram of proposed model is represented in Fig. 2.

Fig. 2 Circuit diagram

Smart Crop Prediction and Assistance

275

6 Working of the Proposed System: This paper presents a system to predict suitable crop on a field based on various significant features of field and atmospheric conditions. Along with prediction, the proposed system also has fertilizer recommendation system to enhance the crop production. And there is one feature incorporated to facilitate farmer by informing about the need to irrigate the crop. Working of the proposed system has the following module: Module 1: Prediction of rainfall, temperature and humidity based on the historical data as well as real-time data of 15 days using time series analysis and ML framework. Module 2: Prediction of suitable crop for the selected field based on the soil nutrient composition and predicted rainfall, temperature and humidity. Module 3: Recommendation of the fertilizer composition based on the current soil nutrient composition and the suggested crop to enhance the crop growth. Module 4: Send message to the farmer whenever there is need of water supply in the field based on the field condition measured using sensor. This module can be implemented in Python using REST API.

7 Conclusion Agriculture sector is very sensitive to natural factors that cannot be easily sensed by the lay man. Thus, this proposed system is an attempt to help in increase in the crop production by making correct decision with respect to selection of crop, fertilizer used and with proper irrigation of the crop. Though many IoT and ML-based systems have been proposed by many researchers, in this system, a systematic methodology is proposed for the prediction of the climatic condition because climatic condition of a specific duration keeps on changing year by year. Thus, the proper study of change in climatic condition based on the pattern followed by historical data along with soil nutrients can predict efficiently the suitable crop. This proposed system can be used by a lay man without having any technical insight of the system. A simple interface can be provided to facilitate the user to make use of it. Conflict of Interest: The author(s) declare that there is no conflict of interest regarding the publication of this manuscript.

276

A. Gupta and A. Gupta

References 1. R. Varghese, S. Sharma, in Affordable Smart Farming Using IoT and Machine Learning. 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS) (2018), pp. 645–650. https://doi.org/10.1109/ICCONS.2018.8663044 2. G. Naveenbalaji, V. Nandhini, S. Mithra, N. Priya, R. Naveena, IOT based smart crop monitoring in farm land. Imperial J. Interdisc. Res. (IJIR) 4 (2018) 3. M. Lee, J. Hwang, H. Yoe, in Agricultural Production System Based on IoT. 2013 IEEE 16th International Conference on Computational Science and Engineering (2013), pp. 833–837. https://doi.org/10.1109/CSE.2013.126 4. T. Satish, T. Bhavani, S. Begum, Agriculture Productivity Enhancement System using IOT (2017) 5. J. Ross Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993) 6. A. Radhika, A. Narendiran, Kind of crops and small plants prediction using IoT with machine learning. Int. J. Comput. Math. Sci. 93–97 (2018) 7. W Gay, DHT11 Sensor (2014). https://doi.org/10.1007/978-1-4842-0769-7_1 8. www.tensorflow.org 9. A. Gupta, D. Nagda, P. Nikhare, A. Sandbhor, Smart crop prediction using IoT and machine learning. Int. J. Eng. Res. Technol. (IJERT) NTASU 2020 09(03) (2021) 10. https://mechatrofice.com/arduino/automatic-irrigation-system

An Exploration of Machine Learning and Deep Learning-Based Diabetes Prediction Techniques Atiqul Islam Chowdhury and Khondaker A. Mamun

Abstract Diabetes is now one of the world’s leading chronic diseases, affecting the middle-aged and elderly in most cases. This disease will gradually transform a person into death. There is an imbalance in blood glucose with the consequence of this disease that prompts the production of lower insulin. Medical science for the treatment of this disease is now advancing steadily. In addition to this, research focused on artificial intelligence (AI) is now advancing to define the stage of diabetes so that steps can be taken by everyone. A state-of-the-art analysis of various techniques for predicting diabetes is seen in this paper. For the last decade, several techniques based on machine learning (ML) and deep learning (DL) have been focusing on diabetes prediction. This research shows a summary of the published literature on the prediction of diabetes in the last six years. A recommendation system for observing the health of a patient through a web portal is proposed at the end of this article. Keywords Diabetes · Machine learning · Deep learning · Applications · Prediction · Prescriptive analysis

1 Introduction A healthcare system is one kind of process by which the health care is properly organized and delivered to the huge population in a better way. Healthcare system basically focuses on public health informatics. If any preventive measures are taken from analyzing the data of diabetes and hypertension, people can live a better and happy life. The purpose of a healthcare system is to ennoble the health of the population in a most effective way with the help of available resources and competing needs. There are many more diseases in the whole world. But nowadays, most people A. I. Chowdhury (B) · K. A. Mamun United International University, Dhaka, Bangladesh e-mail: [email protected] K. A. Mamun e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_23

277

278

A. I. Chowdhury and K. A. Mamun

suffer from diabetes which is very dangerous. There is no permanent solution for this disease, but a good self-controlling can help a person to reduce the rate of diabetes. Healthcare information systems is now developing gradually for research and analysis sector in order to assist in making medical decisions. If a data-driven decision is possible in the system, then many lives will be saved by this system. There are so many destructive diseases in the world like cancer and diabetes which cause rapid death. But those are curable if any diagnosis system is introduced into the human body at the first stage [1]. Being one of the highly populated countries, several people are suffering due to obesity, cardiovascular diseases, diabetes, and many other noncommunicable diseases [2]. And nowadays, diabetes is more dangerous because of its availability to many people. Diabetes mellitus which is known as diabetes is a metabolic disease that is the reason for high blood sugar. The vast majority of cases of diabetes fall into two broad categories Type 1 diabetes mellitus (T1DM) [3] and Type 2 diabetes mellitus (T2DM) [4]. High blood sugar from diabetes can damage the nerves, kidneys, and other organs of any person if it is untreated. So this disease is harmful to many people in many ways. Having too much glucose in your blood can create health problems, and it will be harmful day by day. More than 25 papers of ML/DL techniques in diabetes detection are analyzed here. Hopefully, this research will be helpful for the researchers to get a clear overview of different techniques on diabetes. The next section is about the review criteria behind this research. Section 3 will describe the ML-based techniques on diabetes prediction. Section 4 is about DL-based techniques analysis. After these, recommendation part will be discussed in Sect. 5. And the last section is about the conclusion of this paper.

2 Methodology This paper represents a state-of-the-art review of different diabetes prediction techniques. In this review, a literature portion is collected, screened, selected, reviewed, and assessed with a pre-specified objective for collection unbiased evidence and reaching an impartial conclusion. In this review, our target is to analyze research papers of ML and DL techniques on diabetes prediction. We also focused on the application-based overview for the prediction system. Behind this research, we had to analyze some questionnaires on which previous analysis issues and their limitations were found. The questions are given below: • What types of techniques are previously used for the detection of diabetes? • What are the age ratio of the generations affected by diabetes? • And what are the current methods or techniques used to detect diabetes? Before going through the paper, the analysis followed some criteria like inclusion and exclusion criteria so that it would be better to get the appropriate idea on this research. The inclusion criteria are defined as follows:

An Exploration of Machine Learning and Deep Learning …

279

• Researches must be published on diabetes prediction from 2015 to 2020. • Studies must include ML- and DL-based prediction. • Studies must have experimental findings in predicting diabetes data. Besides this inclusion criteria, there are some exclusion criteria which are followed for this review. These are: • Any other review/survey paper on diabetes prediction is excluded from this review. • Researches published in any language other than English are excluded. • Articles of predatory journals are excluded from this review. After the analysis of these criteria, there are many journals and conferences articles were found.By exploring the articles from these different databases, it was possible to categorize the diabetes prediction techniques into the following dimensions: • • • • •

Detection of Type 1 and Type 2 diabetes both separately and combined. Risk prediction for diabetes disease. Development of ML and DL model in predicting diabetes. Development of mobile and web-based applications regarding this area. Early detection approach by optimal feature selection.

The total analysis based on different criteria is described in this section, and the next sections will cover a brief analysis of different techniques and approaches.

3 Review on ML-Based Research Diabetes can be happened due to the consumption of highly processed food, bad consumption habits, etc. Machine learning has an immense effect on improving diabetes care in the recent era. There were various computerized information systems that were outlined utilizing diverse classifiers method for diagnosing diabetes. And there are a lot of researches that were happened in the detection of diabetes using machine learning and deep learning techniques. ML was employed based on steps of feature extraction, feature selection, and classification for the diabetes prediction process [5]. Popular ML algorithms are logistic regression (LR), decision tree (DT), support vector machine (SVM), Naive Bayes (NB), k-nearest neighbor (kNN), random forest (RF), and XGBoost which are used mainly in this diabetes prediction. Among these, NB and RF algorithms show good result according to analysis. Rather than these techniques, some combined algorithms were proposed also which are sometimes good in prediction. If we see Fig. 1, it will be clear which ML algorithms are mostly used in the prediction system. And, Table 1 shows the detailed information of the ML-based researches.

280

A. I. Chowdhury and K. A. Mamun

Fig. 1 Analysis of ML techniques

4 Review on DL-Based Works Different computer-based approaches are now popular in the healthcare system as well as diabetes prediction system. Deep learning approaches are now popular in this area as the medical system is now vast and there are lots of data. Deep neural network (DNN), recurrent deep neural network (RNN), deep belief network (DBN), convolutional neural network (CNN), multilayer perceptron neural network (MLP), artificial neural network (ANN), long short-term memory (LSTM), etc., are the most popular methods nowadays in the diabetes detection sector. These algorithms are much more efficient regarding many detection processes. The accuracy of ML is not sufficient in many cases, so the DL methods play a good role in these issues. Figure 2 shows the analysis of DL techniques used in the prediction system. And, Table 2 shows the detailed information of the DL-based researches.

5 Recommendation From the review, it is clear that there was a lot of work done on diabetes prediction. Some web and mobile applications had also been developed regarding this. But there is no smooth recommendation system till now. So, in this research, authors would like to suggest the development of a diabetes prediction system along with a prescribed system. That means a combination of predictive and prescriptive analysis can make the future system more valuable. This is also sometimes called a recommendation system regarding healthcare issues. The recommendation system is needed for health care because any patient has a right to know his/her health situation, and then, how he/she can overcome the issue, control the issue, is an important thing to know. If any preventive measures are taken from analyzing the data of diabetes and hypertension, people can live a better and happy life.

An Exploration of Machine Learning and Deep Learning …

281

Table 1 Review based on ML algorithms Refs.

Year

Dataset

Techniques

Result

Citation

[6]

2018

Pima Indians Diabetes Database

DT, SVM, NB

Accuracy- NB: 76.30%, SVM: 65.1%, DT: 73.82%

291

[7]

2019

Pima Indians Diabetes Database

DT, NB, RF

Accuracy- DT: 88%, NB: 91%, RF: 94%

54

[8]

2018

Pima Indians Diabetes Database

DT, NB, RF, SVM, LR, ANN

Accuracy:- NB: 23 74%, RF: 71%, LR: 70%, ANN: 68%, SVM: 73%, DT: 71%

[9]

2016

Global Dataset (Combination of all available datasets)

SVM

Accuracy: 72.93%

29

[10]

2016

Pima Indians Diabetes Database

Probabilistic neural network (PNN)

Training Acc: 89.56%, Testing Acc: 81.49%

64

[11]

2017

Pima Indians Diabetes Database

Two-class neural network

Accuracy: 83.3%

11

[12]

2017

Not mentioned

NB

Accuracy: 99.51%

88

[13]

2020

Not mentioned

Noise reduction-based technique using K-means clustering

Accuracy: 97.53%

10

[14]

2019

Dataset of Khulna Diabetes Center, Bangladesh

Two ensemble ML algorithms: bagging and decorate

Bagging: 4 95.59%, Decorate: 98.53%

[15]

2015

Pima Indians Diabetes Database

Back propagation Accuracy: 91% neural network and LevenbergMarquardt optimizer

21

[16]

2017

NHANES-0506, NHANES-0708 and NHANES-0910

Perception, ensemble perception

23

[17]

2019

Pima Indians Diabetes Database

SVM, NB, KNN, Accuracy: SVM: DT, RF 77.73%, RF: 75.39%, NB: 73.48%, DT: 73.18%, KNN: 63%

Accuracy: Perception: 72%, Ensemble perception: 75%

107

282

A. I. Chowdhury and K. A. Mamun

Fig. 2 Analysis of DL techniques Table 2 Review based on DL algorithms Dataset Techniques Refs. Year [18]

2018

Pima Indians Diabetes Database

RNN

[19]

2019

Pima Indian Diabetes Database

DNN

[20] [21]

2020 2017

Not mentioned Glucose monitoring (CGM) signals

[22]

2020

[23]

2018

Pima Indian Diabetes Database Electrocardiograms (ECG)—Private

DBN CNN, MLP, continuous logistic regression (CLR) ANN

[24]

2019

Electronic health records

[25] [26]

2019 2016

UCI Diabetes Data Not mentioned

CNN, CNN-LSTM

Result

Citation

Accuracy: Type 1 diabetes: 78%, Type 2 diabetes dataset: 81% Accuracy: fivefold: 98.04%, tenfold: 97.27% Accuracy: 81.20% Accuracy: CNN: 77.5%, MLP: 72.5%, CLR: 65.2% Accuracy: 85.09%

40

Accuracy: CNN: 90.9%, CNN-LSTM: 95.1% Accuracy: 84.28%

Synthetic minority oversampling technique (SMOTE) DNN Accuracy: 86.26% ANN Accuracy: 81%

54

10 23

18 84

43

63 31

Analyzing the available data to find out which practices are most successful helps in reducing costs and improve the health of the community by preserving the flow of improvements in health care toward outcomes and value-based payment measures. So a prescribed/recommendation system is needed to improve the condition of health of any patient. In the recommendation system, the improvement of a patient’s health is mainly observed. The recommendation system has three dimensions to measure, which are:

An Exploration of Machine Learning and Deep Learning …

283

• Improving the patient experience of care. • Improving the health of populations. • Reducing the per capital cost of health care. The main theme behind this research is any predictive system can be built up using a hybrid ML model. If we monitor the health system of a person, then we can give better optimization result. And for a non-diabetic person, the recommendation system is also needed to prevent the diabetes problem. Basically, the recommendation system is like an online prescribed format by which any diabetic or non-diabetic patient can get the update of health condition and get the idea what to do in the future to control diabetes or prevent it.

6 Conclusion With the assistance of technology, the healthcare system is now evolving more rapidly. And there are many serious diseases like cancer, diabetes, stroke, etc., nowadays. But the majority of people are diabetes sufferers. Not all the diseases need to be tracked all the time, but diabetes is a condition that needs to be controlled because it is a disease of the silent killer type. Nowadays, technology provides a better solution in the health care and diabetes management system is now focused much more. If the parameter of diabetes is monitored, then it will reduce the risk of other correlated diseases. In previous researches, prediction processes were done using a different classification algorithm. But there were sometimes limitations of data, sometimes algorithm did not give the best outcome for the lacking of data. So a complete solution to diabetes prediction, its issue and prescribed analysis will come up for the betterment of any person.

References 1. M. Al Helal, A.I. Chowdhury, A. Islam, E. Ahmed, Md.S. Mahmud, S. Hossain. An optimization approach to improve classification performance in cancer and diabetes prediction, in 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE) (IEEE, 2019), pp. 1–5 2. M. Faruque, M.B. Mia, M.H. Chowdhury, F. Sarker, K.A. Mamun, Feasibility of digital health services for educating the community people regarding lifestyle modification combating noncommunicable diseases, in International Conference on Human-Computer Interaction (Springer, 2019), pp. 333–345 3. E. Georga, V. Protopappas, A. Guillen, G. Fico, D. Ardigo, M.T. Arredondo, T.P. Exarchos, D. Polyzos, D.I. Fotiadis, Data mining for blood glucose prediction and knowledge discovery in diabetic patients: the metabo diabetes modeling and management system, in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2009), pp. 5633–5636

284

A. I. Chowdhury and K. A. Mamun

4. B.M. Patil, R.C. Joshi, D. Toshniwal, Association rule for classification of type-2 diabetic patients, in 2010 Second International Conference on Machine Learning and Computing (IEEE, 2010), pp. 330–334 5. G. Swapna, R. Vinayakumar, K.P. Soman, Diabetes detection using deep learning algorithms. ICT Express 4(4), 243–246 (2018) 6. D. Sisodia, D.S. Sisodia, Prediction of diabetes using classification algorithms. Procedia Comput. Sci. 132, 1578–1585 (2018) 7. N. Yuvaraj, K.R. SriPreethaa, Diabetes prediction in healthcare systems using machine learning algorithms on hadoop cluster. Cluster Comput. 22(1), 1–9 (2019) 8. S.M. Hasan Mahmud, Md.A. Hossin, Md.R. Ahmed, S.R. Haider Noori, Md.N.I. Sarkar, Machine learning based unified framework for diabetes prediction, in Proceedings of the 2018 International Conference on Big Data Engineering and Technology (2018), pp. 46–50 9. A. Negi, V. Jaiswal, A first attempt to develop a diabetes prediction method based on different global datasets, in 2016 Fourth International Conference on Parallel, Distributed and Grid Computing (PDGC). (IEEE, 2016), pp. 237–241 10. Z.. Soltani, A.. Jafarian, A new artificial neural networks approach for diagnosing diabetes disease type ii. Int. J. Adv. Comput. Sci. Appl. 7, 89–94 (2016) 11. S. Rakshit, S. Manna, S. Biswas, R. Kundu, P. Gupta, S. Maitra, S. Barman, Prediction of diabetes type-ii using a two-class neural network. In International Conference on Computational Intelligence, Communications, and Business Analytics (Springer, 2017), pp. 65–71 12. B. Ali´c, L. Gurbeta, A. Badnjevi´c, Machine learning techniques for classification of diabetes and cardiovascular diseases, in 2017 6th Mediterranean Conference on Embedded Computing (MECO). (IEEE, 2017), pp. 1–4 13. Saleh Albahli, Type 2 machine learning: An effective hybrid prediction model for early type 2 diabetes detection. J. Med. Imaging Health Inf. 10(5), 1069–1075 (2020) 14. Md.T. Islam, M. Raihan, S. Raihan, I. Akash, F. Farzana, N. Aktar, Diabetes mellitus prediction using ensemble machine learning techniques, in International Conference on Computational Intelligence, Security and Internet of Things (Springer, 2019), pp. 453–467 15. M. Durairaj, G. Kalaiselvi, Prediction of diabetes using back propagation algorithm. Int. J. Emerging Technol. Innov. Eng. 1(8) (2015) 16. R. Mirshahvalad, N.A. Zanjani, Diabetes prediction using ensemble perceptron algorithm, in 2017 9th International Conference on Computational Intelligence and Communication Networks (CICN) (IEEE, 2017), pp. 190–194 17. N. Sneha, T. Gangil, Analysis of diabetes mellitus for early prediction using optimal features selection. J. Big Data 6(1), 13 (2019) 18. A. Ashiquzzaman, A.K. Tushar, Md.R. Islam, D. Shon, K. Im, J.-H. Park, D.-S. Lim, J. Kim, Reduction of overfitting in diabetes prediction using deep learning neural network, in IT Convergence and Security 2017 (Springer, 2018), pp. 35–43 19. S.I. Ayon, Md. Islam et al., Diabetes prediction: A deep learning approach. Int. J. Inf. Eng. Electron. Bus. 11(2) (2019) 20. K. Vidhya, R. Shanmugalakshmi, Deep learning based big medical data analytic model for diabetes complication prediction. J. Ambient Intell. Hum. Comput. 1–12 (2020) 21. A. Mohebbi, T.B. Aradóttir, A.R. Johansen, H. Bengtsson, M. Fraccaro, M. Mørup, A deep learning approach to adherence detection for type 2 diabetics, in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, 2017), pp. 2896–2899 22. N. Pradhan, G. Rani, V.S. Dhaka, R.C. Poonia, Diabetes prediction using artificial neural network, in Deep Learning Techniques for Biomedical and Health Informatics (Elsevier, 2020), pp. 327–339 23. G. Swapna, S. Kp, R. Vinayakumar, Automated detection of diabetes using CNN and CNNLSTM network and heart rate signals. Procedia Comput. Sci. 132, 1253–1262 (2018) 24. B.P. Nguyen, H.N. Pham, H. Tran, N. Nghiem, Q.H. Nguyen, T.T.T. Do, C.T. Tran, C.R. Simpson, Predicting the onset of type 2 diabetes using wide and deep learning with electronic health records. Comput. Methods Prog. Biomed. 182, 105055 (2019)

An Exploration of Machine Learning and Deep Learning …

285

25. K. Kannadasan, D.R. Edla, V. Kuppili, Type 2 diabetes data classification using stacked autoencoders in deep neural networks. Clin. Epidemiol. Glo. Health 7(4), 530–535 (2019) 26. S. Joshi, M. Borse, Detection and prediction of diabetes mellitus using back-propagation neural network, in 2016 International Conference on Micro-Electronics and Telecommunication Engineering (ICMETE) (IEEE, 2016), pp. 110–113

Machine Learning-Based Smart Tourist Support System (Smart Guide) Prasanna Vikasitha Rathnasekara, Anuradha Sadesh Herath, Avishka Heshan Abeyrathne, Prasadini Ruwani Gunathilake, and Samantha Thelijjagoda

Abstract The tourism industry has become one of the fastest growing industries in the world’s revenue generation. In order to conquer global tourism, Sri Lanka needs to adapt to modern trends to provide a better service to travelers. The ultimate goal of our project is to support the betterment of the tourism industry by suggesting best locations for users using preferences such as age group, gender, religion, country, traveled month, travel group type(s), food preferences, suggesting best routes and transportation methods for travelers, planning trips according to traveler’s preferences, customizing and managing time throughout the journey. Moreover, the proposed system provides the opportunity for hoteliers and other related service providers to customize their business to suit the needs of tourists. We obtained past visited dataset from Sri Lankan Tourism Board and also we collected data from tourists using a google survey via a tourism agency. In this study, we propose a classification of AI models to predict the best locations and best transportation methods. The classification model selected for this study is random forest, which has exhibited an accuracy level of 90% in location prediction and 86% in transportation method prediction. The other one is exponential smoothing time series models, which has shown an accuracy of 84.74% level. To plan the trip optimistically, an algorithm that generates trip plans according to user preferences is implemented. Keywords Sri Lankan tourism · Location-based tourist supporting · Tourist service providers · Multi-label classification model · Univariant data time series model · Multivariant data time series model

P. V. Rathnasekara · A. S. Herath · A. H. Abeyrathne · P. R. Gunathilake (B) · S. Thelijjagoda Sri Lanka Institute of Information Technology, Malabe, Sri Lanka e-mail: [email protected] S. Thelijjagoda e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_24

287

288

P. V. Rathnasekara et al.

1 Introduction In the past few decades, tourism has become one of the fastest-growing industries in the revenue generation of our country. In 1960s, Sri Lanka entered the international tourism area and since then government involvement has been the key factor for the development of tourism in Sri Lanka [1]. Over the years tourism industry in Sri Lanka has developed significantly. Today, tourism has become the sixth foreign exchange earner in the Sri Lankan economy. Therefore, the tourism industry, which can be also called the national economy, can make a great impact on Sri Lanka’s economy [2]. It is known as the national economy because it contributes directly to foreign exchanges earnings. For centuries, Sri Lanka has been a popular place of attraction for foreign travelers [3]. This paper focuses on tourist’s individual preferences for trip planning and guiding. Implementing such a system the tourists can plan their trips efficiently and optimistically. Our proposed system accomplishes the following: Suggest best recommended location to the travelers, provide information about location, suggest alternative locations around while the trip, and other travelers are allowed to become aware of those locations. Suggest best routes and transportation methods, arrange transportations as per traveler’s request, let travelers suggest alternative routes to the system, and other travelers get informed about those routes. Trip planning according to traveler’s preferred location and preferred routes, they are given opportunity to customize and manage the time trout the journey. The proposed system gives information about the travelers for the supportive service providers such as hoteliers and other related service providers. Then, they can update with timely trends and latest tourism-related information. This paper discusses how the proposed system use to machine learning classification methods and time series models to achieve all of the above.

2 Literature Study There are several mobile applications and web platforms; there can be found which provide guidance and support for independent travelers, when it comes to the Sri Lankan context but they do not fulfill some important aspects. 1. ‘GuideMe’ Research Paper It has implemented a system using social interaction. When a person visits a location, a tourist guide application recommends useful information according to current location, preferences, and past visits. Afterward, the tourist guide allows the users to provide feedback about each visit. In this research, the author has mentioned the development and the key features of touristic locations [4]. Each user may choose places of tourist interests; receive suggestions of hidden touristic places according to the user’s recommendations, and to perform its own

Machine Learning-Based Smart Tourist Support System (Smart Guide)

289

recommendations. The recommendations were carried out using the Mahout library. In contrast with previous recommender-based tourist guides, the key features of GuideMe are its integration with social networks and also the unique options set in the application. 2. Tripadvisor Tripadvisor is the world’s largest travel guidance platform; it provides its service, from planning to booking to taking a trip. Travelers across the globe use the Tripadvisor site and app to discover where to stay, what to do, and where to eat, to find deals on accommodations, book experiences, reserve tables at delicious restaurants, and discover great places nearby based on guidance from those who have been there before [5]. However, this platform does not provide a customized service to the users by considering the user’s preferences. The user has to go through the recommendations and reviews given by the other travelers and plan their trip. 3. Sri Lanka Travel Guide Offline Sri Lanka Travel Guide Offline is an offline travel guide, which recommends detailed articles around the country in relation to the user’s current location. Get around guides, phrase lists, warnings on how to stay safe and healthy, restaurants and hotel recommendations, local costs and taxes, road rules, and a variety of other info are carefully stored in one app [6]. However, this application provides neither customized service for the users to plan the trip nor access to the supportive services. 4. This is an application that provides information about travel hotspots in Sri Lanka, their locations on a map, and nearby accommodation service providers [7]. There are several drawbacks to this application as well. As this application is only available in the Sinhala language, it does not reach international users, and it does not provide a customized service and platform for the users to plan their trip and connect to the service providers through the application. 5. Travel Sri Lanka—Tourism Guide This application contains descriptions of travel hotspots in Sri Lanka [8]. Other than that, this application does not provide any other service to its users. According to the literature, it is proven that there is no existing system or platform in Sri Lanka that provides a customized service to independent travelers which focuses on their individual preferences as well as real-time conditions when it comes to trip planning and guiding. Implementing such a system would address the pain point of the travelers who are willing to plan their trip efficiently and optimistically. 6. Researches In recent years, many kinds of research papers on tourist support systems are published. But many kinds of research have been implemented to consider only the route times and road traffic [9]. When considering previous research studies, the scheduling method exclusively focuses on roads, road traffic, railways, and railway traffic [10]. So they can get an idea about the real-time roads and railways

290

P. V. Rathnasekara et al.

situations and give predictions. Also, they did not consider the scheduling period [11]. Some researched systems schedule trips on points of interest (POI) option and the trip plan with only contain route diagram [12] as tourist decision support. In this, the traveler can get an idea about route plans and get an idea about visiting places. Therefore, the Smart Guide mobile application is proposed as a solution for the identified problem, which comes accomplishing the following objectives. • Analyze And Suggest Location with User Preference – Recommend locations to the travelers – Provide information about locations – Let travelers suggest alternative nearby locations to the system and other travelers get informed about those locations. • Predict Preferred Transportation Methods And Suggest Best Routes – Recommend transportation methods and routes to the travelers – Transportations as per traveler’s request – Let travelers suggest alternative routes to the system and other travelers get informed about those routes. • Predict and Generate Time Schedule on Effective Time Managing with User Preferences – – – – –

According to the traveler’s preference, trips are arranged considering Best travel locations Best traveling routes and transportation methods Availability of supportive services The time schedule generate by considering all user aspects, other country and environmental conditions and travelers actions – Let user customize and manage the trip via the system. • Predict And Analyze Seasonal Tourist Arrivals And Give Suggestions Tips to Supportive Service Providers – During the trip, the traveler is given information about the supportive service providers who provide – Accommodation/resting – Transportation – Let service providers register into the system – Inform service providers when travelers require their service – Update service providers with timely trends and latest tourism-related information. With the increasing user interaction, the proposed system will expand its databases and efficiency of the services accordingly.

Machine Learning-Based Smart Tourist Support System (Smart Guide)

291

3 Methodology As discussed above, an application is proposed to solve the issues faced by the travelers coming to Sri Lanka, which are not addressed by the existing systems. Here, it has identified the following main requirements that the proposed system should address essentially. • A traveler goes on a trip mainly to experience the travel destination and the journey. Therefore, the system must be capable of identifying the best travel locations and routes according to the user preference. • The system must identify traveler’s preferred types of locations and transportation types and allow the traveler to plan the entire trip. • Provide supportive services to the travelers throughout the journey where it gives the best travel experience to the traveler. Following the above requirements, the system is developed to accomplish four main functions. A. Analyze And Suggest Location with User Preference B. Predict Preferred Transportation Methods And Suggest Best Routes C. Predict And Generate Time Schedule on Effective Time Managing with User Preferences D. Predict And Analyze Seasonal Tourist Arrivals And Give Suggestions Tips to Supportive Service Providers.

3.1 Analyze and Suggest Location with User Preference Locations or destinations take an important place in a trip. There are plenty of attractive locations in Sri Lanka that travelers tend to visit. However, it is important to sort out those numerous locations for the travelers according to their preference and assist them to select the ideal locations that they can travel within the limited time of their journey. Once the locations are sorted out clearly, the travelers can select the travel locations for their trip on their own. The proposed system does a background study about the traveler in order to identify their preferences, and then the application will select the suitable locations for the travelers according to their preferences. Here, the application does not force its users to travel in the system’s proposed locations, but it shows the suitable locations with a high priority. Initially, these data were collected from the Sri Lanka Tourism Board (SLTB). However, we were not able to collect all these facts from SLTB because they do not have all of these data. Therefore, we had to use the survey method to collect data to train and test the ML model. For that, we had to collect foreign tourist email details from the SLTB. Afterward, we sent the survey form to the email addresses of the tourists. Next, we published it in websites of traveling agencies and in social media pages and groups to gather data from tourists those who use Sri Lanka tourist (SLT)

292

P. V. Rathnasekara et al.

services. The major aim of gathering data by means of these methods is to create an accurate dataset for testing and training ML algorithms. Unwanted data was removed from the dataset using the data preprocessing technique. Because the data collection contained both numerical and categorical data, a preprocessing technique was used to normalize the dataset using the Sklearn module. The training dataset was selected at random from the complete dataset, while the test dataset was selected at random from the remaining 20%.

3.2 Predict Preferred Transportation Methods and Suggest Best Routes To move from one location to another, travelers can use numerous routes and transportation methods. Those can vary with the preference of the traveler. The proposed application sorts out the suitable routes and transportation methods for the travelers which gives them the best traveling experience. At the same time, it considers the traveler’s preferences on different transportation methods as well. Data were collected in the first stage using the survey approach, which included sending an online form to users’ email addresses obtained from the Sri Lanka Tourism Board (SLTB). To implement this, the dataset is collected that includes travelers’ details and the suitable transportation type for each of them. Using the data, classification, and clustering AI models are trained. They identify the suitable route and transportation methods to travel from one place to another, when a given traveler’s details are input to the application. Apart from this, the application allows its users to suggest new routes to the application. This causes the application to expand with the users’ inputs. As the number of users increases, the application broadens at the same time. As the AI models are developed to predict the user-preferred location types and transportation methods, the architecture of the practical implementation of the AI model in the application is described in Fig. 1.

3.3 Predict and Generate Time Schedule on Effective Time Managing with User Preferences The trip should be planned for the number of days the traveler stays within the country where he/she can spend their time optimistically during the journey. When it comes to trip planning, several factors should be considered in order to plan the trip well. The followings are the main factors that need to be considered. • Starting location of the journey • Ending location of the journey • Travel hours per day

Machine Learning-Based Smart Tourist Support System (Smart Guide)

293

Fig. 1 System architecture: application of AI models in the system

• Number of days to travel • In location time. The above inputs will be taken from the user/traveler. To figure out the time spent traveling from one location to another, the system considers the following factors. • Weather condition • Traffic • Transportation method used. Collaborating all the above information, the application comes up with the trip schedules with sort outed locations which allows the travelers to spend their time efficiently and effectively during the time they spend within the country.

3.4 Predict and Analyze Seasonal Tourist Arrivals and Give Suggestions Tips to Supportive Service Providers For the efficiency and the optimality of the trip, the traveler could be able to reach supportive services with less effort. Therefore, the proposed application directs users to supportive services through the application. To accomplish this, the application collaborates with the service providers in the tourism industry. Service providers are registered to the application and travelers can connect with the service providers through the application. To provide an efficient service to travelers, the application empowers service providers with the required information. The system forecasts the upcoming trends of the travelers’ arrivals and other several trends of the tourism industry and provides that information to the service provides where they can get ready for the upcoming trends. For this, the proposed system has developed the following time series models.

294

P. V. Rathnasekara et al.

Fig. 2 System architecture: application of time series models in the system

• Location-vise time series models • Season-vise time series models. Using those models, the system predicts the count of tourists’ arrival for the upcoming time periods, and service providers get informed with those data. As the time series models are developed to analyze the timely trends of the tourists’ arrival, the architecture of the practical implementation of the time series models in the application is described in Fig. 2. With the contribution of all these four functions, the overall solution is built which comes as the Smart Guide application.

4 Results and Discussion The proposed system has come up with a solution for identifying the user-preferred travel locations and transportation methods and suggests them to the users by predicting users’ preferences. For this, a dataset was collected from the foreigners who have traveled in Sri Lanka which consists of their details, locations they prefer to travel and transportation methods they prefer to use. Then, it trained classification models to make predictions. To train the classification models, the following features were taken as the inputs. • • • • • • • •

Traveled month Country Primary language Age Gender Religion Travel group type/s Food preference.

Machine Learning-Based Smart Tourist Support System (Smart Guide) Table 1 Accuracies of location type forecasting models and transportation methods forecasting methods

295

Model

Location type (%) Transportation method (%)

Decision tree

76

80

Random forest 90

86

83

75

Extra tree

To predict the preferred location types, the classification model is developed by taking the preferred location types as the output class. There was more than one preferred location type selected, therefore, to classify the data, it required a multilabel classification (MLC) model. To predict the preferred transportation methods, the classification model is developed by taking the preferred transportation methods as the output class. There also were more than one preferred transportation method selected, therefore, to classify the data, it also required a multi-label classification (MLC) model. To develop the classification models to predict the preferred location types and the preferred transportation methods, the experiments were done with decision tree, random forest, and extra tree classifier models. Each model exhibited the following accuracy levels in Table 1, in order to predict preferred location types and preferred transportation methods. The application assists the users to manage their trips by providing trip schedules throughout the journey. It allows the user to optimize the time they spend within the country by managing the trip in the most convenient way. As the users select their preferred locations and transportation methods on their own, the trip schedules must be customized according to the user selections. In order to create the trip schedule, it should figure out the time that might spend to travel from one location to another. For this, it mainly needs to consider the following three facts. • Weather condition • Traffic • Transportation method used. Google API provides the time that can be spent to travel from one location to another by considering the above three factors. In order to schedule the trip, an algorithm is applied in the application. The algorithm is described in Fig. 3. The inputs taken for the algorithm are number of days of travel, estimated travel hours per day, starting location, destination, and the next location the traveler likes to visit. Then, it calculates the total travel hours using the number of days travel and the estimated travel hours per day. Starting location is taken as the current location at the initial step. Using the user inputs and the data grabbed from the Google API, it calculates the travel time from the current location to the next location and the travel time from the next location to the destination. Using that information, the algorithm directs the decision. If the travel time from the next location to the destination is less than the total travel hours (D), then the following decisions are made.

296

P. V. Rathnasekara et al.

Fig. 3 Trip planning algorithm

• The selected next location is added to the trip • Total travel hours is deducted by the travel time from the current location to the next location • Selected next location is taken as the current location • The user is asked to add the next location. If the given condition is not satisfied, the selected next location cannot be added to the trip plan. The user is asked to add another location as the next location. In order to assist the service providers who, get registered to the proposed system, time series models were developed which forecast the tourist arrival to Sri Lanka for the upcoming time periods by analyzing the past tourist behaviors. The time series models were developed in the following four aspects. A. Tourist Arrival by Month Overall tourist arrival by month was studied from the beginning of the year 2000 and to the end of the year 2019. A univariant dataset was occupied for the analysis, and the developed univariant model could forecast the future arrival of tourists by month. B. Seasonal Variation of Tourist Arrival Using a univariant dataset that consists of the seasonal variation of tourist arrival from 2001 to 2019, the univariant model is developed. The developed model could forecast the variation of tourist arrival for the upcoming seasons. C. The Monthly Occupancy Rate in Tourist Hotels by Regions in Sri Lanka The monthly occupancy rate of tourist hotels in several regions in Sri Lanka was studied from the beginning of the year 2011 to the end of the year 2019. Using these multivariant data, a time series model was developed to forecast the future occupancy in tourist hotels in Sri Lanka.

Machine Learning-Based Smart Tourist Support System (Smart Guide)

297

D. Monthly Tourist Arrival by Region of Resident The number of travelers who arrived from different regions of resident in the world was studied from the beginning of the year 2018 to the end of the year 2019. Using these multivariant data, a time series model was developed to forecast the arrival of tourists from different regions for the upcoming months. With research on existing time series models available, for the univariant datasets, it could be chosen the following time series models. • Seasonal Auto-Regressive Integrated Moving Average (SARIMA) • Exponential Smoothing The above models were used to develop the time series models for the tourist arrival by month and seasonal variation of tourist arrival. The R2 value of the trained models was taken as the accuracy and the mean absolute percentage error is taken as the error of the models. a. Tourist Arrival by Month To analyze tourist arrival by month, a dataset has been used which consists of the number of tourists who arrived in Sri Lanka from the beginning of the year 2000 to the end of the year 2019. Due to the COVID 19 global pandemic, the tourist arrival to Sri Lanka in 2019 has been drastically dropped in a huge amount. The impact of this circumstance is clearly displayed in Fig. 3. This unexpected sudden change in the dataset makes a considerably high impact on the accuracy of the time series models developed. Since this drastic drop is presented right at the end of the dataset, the impact of this circumstance is basically reflected in the model evaluation. i. Tourist Arrival by Month with Exponential Smoothing The above dataset (Fig. 4) is been used to train the exponential smoothing model in order to analyze the tourist arrival by month. Data from the beginning of the year 2000 to the end of the year 2018 were used to train the model, and the data from the beginning to the end of the year 2019 were used to evaluate the model. Figure 5 displays the outcome of the model training. In Fig. 5, it can be seen that the model forecast is displayed in blue, but the respective actual values display a far difference from the forecasted values. The sudden impact of the COVID 19 global pandemic caused this unnatural behavior of the dataset, and it was unable to do a fair model evaluation with these data. In order to evaluate this model, it has ignored the data that are drastically impacted by the COVID 19 global pandemic. With the evaluation, the exponential smoothing model has represented an accuracy level of 84.74% where the error is 6.96%. ii. Tourist Arrival by Month with SARIMA To develop the SARIMA model, the same dataset was used for the exponential smoothing model. In order to train a dataset using SARIMA, it should be a stationary dataset. By studying Fig. 3, it can see that the dataset is not

298

P. V. Rathnasekara et al.

Fig. 4 Tourist arrival by month

Fig. 5 Tourist arrival by month with exponential smoothing

stationary. This is proven by the P value obtained by the dataset, which is o.8457. If the P value is closer to 1, then the dataset cannot be considered as a stationary dataset. Therefore, it has been used differentiation techniques in order to make the dataset stationary. After applying differentiation techniques, it has obtained 0.0092 as the P value which is closer to 0. The outcome of the model is displayed in Fig. 6. With the evaluation, the SARIMA model has obtained an accuracy value of 78.17% where the error is 53.81%. As the exponential smoothing model represents a higher accuracy level, it is proven that the exponential smoothing model is the most efficient method that can be used to forecast the tourist arrival by month in Sri Lanka. b. Seasonal Variation of Tourist Arrival To train the exponential smoothing model for the seasonal variation of tourist arrival, the data from the beginning of the year 2001 to the end of the year

Machine Learning-Based Smart Tourist Support System (Smart Guide)

299

Fig. 6 Tourist arrival by month with SARIMA

2018 were used. The outcome of the trained model is represented in Fig. 7. The exponential smoothing model is evaluated by the data from the beginning to the end of the year 2019. Here also, the impact of the COVID 19 global pandemic was exhibited. In order to get a fair validation, the data impacted by the pandemic was ignored. i.

Seasonal Variation of Tourist Arrival with Exponential Smoothing To train the exponential smoothing model for the seasonal variation of tourist arrival, the data from the beginning of the year 2001 to the end of the year 2018 were used. The outcome of the trained model is represented in Fig. 7.

Fig. 7 Seasonal variation of tourist arrival with exponential smoothing

300

P. V. Rathnasekara et al.

The exponential smoothing model is evaluated by the data from the beginning to the end of the year 2019. Here also, the impact of the COVID 19 global pandemic was exhibited. In order to get a fair validation, the data impacted by the pandemic was ignored. The accuracy of the model was represented as 78.9% where the error is 9.90%. ii. Seasonal Variation of Tourist Arrival with SARIMA To train the SARIMA model for the seasonal variation of tourist arrival, the data from the beginning of the year 2001 to the end of the year 2018 were used. As this dataset represents the variation of the tourist arrival, this is obviously a stationary dataset. The P value of the model was represented as 1.7336 e−07 which is almost closer to 0. The model is evaluated by the data from the beginning to the end of the year 2019. The accuracy of the model was represented as 84% where the error was 16.29%. Although the accuracy of the SARIMA model (84%) is higher than the accuracy of the exponential smoothing model (78.9%) by 5.1%, the error value of the SARIMA model (16.29%) is also higher than the error value of exponential smoothing model (9.90%) by 6.39%. Therefore, the exponential smoothing model can be selected as the most efficient model to forecast the seasonal variation of tourist arrival to Sri Lanka. For the multivariant datasets, it could apply neural basis expansion analysis time series forecasting (N-BEATS Forecasting) which represented a good accuracy in developing time series models with multivariant data. iii. The Monthly Occupancy Rate in Tourist Hotels by Regions in Sri Lanka With N-BEATS Forecasting, it exhibited an average accuracy level of 81% in order to forecast the monthly occupancy rate in tourist hotels by region in Sri Lanka. iv. Monthly Tourist Arrival by Region of Resident In order to forecast the monthly tourist arrival by region of residence, NBEATS Forecasting exhibited an average accuracy level of 83%.

5 Conclusion The Smart Guide application was developed as a solution for the newly arising requirements in the tourism industry in Sri Lanka to assist tourists with an enhanced system with advanced technologies. The developed system focuses on assisting the travelers who visit Sri Lanka to select the most suitable travel locations and transportation methods according to their preferences, plan their trips in order to spend the time they stay within the country efficiently, provide supportive services when they require and to assist service providers in the tourism industry with the latest data in order to let them make decisions efficiently. In order to predict user-preferred location types and transportation methods, multi-label classification models were used.

Machine Learning-Based Smart Tourist Support System (Smart Guide)

301

Random forest models were employed to predict preferred location types with an accuracy level of 90% and to predict preferred transportation methods with an accuracy level of 86%. In order to train the models, a dataset was used which was collected from the travelers who have traveled to Sri Lanka. For the better enhancement of the created models, it is proposed to use a dataset that includes data of travelers from an extended number of countries. For the trip planning, an algorithm was implemented to make decisions along with the data grabbed from the Google API. Time series models were developed to analyze the trends of tourists’ arrival in Sri Lanka. For the univariant data, exponential smoothing time series models were selected, which had shown 84.74% of accuracy level to analyze tourist arrival by month, 78.9% of accuracy level to analyze the seasonal variation of tourist arrival. For the multivariate data, N-BEATS forecasting was used and forecasted the monthly occupancy rate in tourist hotels by regions in Sri Lanka with an average accuracy level of 81% and the monthly tourist arrival by region of resident with the accuracy level of 83%. Due to the COVID 19 global pandemic, the global tourism industry was dropped in large numbers. This has been caused by high variations in the time series models which showed huge differences from the previous trends. Therefore, it is required to retrain the time series models with the newest data in to align the models with the newest trends. It can be concluded that the study has come up with novel solutions which fulfill its objectives successfully.

References 1. Sri Lanka tourism problems and development, UKEssays.com. [Online]. Available: https:// www.ukessays.com/essays/tourism/sri-lanka-tourism-challenges-and-opportunities-tourismessay.php (2021). Accessed 26 Feb 2021 2. Tourism service quality and tourist re-visit intention to Sri Lanka: a mediating role of brand image. [Online]. Available: https://www.researchgate.net/publication/336364658_Tourism_s ervice_quality_and_tourist_re-visit_intention_to_Sri_Lanka_a_mediating_role_of_brand_i mage. Accessed 20 July 2021 3. Tourism in Sri Lanka, En.wikipedia.org. [Online]. Available: https://en.wikipedia.org/wiki/Tou rism_in_Sri_Lanka (2021). Accessed 26 Feb 2021 4. A. Umanets, A. Ferreira, N. Leite, GuideMe—a tourist guide with a recommender system and social interaction (2021) 5. US Press Center | About Tripadvisor, MediaRoom, [Online]. Available: https://tripadvisor.med iaroom.com/us-about-us (2021). Accessed 20 July 2021 6. Play.google.com, [Online]. Available: https://play.google.com/store/apps/details?id=com.gui dewithme.sri.lanka (2021). Accessed 20 July 2021 7. Play.google.com, [Online]. Available: https://play.google.com/store/apps/details?id=net.trista nsoft.srilankasinhala (2021). Accessed 20 July 2021 8. Play.google.com, [Online]. Available: https://play.google.com/store/apps/details?id=com.liy anage.travelsl (2021). Accessed 20 July 2021 9. D.F. Vargas, D.A. Sendales, Optimal scheduling trip plan for tourist. IEEE Xplore, 07 Jan 2016. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/7727111. Accessed 20 Feb 2021

302

P. V. Rathnasekara et al.

10. O.K. Kwon, C.D. Martland, J.M. Sussman, Routing and scheduling temporal and heterogeneous freight car traffic on rail networks. ScienceDirect, 02 June 1998. [Online]. Accessed 1 2021 11. C. Zhang, H. Liang, K. Wang, Trip recommendation meets real-world constraints: POI availability, diversity, and traveling time uncertainty. DL, 09 2016. [Online]. Available: https://doi. org/10.1145/2948065. Accessed 17 Feb 2021 12. W. Souffriau, P. Vansteenwegen, in Tourist Trip Planning Functionalities: State–of–the–Art and Future (SpringerLink, 2010). [Online]. Available: https://doi.org/10.1007/978-3-642-169854_46. Accessed 21 Feb 2021

Analysis of User Inclination in Movie Posters Based on Color Bias Harshita Chadha, Deeksha Madan, Deepika Rana, and Neelam Sharma

Abstract Visual bias can be described as the phenomenon of showing a preference for a particular visual stimulant based on some inherently unique characteristics possessed by it. This predisposition is quite apparent in everyday life: different people respond differently to the same visual information. The careful utilization of this phenomenon can be extremely utilitarian. This is apparent from the recent widespread acknowledgment of visual bias, and its manipulation to bring about salutary business effects. Today, visual bias utilization can be seen at play in a number of different industries ranging from advertisements to recommendation systems of OTT giants. Despite the fact that the existence of visual bias is now widely accepted, few empirical studies have been conducted to confirm the same. The present article and the ensuing research serve to bridge that gap. In our article, we elucidate the salient points of the research. We conducted using movie posters as a means of estimating and studying the effects of visual bias. We built a Web application to survey and collect user ratings for movie posters belonging to different genres and thus having different visual effects. Based on the user’s input, a bias mapping was done, and the result of which was the genre that the user was most visually partial to. Contingent on that, neural style transformation was applied to movie posters, and the augmented results were presented to the user to rate. By comparing the initial genre-wise ratings to the transformed genre ratings, the extent of visual bias was detected and thus analyzed. Our study not only confirmed the empirical existence of such bias, but the detection methodology outlined here may serve as a visual bias manipulation tool that can utilize real-time machine learning to optimize AdSense and the likes. Keywords Visual bias · Neural style transfer · Convolution neural networks

H. Chadha · D. Madan · D. Rana · N. Sharma (B) Maharaja Agrasen Institute of Technology, New Delhi, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_25

303

304

H. Chadha et al.

1 Introduction The current study aims to analyze the extent of the effect of visual biases in user decision-making. We also outline a detection methodology that can be expanded upon to create a machine learning driven bias manipulation system. To achieve this, we utilize movie posters as visual stimulants. In our study, we used a Web site to collect relevant data from subjects. The site visitors were presented with movie posters from across different genres. To avoid errors from linguistic bias, the choice of using posters in languages unfamiliar to the subject pool was made. The languages chosen included Korean, Turkish, Telugu, and French. On visiting our data collection portal, the user was presented with five movie posters from across five different genres, namely: horror, thriller, romance, comedy, and action. They were then asked to rate these on a scale of 1–5 with the condition that no two posters had the same rating number. Based on these ratings, a bias mapping was done. This indicated the preferred movie genre for a particular user. The bias results were then used as a basis to apply the style transformation algorithm [1] on a fresh set of images from the same five genres. The users were then asked to rate these style transformed images on a scale of 1–5 again. The difference in the cross-genre rating before and after the neural style transfer was then used to estimate the visual bias in each particular test case. The research results indicate a positive correlation between user rating and their preferred biases. In the present article, we serve to describe in detail the abovementioned methodology and the mathematical model employed to empirically estimate bias existence. We also serve to present a machine learning-based methodology that may be used to utilize user biases and create an automated bias manipulation system that may be coupled with recommendation engines to enhance business.

1.1 Literature Survey A bias may be defined as a disproportionate weight in favor of or against an idea or a thing [2]. Contingent on this definition, visual bias can thus be described as the phenomenon of showing a preference for a particular visual stimulant based on some inherently unique characteristics. The effect of such biases on human behavior has in recent times become a hot topic of discussion because of its applicability in both the financial and social spheres. In the past, research [3] has shown that visual biases such as size, salience, position, emotional valence, predictability, and a number of information elements play a significant role in the larger decision-making process. Moreover, the manifestations of visual bias in the diverse social sphere in the form of racial prejudice have also become a widely considered topic [1, 4]. Recently, the field of visual bias study has garnered much interest due to the tremendous untapped potential, it has of increasing revenues for online business and streaming platforms, etc.

Analysis of User Inclination in Movie Posters Based on Color Bias

305

In terms of the consumer space, the most obvious example of visual bias exploitation is the OTT platform Netflix’s ever-changing thumbnails. Based upon the kind of genre and actor preferences that a user’s movie selection may display, the company switches around pre-made thumbnails to maximize user engagement. However, despite the recent interests, the field is particularly novel, and no dedicated studies have yet been conducted to explicitly measure empirical evidence of such bias and manipulate it suitably without prior human intervention. In this sense, our study is a pioneering one [5, 6]. The presented study serves both to mathematically measure and map out visuals. While it is acknowledged that such biases exist in multiple areas of the consumer space, the present study is limited to movie posters for its considerations. In order to assist such detection, the use of neural style transfer as a machine learning technique has been done [7]. The neural style transfer technique is a convolution neural network application that helps to transfer the aesthetic styles of one image onto another one. Here, this has been used in order to overcome the inherent multi-poster production costs that rudimentary visual bias incorporations may bring about. The existing poster repositories can be simply used to transfer aesthetic design features between two images [8, 9]. Once performed the bias, values may be incorporated into recommendation systems used by AdSense and OTT platforms to enhance their suggestion accuracy.

2 Study Prerequisites The methodology followed to establish the preliminary requirements for the study is illustrated in the process flow diagram (Fig. 1). First, using a Web scraping script, a project scope-specific database was created. For this purpose, the international movie database (IMDB) Web site was used. Furthermore, the titles scraped were language non-comprehensible by the test subjects of the study/users of the utility to make sure that the inputs are based strictly on visual biases and no other parameters. These movie posters were then segregated into different categories on the basis of their genres. In total, four languages were considered including Turkish, French, Korean, and Telugu. For each of these languages, 200 images were collected; this amounts to about 40 images per genre for the five genres. In the end, the total size of the dataset in terms of images was 800 images. Following this segregation, possible genre-bias combinations were anticipated. 20 such combinations were identified. Once this grouping was established, the 20 possible genre-bias combinations were used to apply neural style transfer to the movie poster data bank. This is indicated in Fig. 3. For each row in the table below, a random image was selected from the genre indicated in the bias column and used as the style image. This style was then applied to all the content images in the genres where there is a green check mark. This was done for all possible bias-genre combinations (Fig. 2) so that all cases of future user input could be anticipated and catered for.

306

H. Chadha et al.

Fig. 1 Flowchart of study methodology components

For style application, the neural style transfer algorithm was used. Neural style transfer or NST builds on the key idea that it is possible to separate the style representation and content representations in a [10–13] convolution neural network (CNN), learned during a computer vision task (e.g., image recognition task). Following this concept, NST employs a pre-trained convolutional neural network (CNN) to transfer styles from a given image to another. This is done by defining a loss function that tries

Analysis of User Inclination in Movie Posters Based on Color Bias

307

Fig. 2 Bias-genre combinations to pre-prepare neural style transfer data

Fig. 3 Neural style transform algorithm’s working illustrated on a data sample form the study

to minimize the differences between a content image, a style image, and a generated image [8, 9]. An example of this is illustrated in the image below: For the present study, the algorithm was applied using the Pytorch library for about 3500 epochs for optimal results. A pre-trained VGG19 Net is used to extract content or style features from a passed in image. The idea of content and style losses are used and then used iteratively to update the target image until the required result is obtained. Finally, a Web site was created that could query the created database in real time and collect the data relevant to empirical establishment of visual biases. The data collection procedure and the mathematical model used to estimate the bias are presented in the sections that follow.

3 Data Collection and Detection of Implicit Biases In the study, the user via the Web interface was presented with movie posters from across six genres, namely: horror, comedy, drama, romance, crime, and thriller. They gave their inputs in two stages. First, they were then asked to rate these posters on a scale of 1–5. Based on these user ratings, the genre that the user voted most favorably for was considered as the archetypical example of the user’s preference. This marks

308

H. Chadha et al.

the first stage of the methodology: the detection of the implicit bias. Next, the results obtained from the detection of implicit biases were used to query the neural style transformed dataset and obtain five cross-genre movie posters based on the bias of the test case. These were presented to the user and they were prompted to rate these images on a scale of 1–5 once again. For instance, if the user was found to have an inclination for horror movies in the first step, the Web portal then presented to the user horror style transformed images from across the five genres. The choice of applying the neural style transform algorithm in advance was made to ensure that the data collection process was quick. For most industrial applications, this transformation may be performed in real time for storage saving purposes. This marks the second stage of data input. Ultimately, the calculated difference between the user ratings in the first stage and the new style transformed ratings in the second stage was taken as a measure of the visual bias in user perception. Once such behavior is learned, similar style transformed images may be used to tempt the user to make relevant purchases or click on certain advertisements and such.

4 Empirical Model for Bias Detection In order to mathematically obtain proof of bias existence, the use of a positive correlation factor was made. The positive correlation factor, here, may be defined as a value arrived at with the use of the aggregation function defined below that takes the difference between the stage 1 rating and the stage 2 rating as an input. X = [x2 (i ) − x1 (i )], i ∈ G 5 P.C.F =

i=1 [x(i )

5

− x]

(1)

(2)

In Eq. (1) defined above, x 2 (i) represents the stage 2 rating for the style transformed image and x 1 (i) represents the stage 1 rating for the original image. In both these cases, i represents the genre under consideration and can have one of the five values contained in set G indicated by Fig. 4 at any instance of time. X in Eq. (1) is a 1 × 5 dimensional matrix that holds the correlation difference among the corresponding genres in stage 1 and 2 images. In Eq. (2), P.C.F represents the positive correlation factor and is calculated by estimating the mean deviation of the values contained in matrix X. For each of the users, the P.C.F value is estimated and lies on a scale of 1–5. The value of this factor is used as an indicator of the empirical magnitude of visual bias detected in the user, and the results of this for the study are discussed in the next section. Table 1 contains definition of the used terms and abbreviations. Intuitively, since no modulus has been used in the mathematical model, a negative value of P.C.F shall indicate a lower score given to style transformed images. This would point toward the absence of visual bias in the test case. Conversely, obtaining

Analysis of User Inclination in Movie Posters Based on Color Bias

309

Fig. 4 Illustration of internal equation number mapping to genre category

Table 1 Definition of the used terms and abbreviations Abbreviation/Term

Definition

Visual bias

Visual bias can thus be described as the phenomenon of showing a preference for a particular visual stimulant based on some inherently unique characteristics

P.C.F

Positive correlation function. The empirical factor used to indicate the visual bias’s numerical value in the present article

CNN

CNN stands for convolution neural networks. These are a class of deep artificial neural networks that are used for image analysis and visual interpretations

Neural style transfer

NST or neural style transfer employs a pre-trained convolutional neural network (CNN) to transfer styles from a given image to another. This is done by defining a loss function that tries to minimize the differences between a content image, a style image, and a generated image

Recommender systems A recommender system, or a recommendation system, is a subclass of information filtering systems that seeks to predict the “rating” or “preference” a user would give to an item

a positive value of P.C.F is taken as an indicator of positive bias existence since it would signal toward a positive difference as indicated by the equations above. Further, it is assumed that the larger the positive value of the P.C.F in magnitude, the more pronounced the visual bias correlation.

5 Results As the final result of the study, one positive correlation factor value or P.C.F value per test subject was estimated. The graph (Fig. 5) shows an illustration of the distribution of this P.C.F value. As can be seen, for a maximum number of data points, the value of the positive correlation factor or P.C.F is found to be between 4 and 5 and is strongly suggestive of a bias existence. The number of test case results for P.C.F in the range of 1–3, on the other hand, is significantly lower. The percentage distribution (approximate) of the test cases as a function of the P.C.F range is also illustrated (Table 2). All the test results are strongly indicative of the existence of a positive correlation between genre style transform and the

310

H. Chadha et al.

Fig. 5 Positive correlation function distribution in terms of value range and corresponding frequencies

Table 2 Percentage of test subjects on each P.C.F value range

P.C.F range

Percentage of test cases (%)

0–1

0

1–2

5.5

2–3

4.5

3–4

30

4–5

60

subsequent user rating based on genre mapping. This helps conclude the tangible existence of a visual bias.

6 Conclusions and Future Scope In this article, we successfully presented empirical evidence of visual bias by illustrating the conducted study. Moreover, a methodology was put forth which may be employed in tandem with modern recommendation systems to utilize the bias thus detected. Favorable detection of visual bias as indicated by the study can go on to have many formidable implications. The style transfer methodology used in the project may be extended to a Web application utility in the future. This may help online businesses increase the likelihood of product purchasing. Streaming services like Netflix and the performance art companies may also serve to benefit. This is because the methodology used eliminates the need to have multiple posters to cater to diverse needs and can help generate bias-specific outputs that maximize user engagement. Similarly, the online advertising industry also stands to profit.

Analysis of User Inclination in Movie Posters Based on Color Bias

311

References 1. Outsmarting human minds: a project at Harvard University, About | Outsmarting Human Minds: A Project at Harvard University (2021) [online] Available at: https://outsmartinghumanminds. org/about/. Accessed 7 October 2021 2. Wikipedia Contributors. Bias, in Wikipedia (2021, August 21). https://en.wikipedia.org/wiki/ Bias 3. J.L. Orquin, S. Perkovic, K.G. Grunert, Visual biases in decision making. Appl. Econ. Perspec. Policy 40 (2018).https://doi.org/10.1093/aepp/ppy020 4. Bias in visual perception, in Amodio Lab (2019, August 30). https://amodiolab.org/research/ bias-in-visual-perception/ 5. G. Barton, Why your Netflix thumbnails change regularly, in Vox (2018, November 21). https:// www.vox.com/2018/11/21/18106394/why-your-netflix-thumbnail-coverart-changes 6. D. Roth, The secret behind Netflix’s personalized thumbnails, in Looper.Com (2020, November 8). https://www.looper.com/274997/the-secret-behind-netflixs-personalized-thumbnails/ 7. P.K. Singh, P.K. Pramanik, A.K. Dey, P. Choudhury, Recommender systems: an overview, research trends, and future directions. Int. J. Bus. Syst. Res. 15, 14–52 (2021) 8. L. Gatys, A. Ecker, M. Matthias, A neural algorithm of artistic style (2015). arXiv. https://doi. org/10.1167/16.12.326 9. T. Ganegedara, Intuitive guide to neural style transfer—towards data science. Medium (2020, December 5). https://towardsdatascience.com/light-on-math-machine-learning-intuit ive-guide-to-neural-style-transfer-ef88e46697ee 10. The Artist Editorial, Color psychology: why we love some and hate others? Popxartist (2020, October 17). https://www.theartist.me/design/the-psychology-of-color 11. J. Park, D. John, Judging a book by its cover: the influence of implicit self-theories on brand user perceptions. J. Consum. Psychol. 28, 56–76 (2018). https://doi.org/10.1002/jcpy.1014 12. B. Pickard-Jones, How our unconscious visual biases change the way we perceive objects, in The Conversation (2019, January 15). https://theconversation.com/how-our-unconscious-vis ual-biases-change-the-way-we-perceive-objects-109039 13. B. Odegaard, D. Wozny, L. Shams, Biases in visual, auditory, and audiovisual perception of space. PLoS Comput. Biol. 11, e1004649 (2015). https://doi.org/10.1371/journal.pcbi.1004649

The Changing Role of Information Technology Management in the Era of Artificial Intelligence Abeer A. Aljohani

Abstract Artificial intelligence (AI) has proven its popularity in the age of Industry 4.0 through reforming the area of information technology (IT). It intends to transform information technology infrastructure into intelligent systems, restoring AI importance in the IT industry. AI may play a critical role in information technology because it is primarily about virtual machines, applications, and data communication protocols. Presently, AI has established itself as a brand that identifies innovative techniques and its implications in a variety of disciplines. IT is changing at a rapid rate. This rapid change has become a continuous strength, but change has always been extremely problematic. It requires rapid change in information technology management. Change is not easy to adopt as we strongly fights with any change and the similar case also correct for business organizations. For business organizations, new change can associate to more dangers and more probable for business interruptions if it is not managed efficiently with a pre-defined information technology management policy and procedures. In business organization, information technology management requires proper training, procrastination, internal support, vendor support, new measures, and inducement. This paper provides an overview of the changing role of information technology management in in the era of artificial intelligence. Keywords Information technology · IT management · Business organization · Security threat · Artificial intelligence · Digitalization

1 Introduction Technology is evolving at a tremendous speed. Technological developments involve not only innovative equipment, applications, as well as communication systems for client computers, but also information technology (IT) experts’ management and building tools. Product innovations appear to be appearing with significantly larger zeal than ever before in computer science history. According to industry analysts, A. A. Aljohani (B) Applied College, Taibah University, Medina, Saudi Arabia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_26

313

314

A. A. Aljohani

today’s IT management is being hampered by rapidly changing technology [1, 2]. Because of the long and complex duration of IT acquirement and deployment, the advent of spectacular, innovative, and beneficial ITs, as well as the attempting to pass of others, may occur on massive development initiatives. Business owners, professions, and trades in today’s modern information technology world rely entirely on their advanced technologies to be more able to adapt to changing strategic goals, competitive conditions, and accomplishments. They are less concerned with the innovations used by IT organizations and more concerned with how the IT organization will impact or enabling a change in market model. Security and privacy for information systems is also a must-have concern for IT management stakeholders [3]. Currently, a latest innovative IT may be established and superseded prior to project completion. Attempting to exploit the opportunities of innovative IT while avoiding the risks of crazes can present a multifaceted threat to IT management. Flaws can be costly, as well as IT management teams cannot be experts in all areas of emerging information technology [4]. Moreover, due to the growing strategic importance of IT, the need to finish it effectively is felt with tenacity. Assuming this is the case, research is required to assist IT managers in appreciating, planning for, and controlling the effect of shifting IT on their IT organization. In particular, investigation is required to represent the challenges and issues confronting the IT supplier. The information technology service management market was estimated at USD 2627.07 million in 2020 and is predicted to grow at a 7.7% CAGR from 2021 to 2026, to USD 4246.82 million. Information technology service management (ITSM) is a framework for delivering IT services to businesses. These technologies help a company expand by connecting its IT procedures and services with its business goals. For both medium and lengthy management planning, there had been a change in attention toward controlling IT services as well as their implementations. Based on the options and functionalities given by ITSM solutions, businesses spend extensively in them to replace their old legacy systems. Improvements and upgrades made by competitors could have a significant impact on future market share changes [5]. The following Fig. 1 shows the information technology service management market growth rate by world-wide geography from 2020 to 2025. Information systems employees are critical to the performance of commercial organizations, especially given the current impacts of the computing transformation on every element of professional and economic life [6–8]. This technological revolution is referred to as a "disruption" event. Undoubtedly, many businesses today are grappling with the dilemma of how to foresee the unanticipated challenges posed by technological advancements that could damage their operations. This article emphasizes on the critical importance that communication and digital technology organizations must contribute to the creation and growth of organizations, as well as the growing necessity to completely incorporate ICT into workplace organizational instructional strategies. Technology employees have long been chastised for their inability to work as part of a team, and they are frequently viewed as an outlier in the workplace. This is indeed a socio-cultural background issue, and it is one of the two

The Changing Role of Information Technology …

315

Fig. 1 IT service management market—growth rate by geography (2020–2025)

key frontiers that businesses must contend with in order to grasp the new, expanding potential of technology and remain competitive in a global environment. The technological innovation and businesses’ revolutionary use of technology resulted in technical advancements to optimize and address the industry’s key concerns. Across all the technology implementations, AI is at the heart of every industry’s development, with information systems at the top of the list. Incorporating AI systems into IT helped designers work more efficiently, while also ensuring quality and increasing productivity. And, because to AI’s sophisticated algorithmic capabilities, massive development, implementation, and implementation of IT systems that were before inconceivable are now possible. This paper consists of four sections. Section 2 discusses the rapid change of information technology in business sector. Section 3 discusses the different stages of information technology management. Section 4 presents the discussion about the current study. Section 5 finally concludes the paper.

2 Rapid Change of Information Technology In many firms, IT has evolved into a strategic resource. As a consequence, IT administration has become significantly more important. IT, on the other hand, is rapidly evolving. The issues that IT administrators face are becoming more complicated as a result of this development. Previous study has suggested that IT administrators utilize certain coping techniques to deal with these issues on purpose. Both research and practice understand the impact of rapid information technology transition on information technology management. Investigators have agreed that IT is evolving at an exceptional rate [9]. The use of information technology in business has resulted in increased investment in digital technologies and allowed for this

316

A. A. Aljohani

transformation [10]. As the number of types of information technology grows, IT professionals are being compelled to learn a wide range of new capabilities. Furthermore, as infrastructure, presentation, and data-sharing standards strengthen, new ITs are projected to increase in quantity over the next few years [11–15]. Although many IT companies keep focusing on symptoms and treatment investigations, others have shifted their attention to healthcare administration. In recent years, health care has become more popular, with several emerging tech companies extending virtual healthcare solutions. Individuals are receiving health care through new technologies like as virtual visits and chatbots, particularly during COVID19. In addition, several businesses focus their healthcare technology on patients instead of physicians. A symptoms detector application, for instance, was originally intended for physicians but has since changed its language and presentation to prioritize providing patients with information about their symptoms. According to investigations, this development is posing a challenge to IT administrators. The second biggest concern, according to a research of 50 US chief executive officers, is quickly changing IT. Maintaining with fast evolving IT was the second most critical issue for IT managers in a study of Canadian IT managers focused on determining the biggest concerns [15–17]. Keeping on top of new technology is crucial or critical, according to 61 percent of European IT professionals polled. Due to the obvious requirement to adopt information technology, these hesitations about new IT are likely to grow. As a result, in one study, IT executives in the USA public sector said that digital transformation was a top issue for them. Irrespective of any aggravating causes, changing IT is certainly a significant challenge affecting IT managers all around globe. The framework of information technology management is depicted in Fig. 2. Fig. 2 Information technology management Software

Data Center

Remote Assets

Hardware

IT Manag ement

Network

The Changing Role of Information Technology …

317

3 Stages of Information Technology Management Information technology management is the efficient and appropriate blends project planning administration, best practices with the art of management engagement, adaptation, possibilities, and much more. The first step is to choose an information technology management approach that matches the team’s capabilities and your company’s goals. Figure 2 depicts the many stages of information technology management.

3.1 Start Phase This phase of IT management is very crucial as it starts the implementation of information technology in business organization. This phase assessed whether the implementation of IT is a good use of assets and whether this action outcome will satisfy a business need.

3.2 Planning Phase This phase involves different plan and procedures for information technology management. This involves the information technology agreement with business organization. This provides a good understanding of information technology dependencies and how missed milestones could impact overall business of organization.

3.3 Execution Phase This phase deals with the proper execution of information technology within the business organization. Different stakeholders and IT management expert daily organize conferences to discuss status and any information technology complications.

3.4 Monitor and Control Phase This phase is also a crucial phase of IT management. This phase guarantee the business organization about successful implementation and working of information technology resources. This phase enables the managers to monitor progress in real time.

318

A. A. Aljohani

Exit Execution

Monitor & Control

Planning Start

Fig. 3 Stages of information technology management

3.5 Exit Phase This is the final and last step of information technology management. This phase reflects that responses regarding IT management like what went well, what could have been improved, and what you would change next time. Figure 3 shows the different significant stages of information technology management.

4 Discussion Artificial intelligence is altering enterprise by enabling firms to provide quicker, smarter, more comprehensive, and unexpected insights. These simplify and streamline processes leading to organizational effectiveness and deliver the precision and agility of insights needed to alter decision-making and generate financial consequences. It also improves user experiences, which leads to more revenue. IT aids in the development and expansion of the trade and commercial sectors, as well as the generation of maximum production. With advancements in information technology, the timeframe it takes for diverse industries to produce business is now reduced. It offers electronic security, durability, and telecommunication efficiency. Almost every business operation is supported by technology. Automation, data analysis, and always-on connection have made it possible to achieve previously unimaginable capabilities as well as efficiencies. Separating technology from day-to-day business activities may be challenging [18–20]. Simultaneously, when system breaks down or underperforming, a company becomes vulnerable. A downed network, missing data, or computer viruses can all have a significant impact on daily operations. Information technology management methods ensure that systems are safe, easily accessible, and function at their best. One of the most significant advantages of technology is that users can compress all of the information that used to maintain in storing files or storage rooms and store it securely on a small hard disk. While storage is one aspect where IT management offers obvious advantages, efficiency is also another. Considering so much information on the Web, one can get almost whatever need with a single click of a button.

The Changing Role of Information Technology …

319

Having a centralized location to find and keep relevant information makes human lives so very much simpler in a variety of ways [21]. Digital lockers, a new type of storage system that allows individuals to store or remove their papers, are already being used by businesses. To effectively communicate, the IT department has put in place a solid communication infrastructure. The usage of Internet and technology improves educational quality. The pedagogical approach of education and learning has improved, and information technology has aided in the improvement of school infrastructure, student engagements, and instructional practices [22–25]. Students should be encouraged to learn utilizing current technology and are placing a greater emphasis on online instruction. Their teaching methods rely on live communication between teachers and customized classes for children with special needs. IT has an impact on practically every aspect of life, including employment, education, leisure, and health care. Every area, from governments to universities, makes use of technology to obtain the best outcomes. Practitioners also utilize information technology to double-check record entries, patient records, and recommended doses to ensure that they are moving in the right direction. IT is also being used in agriculture to boost efficiency. Satellites are linked to agriculture in order to forecast monsoons and smog. Drone technology allows for bulk data collecting, land surveying, pesticide application, seed planting, watering, and fertilizer application.

5 Conclusion AI and IT both are advancing at breakneck speed. Moreover automated systems are reviving old notions about how to improve information technology infrastructure so that they can conduct better functions. AI is a launching pad for the IT sector to change its technologies into smarter ones in order to scale IT capabilities. Lacking information, AI is useless. Data is the lifeblood of the current technological era. The increasingly profound, large, and complicated the data warehouses and flows grow, the more important AI becomes. AI is assisting enterprises in accelerating results, driving company growth, making smarter, and more effective decision across a wide range of enterprises and use scenarios by utilizing data. This study discussed the changing nature of information technology management in this artificial intelligence era today. This paper provides an inclusive explanation of the management of IT in relations of quickly shifting IT. Though independently, no challenge or managing instrument is predominantly new, this study presented an organized view of information technology management and the uses of the managing mechanisms. As a result, the research presented here provides significant contributions. It gives scholars in this subject a literature review as well as a list of potential obstacles to help them comprehend their own challenges. It provides them with a new set of management methods to assist them in finding substitutes for fast evolving information technology. It also provides them with exact preparations for coping with shifting IT management.

320

A. A. Aljohani

References 1. B.H. Boar, Logic and information technology strategy: separating good sense from nonsense. J. Syst. Manag. 45(5), 16 (1994) 2. S. Paul, European IS managers get down to business. Datamation 40(5), 78 (1994) 3. M.T.J. Ansari, D. Pandey, M. Alenezi, STORE: security threat oriented requirements engineering methodology. J. King Saud Univ. Comp. Inf. Sci. (2018) 4. D. Carey, Rating the top MIS issues in Canada. Can. Data Syst. 24(6), 23–25 (1992) 5. Information Technology Service Management Market | 2021–26 | Industry Share, Size, Growth—Mordor Intelligence. (n.d.). Retrieved November 24, 2021, from https://www.mor dorintelligence.com/industry-reports/information-technology-service-management-market 6. A.M. Langer, in Analysis and Design of Information Systems (Springer Science & Business Media, 2007) 7. A.M. Langer, in Information Technology and Organizational Learning: Managing Behavioral Change in the Digital Age (CRC Press, 2017) 8. J. Lipman-Blumen, in The Connective Edge: Leading in an Interdependent World (Jossey-Bass, 1996) 9. E. Geisler, Managing information technologies in small business: some practical lessons and guidelines. J. Gen. Manag. 18(1), 74–81 (1992) 10. R.I. Benjamin, J. Blunt, Critical IT issues: the next ten years. MIT Sloan Manag. Rev. 33(4), 7 (1992) 11. R.I. Benjamin, E. Levinson, A framework for managing IT-enabled change. Sloan Manage. Rev. 34(4), 23–33 (1993) 12. S.L. Caudle, W.L. Gorr, K.E. Newcomer, Key information systems management issues for the public sector. MIS Quar., 171–188 (1991) 13. R. Kumar, M. Alenezi, M.T.J. Ansari, B.K. Gupta, A. Agrawal, R.A. Khan, Evaluating the impact of malware analysis techniques for securing Web applications through a decisionmaking framework under fuzzy environment. Int. J. Intell. Eng. Syst. 13(6), 94–109 (2020) 14. R. Kumar, M.T.J. Ansari, A. Baz, H. Alhakami, A. Agrawal, R.A. Khan, A multi-perspective benchmarking framework for estimating usable-security of hospital management system software based on fuzzy logic, ANP and TOPSIS methods. KSII Trans. Internet Inf. Syst. (TIIS) 15(1), 240–263 (2021) 15. M.T.J. Ansari, A. Baz, H. Alhakami, W. Alhakami, R. Kumar, R.A. Khan, P-STORE: extension of STORE methodology to elicit privacy requirements. Arab. J. Sci. Eng., 1–24 (2021) 16. A. Attaallah, M. Ahmad, M.T.J. Ansari, A.K. Pandey, R. Kumar, R.A. Khan, Device security assessment of Internet of healthcare things. Intell. Autom. Soft Comp. 27(2), 593–603 (2020) 17. T.J. Ansari, D. Pandey, An integration of threat modeling with attack pattern and misuse case for effective security requirement elicitation. Int. J. Adv. Res. Comp. Sci. 8(3) (2017) 18. W. Alosaimi, M.T.J. Ansari, A. Alharbi, H. Alyami, A.H. Seh, A.K. Pandey, …, R.A. Khan, Evaluating the impact of different symmetrical models of ambient assisted living systems. Symmetry 13(3), 450 (2021) 19. M. Zarour, M. Alenezi, M.T.J. Ansari, A.K. Pandey, M. Ahmad, A. Agrawal, … R.A. Khan, Ensuring data integrity of healthcare information in the era of digital health. Healthcare Technol. Lett. 8(3), 66 (2021) 20. W. Alhakami, A. Binmahfoudh, A. Baz, H. Alhakami, M.T.J. Ansari, R.A. Khan, Atrocious impinging of COVID-19 pandemic on software development industries. Comp. Syst. Sci. Eng., 323–338 (2021) 21. M.T.J. Ansari, N.A. Khan, Worldwide COVID-19 vaccines sentiment analysis through Twitter content. Electron. J. Gen. Med. 18(6), em329 (2021) 22. W.C. Ho, Use of information technology and music learning in the search for quality education. Br. J. Edu. Technol. 35(1), 57–67 (2004) 23. M.T.J. Ansari, D. Pandey, Risks, security, and privacy for HIV/AIDS data: big data perspective, in Big Data Analytics in HIV/AIDS Research (IGI Global, 2018) pp. 117–139

The Changing Role of Information Technology …

321

24. N. Smeda, E. Dakich, N. Sharda, The effectiveness of digital storytelling in the classrooms: a comprehensive study. Smart Learn. Environ. 1(1), 1–21 (2014) 25. F.W. Amenyedzi, M.N. Lartey, B.M. Dzomeku, The use of computers and internet as supplementary source of educational material (2011)

Study of Document Clustering Algorithms Applied on Covid Data Sweety Suresh, Gopika Krishna, and M. G. Thushara

Abstract The advancement in the technology rises online unstructured data. As the data grow rapidly, tackling the information is becoming hard. There is a demand to maintain these unstructured data to gather important insights. Clustering of the text documents has become leading edge over Internet. Document clustering is mainly described as grouping of the similar documents. It plays vital role for establishing massive information. The paper shows an overview of study done on different clustering algorithms on covid data. The study of the semantic links between words and concepts in texts aids in the classification of documents based on their meaning and conception. The clusters were visualized using the k-means clustering technique, which was then evaluated using t-distributed stochastic neighbor embedding (t-SNE) and principal component analysis (PCA).

1 Introduction Fast digitalization has resulted in a surge of pages or documents with unstructured text. Text classification, information retrieval, public perception surveys, decisionmaking assistance, and response and recovery are just a few of the services available. Might indeed benefit from the massive volume of textual data. Conveniently, clustering data into meaningful clusters is one of the most reliable approaches to managing this massive volume of data. Clustering is a data mining approach that is used to sort, consolidate, optimize, and differentiate text information. The practice of accumulating papers is known as document clustering, which is comparable to S. Suresh (B) · G. Krishna · M. G. Thushara Department of Computer Science and Applications, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India e-mail: [email protected] G. Krishna e-mail: [email protected] M. G. Thushara e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_27

323

324

S. Suresh et al.

a cluster. Assembling various text-based resources is becoming exceptionally difficult. Document clustering is a collection of machine learning algorithms geared at conveniently clustering documents into clusters that are nearly equal to documents in other clusters. Document clustering has become a centralized approach seen in several text mining and information retrieval applications [1]. Current document clustering research efforts have begun to focus on the construction of a more efficient clustering that takes into account the semantics between terms in texts to improve the clustering results [2]. Understanding the linguistic links between words, concepts, and ideas inside documents aids in the grouping of texts that support the notion and its meaning. The purpose of this research is to look at several papers that propose agglomeration terms of linguistic resemblance, rather than to highlight an often relevant point from each. Through well data, text mining works best. The text mining approach starts with a group of papers and then evaluates the text format and contents to determine which one should be preprocessed. Following that, a document evaluation is done, but all these approaches are continued till the document’s contents or information are extracted. Researchers in this field employ text document clustering (TDC) as a useful and efficient approach [3]. We will look at several forms of data clusters, such as k-means and hierarchical clusters, and how they are used in semantic networking. Clustering is a well-known data mining approach, with a variety of algorithms presented by various academics [4]. The primary purpose of this study is to analyze an outlook and survey of various data clustering approaches. The paper is categorized into three parts: The methodology of several conventional feature extraction and clustering techniques, as well as a brief overview and the interpretation, is emphasized in the literature survey.

2 Related Works The case study’s findings from the relevant studies are addressed below. Text mining is described as the extraction of particular patterns and information from text content. Text mining identifies relevant information on sentiment, purpose, and application. This work provides a text mining program that performs document clustering and text summarizing, as well as other tasks [5]. It is an unsupervised automated document clustering approach that divides documents into homogenous groups using the document similarities criteria [6]. The linguistics analysis defines the strategy for comprehending human communication, as well as the methods and circumstances that support it. Semantic technology analyzes a sentence’s logical structure to determine the most relevant aspects in the text and comprehend the topic discussed in the text. It mainly deals with enhancing the comprehension of connections between various textual concepts. This mainly entails the collecting and categorizing of data into groups which make it easier for a user to read and retrieve information when we use the broad term of data clustering.

Study of Document Clustering Algorithms Applied on Covid Data

325

The most often used algorithms for document clustering are agglomerative hierarchical clustering and k-means. Hierarchical clustering algorithm arranges groupings in a tree as well as a hierarchy to make searching easier [7]. The cluster hierarchy allows for a rapid search of the topics of interest [8]. To find the centroid with the lowest inertia or within-cluster sum of squared criteria [9], the k-means algorithm is been used. Clusters of items typically reside in subspaces rather than the full space in high-dimensional data, and hence, for clustering, high-dimensional [10] objects in subspaces were developed by using a novel k-means-type technique. The author of [11] presents a comparative analysis of several keyword extraction algorithms as well as clustering algorithms, with the goal of modeling a new prototype of information keyword and classifying texts based on numerous major research topics in Computer Science. The author studied and analyzed key extraction techniques TextRank, PositionRank, KEA, and multi-purpose automated topic indexing (MAUI) in [12]. The overview of the text is provided by Keywords, keys. Keywords and key phrases aid comprehension of the material presented in the research paper [13]. The number of cluster algorithms is evaluated by the author, including spectral, k-means, and mean shift [14] to identify the most potent and accurate. The author discusses his program CredSat, which aids in the evaluation of big data trustworthiness. Another study discusses how to rate big portfolios of variable rente contracts using information aggregation and machine learning methods. Self-supervised contrastive learning and unsupervised data augment by partial contrastive learning methods attain state-of-the-art outcomes in clustering accuracy when compared to previously developed unsupervised clustering methodologies. The author of [15] provides two methods for document clustering learning: selfsupervised contrastive learning and unsupervised data augment by partial contrastive learning. Methods attain state-of-the-art outcomes in clustering accuracy when compared to previously developed unsupervised clustering methodologies. The author elaborates to create content-specific topic-based clusters that are targeted [6]. They can assist users in finding subjects in a collection of papers more quickly. The author evaluates the fuzzy c-means method which is more efficient in terms of running time [16]. There are two primary processes in fuzzy text document clustering. It involves document clustering and feature extraction. In the document clustering procedure, the fuzzy c-means method is used. It is a method for organizing data points in a multidimensional space into a particular number of clusters. An incremental clustering technique is shown [17]. It emphasizes the significance of obtaining data from a dynamic text. The approach for incremental clustering is order agnostic. For a clustering method, order independence is a desired attribute. The suggested method has been put to the test in terms of complexity and cost. The document retrieval algorithm must be suitable for this dynamic environment. DBSCAN is a clustering technique that finds core items and divides them into groups equal to the number of core objects discovered. A densely populated item is referred to as a core object [18]. The algorithm considers certain items to be noise if

326

S. Suresh et al.

they have no dense regions. It connects core items with their surroundings to generate dense zones known as clusters. Documents are embedded in vector space using the embedding vector representation approach [19]. To improve clustering accuracy, this strategy increases the distance between texts in the same category. This allows for a more accurate categorization of comparable papers. The semantic clustering algorithms recurrent neural network (RNN) and convolution neural network (CNN) have been studied [20]. Since the prior technique relied on a single similarity measure with only one reference, the findings were less accurate. The paper proposes that an existing method be improved by including a multi-view reference [21]. The study presents a question answering system that uses a combination of word sense disambiguation (WSD) and semantic role labeling to answer questions (SRL) [22]. A method for clustering text documents the weighted bidirectional encoder representation from transformers (BERTs) model, which embeds, weights, and clusters documents. It creates a cluster of comparable clusters. To begin, the contextualized phrase embedding is generated using the BERT [23] is a language representation model that has been pretrained. Then, based on the named entity, two-sentence level weighting techniques, WA and WR, are created to improve performance. Then, the k-means clustering technique is used to locate groups of texts that are comparable [9]. The author proposes a topic modeling approach based on Latent Dirichlet Allocation (LDA) [24]. The author of the study article [25] shows how to use methods like labeling and route similarity analysis to gather various medical-related data and determine the relationship between them. The DAKE rule, a BiLSTM-CRF [11] model accumulated with document-level attention, and a gating method are explored in the work for optimizing the extraction of keywords from research documents [23]. The study discusses how an unsupervised graph-based model known as position rank [11] is used to generate keyword scores by assimilating information from all of the positions of the word’s occurrences into a biased Page-Rank.

3 Study on Semantic Document Clustering Unstructured or semi-structured data types including emails, text documents, and Web pages can also be handled by using text mining. The classification of documents according to their conception and significance is mainly performed by studying the semantic relationships between words and concepts in documents assists in the classification of documents according to their conception and significance. To improve Web mining, the suggested method is based on the deployment of a semantic-based document clustering algorithm [26]. Text mining is a rather effective approach in domains involving text management. The text mining program begins by examining

Study of Document Clustering Algorithms Applied on Covid Data

327

the text format and contents of a group of documents before selecting one to preprocess. After that, text analysis is performed, and the process is continued until the document’s keywords or information are retrieved.

3.1 Architecture The architectural model in Fig. 1 depicts the prototype’s operation. This creates a system that uses a semantics-based search algorithm to extract certain tags from each text in the database [14]. The classification algorithm then examines these tags and generates a graphical model that displays the number of documents that have a certain tag.

3.2 WordNet Ontology Mapping The WordNet ontology will be employed. The nouns, adjectives, verbs, and adverbs in the vast English lexical database are divided into groups of synonyms each representing a separate notion. As a result, after preprocessing, determine the frequency of keywords in the document and then use the tf-idf equation to calculate the weight. The most common terms are eliminated by using TF-IDF technique and which also extracts only the most relevant terms from the corpus [27] Tf-idf (term frequent— inverse document frequency) is a measure of the relevance of word inside document

Fig. 1 Architectural model of analyzed system

328

S. Suresh et al.

[11]. Following the formation of weights across all terms, the semantic weights of keywords are plotted to the WordNet ontology, so various ideas are created. Synonym similarity is used to construct concepts by dynamically mapping keywords to the online WordNet lexicon. The WordNet lemmatizer makes use of a large, accessible lexical repository within the English language, which ensures proper functioning syntactic relationships between words [28].

3.3 Similarity Measure The degree of similarity papers must be determined in a clustering study. Euclidean distance, Manhattan distance, cosine similarity, and other similarity metrics are available to determine the similarity between two texts. To compare two articles based on a variety of metrics, the cosine similarity metric is being used.

3.4 Evaluation Methods There are two types of validation methods for assessing document clustering algorithms: intrinsic and extrinsic measurements [29]. Extrinsic measures are used to evaluate clustering accuracy when class labels are not accessible. When class labels are unavailable, intrinsic measurements are employed. For enhancing cluster accuracy, efficiency, and results, a variety of assessment metrics are provided, including entropy, recall, accuracy, F-measure, silhouette coefficient, purity, and inverse purity are some of the terms used in statistics. The clustering operation is completed.

4 Methods and Algorithms 4.1 Clustering Algorithms The suggested method for document clustering consists of four steps: data training with preprocessing, testing with preprocessing and TF-IDF, feature selection phase, and classification module.

Study of Document Clustering Algorithms Applied on Covid Data

329

4.2 Data Training Phase with Preprocessing To construct a training dataset, each module does data preprocessing. Then, initially, upload the dataset’s training directory. After that, porter’s stemmer, tokenization, and stop word elimination will take place. Finally, TF-IDF will offer the current vector’s availability and save it in a feature database.

4.3 Testing Phase with Preprocessing and TF-IDF Initially, upload the pdf and picture dataset test directories. Until the IDF score is calculated, the testing period is similar to the training phase.

4.4 Feature Selection Phase Using an optimization strategy, this module pulls the feature from all buckets. The initial pheromone must be established [20]. The pheromone will choose the strong node and neighbors for selection.

4.5 Classification/Clustering Module Find the training dataset that includes domain and feature information. When the appropriate clustering algorithm is run, it will request variation as well as generation and then optimize the results. Finally, the similarity score will group or categorize each bucket into its domain.

5 Analysis and Results Separating technical and scientific documents is a time-consuming task. Manual categorization leads to inaccuracy and error. Using machine learning, unsupervised clustering algorithms helps to get a better outcome. It allows for more efficient and cost-effective operations, as well as a reduction in human workload. The elbow point in the sum of square error (SSE)/inertia diagram depicts the point at which SSE or inertia begins to decrease linearly. In Fig. 2, SSE is represented on the Y -axis, while the number of clusters is represented on the X-axis. In Fig. 2, it can analyze that as the number of clusters reaches 14, the SSE begins to decrease linearly.

330

S. Suresh et al.

Fig. 2 Cluster center plot

The proposed framework in [19] evaluated cluster patent documents for scientific analysis. It includes two steps which are the patent embedding step and the patent clustering step. The dataset for the patent analysis used from KIRPS and KISTA. It includes several categories such as cars, camera, and CPU [19]. The DBSCANDBSCAN is a program that scans your computer for (Density-based spatial clustering of applications with noise). It employs information gain as a feature selection approach and measures the DBSCAN algorithm’s ability to integrate all of the samples from the dataset into clusters. They employed the WEKA framework [18], which makes all six clustering phases easier. Agglomerative hierarchical clustering and k-means are the two most common approaches to document clustering. Even though hierarchical clustering provides a higher-quality clustering strategy, it is not often used due to its time complexity quadratic [21]. On the other hand, k-means and its variations have a time complexity proportionate to the number of documents but are thought to produce poorer clusters. The statistical approach of t-distributed stochastic neighbor embedding (t-SNE) is used to visualize high-dimensional data. In Fig. 3 clustered data represent using t-SNE. The cluster plotted using principal component analysis (PCA) is shown in Fig. 4. It is a technique for lowering the dimensions of a dataset while maintaining the majority of its variation. In comparison to PCA, two-dimensional data obtained by the t-SNE demonstrated improved visualization and clustering quality [30]. On the datasets, k-mean clustering with tf-idf is used. Getting a k number of centroids and grouping close items to the centroids is how the k-means clustering algorithm works [31]. The process of improvising these algorithms was thoroughly detailed. The measurements used to evaluate the findings are purity and entropy,

Study of Document Clustering Algorithms Applied on Covid Data

331

Fig. 3 Clustered data using TSNE

respectively. Improved purity is a crucial criterion for assessing the quality of clusters. Clusters of the same type of document are thought to be purer. The second metric is entropy, which estimates the number of disorganized objects in a cluster [21].

6 Conclusion The paper goes through semantic clustering as well as several clustering algorithms. It analyzes a document dataset to explain the technique and implementation. The document dataset in this case consists of covid-19 research articles. This information is used to conduct a comparative survey. The research aids in the discovery of a clustering technique. This is useful in the field of text mining, in which researchers are mostly concerned with document management techniques. This aids in automated document search by allowing for multiple keyword and domain criteria. Further study will also contribute to the establishment of a dashboard for the display of covid-19 research to facilitate information retrieval.

332

S. Suresh et al.

Fig. 4 Clustered data using PCA

References 1. M.M. Thakor, D. Bhalodiya, A review of privacy preserving in document clustering 2. S.K. Ahammad Fahad, W.MS. Yafooz, Review on semantic document clustering. Int. J. Contemp. Comput. Res. 1(1), 14–30 (2017) 3. I. Arın, M.K. Erpam, Y. Saygın, I-twec: interactive clustering tool for twitter. Expert Syst. Appl. 96, 1–13 (2018) 4. M. Rafi, M.S. Shaikh, An improved semantic similarity measure for document clustering based on topic maps. arXiv preprint arXiv:1303.4087 (2013) 5. J.L. Neto, A.D. Santos, C.A.A. Kaestner, D. Santos, et al., Document Clustering and Text Summarization, (Neto Alexandre, 2000) 6. R. Parimala, K. Gomathi, Topnmf: topic based document clustering using non-negative matrix factorization. Indian J. Sci. Technol. 14(31), 2590–2595 (2021) 7. B.C.M. Fung, K. Wang, M. Ester, Hierarchical document clustering, in Encyclopedia of Data Warehousing and Mining, 2nd edn. (IGI Global, 2009), pp. 970–975 8. R. Encarnação, P. Gomes, Incremental and hierarchical document clustering 9. I. Pauletic, L.N. Prskalo, M.B. Bakaric, An overview of clustering models with an application to document clustering, in 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (IEEE, 2019), pp. 1659–1664 10. L. Jing, M.K. Ng, J.Z. Huang, An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans. knowl. Data Eng. 19(8), 1026–1041 (2007)

Study of Document Clustering Algorithms Applied on Covid Data

333

11. S.R. Nair, G. Gokul, A.A. Vadakkan, A.G. Pillai, M.G. Thushara, Clustering of research documents-a survey on semantic analysis and keyword extraction, in 2021 6th International Conference for Convergence in Technology (I2CT) (IEEE, 2021), pp. 1–6 12. E. Gopan, S. Rajesh, G.R. Vishnu, M.G. Thushara, et al., Comparative study on different approaches in keyword extraction, in 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC) (IEEE, 2020), pp. 70–74 13. M.G. Thushara, T. Mownika, R. Mangamuru, A comparative study on different keyword extraction algorithms, in 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC) (IEEE, 2019), pp. 969–973 14. M.F. Areed, Python-based graphical user interface for automatic selection of data clustering algorithm. Int. J. Future Gener. Commun. Network. 13(3), 451–461 (2020) 15. H. Shi, C. Wang, Self-supervised document clustering based on bert with data augment. arXiv preprint arXiv:2011.08523 (2020) 16. T.F. Gharib, M.M. Fouad, M.M. Aref, Fuzzy document clustering approach using wordnet lexical categories, in Advanced Techniques in Computing Sciences and Software Engineering (Springer, 2010), pp. 181–186 17. F. Can, N.D. Drochak, Incremental clustering for dynamic document databases, in Proceedings of the 1990 Symposium on Applied Computing (IEEE, 1990), pp. 61–67 18. Y. Cai, J.S. Yuan, Text clustering based on improved dbscan algorithm. Comput. Eng. 37(12), 49–50 (2011) 19. J. Kim, J. Yoon, E. Park, S. Choi, Patent document clustering with deep embeddings. Scientometrics (2020), pp. 1–15 20. P.B. Sonawane, P.M. Chawan, Semantic document clustering using recurrent neural network (2020) 21. A. Gunta, R. Dubey, An improved document clustering approach with multi-viewpoint based on different similarity measures, in 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS) (IEEE, 2018), pp. 152–157 22. L.R. Pillai, G. Veena, D. Gupta, A combined approach using semantic role labelling and word sense disambiguation for question generation and answer extraction, in 2018 Second International Conference on Advances in Electronics, Computers and Communications (ICAECC) (IEEE, 2018), pp. 1–6 23. Y. Li, J. Cai, J. Wang, A text document clustering method based on weighted bert model, in 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), vol. 1 (IEEE, 2020), pp. 1426–1430 24. V. Gangadharan, D. Gupta, Recognizing named entities in agriculture documents using lda based topic modelling techniques. Procedia Comput. Sci. 171, 1337–1345 (2020) 25. G. Veena, R. Hemanth, J. Hareesh, Relation extraction in clinical text using nlp based regular expressions, in 2019 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), vol. 1 (IEEE, 2019), pp. 1278–1282 26. M.S.D. Mahalle, K. Shah, Document clustering by using semantics. Int. J. Sci. Eng. Res. 3(9) (2012) 27. P. Bafna, D. Pramod, A. Vaidya, Document clustering: Tf-idf approach, in 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT) (IEEE, 2016), pp. 61–66 28. I.-M. R˘adulescu, C.-O. Truic˘a, E.-S. Apostol, C. Dobre, Enhancing scientific collaborations using community detection and document clustering, in 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP) (IEEE, 2020) pp. 43–50 29. P. Yang, Y. Yao, H. Zhou, Leveraging global and local topic popularities for lda-based document clustering. IEEE Access 8, 24734–24745 (2020) 30. B.M. Devassy, S. George, Dimensionality reduction and visualisation of hyperspectral ink data using t-sne. Forensic Sci. Int. 311, 110194 (2020) 31. H.-S. Park, C.-H. Jun, A simple and fast algorithm for k-medoids clustering. Expert Syst. Appl. 36(2), 3336–3341 (2009)

Course and Programme Outcomes Attainment Calculation of Under Graduate Level Engineering Programme Using Deep Learning S. Manikandan, P. Immaculate Rexi Jenifer, V. Vivekanandhan, and T. Kalai Selvi Abstract Outcome-based education (OBE) is playing important role for producing successful engineering graduates. OBE provides an engineering graduates that are employable and accepting their graduation as globally competent. After end of the graduation, this outcome-based education will enable graduates to compete the global market. This paper gives course outcome (COs), programme outcome (POs) and programme-specific outcome (PSOs) attainment calculation of under graduate level engineering course. Here, we are taking cloud infrastructure and computing core course for B. Tech Information Technology programme. This paper explains detailed view of course plan, knowledge levels, delivery methods and assessment. The direct and indirect methods are used for assessment and attainment process. At the end, based on attainment results, we find the graduates level and programme outcome attained or not. The course end reflective report provides information about course attained or not. Also, this end report will be used for next academic year course input. At the end of this paper, we can easily track the performance and attainment of POs and PSOs. Keywords Outcome-based education · Course outcomes · Programme outcomes · Programme-specific outcomes · Attainment

S. Manikandan (B) E.G.S. Pillay Engineering College, Nagapattinam, Tamil Nadu, India e-mail: [email protected] P. Immaculate Rexi Jenifer Anjalai Ammal Mahalingam Engineering College, Kovilvenni, Tamil Nadu, India V. Vivekanandhan Adhiyamaan College of Engineering College, Hosur, Tamil Nadu, India T. Kalai Selvi Erode Sengunthar Engineering College, Thudupathi, Perundurai, Tamil Nadu, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_28

335

336

S. Manikandan et al.

1 Introduction India started to implement outcome-based education in all the higher educational institution and signed memorandum of understanding between Washington Accord and National Board of Accreditation of improving graduating attributes quality. Current graduates need to compete with global standard, so outcome-based education system is mandatory requirements. Assessment is most important factor for implementing OBE. The overall attainment is calculated by using programme educational objectives (PEOs), programme outcomes (POs), programme-specific outcomes (PSOs) and course outcomes (COs). PEOs attainment is calculated after graduation after four year how the graduates are there in the society or they are globally competent or not [1, 2]. POs and PSOs attainments are calculated after graduations. Course outcomes attainment is calculated at each semester end of the courses. Course outcomes attainment is very important factor calculating POs and PSOs. POs are taken from graduates attributes (GAs). The GAs for under graduate engineering programme are given by NBA, so we are using as the same. Also, AICTE current exam reforms policies performance indicators are also measuring competencies [1]. Programme-specific outcomes are specific to the undergraduate programme. Direct and indirect assessment tools are used for measuring the attainment of course outcomes. Direct attainment tools are continuous assessment test, end semester examination, assignments, case studies, technical seminar, etc. The indirect assessment tools are surveys, feedbacks, etc. Course end survey is collected at end of each course and that will be considered for CO attainment. Alumni, employer and graduate exit survey is used for calculating POs and PSOs attainment [3]. In this paper, we used B.Tech Information Technology Programme Outcomes, programme-specific outcomes and performance indicator of E.G.S. Pillay Engineering College, Nagapattinam. Cloud infrastructure and computing course is taken for course attainment calculation. This paper discusses course outcomes mapping with POs and PSOs, course end survey, assessment questions mapping and attainment calculations.

2 Course Outcomes, Programme Outcomes and Programme-Specific Outcomes Course outcomes are narrow statement which gives end of this course student will be able to develop knowledge, skill and attitude of that course. Course outcomes are mapped with programme outcomes and programme-specific outcomes (Table 2) [4, 5]. The below table (Table 1) shows that course outcomes for cloud infrastructure and computing course offered for seventh semester B.Tech Information Technology. The cognitive levels are knowledge levels taken from revised blooms taxonomy which can be achieved by using test [6], assignments [7], tutorials [8], case studies [9],

Course and Programme Outcomes Attainment Calculation of Under …

337

Table 1 Course outcomes competency and cognitive levels Competency

Cognitive level

CO1

Develop cloud computing architecture, infrastructure and delivery models using various cloud services

Apply

CO2

Build virtual machines using their types, tools and operations at storage, network and compute levels

Apply

CO3

Deploy virtual machines using various cloud platforms

Analysis

CO4

Deploy various programming model to implement cloud infrastructure and platform

Analysis

CO5

Build the appropriate cloud security services to implement real-time Apply cloud models

mini-projects, activities, events [10], etc. Remembering, understanding and apply are lower order thinking skills (Table 1). Analysis, evaluate and create are higher order thinking skills. The programme outcomes and programme-specific outcomes are taken from AICTE reference [11, 12]. Support provided by COs to Pos/PSOs: L = lightly(1); M = Moderately(2); S = Substantially(3).

3 Course Plan, Delivery and Assessment Method In this section, we discussed course plan, delivery method and assessment method for cloud infrastructure and computing course. Course is also called lecture plan which gives information about topic, number of hours and teaching methods [13, 14]. We give information each topic targeting which course outcomes also mentioned. Table 3 shows lecture plan and delivery methods. In below, course outcome-wise topics and number hours are mentioned. Each course outcomes are taken, and delivery and assessment methods are shown in below. Also, each topic teaching methods are given. Each module completion number of lecture hours, tutorial hours, and laboratory hours are mentioned. This course is tutorial-based course, so total teaching hour is 60 and credit is 4 (Lecture: 3, Tutorial: 1). So, faculty members and students can easily understand what kind knowledge, skill and competency to each topic. Also, they get clear view about assessment and evaluation.

4 Attainment Calculation—Process Direct methods display the student’s knowledge and skills from their performance in the continuous assessment tests, end-semester examinations, presentations, classroom assignments, etc. (Table 4). These methods provide a sampling of what students

3

3

3

3

CO2

CO3

CO4

CO5

POI

3

Comp

CO1

2

2

2

2

2

PO2

2

2

2

2

2

PO3

1

1

1

1

1

PO4

2

2

2

2

2

PO5

Table 2 Course outcomes to PO and PSO mapping PO6

1

1

1

–

–

PO7

–

–

–

–

–

POS

–

–

–

–

–

PO9

–

–

–

–

–

P10

–

–

–

–

–

POII

–

–

–

–

–

PO12

–

–

–

–

–

PSO 1

3

3

3

3

3

PSO 2

3

3

3

2

2

PSO 3

2

2

2

1

1

338 S. Manikandan et al.

Course and Programme Outcomes Attainment Calculation of Under …

339

Table 3 Lecture plan and delivery methods CO

Level

Delivery method

CO1

Apply

Lecture with discussion and lab—problem solving

CO2

Apply

Lecture with discussion and lab—problem solving

CO3

Analysis

Tutorial, lab and case study—problem solving

CO4

Analysis

Tutorial, lab and assignment—problem solving and collaborative

CO5

Apply

Tutorial, lab and case study—problem solving and project-based learning

Table 4 Assessment pattern

THEORY COURSES Direct assessment—CAT and end semester Continuous assessment

40

Distribution of marks for CA: Test I (10) Test II (10) Test III (10) Activity 1 (5)—tutorial Activity 2 (5)—case study End semester examination

60

Total marks

100

Indirect assessment: course end survey

know and/or can do and provide strong evidence of student learning. Direct assessment method: using measurable performance indicators of student. Test, Assignments, Tutorials, Laboratory Test and Project marks are taken for final attainment calculations. The attainment levels are explicitly shown to the students by virtue of correlation and the average performance levels in the university examinations and the student’s performance in internal assessment; it is enlightened that the performance in careful internal assessment would upgrade and excel their performance in the end semester examinations. The attainment calculations are shown in below. In this Fig. 1 shows that the examination marks are mapped with each course outcomes. The course outcome attainment is calculated by using direct and indirect assessment method. For direct assessment we used continuous assessment test marks, End semester marks, assignment, open book test, case studies, laboratory experiments marks and Indirect assessment Course end survey (at the end of each semester feedback is collected from student). Each questions mark is mapped with course outcomes, and total mark is calculated for each COs. The CO target is calculated by using previous academic year course attainment percentages. For cloud computing course, target is fixed; 70% means previous 3 academic years; this course average percentage is 60% to 70%. The students’ performance is calculated for each students, and percentage is obtained those who are all got above the target percentages.

340

S. Manikandan et al.

Fig. 1 Attainment process

Fig. 2 shows attainment results of course outcome wise. Here, the number of students attended for cloud infrastructure and computing course is 60, and attainment target is 70%. The course outcome benchmark is calculated from number of questions asked in test, assignments, lab experiments, etc. For the calculation number students are got 70% and above in each course outcomes taken and obtain the attainment. From the above course end reflective report said that CO2, CO3 and CO5 are above 70% so level is 3 and CO1 and CO4 60–69% so the level is 2. If level is below 1 means that course outcome is not attainment, so in next academic year further action or implementation is required. All the course outcomes are achieved means same process will be continued next academic years. The PO and PSO attainment marks are taken from questions mark sheet. The highest knowledge levels are only taken into PO and PSO attainment calculations. Each performance indicators are marked in highest level questions and that can be used for attainment process. Fig. 3 shows cloud infrastructure and computing course PO and PSO attainment. The marks are taken from highest knowledge level questions. Target is fixed 70%, PO1, PO2, PO5, PSO1, PSO2 and PSO3 are above 70%, and PO3 and PO4 are 60%. The process will be followed for all the courses. Here, PO and PSO attainment is

Course and Programme Outcomes Attainment Calculation of Under …

341

Fig. 2 CO attainment results

given as sample only. End of the programme all the courses and surveys are required for attainment calculations.

342

S. Manikandan et al.

Fig. 3 PO and PSO attainment

5 Conclusion This paper provides insight about course outcomes attainment calculations using detailed lecture plan and assessment methods. Each course outcomes are tested by using different assessment methods. All the questions and test items are mapped with COs and knowledge levels. So, target is fixed for each course, and attainment is obtained by using students’ performance. Using performance indicators, PO and PSO attainments are calculated. After the CO attainment course end reflective report is prepared, and it is the input for next academic year handling the same course. Based on this, we will change assessment pattern and delivery methods. So, we can easily track the students’ knowledge and competency.

References 1. AICTE Exam Reform policy (2018). https://www.aicteindia.org/sites/default/files/Examinati onReforms.pdf 2. National Board of Accreditation. https://www.nbaind.org/publications 3. Abet Accreditation. https://www.abet.org/accreditation/ 4. A. Karimi, K. Clutter, A. Arroyo, in An Example of Course and Program Outcome Assessment, Proceedings of the 2004 American Society for Engineering Education Annual Conference and Exposition Copyright © 2004, American Society for Engineering Education 5. M. Vanjale, S. Shelar, P.B. Mane, in Assessment of Course Outcomes (COs) in University Affiliated Engineering Programs—Case Study of Course Outcome Attainment. 2015 IEEE 3rd International Conference on MOOCs, Innovation and Technology in Education (MITE) 6. S. Kaul, R.D. Adams, Learning Outcomes of Introductory Engineering Courses: Student Perceptions. American Society for Engineering Education (2014)

Course and Programme Outcomes Attainment Calculation of Under …

343

7. S. Manikandan, M. Chinnadurai, Evaluation of students performance in educational sciences and prediction of future development using tensorflow. Int. J. Eng. Educ. 36(6), 1783–1790, (2020) (0949–149X/91, TEMPUS Publications, Printed in Great Britain) 8. S. Manikandan, M. Chinnadurai, Intelligent and deep learning approach OT measure E-learning content in online distance education. Online J. Distance Educ. e- Learning 7(3) (2019), ISSN: 2147–6454 9. O. Eris, D. Chachra, H.L. Chen, S. Sheppard, L. Ludlow, C. Rosca, T. Bailey, G. Toye, Outcomes of a longitudinal administration of the persistence in engineering survey. J. Eng. Educ. 99, 371–395 (2010) 10. E. Valveny, R. Benavente, A. Lapedriza, M. Ferrer, J. Garcia-Barnes, G. Sanchez, (2012) Adaptation of a computer programming course to the ESHE requirements: evaluations five years later. Eur. J. Eng. Educ. 37, 243–254 (2012 11. Accreditation manuals for UG Engineering programs, National Board of Accreditation, http:// www.nbaind.org/En/1065-archives-accreditation-documents.aspx 12. P. Polimetla, C. Sailu, R. Kancharla, in Assessment of Engineering Programs—Indian Context in Global Outcome Based Education Paradigm. 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) 19–20 Dec. 2014, pp. 22, 26 13. L. Suskie, Assessing Student Learning: A Common Sense Guide, Bolton (Anker Publishing, MA, 2004) 14 Dr. D.K. Paliwal, Dr. A. Koteshwara Rao, Dr. S. Bhaskar, Dr. A. Abudahir Dr. S. Rajakarunakaran, Outcome Based Accreditation. Three Day Workshop for Evaluators/Resource Person. http://www.nbaind.org/files/workshops/Three%20days%20workshop% 20-%20Outcome%20based%20education.pdf

Multilayer Communication-Based Controller Design for Smart Warehouse Testbed Ngoc-Huan Le, Minh-Dang-Khoa Phan, Duc-Canh Nguyen, Xuan-Hung Nguyen, Manh-Kha Kieu, Vu-Anh-Tram Nguyen, Tran-Thuy-Duong Ninh, Narayan C. Debnath, and Ngoc-Bich Le

Abstract Regarding a smart warehouse, building an industrial control system (ICS) that works effectively with the requirements is essential. An ICS, on the other hand, is a broad category of command and control networks and systems that support a wide range of industrial processes. They include SCADA systems, distributed control systems (DCS), process control systems (PCS), safety control systems (SIS), and N.-H. Le · D.-C. Nguyen · X.-H. Nguyen Mechanical and Mechatronics Department, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] D.-C. Nguyen e-mail: [email protected] X.-H. Nguyen e-mail: [email protected] M.-D.-K. Phan EIU Fablab, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] M.-K. Kieu · V.-A.-T. Nguyen · T.-T.-D. Ninh Becamex Business School, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] V.-A.-T. Nguyen e-mail: [email protected] T.-T.-D. Ninh e-mail: [email protected] N. C. Debnath School of Computing and Information Technology, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] M.-K. Kieu School of Business and Management, RMIT University, Ho Chi Minh City, Vietnam N.-B. Le (B) School of Biomedical Engineering, International University, Ho Chi Minh City, Vietnam e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_29

345

346

N.-H. Le et al.

other, generally smaller control system designs like programmable logic controllers (PLCs). In this paper, a typical 4 layers ICS, including a warehouse management system (WMS) software, was introduced to control all processes in the supply chain of a smart warehouse testbed. The WMS software initially achieved the set goals in terms of basic tasks. Furthermore, ICS worked well without incidents such as blocked or delayed flow of information, inaccurate information sent to system operators, etc. Keywords Multilayer industrial communication · Smart warehouse · Industrial control system · IoT · Industrial testbed

1 Introduction Vietnam’s logistics business has expanded at a breakneck pace in recent years. As a result, warehouses and warehousing services are crucial for enhancing logistical competitiveness in Vietnam [1]. Warehouse services have limitations in simple storage, processing, value-adding, and piece-picking [2]. One of the issues and bottlenecks to the development of warehousing services is using new technologies to warehouse operations, such as IoT, deep learning, etc. Testbeds are critical platforms for rigorous and repeatable testing of concepts, computational tools, new technologies, and systems [3]. They are made up of both software and hardware. Manufacturing organizations can employ testbeds to put innovative technologies to the test in a real-world setting before applying them [4]. Testbeds have also been shown to be beneficial in the academic field. According to the authors [5], the concept of a testbed has aided the accumulation of practical and theoretical knowledge. An ICS is composed of a set of control components (electrical, mechanical, hydraulic, and pneumatic) that work together to achieve a particular industrial goal. The desired output or performance specification is part of the system’s control. Control might be completely automated or include a human. Many control loops, human–machine interfaces (HMIs), remote diagnostics, and maintenance tools may be found in a typical ICS, which are created utilizing a variety of network protocols. ICS is often used to manage industrial operations. WMS is one of the most successful techniques for managing a smart warehouse. It prioritizes the stability of an organization’s supply chain. The fundamental goal is to properly control all supply chain operations [6]. In [7, 8], the authors describe how an RFID-based inventory management system may interact with an autonomous retrieval and storage mechanism without the need for human involvement. The systems can automatically gather data from different linkages and offer high-speed and accurate supply to the warehouse ERP/WMS software.

Vietnam National University Ho Chi Minh City, Thu Duc City, Ho Chi Minh City, Vietnam

Multilayer Communication-Based Controller Design for Smart …

347

In this paper, we introduce in detail the design of a typical ICS built to control an actual smart warehouse testbed. Similarly, WMS software has been created to collect data, control, and optimize warehouse management.

2 System Description and Input Analysis In this project, a smart warehouse testbed was developed, including (1) a pallet circulation for data collecting system (Fig. 1a), (2) automated guide vehicles (AGVs) (Fig. 1b) for traversing between storage racks, and (3) the conveyor system for circulating pallets across the system by employing horizontal conveyors to feed pallets into the system, saving pallets as they leave the system, and circulating pallets out of the system to return to the feeding area (Fig. 1c). The testbed operation can be summarized as follows: (1) When an order is received, the WMS software uses an optimization algorithm to find the best pallet placement. The RFID writer will then issue management codes to the matching number of pallets (each pallet is attached with one RFID tag) and place them on the horizontal conveyor system; (2) at regular intervals, the pallets travel to the separator

a)

b)

c) Fig. 1 a Warehouse system, b AS/RS AGV with telescopic fork, and c conveyor system

348

N.-H. Le et al.

equipment, which then moves them into the storage area. The RFID reader will then scan the RFID tag on the pallet as it comes in front of each rack to determine which rack the pallet is kept in. The process is repeated until all pallets have been stored in the warehouse; (3) when the pallets must be removed following the customer’s request, the WMS software will run an algorithm to identify the sequence in which the pallets must be removed. The AGVs will transport the pallets to the vertical conveyors. Next, the pallets will be transferred to the horizontal conveyor by the vertical conveyors. Before the intersection of the vertical and horizontal conveyors, a sensor and a lever block the moving pallets until the pallets from the vertical conveyor are already inside the horizontal conveyor, preventing a collision between the existing pallet and the pallet about to enter the conveyor. The output pallet will then be held on the horizontal conveyor until a fresh order is received.

3 System Design Figure 2 depicts the functional diagram of the proposed controller. In terms of multilevel management, the system requires the following layers: (1) Field layer manages seven optical sensors, seven RFID readers, two RFID read/write stations, 15 pneumatic cylinders, 20 DC motors, seven stepper motors, and 14 AC servo motors; (2) the control layer manages eight S71200 PLCs, 14 AC servo motor drives, seven stepper motor drives, power drives for 20 DC motor, and 15 air cylinders; (3) the HMI layer manages and monitors the operating parameters of the system, and (4) the management layer is warehouse management software (WMS). The system needs the following functions regarding the required control feature: (1) upload data to the cloud to serve the WMS and other analysis purposes, including AI; (2) three axis’s movement of seven AGVs. Each AGV needs two servo motors and corresponding drivers to control horizontal and vertical motion, one stepper and driver to execute the movement of the telescopic fork and nine optical sensors for homing purposes and limiting travel of three movements. These devices are managed by one PLC S7-1200; (3) conveyor operation to ensure pallets circulation. With this requirement, the system needs seven optical sensors and seven pneumatic cylinders to recognize and block the pallets on the main output conveyor to avoid collisions with moving out pallets from AGV’s outlet conveyors. A separator station consisting of an optical sensor and a pneumatic cylinder is installed at the inlet of the main feeding conveyor to help regulate the number of pallets supplied to the inlet conveyor of the AGVs; (4) code reassignment. To fulfill this task, the system needs 1500 RFID tags permanently attached to the respective pallets, seven sets of RFID readers and a pneumatic cylinder are necessary to read the code and redirect pallets to the corresponding AGV’s inlet conveyors. Two RFID read/write stations at the main conveyor’s outlet and inlet to clear the old code and assign a new code, respectively. (5) Supervisory, management, and connection requirement of AGV’s controllers and conveyor system’s controller. To fulfill this requirement, an Ethernet communication network configuration was utilized.

Multilayer Communication-Based Controller Design for Smart …

349

Fig. 2 Controller schematic diagram

Recently, the MQTT protocol was developed for the Internet of Things (IoT) applications. MQTT is supported by many software languages and cloud service providers. Furthermore, MQTT is more secure than Modbus TCP [9]. MQTT was deployed as the connection protocol between the WMS application and PLCs in this project.

4 Results and Discussion Controller design results With the aforementioned hardware configuration required, the electrical connection diagram of AGV and the conveyor system were proposed and shown in Fig. 3. As described, to ensure the requirements, stability, and reliability of the entire control system, the key devices were selected from reputable manufacturers. Specifically, the PLC controllers were selected from the Siemens manufacturer and are among the latest configuration (i.e., S7-1200). Similarly, AC server motor and driver were selected compatibly from the Mitsubishi manufacturer and so on. The installation process showed that implementing these circuit diagrams is relatively straightforward except for one consideration of servo driver exciting signals. Specifically, an NPN to PNP converter should be applied between the PLC’s high-speed pulse outputs and the servo driver’s pulse inputs to mismatch the incompatible voltage level. Figure 4 presents the control flowchart of the AGV and conveyor system. As demonstrated in Fig. 4a, the AGV executes the main commands including “Storing”, “Retrivaling”, “Home X”, “Home Y”, and “Home Z”. In which, to execute

350

N.-H. Le et al.

a)

b)

Fig. 3 a AGV and b conveyor system electrical connection diagram

the “Storing” instruction, the AGV needs to perform the subcommands including “Taking from conveyor”, “Moving to selected position”, and “Putting to selected position”. Similarly, to achieve the “Retrivaling” instruction, the AGV needs to run the subcommands including “Moving to selected position”, “Taking from selected position”, “Move to conveyor”, and “Putting to conveyor”. In a simpler routine, the “Home X” and “Home Y” commands are executed through two subcommands, “Moving to home X” and “Moving to Home Y,” respectively. These commands were organized into subroutines. Due to the complexity of the program structure as well as a large number of subroutines, subroutines’ detail is not presented here. Simultaneously, as depicted in Fig. 4b, the conveyor system has three main groups of activities that operate independently and in parallel. The first is to control the delivery of pallets to the AGV inlet conveyor (AIC). To do this, the controller checks the pallet’s RFID code at each AIC; if the state of the corresponding RFID bit is ON, then the corresponding cylinder will be activated for one second to redirect the pallet to the corresponding AIC. The second group of control operations is to block pallets on the main output conveyor if there is a risk of collision with the pallet coming out of the AGV outlet conveyor (AOC). The controller checks the status of seven optical sensors mounted on the main output conveyor prior to the AOCs to perform this task. Suppose the kth AOC (i.e., k = 1 to 7) is transporting the pallet, and the corresponding kth optical sensor is activated. In that case, the corresponding cylinder is activated to block the pallet until the kth AOC stops working. Finally, the third control operation is the control of the separator unit at the input of the main feeding conveyor to moderate the number of pallets available on the main feeding conveyor. The success of the corresponding codes demonstrated the feasibility of these flowchart algorithms.

Multilayer Communication-Based Controller Design for Smart …

(a)

351

(b)

Fig. 4 Control flowchart of a AGV and b conveyor system

Implementation results Figure 5 depicts the AGV’s controller components. As shown, these essential devices were designated from reputable manufacturers; consequently, these devices ensure the system’s designed operating features such as good communication over Ethernet standard, high-speed pulse output function to drive AC servo and stepper motors work properly, the RFID codes read/write function is stable; the pallet circulation of the conveyor system works well. Figure 6 illustrates the WMS graphic user interface (GUI). The WMS application initially achieved the set goals in terms of basic tasks such as receiving orders, assigning codes to packages according to basic rules (for example, near to first, far to

Fig. 5 AGV controller components

352

N.-H. Le et al.

Fig. 6 E-WMS interface

last, and goods of the same company prefer to arrange in one rack), marking where the cells are occupied or pending or empty, and performing basic statistics.

5 Conclusion and Future Work The paper described in detail how a smart warehouse testbed works in practice. The system consists of a warehouse with seven racks, a pallet circulation system, AGVs that move pallets, and an RFID unit (reader/writer/tag) that manages the necessary information of the pallets. The design of an ICS, including motors, RFID, PLCs, sensors, cylinders, etc., and the flowchart of the control algorithm of the main objects (AGVs, conveyors, and WMS) is clearly described. The system has completed a rack system that includes all the components of a typical four-layers ICS. A WMS software has also been conducted to control all smart warehouse operations. The WMS software accomplished all basic tasks such as receiving orders, assigning management codes to packages according to basic rules of arrangement, marking where the cells are occupied, pending, or empty, and performing basic statistics. Initial results show that ICS configuration (multilayer controller) is optimal, leading to accurate control, and data collection as expected. The system will be completed with the hardware system of a full testbed as well as the ability to optimize the operational management of WMS using AI applications as soon as the epidemic situation is under control.

Multilayer Communication-Based Controller Design for Smart …

353

Acknowledgements This research is financially supported by Eastern International University, Binh Duong Province, Vietnam.

References 1. L.C. Blancas, Rapid growth, Challenges and Opportunities in Vietnam’s Logistics Limited Connectivity (2014) 2. World Bank, Trade and Transport Facilitation Assessment: A Practical Toolkit for Country Implementation. World Bank Study (2010) 3. P. Boynton, Summary Report—Measurement Challenges and Opportunities for Developing Smart Grid Testbeds (Maryland, 2015) 4. O. Salunkhe, M. Gopalakrishnan, A. Skoogh, Å. Fasth-Berglund, Cyber-physical production testbed: literature review and concept development. Procedia Manuf. 25, 2–9 (2018) 5. V. Kaczmarczyk, O. Baštán, Z. Bradáˇc, J. Arm, An industry 4.0 testbed (self-acting Barman): principles and design. IFAC-Papersonline 51(6), 263–270 (2018) 6. M. Dhouioui, T. Frikha, in Intelligent Warehouse Management System. 2020 IEEE International Conference on Design and Test of Integrated Micro and Nano-Systems (DTS) (2020), pp. 1–5 7. J. Yang, in Design and Study of Intelligent Warehousing System Based on RFID Technology. 2019 International Conference on Intelligent Transportation, Big Data and Smart City (ICITBS) (2019), pp. 393–396 8. T. Adiono, H. Ega, H. Kasan, C.S. Harimurti, in Fast Warehouse Management System (WMS) Using RFID Based Goods Locator System. 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE) (2017), pp. 1–2 9. D. Thangavel, X. Ma, A.C. Valera, H. Tan, C.K. Tan, Performance evaluation of MQTT and CoAP via a common middleware. IEEE ISSNIP 1–6 (2014)

Development of DeepCovNet Using Deep Convolution Neural Network for Analysis of Neuro-Infections Causing Blood Clots in Brain Tumor Patients: A COVID-19 Post-vaccination Scenario Kunal S. Khadke Abstract COVID-19 outbreak is ones in hundred years’ experience for each human being around the globe. Frequent lockdowns and unpredictable days shaken whole world. While researchers are struggling for genome sequencing and understanding the changes occurring in virus sequences, on the other hand, common people are struggling to control their fear about the future in all aspects. After the successful research for vaccination, the medical experts analyzed the post-COVID-19 impacts on various health fronts like heart failures, thrombosis, impact on brain, and many more complications. Out of which, identification of post-COVID-19 impact on brain took more time to understand the exact way the virus is affecting because psychological behavior is the first symptom and that takes keen observation to suggest the possibility of neurological infections. But, by that time, the illness reaches to more serious complications. Also, post-vaccination evidence shows that the blood clot formations becoming a new challenge for brain tumor patients. The blood clots in nervous systems are so tiny that by MRI/CT, it is not possible to differentiate between cerebral fluids. Hence, it becomes necessary to operate patient immediately with a clear vision facility for blood clots. Hence, this paper suggests the new deep learning algorithm which can be a great solution for image analysis with high level of accuracy. The proposed deep CNN module further can be used as a software package for needle camera for robotic assisted surgery which in turn saves time for image analysis and direct location of tumor can be identified during live camera surgery. Keywords COVID-19 pandemic · Brain tumor · Convolution neural network · Deep learning · Thromboembolism · Segmentations · Post-vaccination

1 Introduction Various evidences reveal that the COVID-19 outbreak has serious physiological and sociable issues. Presently, there is a pervasive consciousness of uncertainness K. S. Khadke (B) Sri SatyaSai University of Technology and Medical Sciences, Sehore, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_30

355

356

K. S. Khadke

throughout the future and knowledge the fact that the outbreak is not yet over [1]. Many of us who all have committed our lives to fighting health issues understand that science is the key point to stopping this devastation; however, we are definitely conscious of the truth that the structure of the globe we live will be noticeably transformed by this outbreak [2]. Within various computational strategies, deep learning can be employed to perfect image analysis, amino acid construct prediction, as well as drug repurposing. Deep learning has likewise been employed in spread out forecasting for Epidemiology. In other words, deep learning solutions have further more potential to deal with many challenges interrelated to COVID-19 [3]. Medical imaging, just like X-ray as well as computed tomography (CT), can perform a significant part in the rapid generation of COVID-19 patients that can enable the on-time treatment of the patients. Convolutional neural network (CNN) can be employed for extraction of the features from chest X-ray images for the prediction [4]. Robotic surgery treatment can be a satisfactory alternative for brain tumor treatment, as robotaided surgery has more accuracy [5]. Various sufferers report operational issues after evident recovery from COVID-19. This medical condition is called as long-COVID. It has been hypothesized that this instability and trouble may be interrelated to the brain swelling linked with the neurotropism of SARS-CoV-2 [6–8]. Research noticed overlapping relationships among seriousness of infection within the acute COVID19, brain structure as well as function, and so intensity of depressive disorder and post-traumatic misery in survivors [9].

1.1 Background In year 2019–2020, the whole world was waiting for vaccine for COVID-19. But as of year 2021, many vaccines drives are in progress throughout world. Now, the postCOVID-19 complications are raising head. It is now proved that COVID-19 infection has been associated with hypercoagulability [10]. Thromboembolism means blood clot formation in blood vessels. As blood clots move in body, it may block heart veins, brain veins, etc., which can be life threatening if kept untreated. Many longCOVID patients are suffering with neurological complications where some of facing post-vaccination thromboembolism symptoms. At some extent, blood thinner medications can handle clots but in some cases, surgical session requires. Also, in case where patients with comorbidities have limitations to consume blood thinner need to undergo MRI/CT scans to take a decision about treatment. In case where patients already suffering with brain tumor, periodical brain MRI is needed.

1.2 Merits of Deep CNN for Robotic Surgery The proposed research is focusing on development of algorithm using convolution neural network (CNN) framework, which can be used as a software package to

Development of DeepCovNet Using Deep Convolution Neural Network…

357

interface robotic surgical needle camera to view brain micro-structure during the surgery. As discussed before, the blood clots formation is a serious issue for any brain tumor patient. It is necessary to analyze the clot size and path. These blood clots cannot be seen easily in MRI report and proposed method can identify the pixel level blood clot moves. Also, in case of COVID-19 patients with pre-existing brain tumor, conditions can be very serious, and only option remain is immediate surgery. Hence, proposed approach can be very useful for robotic surgery to identify brain tumor and classify the blood clots. Advantages of the proposed system for robotic surgery are: (1) (2) (3) (4)

Ongoing surgical assistance with more clear visibility of brain tumor. Precise classification of micro-structural lesions. In case of post-surgical MRI analysis, identification of tumor boundary is clearer. Proposed CNN will provide a speedy analysis by lowering the number of pixel clusters.

As discussed, worldwide research is going on to identify the impact on brain because of COVID-19 virus and few post-vaccination complications like blood clot formation. Hence, it is necessary to develop a system which can provide fast MRI analysis with more precision. In modern medical systems, robotic surgeries are more accurate, and various researches in the field of robotic software are in progress. Deep neural networks are being used for analysis, predictions, and drug discoveries. In case of manual image analysis, there is a possibility of human errors, and hence, automated tools for brain analysis are necessary. Deep CNN can be a very promising solution for accurate image analysis. The most important element is survival of brain tumor patient. In case of normal brain tumor patient, we can have historical MR image data or we can say it as a patient file is already under study. But, as discussed earlier, patients with brain tumor and with long-COVID symptoms and/or suffering with post-vaccination thromboembolism are at highest risk as to identify symptoms of thromboembolism by any patient takes time. The high risk is because of blood clots which are moving toward the heart of brain veins. As soon as such blood clots flow through tiny veins, there is the possibility of sudden strokes which can be life threatening. Looking at such health threats, it becomes necessary to develop more precise solution for brain tumor analysis. Deep CNN algorithm can provide more prominent brain tumor classification as well as tumor segmentation. Early identification of lesion is important to improve the survival rate of patient. Hyper-parameter tuning for dice scores (DSC), sensitivity, and positive prediction values (PPV) in deep CNN can be improved programmatically to optimize the image visibility. At each layer of the CNN, there are kernels or filters that are convolved with the image based on which further optimization can be done. By using deep CNN, minimal invasive robotic surgery is possible as needle camera captures the image which can be inserted at the exact boundary of tumor or near to the blood clot. This can be a good option for craniotomy surgery where hole is drilled in skull portion to drain the blood clot and/or tumor fluid [11]. Lately, the smaller accuracy in prediction models as well as

358

K. S. Khadke

crucial aspect of the therapeutic data evaluation compelled experts toward innovative strategies of brain tumor prognosis to boost classification precision [12]. The paper is organized as Sect. 2 presents the brief literature review for analyzing existing methods adopted for biomedical/neurological research using deep learning CNN architectures, Sect. 3 presents the proposed algorithm “DeepCovNet” in which the execution of algorithm is explained, Sect. 4 is results and discussion where proposed algorithm training and validation are discussed, and finally Sect. 5 concludes the research paper.

2 Related Work In this section, we present the recent research in the domain of deep learning and post-vaccination complications. Data science is proving as a speedier analysis tool in this pandemic to boost the decision-making. Scientists throughout the world are now involved with post-vaccination symptom analysis also, and line of action is being setup for future pandemic possibilities. As per [13], various patients with serious Corona virus disease 2019 (COVID-19) are unresponsive and are with surviving imperative health issues. Even though different structural brain malocclusions have been identified, their particular influence on brain function as well as consequences for analysis is being examined. As per [14], operational neuroimaging, which has prognostic relevancy, has yet to be investigated in this populace. Author described a patient with severe COVID-19 who, even though extended unresponsiveness as well as structural brain malocclusions, exhibited complete functional network functioning, and then weeks after recovered the potential to follow instructions. Research in [15] revealed that scientifically more significant, intense ischemic infarcts, as well as intracranial hemorrhage have been observed in all these patients. In [16], author recommended a brain tumor recognition approach employing edge detection-based fuzzy logic and U-NET. The recommended tumor segmentation system improves both visibility and accuracy of classification. Author utilized transformer in [17] that can assist from universal data modeling applying self-attention systems has been effective in natural language processing as well as 2D image classification recently. Even so, together, local as well as global aspects are important for dense prediction steps, specifically for 3D medical image segmentation. In this paper, author used transformer in 3D CNN for MRI brain tumor segmentation as well as proposed a novel network known as TransBTS structured on the encoderdecoder framework. In [18], the author applied fully convolutional neural networks for segmentation. Hard mining is carried out at the time of training to train for the complicated cases of segmentation steps by elevating the dice similarity coefficient (DSC) tolerance to select the hard cases as epoch boosts. As per recent review presented in [19], irrespective of all the features and performance of the COVID19 vaccines pointed out in latest clinical trials, several post-vaccination adverse effects just like lymphadenopathy (LAP) were noticed. This paper examined all analyses with imaging results presentation of LAP after COVID-19 vaccination. In

Development of DeepCovNet Using Deep Convolution Neural Network…

359

[20], author explained the lesions noticed on magnetic resonance imaging (MRI) of three patients who formulated neurological symptoms and signs subsequent to obtaining the AstraZeneca vaccination. This evidently proves the brain issues. As per rigorous literature study related to COVID-19 for post-vaccination, we found that it is now confirmed that in some cases, post-vaccination can lead to unpredictable results. The proposed research is the first which targeted the deep CNN for analysis of post-vaccination complications for brain tumor patients. As there is no dataset available for post-vaccination brain tumor patients, IIARD-PostV-2021 dataset is used for proposed analysis. For the analysis of patients with only brain tumor and no COVID-19 history, BRATS-2020 dataset is used.

3 Proposed Method As mentioned earlier, the proposed research can be a first analysis for postvaccination brain tumor patients; this section discusses the proposed algorithm. IIARD-PostV-2021 MRI dataset is obtained from IIARD repository which contains 78 patient data. The BRATS-2020 dataset is obtained from Kaggle. The proposed algorithm is developed with the motive to analyze the brain tumor segmentation as well as to identify most prominent blood clot flowing through veins. Steps for proposed algorithm “DeepCovNet” are mentioned below: Algorithm 1: DeepCovNet Input Dataset: BRATS-2020, IIARD-PostV-2021 Output: Segmented brain tumor image set and classification of T1, T2, and T1ge images and location of most prominent blood clot (1) (2)

Register the MRI/CT images for patients with post-vaccination status. Input labeled training data of BRATS-2020 dataset as [HGG], [LGG], and [Combined] for segmentation and IIARD-PostV-2021 datasets for blood clot analysis and comparative analysis. (3) Process raw training data by converting MR images into “.nii” format using MATLAB. (4) Extract feature vectors and map to high-dimensional space. (5) Apply Gaussian distribution and form a cluster of white and gray pixels. (6) Apply convolution layers with constant stride 2 for HGG, LGG, and combined dataset. (7) Set and execute max-pooling layers to tenfold first for segmentation and later for segmented tumor portion to locate blood clot. (8) Apply ReLU layer filter to gray pixels cluster for extraction of blood clot boundary by storing pixel count. (9) Initiate de-convolution first for segmented tumor blood clot and then add check points. (10) Store the segmentation and classified T1, T2, and T1ge images for further augmentation.

360

(11) (12) (13) (14)

K. S. Khadke

Set epoch size to 100. Conduct training and validation with epoch 100 on GPU. Compute the performance of hyper-parameters—DSC, sensitivity, and PPV. Compare the performance for complete, core, and enhanced brain tumor for HGG, LGG, and combined dataset.

For development and execution of proposed system hardware used as Windows 10, GPU machine with 12 GB memory size. Algorithmic coding was done by using Python 3.6 version. The dataset used is BRATS-2020. The dataset contains 369 samples and 4 slices for pre-operative conditions [21]. Figure 1 shows the flow of execution for proposed research “DeepCovNet” algorithm. For any dataset, image registration is a very crucial step where patient data is collected for development of (image/text) dataset. As discussed earlier, COVID19 brain MRI database is still under development and in case of post-vaccination of brain tumor patients, there is no dataset available. Hence, for proposed analysis, IIARD-PostV-2021 dataset used which comprised of 79 patients who are suffering with brain tumor/neurological complications. For training and testing, BRATS-2020 dataset, which was obtained from Kaggle, was employed. The proposed algorithm is executed by convolution neural network. The registered images are MR images and for input to deep CNN compressed file format required is “.nii” hence, as a step to pre-process the images conversion of MRI to “.nii” is done with MATLAB module programming. Further, algorithm initially processed the feature extraction of brain tumor which subsequently given as input to convolution layer formation with stride size 2. Following to convolution for extracted tumor, a second convolution is applied for identification of blood clots in tumor. At optimum max-pooling, the de-convolution is executed in a reverse manner, as the first blood clot pixel cluster is de-convoluted and later the whole brain tumor is de-convoluted. By, two-stage convolution and de-convolution, we got two checkpoints where visibility of tumor and visibility of blood clot are clearer. After above process, training and validation was done for BRATS-2020 dataset to get classification results for HGG and LGG for post-COVID-19 samples. The comparison of blood clot identification is conducted for non-vaccinated and post-vaccinated patient data using both datasets. This is the first study in the field of Deep CNN to analyze the post-vaccination analysis for brain tumor patients. The next Sect. 4 reveals the results.

4 Results and Discussion As discussed in a previous section of this paper, we conducted training and validation for proposed algorithm DeepCovNet using deep CNN framework. The recurring convolution and de-convolution give better results with constant max-pooling layer count. The data training is conducted for HGG, LGG, and lastly for combined dataset. The values of hyper-parameters are enhanced with the proposed algorithm. Similarly,

Development of DeepCovNet Using Deep Convolution Neural Network… Fig. 1 Schematic representation of the proposed framework

361

362

K. S. Khadke

the validation of dataset is executed. As the segmented tumor (as shown in Fig. 2) is extracted in first level of convolution the first checkpoint is generated and further, the second level of convolution with strides 2 with constant max-pooling values executed for blood clot identification as shown in Fig. 3. Thus, proposed algorithm is able to show the gray pixel cluster of blood clot in brain tumor which is very useful for robotic needle camera surgery as well as useful for decision-making for surgical line of action. Based on training and validation conducted, Table 1 depicts the dataset analysis results. The performance of the proposed algorithm is evaluated on the basis of hyperparameter values as given in Table 2. As the identification of blood clot is important, the DSC is 0.91 and sensitivity obtained in 0.92, which reveals that the visibility is better with the proposed algorithm. The proposed algorithm performance is compared with the existing system in [22] and comparative analysis is given in following Table 3. As per the hyper-parameter comparison, it is proven that the proposed DeepCovNet algorithm performance is better to identify blood clots from brain MR image and also segmentation and classification of T1, T2, and T1ge provides more accurate results with epoch size 100. Figure 4 shows the percentage of segmentation done for BRATS-2020 dataset with novel CNNPostCoV algorithm. The training epochs are 100 for both training and validation of brain tumor images for identification of blood clots.

Fig. 2 Proposed algorithm MR image segmentation output

Fig. 3 Blood clot identification in extracted tumor using proposed algorithm

Table 1 Training and validation setup

Targeted data

Training set

Validation set

HGG

259

49

LGG

110

29

HGG + LGG

369

49 + 29 = 78

Development of DeepCovNet Using Deep Convolution Neural Network…

363

Table 2 Performance comparison of dataset Dataset

Method Proposed system

Xue et al. [22]

Percentage of segmentation

0.9

Sensitivity Core

Enhanced

Complete

Core

Active

HGG

0.91

0.96

0.14

0.92

0.90

0.91

LGG

1.82

3.66

1.56

1.52

3.60

1.74

Combined

0.93

2.78

2.09

0.91

2.76

2.17

HGG

0.89

0.95

0.15

0.87

0.90

0.21

LGG

1.78

3.54

1.69

1.45

3.58

1.86

Combined

0.97

2.38

2.35

0.94

2.78

2.23

Table 3 DeepCovNet comparison analysis

1

DSC Complete

Hyper-parameters

Proposed system

Existing system [22]

DSC

0.91

0.83

Sensitivity

0.92

0.86

PPV

0.92

0.84

0.91

0.92

0.92

0.8

Proposed System (%)

0.7 0.6

Existing system

0.5 0.4 0.3 0.2 0.1 0 DSC Sensitivity PPV Output hyper parameters

Fig. 4 Performance comparison between proposed system and existing system

As shown in Fig. 5, the execution time for whole, LGG and HGG dataset, is recorded on GPU. As preprocessing for MR images is done separately using MATLAB module, the processing time for training and validation is lowered. The comparison is done between BRATS-2020 and IIARD-PostV-2021 post-vaccination patient dataset. As a number of samples are more in BRATS-2020, the time taken for execution is more. Otherwise, both datasets are able to execute in same speed.

364

K. S. Khadke

9 8

GPU Execution Time (Hrs)

8 7 BRATS-2020 Dataset

6 5 4

3.5

3.9

IIARD-PostV2021 Dataset

3 2 1 0

Whole LGG Dataset Dataset Type of Data Cluster

HGG Dataset

Fig. 5 Dataset comparison

5 Conclusion As long-COVID and post-vaccination health observations are new challenges, the proposed system is able to handle issue of thromboembolism, i.e., blood clot formation. The key challenge for medical professionals is to come to know that patient is suffering with thromboembolism because it does not show many symptoms and slowly blood clot formation reaches to block the veins which further cause the strokes following life threatening incidences. Hence, periodical brain MRI is essential. As identification of blood clots is much more expensive process, the proposed CNNPostCoV algorithm can be a very cheap option. Results of training and validation of BRATS-2020 revealed that the hyper-parameter performances are more accurate as compared to the existing system, which is the result of two-staged convolution and de-convolution of brain tumor and for blood clot. The classification of T1, T2, and T1ge holds significance in line with medical treatment decision, which can improve the survival of patient by understanding at which stage blood clots are being produced. The research can further be developed for pre-vaccination and post-vaccination dataset development for brain tumor patients as well as healthy people.

Development of DeepCovNet Using Deep Convolution Neural Network…

365

References 1. L. Sher, The impact of the COVID-19 pandemic on suicide rates. QJM: Int. J. Med. 113(10), 707–712 (2020) 2. S. Filetti, The COVID-19 pandemic requires a unified global response. Springer Endocrine 68(1), 020–022 (2020) 3. C. Shorten, T.M. Khoshgoftaar, B. Furht, Deep learning applications for COVID-19. Springer, J. Big Data 8(1), 1–54 (2021) 4. M. Umer, et al., COVINet: a convolutional neural network approach for predicting COVID-19 from chest X-ray images. J. Amb. Intell. Human. Comput. 1–13 (2021) 5. M. Kocher, et al., Applications of radiomics and machine learning for radiotherapy of malignant brain tumors. Springer, Strahlentherapie und Onkologie 196, 856–867 (2020) 6. E. Guedj, et al., 18 F-FDG brain PET hypometabolism in patients with long COVID. Eur. J. Nucl. Med. Mol. Imag. 1–11 (2021) 7. F. Benedetti, et al., Brain correlates of depression, post-traumatic distress, and inflammatory biomarkers in COVID-19 survivors: a multimodal magnetic resonance imaging study. Brain Behav. Immun.-Health 100387 (2021) 8. J. Zhang, et al., Implementation of a novel Bluetooth technology for remote deep brain stimulation programming: the pre–and post–COVID-19 Beijing experience. Movement Disorders (2020) 9. B. Oronsky, et al., A review of persistent post-COVID syndrome (PPCS). Clin. Rev. Allergy Immunol. 1–9 (2021) 10. G.P. Castelli, et al., Cerebral venous sinus thrombosis associated with thrombocytopenia postvaccination for COVID-19. Critical Care 25(1), 1–2 (2021) 11. H. Mzoughi, et al., Deep multi-scale 3D convolutional neural network (CNN) for MRI Gliomas brain tumor classification. J. Dig. Imag. 33, 903–915 (2020) 12. N. Noreen, et al., A deep learning model based on concatenation approach for the diagnosis of brain tumor. IEEE Access 8, 55135–55144 (2020) 13. D. Fischer, et al., Intact brain network function in an unresponsive patient with COVID-19. Ann. Neurol. 88(4), 851–854 (2020) 14. A. Bhattacharya, et al. Predictive analysis of the recovery rate from coronavirus (COVID-19). in Cyber Intelligence and Information Retrieval eds by J.M.R.S. Tavares, P. Dutta, S. Dutta, D. Samanta. Lecture Notes in Networks and Systems, vol. 291. Springer, Singapore. https:// doi.org/10.1007/978-981-16-4284-5_27 15. A. Radmanesh, et al. Brain imaging use and findings in COVID-19: a single academic center experience in the epicenter of disease in the United States. Am. J. Neuroradiol. 41(7), 1179– 1183 (2020) 16. S. Maqsood, D. Robertas, M.S. Faisal, in An Efficient Approach for the Detection of Brain Tumor Using Fuzzy Logic and U-NET CNN Classification. International Conference on Computational Science and Its Applications. Springer, Cham (2021) 17. W. Wang, et al., in Transbts: Multimodal Brain Tumor Segmentation Using Transformer. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham (2021) 18. V.K. Anand, et al., in Brain Tumor Segmentation and Survival Prediction Using Automatic Hard Mining in 3D CNN Architecture. International MICCAI Brainlesion Workshop. Springer, Cham (2020) 19. P. Keshavarz, et al., lymphadenopathy following covid-19 vaccination: imaging findings review. Academ. Radiol. (2021) 20. D.G. Corrêa, et al., Neurological symptoms and neuroimaging alterations related with COVID19 vaccine: cause or coincidence? Clin. Imag. 80. 348–352 (2021) 21. N. Sohail, et al., Smart approach for glioma segmentation in magnetic resonance imaging using modified convolutional network architecture (U-NET). Cybern. Syst. 1–16 (2020) 22. J. Xue, et al., Hypergraph membrane system based F2 fully convolutional neural network for brain tumor segmentation. Appl. Soft Comput. 94, 106454 (2020)

Machine Learning Models to Predict Girl-Child Dropout from Schools in India Annapurna Samantaray, Satya Ranjan Dash, Aditi Sharma, and Shantipriya Parida

Abstract Good education is the foundation of developing nation. Government and number of NGOs are continuously working to spread awareness about importance of education. Smile foundation’s “Each One Teach One” teaching philosophy is one such initiative. The paper proposes a model to predict dropout rate among school girls, which will further help school authorities, government agencies and NGOs to reduce school dropout at an early stage. For prediction, two machine learning techniques, Naïve Bayes classifier and logistic regression model, have been compared. Keywords Dropout · Education · Bayesian classification · Logistic regression

1 Introduction “Girl education is a way to develop the nation”. Girl-child dropout from schools has been a major social concern in India since ages. Different social and economical constraints directly or indirectly influence dropout rate among students [1–5]. The purpose of this study is to establish a predictive model to assess the dropout rate among school girls. Comparing the relative importance of different directly influencing factors such as family issues, problem with teachers, lack of interest in studies, gender discrimination [6] or indirectly influencing factors such as demographic or background A. Samantaray Indraprastha Institute of Information Technology, Delhi, India S. R. Dash (B) School of Computer Applications, KIIT University, Bhubaneswar, India e-mail: [email protected] A. Sharma Jaypee Institute of Information Technology, Noida, India S. Parida (B) Idiap Research Institute, Martigny, Switzerland e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_31

367

368 Table 1 Weight of different factors

A. Samantaray et al. Factors

Weight values (θ 1 , … , θ 7 )

Proximity of school

0.229

Financial burden

0.615

Household work

0.185

Parents do not value education

0.134

Students not interested in studies

0.149

Problem with teachers

0.057

Lack of basic amenities

0.067

factors, lack of Infrastructure, and basic amenities [7] such as mentioned in Table 1, a questionnaire-based survey is conducted among school girls. This paper analyzes two machine learning techniques, namely Bayesian classification and logistic regression model. Within the dataset gathered from questionnaire-based survey, cases were randomly assigned to training and testing datasets. Same testing dataset was used to test the accuracy of both the models. Classification methods are used to predict binary class target variables. Both Bayesian and logistic regression are linear classifiers. Here the dependent variable is binary. So classification and regression are two machine learning techniques used for prediction, where regression predicts a value from a continuous set, whereas classification is a supervised learning technique which gives the known dataset or relationship predicts the belonging to the class.

2 Related Work The number of different factors leading to dropout has been studied in the last few years. Though India is one of the fastest growing economies in the world, larger section of Indian population is still poor. Agriculture has been the major source of income in the Indian economy, and a sizeable amount of population works as a manual labor for their living. Agricultural work is seasonal; the families need to migrate frequently to different cities and states, thus forcing girls to drop out from school [8, 9]. Even if these girls join school in the migrated state, due to language difference, they need to struggle a lot to cope up with their studies. The communication gap plays a crucial role in increasing the dropout rate. Also migrant girls are admitted to schools according to their age, even if did not go to school previously. Thus, they lack behind local students, thereby losing interest in studies. A Danish study considering equal dataset of pupils for training and testing evaluated different machine learning techniques [10] to predict the dropout in the coming three months among Danish high school pupils. Authors observed that random forest classifier performed best among all with an accuracy of 93.5%, followed by SVM, CART and Naïve Bayes which achieved the minimum accuracy.

Machine Learning Models to Predict Girl-Child Dropout from Schools…

369

Alexander discusses background characteristics [11] family factors [6] such as parents’ attitude and values and socialization practices; child’s attitude toward self and school including school experiences such as tests, marks, and his academic performance; and engagement behaviors of child such as TV time. Author implemented logistic regression analysis to predict the dropout rate. Neighborhood plays an important role in the sole foundation of a child’s personality. Author in [12] discusses effect of different neighborhood on school dropout using sensitivity analysis. It was observed that students living in area with high poverty are likely to drop out and have teenage pregnancy as compared to those living in the areas of low poverty. This paper describes how family members (financial issues, girls migration), villages environment (social issues: child marriage, importance of education for girls), and school-related issues (poor education, harassment by teacher/classmate) together lead to a vital reason for girl’s school dropout and absenteeism in Karnataka. Because of continuity of depressive activity in childhood, girls may be dropped school or suspended or expelled. Using longitudinal analysis over social issues, school problems and delinquency with young adulthoods find risk of depression in adulthood [13]. School engagement is a major factor for academic career as well as personal development. This paper has considered some attributes (family, peers, school, age, sex) for analysis that depend on which school engagement measured. For successful school engagement, the percentage of dropout is less. The analysis also shows that for girls, peers support and emotional engagement are high than boys who lead to successful school engagement [14]. The cultural values and beliefs have negative impact on girl-child education in Africa. Some incidents like girls used as wealth, child marriage and getting unwanted pregnancy, wife replacement and many more show how girls are abused by culture as well as reason for girls’ school dropout [15]. This study shows that in Turkey, high rate of physical and mental abuse and neglect within the family is a major cause for children school dropout. Economic abuse means forcefully taken away from school for family support, which is also a reason for children school dropout [16]. In Cooch Behar district of West Bengal, Muslim girls are dropped out because of education of mother, family income, help in household activity, social prohibition, financial problem, home environment and some critical thinking of parents (problem in marriage, denials to religious intensity, conflict with traditional values) about primary education [17]. In Zimbabwe also, girls are dropped out for family problems, personnel illness and security fears because of long distances to school, financial problem for deposit school fees, early marriage and getting pregnancy. Some prevention is also recommended as follows: Girls are not getting marriage before age of 18 years, and education for all (EFA) policy should follow [18]. The factors like push, pull and falling out provide a scheme for dropout. Push factor related to school activity means fail or expel in exam, most of time absent in school. Pull factor is related to jobs and family (married and getting pregnant, financial difficulties, jobrelated issues, poor health). Falling out factor contains your impression about school and schoolmate (did not like school, moved to another city, what classmate thought about you). No Child Left Behind (NCLB) Standard may decrease the rate of dropout [19].

370

A. Samantaray et al.

This paper gives some basic reasons for girl’s dropout at the age of 7–12 years such as family problem, parents education, busy in domestic work to support family, busy in household activity, health problem, did not like school, fail in exam and parents interest toward education. If living style of family is raised, then only rate of girl’s dropout decreased [20]. This paper emphasizes prevention on girl’s dropout from school. Girl-child and women education and empowerment can achieve national evolution [21]. Because of education, women become self-confident, take their own decision and work in workforce and family income increase that affect children’s education, health and nutrition. So there is no financial problem, no parent problem and no household activity which are reasons of dropout from school [22]. This study gives some measure factors which are reasons for high dropout in Kenya (Siaya County) such as fishing, poverty, parental education, lack of sanitary facilities, domestic work, child marriage and getting pregnant, lack of influential women, poor result in exam and negative vibes from dropped out students [23]. In Nigeria, child marriage and adolescent pregnancy, religious misbelieve, some negative rules of culture and poverty are major reasons for which rate of girl’s dropout increase. This paper also focuses on how libraries and information centers improve girl-child education and also handle the problems like religious misbelieve, insufficient role models, child marriage and adolescent pregnancy [24]. The study follows a significant approach of research design and operates questionnaire in assembling data. There were eighty participants for this survey. Girl’s education is influenced by watta satta exchange, marriage engagement and preferential marriage being types of arrangements in Bolni. The effects of such arrangements were unfavorable to their ability to approach school or enroll in school, as a more no. of girls were included in a marriage arrangement that tend to increase in no. of girls dropping out from school. The study proposed simple regression analysis to calculate the correlation between marriage arrangements and girl’s education. It is suggested that parents should be educated to be able to detach from such arrangements and focus on girl’s education. This can be facilitated by school’s social workers, gender activists and community workers [25]. The paper inculcates reasons for girls dropping out of school. These are diverse, ranging from costly fee structure of education, lack of educational amenities, family responsibilities and poverty. Some factors can be considered as sign of threat for dropping out as a result of interplay (interplay option-interaction, coaction) of family circle and school surroundings from some scenarios. These factors of threat are as follows: low educational status of the parents, single-parent family, lack of participation in the class, lack of parental involvement in schooling, improper socialization, grade detention and repetition, inadequate school academic atmosphere, child labor within households, low selfesteem, lack of parental academic support, low economic status of the family/poverty low confidence level and lack of teachers’ support for better learning. Thus, from these factors, it is evident that dropout was not the only event rather one of many events that contributed to circumstances leading into dropouts. From the Ministry of HRD department, we collected some information regarding girl’s dropout. In elementary classes (I–VIII), 26.75% girls dropped out in 2006–07 as well as 52.9% girls dropped out in 2003–04. Similarly, in secondary classes (I–X), 61.5% of girls dropped out

Machine Learning Models to Predict Girl-Child Dropout from Schools…

371

in 2006–07 as well as 64.9% of girls dropped out in 2003–04.Bihar, and Jammu and Kashmir, Orissa, Rajasthan, Uttar Pradesh and West Bengal have highest girl’s dropout rate than other states in India [26]. This paper discussed some issues such as economic, insecurity, cultural and school-related challenges that are considered as the major threats for girl’s education in Afghanistan. Early marriage is considered as a major factor for dropping out of a girl child from school. The factors have been categorized into five main segments: (1) school level factors, (2) poverty, (3) insecurity, (4) cultural factors and (5) overall education quality. Again such segments have been subdivided into other sections. School level factors consider other issues such as school distance, inadequate infrastructure and shortage of female teachers, quality of education, qualified teachers and teacher attitudes toward students (scolding, bad language used by teacher, battering). Similarly, poverty considers other issues such as financial need of the family, less interest of both parents and children in education, household works, the death of the parents, and parents preference toward boys. Harassment is the insecurity in the country. Cultural factors consider some issues such as religion, cultural belief and early or forced marriage [27]. Factors leading to dropout of girls from schools, child abuse, female foeticide, child marriage, female infanticide and child labor are evaluated to encourage girl-child education. The girl child has been dispossessed of her right to proper education by gender socialization, sociocultural, health issues and economical structure. This paper shows how household characteristics and family influence are reasons for school dropout of girls. Also, he noticed that increase in family size means more number of siblings has been measured as predictor of dropping out of girls. Also, some analysis shows that father’s qualification is an important cause to the dropout case. In this survey, the identified reasons are categorized into three parts such as reasons associated with children, reasons associated with school and reasons associated with the household [28]. This paper explains determinants of university dropout in Thailand. The analysis based on some factors like admission type, faculty, admission year, first semester GPA and gender religion in academic years (2007–2011) at Prince of Songkla University, Pattani campus. Here the logistic regression model is used to find out most effective variables on dropout. The model shows the high dropout rate with a low GPA [29]. This paper uses some supervised learning classifiers on Tanzania datasets which has some features. The gender feature has the highest impact on the dropout problem. The classifiers are logistic regression classifier, Knearest neighbors, neural network models associated with multilayer perceptron and ensemble models associated with random forest. The logistic regression classifier performs higher performance than other classifiers [30]. In IGNOU, a huge number of students are taking admission for higher education, but not complete their study. After doing survey, some factors are pointed out which are the reasons of dropout. The reasons are financial problems, social obligations, professional reasons, health problems and some extrinsic problems [31].

372

A. Samantaray et al.

3 Methodology This section explains the general framework as shown in Fig. 1 of the paper; dataset used subsequently discusses the two machine learning techniques compared. The approach follows a four-step process: training dataset, implementation of Bayesian classifier, implementation of logistic regression and finally performance evaluation of the said two techniques. Here same training dataset is used by both Bayesian classifier and logistic regression. In Bayesian classification, the dataset is used by specified algorithm and creates a lookup table. This lookup table is used by testing dataset and by a new entry to provide appropriate classification. According to the requirement of the logistic regression, format of the dataset is changed. This regression method used binary format of data for their calculation. Based on calculation result and threshold setting, data are classified. Then results of both techniques are compared by using confusion matrix.

3.1 Dataset The first step consists of collecting training dataset by inspection for the purpose of training the classifier. To have a clear view about the constraints, questionnaire-based survey was conducted among school girls. Girls who completed at least one year of their schooling were asked to fill this survey, so as to get their academic records as well, which can further bring light to the fact that “whether student is interested in studies or not”, “Academic level is too high or too low”, etc. After a thorough study of all the factors, we selected following seven important features as given in Table 1 which have the highest impact on the dropout rate.

Fig. 1 General framework

Machine Learning Models to Predict Girl-Child Dropout from Schools…

373

Fig. 2 Weight of different features

Figure 2 shows weight of different features mentioned in our study. X and Y axes represent features name and weights range, respectively. It shows Financial_Burden has the highest priority than others, and Problem_with_Teachers and Lack_of_basic_amenities have least priorities than others. The Intercept term (θ 0 ) = −0.322

3.2 Bayesian Classifier The Bayesian classifier [18] assumes that the attributes are conditionally independent and thereby estimating the class-conditional probability provided the class label C k . In our study, it is a two-class classification problem: C 1 , C 2 for yes (dropout), no (no dropout), respectively, this assumption can be stated as follows: P(Y |C1 ) =

m 

P(Yi |C1 )

(1)

P(Yi |C2 )

(2)

i=1

P(Y |C2 ) =

m  i=1

where every attribute set Y = {Y 1 , Y 2 , …, Y m } consists of m attributes. Here m = 8. For this, we first find out probability of each class label.

374

A. Samantaray et al.

P(C1 ) = no. of record ∈ C1 /n

(3)

P(C2 ) = no. of record ∈ C2 /n

(4)

where n is the no. of training records. We can now estimate the conditional probability of every Y i given C k where k = 1, 2. To classify a test record Y, probability for every class C k is as follows: P(Ck |Y ) = P(Ck )

m 

P(Yi |Ck )/P(Y )

(5)

i=1

Since P(Y ) is same for all class label C k , that means here we find out posterior probability as follows: P(C1 |Y ) = P(C1 )

m 

P(Yi |C1 )

(6)

P(Yi |C2 )

(7)

i=1

P(C2 |Y ) = P(C2 )

m  i=1

P(C1 |Y ) > P(C2 )?Y ∈ C1 : Y ∈ C2

(8)

Based on P(C 1 | Y ) and P (C 2 | Y ) values, we can classify the test record Y, i.e., predict to which class C 1 or C 2 it belongs. That means if P (C 1 | Y ) is greater than P (C 2 | Y ), then test record Y belongs to C 1 , otherwise Y belongs to C 2 .

3.3 Logistic Regression Logistic regression [19] is a statistical modeling technique where the output is binary (i.e., of the form yes or no otherwise 1 or 0). Our problem is a binary problem where we can predict whether a student is likely to dropout or not (based on features and its weight). Depending on such results, we can evaluate accuracy, precision and recall. Logistic regression technique is derived from logistic function f (z) which is as follows: z = θ0 + θ1 X 1 + .... + θn X n f (z) =

1 1 + e−z

(9) (10)

Machine Learning Models to Predict Girl-Child Dropout from Schools…

0 T 0 , it predicts that the student will drop out from school in near future.

5 Performance Estimation To estimate the potency of the proposed technique, precision and recall are used. Precision and recall are calculated by using actual and predicted samples that are highlighted in Table 2. For this estimation, standard confusion matrix is used. Each column of this matrix shows predicted samples, and each row of this matrix shows actual samples. Table 2 show the standard confusion matrix. Here TP (True Positive) = the no. of dropout students are correctly classified, also depicted as dropoutstudents in actual. TN (True Negative) = the no. of students who didn’t dropout are correctly classified, also depicted as didn’t drop out students in actual. FP (False Positive) = the no. of students who didn’t dropout are incorrectly classified, as in actual these students dropped out. FN (False Negative) = the no. of students who dropped out are incorrectly classified, as in actual they didn’t dropout.

Table 2 Standard confusion matrix

Predicted Actual

Dropout

Not dropout

Dropout

TP

FN

Not dropout

FP

TN

378

A. Samantaray et al.

TPR (True positive rate/Recall) = TP/(TP + FN) FPR (False positive rate) = FP/(FP + TN) Precision = TP/(TP + FP) Accuracy = (TP + TN)/(TP + FP + TN + FN) Accuracy shows the effectiveness of the proposed approach. It depends on true positive and false negative values. Accuracy of Bayesian classifier is approximately 86%, and logistic regression is approximately 95.5%. Tables 3 and 5 show confusion matrix of Bayesian classifier and logistic regression, respectively. It presents the number of TP, TN, FP and FN values which are evaluated using our approaches. Tables 4 and 6 show performance results of Bayesian classifier and logistic regression, respectively. Results are normalized between 0 and 1. Precision is used as a measure of quality, whereas recall is used as a measure of quantity. Precision and recall are higher in logistic regression technique than Bayesian classifier. FPR is low in both cases. The performance of the classifier can be visualized by receiver operating characteristics (ROC) curve. It shows the relationship between recall and false positive rate. Figure 3 represents the ROC curve of logistic regression. X and Y axes represent the FPR and TPR, respectively. Here we see curve has started bending toward 1 at 0.59. So we take 0.59 as a threshold value in our algorithm (logistic regression). Figure 4 shows performance of logistic regression. Here the X-axis represents various threshold (T 0 ) settings (0.5–0.7), and Y-axis represents range (0–1). Accuracy, FPR and precision are represented by different colors. It shows, with increase in threshold, accuracy and precision also increase but FPR decreases, which indicates Type-1 error is decreased and the chance of getting better performance is increased. Table 3 Confusion matrix of Bayesian classifier Predicted Dropout Actual

Not dropout

Dropout

28 (TP)

19 (FN)

Not dropout

9 (FP)

144 (TN)

Table 4 Performance results of Bayesian classifier

TPR/Recall

FPR

Precision

Accuracy

0.5957

0.0588

0.7567

0.86

Table 5 Confusion matrix of logistic regression

Predicted Actual

Dropout

Not dropout

Dropout

34 (TP)

2 (FN)

Not dropout

7 (FP)

157 (TN)

Machine Learning Models to Predict Girl-Child Dropout from Schools…

379

Table 6 Performance results of logistic regression TPR/Recall

FPR

Precision

Accuracy

0.944444

0.042683

0.829268

0.955

Fig. 3 ROC curve of logistic regression

It also indicates maximum precision tends to less FP (in case of maximum precision, algorithm returns more relevant results than irrelevant results.). So, as the precision increases, FPR decreases. In between 0.56 and 0.6, we get maximum accuracy.

6 Conclusion Women empowerment has always been the most crucial social issue for decades. The Government of India keeps on building up schemes and strategies to empower women, and the year 2001 was declared as the Year of Women’s Empowerment. As far as the women empowerment is concerned, educating a girl child is the foundation to do so. Girls in India are likely to drop out of school more often at an early age due to a number of social and economic factors. This paper proposes a method to predict the likelihood of a student to drop out of school, thus can help the government agencies to respond faster and adapt appropriate measures. Two machine learning algorithms, namely Bayesian classifier and logistic regression, have been compared to predict the most accurate dropout rate. Bayesian classifier is a generative model, whereas logistic regression is a discriminative model. After implementation of the two models, it was observed that overall logistic regression with accuracy of 0.955

380

A. Samantaray et al.

Fig. 4 Graphical representation of performance of logistic regression

more accurately predicts the dropout rate than the Bayesian classifier having accuracy of 0.86.

References 1. M. Kumar, A.J. Singh, D. Handa, Literature survey on educational dropout prediction. Int. J. Educ. Manage. Eng. 7(2), 8 (2017) 2. R.W. Rumberger, Why Students Drop Out of School and What Can Be Done (University of California, 2001), pp. 1–53 3. J.D. Teachman, K. Paasch, K. Carver, Social capital and dropping out of school early. J. Marriage Family 773–783 (1996) 4. R.W. Rumberger, Dropping out of high school: the influence of race, sex, and family background. Am. Educ. Res. J. 20(2), 199–220 (1983) 5. K. De Witte, S. Cabus, G. Thyssen, W. Groot, H.M. van Den Brink, A critical review of the literature on school dropout. Educ. Res. Rev. 10, 13–28 (2013) 6. J. Burrus, R.D. Roberts, Dropping out of high school: Prevalence, risk factors, and remediation strategies. R & D Connections 18(2), 1–9 (2012) 7. E. Begu, Female School Dropout: Gender Differences in Students’ School Retention (2014), pp. 1–31 8. R. Prakash, T. Beattie, P. Javalkar, P. Bhattacharjee, S. Ramanaik, R. Thalinja, ... M. Collumbien, Correlates of school dropout and absenteeism among adolescent girls from marginalized community in north Karnataka, south India. J. Adoles. 61, 64–76 (2017) 9. S. Rovira, E. Puertas, L. Igual, Data-driven system to predict academic grades and dropout. PLoS ONE 12(2), e0171207 (2017) 10. N.B. Sara, R. Halland, C. Igel, S. Alstrup, in High-School Dropout Prediction Using Machine Learning: A Danish Large-Scale Study. ESANN 2015 Proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence (2015), pp. 319–24

Machine Learning Models to Predict Girl-Child Dropout from Schools…

381

11. K.L. Alexander, D.R. Entwisle, C.S. Horsey, From first grade forward: Early foundations of high school dropout. Sociol. Educ. 70, 87–107 (1997) 12. D.J. Harding, Counterfactual models of neighborhood effects: The effect of neighborhood poverty on dropping out and teenage pregnancy. Am. J. Sociol. 109(3), 676–719 (2003) 13. C.A. McCarty, W.A. Mason, R. Kosterman, J.D. Hawkins, L.J. Lengua, E. McCauley, Adolescent school failure predicts later depression among girls. J. Adolesc. Health 43(2), 180–187 (2008) 14. A. Fernández-Zabala, E. Goni, I. Camino, L.M. Zulaika, Family and school context in school engagement. Eur. J. Educ. Psychol. 9(2), 47–55 (2016) 15. C. Amone, A.J. Okwir, D. Akot, Culture and girl-child education in Northern Uganda. PREISSN: 2251–1251 3(6), 570–578 (2013) 16. Z. Sofuo˘glu, G. Sariyer, F. Aydin, S. Cankarde, B. Kandemirci, Child abuse and neglect among children who drop out of school: a study in Izmir, Turkey. Soc. Work Public Health 31(6), 589–598 (2016) 17. S.K. Acharjee, P. Deb, A study on school drop-outs at the primary level among the minority(muslim) girls in cooch behar district of west bengal. J. Intercad. 2(3), 218–221 (1998), ISSN 0971-9016 18. Ngonidzashe Mutanana & Douglas Gasva, Challenges affecting the school retention of the girl child in hurungwe district of mashonaland west province in Zimbabwe. North Asian Int. Res. J. Multidis. 2(9), 1–16 (2016) 19. J.J. Doll, Z. Eslami, L. Walters, Understanding why students drop out of high school, according to their own reports: are they pushed or pulled, or do they fall out? A comparative analysis of seven nationally representative studies. SAGE Open 3(4), 1–15 (2013) 20. T.S. Nagamani, Y. Raja Rajeswari, Factors responsible for dropping out of the school of 7 to 12-year girls. J. Educ. Res. Exten. 24(3), 159–165 (1988) 21. H.E. Garnier, J.A. Stein, J.K. Jacobs, The process of dropping out of high school: a 19-year perspective. Am. Educ. Res. J. 34(2), 395–419 (1997) 22. A.T. Banigo, T.O. Azeez, J.C. Ezelote, Girl-child and women: education and empowerment for sustainable development. J. Poverty, Invest. Dev. ISSN 2422–846X Int. Peer-Rev. J. 38, 18–25 (2017) 23. H.M. Lugonzo, F. Chege, V. Wawire, Factors Contributing to the High Drop out of Girls in the Secondary Schools around Lake Victoria: A Case Study of Nyangoma Division in Siaya County, Kenya 7(4), 049–060 (2017) 24. G.I. Ifijeh, O. Odaro, Issues in girl-child education in Nigeria: implications for library and information support. Gender Behav. 9(2), 4139–4150 (2011) 25. M.G. Mabefam, K. Ohene-Konadu, Access to and dropout of Girls from School: a quantitative analysis of the effects of Marriage arrangements on girl-child education in Bolni. J. Soc. Sci. 9(3), 119 (2013) 26. P. Das, in Process of Girls’ Dropout in School Education: Analysis of Selected Cases in India. Engendering empowerment: education and equality e-conference. united nations girls’ education initiative, New York (2010), pp. 2–10 27. A. Noori, Issues causing girls’ dropout from schools in Afghanistan. Int. J. Innov. Res. Multidisc. Field 3(9), 111–116 (2017) 28. M. Sateesh Gouda, T.V. Sekher, Factors leading to school dropouts in India: an analysis of national family health survey-3 Data. IOSR J. Res. Method Educ. (IOSR-JRME) 4(6), 75–83 (2014) 29. K. Tentsho, R. McNeil, Phattrawan Tongkumchum. Determinants of University Dropout: A Case of Thailand”. Canadian Center of Science and Education, Asian Social Science 15(7), 1911–2025 (2019 ) 30. N. Mduma, K. Kalegele, D. Machuve, Machine learning approach for reducing students dropout rates. Int. J. Adv. Comput. Res. 9(42) (2019) 31. S. Duggal, A study of students’ dropout in non-professional undergraduate degree programmes of IGNOU. Indian J. Open Learn. 25(2), 105–115 (2016)

Keeping the Integrity of Online Examination: Deep Learning to Rescue Towhidul Islam , Bushra Rafia Chowdhury , Ravina Akter Youki , and Bilkis Jamal Ferdosi

Abstract Online education is growing remarkably because of its capability to transfer knowledge and skills remotely. It is playing a key role in this pandemic. However, it poses new challenges that need to be addressed. The examination is an integral part of any education system to judge the learner’s depth of knowledge. In the online examination, keeping the integrity of the examination is very difficult since students and examination proctors remain in remote places. Manually, proctoring several students continuously and consistently using Webcams is a tedious process. Several unethical activities may remain unnoticed. It will be beneficial if machine intelligence can help human proctors. Existing researches are either based on student identification using face recognition without considering different inconsistent activities or fully automated bypassing the intelligence of human proctors. Thus, we propose a semiautomated proctoring system using machine intelligence that helps a human proctor by reporting inconsistent activities and annotated videos. Inconsistent activities during examination are done to deceive in the examination, such as copying from other resources, communicating with other people, or not being present in front of the camera. Such activities can be identified by excessive eye or lip movements or the absence of the student. The proposed system uses videos captured by a Webcam and continuously verifies and logs these three activities. We experimented with two different models using convolutional long short-term memory (convLSTM) and residual network (ResNet50). We obtained validation accuracy of 92% and 97%, respectively, using convLSTM and ResNet50.

T. Islam (B) · B. R. Chowdhury · R. A. Youki · B. J. Ferdosi University of Asia Pacific, 74/A Green Road, Dhaka 1205, Bangladesh e-mail: [email protected] B. R. Chowdhury e-mail: [email protected] R. A. Youki e-mail: [email protected] B. J. Ferdosi e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_32

383

384

T. Islam et al.

Keywords Online examination proctoring · Examination integrity · Machine intelligence · Deep learning

1 Introduction Online learning is quite popular in this age of technology because it offers more convenience and flexibility in acquire knowledge remotely. Due to the adverse effect of COVID-19, all of the educational institutions had to shut down. The only option left is online education to keep the students safe. Besides this pandemic, online education is growing noticeably. A recent survey by Palvia et al. [1] showed that 30% of American students enrolled in at least one online course. Now, it became a necessity when students have limited access to the campus. The examination is an essential component to evaluate the students learning capabilities. But in online, it is crucial to monitor the exam and to preserve the same levels of trust, honesty, and integrity. Several approaches have been utilized to prevent cheating in online exams. Various commercial online exam supervision tools exist such as ProctorU1 that have been widely used and involve human proctor along with machine intelligence. Wahid et al. [2] proposed a system that blocks websites, search engines, and devices, utilizing the system and network configuration to prevent cheating. If the examination is only based on multiple-choice questions (MCQs) or short answers, these systems would be sufficient but not in general. Atoum et al. [3] proposed a multimedia analytics system to detect cheating during an online exam which requires a microphone, one camera facing the examinee, another camera worn by the examinee, and utilized captured video and audio signals to detect anomalies. This system is fully automated bypassing the human proctors. Declaring examinees behavior as “unethical” without considering human factors should be considered as unethical. Therefore, we highly advocate the involvement of a human proctor over machine intelligence to declare any act of examinee as deceitful. Hence, we propose a semiautomated online exam proctoring system to help a human proctor maintain the academic integrity of the examination. The proposed system considers those examinations that include questions requiring descriptive or analytical answers. Observing the behavior of the examinee during the exam may give us clues to identify deceitful acts. Here, we utilized the videos from a mounted Webcam with the computer to identify deceitful acts. The key features of our system are as follows: • We consider three activities of an examinee as indicative of deceitful behavior in an online exam: – remaining absent from the camera, – looking around, and – talking. 1

https://www.proctoru.com/.

Keeping the Integrity of Online Examination …

• • • •

385

Predicted sequence/frames are annotated with the predicted label. A report along with the annotated frames is generated for the human proctor, The report includes the percentage and the time duration of each activity. Based on this report, the human proctor may warn or take necessary steps.

The rest of our paper is organized as follows: in Sect. 2, summaries of related works are included. In Sect. 3, the description of our datasets is presented. Then, in Sect. 4 the details and description of our proposed method are presented. The results and analysis of our proposed system are presented in Sect. 5. Finally, the conclusion and future works are presented in Sect. 6.

2 Related Works Online exam proctoring system is a necessity in online education. Several researchers have proposed several solutions. Here, we will discuss the related works according to the setup of the online examination.

2.1 Classroom Setup In a classroom setup, students appear in the exam together in a room and are monitored by surveillance camera. Soman et al. [4] proposed a method to detect anomalous behavior in classroom setup using grayscale video input from a single camera. They used HOG features and k-nearest neighbor (KNN) classifier to report abnormal behaviors based on predefined anomalies. Ibrahim et al. [5] proposed a system that continuously detects anomalous behavior of students in exam hall by using neural networks and Gaussian distribution with a fixed camera. Classroom setup faces difficulties identifying each student from crowded video input. However, the pattern of detecting anomalous behavior in classroom setup proctoring system might differ from the remote online exam.

2.2 Remote Setup In remote online examination, both examinee and proctor remain in remote places. Hu et al. [6] proposed a proctoring system for remote setup exams using a single Webcam. They used Viola–Jones detector, Haar-based features, and convolutional neural network (CNN) to recognize the face and detect suspicious activities utilizing the examinee’s head pose estimation and threshold-based mouth state judgment. Hadian et al. [7] used audio and video signals captured by a Webcam and fed them into a rule-based inference system to detect deceitful activities. Cote et al. [8] proposed

386

T. Islam et al.

an automated remote proctoring system by summarizing videos of online exams post the exam. The video summaries are from sequences of detected abnormal behavior and then classified into normal and abnormal classes. But, their work focuses on postexam video review; they classify the recorded exam videos into normal and abnormal classes. Atoum et al. [3] proposed an automated online exam proctoring system. They used three sensors—a wearcam, a webcam with an integrated microphone, and support vector machine (SVM) as a classifier to detect anomalous behavior based on the captured audio–visual streams. The drawbacks of this system are its feature to act as fully automated bypassing human judgment and its costliness as it requires multiple devices for monitoring. Especially, the wearcam mounted on the examinee’s head or eyeglass can be uncomfortable.

3 Datasets We utilized both artificial and real-life video data in our work. We have prepared an artificial dataset by imitating activities of looking, talking, and being absent from the camera view. The videos were captured in the home setup of a person and the person acted as an examinee in front of a Webcam. We have created 25 videos for each class and a total of 75 videos. The duration of each video is 10–15 s, and the average frame rate of the videos is 26 frames per second (fps). We have used 80% of artificial data for training and 20% for validation purposes. In the testing phase, our models generate reports on both artificial and real datasets. In the artificial datasets, each video is created mimicking the real-life scenarios where several deceitful acts may present. The real dataset is collected from real online examinations of our university. The average duration of each test video is 65 s, and the average frame rate is 26 frames per second (fps).

4 Methodology In general, an online exam proctoring system is consists of two phases, the preparation phase and the exam phase [3]. Authentication and calibration of all the sensors are performed in the preparation phase. In the exam phase, different types of anomalous behaviors are detected. In our case, we only focused on the exam phase. Our proposed system assists the human proctor during the online exam by generating a report of the anomalous behaviors of the students. The report will be generated focusing on three types of anomalous behavior: absent, looking, and talking. We have tried two models in our proposed system using: convolutional long short-term memory (convLSTM) [9] and residual network (ResNet50) [10]. The block diagram of Fig. 1 describes the working procedure of our proposed system where training videos are divided into three classes: talking, looking, and absent. During an online exam, when an examinee talks, we can detect that act by

Keeping the Integrity of Online Examination …

387

Fig. 1 Block diagram of the proposed model

tracking the movement of the lips. The training videos of the “Talking” class consist of the activities where a student performs a writing activity and after some time the student starts to talk. A common anomalous behavior during an online exam is students looking around to copy. We are considering the situation where an examinee attends the exam by using the Webcam of a computer. Since the viewing area of a Webcam is very constricted, the examinee can take the advantage to keep any helping materials outside of the camera view to copy from those or he/she may try to get help from online resources. We can identify those activities by observing eye movements. The training videos of the “Looking” class consist of such activities. Absent class refers to the activity where a student is completely out of the camera view. This activity occurs when a student goes out of the camera view to take help from a book or notes or for taking help over the phone. The training videos of the “Absent” class contain the act, where a student suddenly disappears and after some time returns to the camera view. By analyzing the prediction value of these three classes, we added an extra class during annotation and report generating, which is a “Normal” class. The “Normal” class defines the activity where a student is performing the writing activity. Extracted frames of the training videos are used as input to the models (convLSTM or ResNet50). Based on the perceived knowledge of the training phase, each model stores weights. In the testing phase, by using the stored weights, each model predicts the class of testing videos. Each testing video is annotated with the predicted class. Based on the prediction, a report is generated for the human proctor.

388

T. Islam et al.

We have chosen convLSTM because of its capability of remembering the information observed from a sequence of images and making a decision based on it. To identify the student’s activity from a video, it is required that we observe the student’s behavior for consecutive frames. In the ConvLSTM, the ConvLSTM2D layers of the model extract feature from the samples. It has 64 output filters in the convolution and a 3 × 3 kernel. Output matrices of this layer are passed to the flatten layer. The flatten layer converts the feature matrix into a vector. The flatten layer is fully connected with a dense layer. Our model has three fully connected dense layers. The first two dense layers have 256 hidden units each and rectified linear units (ReLU) as an activation function. The final dense layer is the output layer; it has three hidden units and softmax as an activation function. In this model, the video input needs to be fed as sequences where each sequence consists of 80 frames. Each sequence is considered as a sample of the dataset. Each sample is labeled with the associated class during sequence making. The convLSTM can not adopt a deep network due to the vanishing gradient problem [11]. By leveraging the advantage of the skip connection, the ResNet50 can work with a deeper network by eliminating the vanishing gradient problem. By utilizing the transfer learning of the ImageNet dataset, the pretrained ResNet50 reduced the computation time on training. Its overall training, testing, and prediction time is relatively lower than the convLSTM model. By removing the top layer of the pretrained ResNet50, a small model was added on top of it. This small model contains an average pooling layer, a flatten layer, a dropout layer, and two dense layers. The average pooling layer reduces the dimension of the feature matrix without eliminating the important features. It has a window size of 7 × 7. The flatten layer converts the feature matrix into a vector. The dropout layer eliminates the overfitting of the model. The first dense layer has 512 hidden units and ReLU as an activation function. The last dense layer is the output layer; it has three hidden units and softmax as an activation function. To train our ResNet50, we have extracted frames from the training videos. Each frame is considered as a sample. All of the samples are labeled with the associated classes.

4.1 Annotation We create annotated videos as output. To annotate a video, the model is loaded with the associated weights. Testing samples are passed to the models. The model returns the probability of each class for each sample in an array, (S p ). The array (S p ) has three elements: probability of the absent (A p ), looking (L p ), and talking (T p ) classes. In our work, we do not have any separate class called the “Normal” class. We identify the normal activities of a student as the absence of the defined anomalous behaviors of talking, looking, and being absent from the frame. To identify the normal activities, we define the following conditions:

Keeping the Integrity of Online Examination …

389

• The student must be present in the frame. Thus, the probability value of the absent class is the smallest among the three classes. • Difference of the probabilities of looking and talking classes is small and is less or equal to a user defined threshold value (Tv ). The procedure of video annotation is depicted in Algorithm 1. At first, we check if a frame contains the normal activity or not as per our set conditions. If yes, then the frame is annotated as normal. Otherwise, the frame will be annotated as the class that has the highest probability value. Algorithm 1: Video Annotation Input : A p , L p , T p , Tv , Ftest Output: Fnor mal , Fabsent , Flooking , Ftalking [ A p - probability of absent class, L p - probability of looking class, T p -probability of talking class, Tv -user-defined threshold value, Ftest -frames of testing videos, Fnor mal -frames annotated with normal

act, Fabsent -frames annotated with absent act, Flooking -frames annotated with looking act, Ftalking -frames annotated with talking act. ]

if (A p < L p ) and (A p < T p ) then if | L p − T p | ≤ Tv then Fnor mal ← Ftest end end else if (A p > L p ) and (A p > T p ) then Fabsent ← Ftest end else if (L p > T p ) then Flooking ← Ftest end Ftalking ← Ftest

4.2 Report Generation Our system generates a report for the human proctor. The report contains the percentage of each activity (Pa ) along with the time duration of each activity (Ta ). We have generated the report by using Eqs. (1) and (2) given below: Pa =

Fa × 100 Na

Ta = Vl × Pa

(1)

(2)

390

T. Islam et al.

Table 1 Results of convLSTM and ResNet50 Models Precision Recall Training Validation Training Validation convLSTM (%) ResNet50 (%)

Training

F1 Score Validation

91

92

91

92

91

92

97

97

97

97

97

97

Here, (Fa ) represents the frequency of a class on the prediction list (Plist ); (Na ) represents the total number of classes on the prediction list (Plist ); (Vl ) represents the length of the video in seconds.

5 Results and Analysis The convLSTM model has achieved 91% accuracy on training and 92% accuracy on validation. On the other hand, the ResNet50 model has achieved 97% accuracy on both training and validation. The precision, recall, and f1-score [12] of both of the models is shown in Table 1. We also compared the performance of the models in generating reports for the testing datasets. To evaluate the generated report by the models, we have manually prepared a report by averaging the percentage of activity and time duration of the activities from all of the testing videos. We are considering this manual report as a standard report. In our experiment, we found that the convLSTM has 22.67%, 14.13%, 19.75%, 17.06% prediction deviation, respectively, on the absent, looking, talking, and normal act. On the other hand, ResNet50 has 3.71%, 11.52%, 0.48%, 14.74% prediction deviation, respectively, on the absent, looking, talking, and normal activities. Thus, ResNet50 can generate a better report.

6 Conclusion and Future Works In this paper, a video analysis-based system is proposed to detect unethical behavior during the online exam. We considered absent, looking, and talking as deceitful activities, and based on this, we created artificial data and collected real-life video data. We worked with the convLSTM and ResNet50 model. We obtained 91% training and 92% validation accuracy using the convLSTM model and compared to that we got 97% on both training and validation accuracy using the ResNet50 model. In our method, after detecting anomaly, the system generates a report to assist the human

Keeping the Integrity of Online Examination …

391

proctor with the time duration and percentage of each activity of candidates. As we have worked with three acts, we would like to work on identifying more deceitful activities and also increase the performance of the model in the future.

References 1. S. Palvia, P. Aeron, P. Gupta, D. Mahapatra, R. Parida, R. Rosner, S. Sindhi, Online education: worldwide status, challenges, trends, and implications. J. Glob. Inf. Technol. Manag. 21(4), 233–241 (2018). https://doi.org/10.1080/1097198x.2018.1542262 2. A. Wahid, Y. Sengoku, M. Mambo, Toward constructing a secure online examination system, in Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication (2015), pp. 1–8. https://doi.org/10.1145/2701126.2701203 3. Y. Atoum, L. Chen, A.X. Liu, S.D.H. Hsu, X. Liu, Automated online exam proctoring. IEEE Trans. Multimedia 19(7), 1609–1624 (2017). https://doi.org/10.1109/tmm.2017.2656064 4. N. Soman, M.N.R. Devi, G. Srinivasa, Detection of anomalous behavior in an examination hall towards automated proctoring, in 2017 Second International Conference on Electrical, Computer and Communication Technologies (ICECCT) (2017), pp. 1–6. https://doi.org/10. 1109/ICECCT.2017.8117908 5. A.A. Ibrahim, G. Abosamra, M. Dahab, Real-time anomalous behavior detection of students in examination rooms using neural networks and Gaussian distribution. Int. J. Sci. Eng. Res. 9(10), 1716–1724 (2018). https://doi.org/10.14299/ijser.2018.10.15 6. S. Hu, X. Jia, Y. Fu, Research on abnormal behavior detection of online examination based on image information, in 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 02 (2018), pp. 88–91. https://doi.org/10.1109/IHMSC. 2018.10127 7. H.S.G. Asep, Y. Bandung, A design of continuous user verification for online exam proctoring on M-learning, in 2019 International Conference on Electrical Engineering and Informatics (ICEEI) (2019), 284–289. https://doi.org/10.1109/iceei47359.2019.8988786 8. M. Cote, F. Jean, A.B. Albu, D. Capson, Video summarization for remote invigilation of online exams, in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) (2016), pp. 1–9. https://doi.org/10.1109/wacv.2016.7477704 9. S. Xingjian, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, W.-c. Woo, Convolutional LSTM network: a machine learning approach for precipitation nowcasting, in Advances in Neural Information Processing Systems (2015), pp. 802–810 10. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778. https:// doi.org/10.1109/cvpr.2016.90 11. H. Wei, H. Zhou, J. Sankaranarayanan, S. Sengupta, H. Samet, Residual convolutional LSTM for tweet count prediction, in Companion of the The Web Conference 2018 on The Web Conference 2018—WWW ’18 (2018), pp. 1309–1316. https://doi.org/10.1145/3184558.3191571 12. C. Sammut, G. Webb, Encyclopedia of Machine Learning (2010). https://doi.org/10.1007/9780-387-30164-8

Steering Wheel Angle Prediction from Dashboard Data Using CNN Architecture Manas Kumar Rath, Tanmaya Swain, Tapaswini Samanta, Shobhan Banerjee, and Prasanta Kumar Swain

Abstract Various innovations on self-driving cars are trending in the automobile industry these days. The general approach for AI applications is to collect the data through various sensors that are fit in a car, process them through appropriate techniques, and then train a model upon which one can try and test the efficiency of the model. Many companies like Tesla, Uber, Waymo (a subsidiary of Google), and Mercedes are already working with a lot of sensor-captured data and high computation power. The sensors range from normal cameras to high-end ultrasonic and LIDAR sensors. Whatever be the data that we capture, the basic intention is to move swiftly through a path given various twists and turns, and other traffic conditions, where the car needs to be correctly steered as per the external environment. We should know that more the data we capture, more shall be the complexity of the system, hence more will be the required computational power for that. In this paper, we present a simple model where we capture data through a front dashboard camera, process it through a CNN, and predict the appropriate steering angles based on the external traffic conditions, where we have acquired a significant level of accuracy. We assume automatic transmission system in our vehicle without clutch and gears. Keywords Self-driving cars · Deep learning · Convolutional neural network · Steering angle · End to end learning M. K. Rath · T. Swain · T. Samanta KIIT, Bhubaneswar, Odisha, India e-mail: [email protected] T. Swain e-mail: [email protected] T. Samanta e-mail: [email protected] S. Banerjee BITS Pilani, Pilani, Rajasthan, India e-mail: [email protected] P. K. Swain (B) MSCB University, Baripada, Odisha, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_33

393

394

M. K. Rath et al.

1 Introduction Any self-driving car as we know needs to be trained on some set of inputs. These inputs may be some directly captured images through various cameras or some other attributes based on values captured from sensors. Every company has its own set of tools and sensors. When using cameras, we can use front-view cameras, side-view cameras, and even videos captured through rear-view cameras. Apart from that we can have light detection and ranging (LIDAR) sensors which can send out beams of light and try to sense the 3D geometry around the car. We may have other sensors like RADARs, GPS sensors, and ultrasonic sensors to gather our data. RADARs are radio frequency-based distance measurement sensors used to detect how far are other vehicles around the car. Ultrasonic sensors are typically used to sense things which are very close to the vehicle. LIDARs are expensive sensors and are very much used by few of the self-driving cars. Other sensors are comparatively cheaper, but cameras are the cheapest form of sensory inputs available to us. These inputs are then fed to some computing box which consists of the trained model. Assuming automatic transmission and not manual transmission, our possible outputs can be acceleration, braking, and steering angles for the car to travel through the road. One may consider other stuffs like wipers if it rains or signals/indicators. A few constraints to be considered are that of following the traffic rules and avoiding collisions with other cars or people. A steering controller with closed-loop feedback has been proposed in [1] using DAVE-2SKY neural network. In [2], we see that a lighter weight model for real-time inferences has been proposed maintaining an equivalent accuracy of the previous segmentation networks. The authors in [3] have trained a ResNet CNN to conduct simulation in the Udacity platform. In [4], we see a brilliant team of researchers from NVIDEA Corporation where they have proposed a CNN using raw image data from the front camera, in which steering commands have directly been mapped to its corresponding input image. A deep learning method has been proposed in [5] where the intention was to enhance the value of information retrieved from samples collected from an expert policy from which the system learns the driving policies. A new CNN model Jacintonet has been proposed in [6] where self-driving has been demonstrated using virtual simulation. It uses the UPB campus dataset for vehicle trajectory planning using car motion and visual inputs. In [7], the authors have focused to maintain the car in lane after acquiring proper steering angles. A path following module has been proposed by authors in [8] where the cars drive automatically on a pre-decided route base on its current position. The authors in [9] have combined VGG16 CNN with an LSTM model, the validation of which has again been carried out on the dataset provided by Udacity. In this paper, we present a simplified version of the problem, where we consider the video captured by the front dashboard camera and try to predict the steering angle using the architecture proposed in [4]. Hence, given a video, we segment it into various images after which we get to have a regression problem of predicting

Steering Wheel Angle Prediction from Dashboard Data …

395

a sequence of steering angles. We have used Sully Chens dataset which is licensed by MIT and gives permission for private and commercial use. It consists of images from a video of 25 min from which images are captured at 30 FPS.

2 Exploratory Data Analysis 2.1 Data Description The dataset consists of images segmented from a video of around 25 min at 30 FPS. This means we have somewhere around 25 × 60 × 30 = 45,000 images. Since the data are temporal in nature, hence, we do not perform a random split between train and test and instead go for time-based splitting. From this, we use 80% of the images to train our model and rest 20% of the images to test it. This means that the training is done on first 20 min samples, and images for the last 5 min are used to test it. In other words, we can say that 36,000 images have been used for train, and rest 9000 images have been used for the test set. The angles that we get are converted into radians. This is a form of normalization as the range of values in case of degrees is large. Radians =

Degrees × π 180

2.2 Visual Description The figure below shows the distributions of train (green) and test (red) data. We can clearly see that 1. There is very less overlap between the two. That is possibly because of the temporal nature of the data. 2. Another point worth noticing is that most often the steering angles are around 0. This shows that the path is not so crowded. 3. Most of the values are roughly between −3 and 2 radians. 4. In the training phase, the car was steering up to around 3.5 radians.

2.3 Mean Baseline Model Since we are trying to solve a regression problem over here, a simple metric for regression problems is the mean square error (MSE). If we consider all our predicted

396

M. K. Rath et al.

Fig. 1 Train and test data distribution

values of the test set to be the mean value of the train set labels, then the MSE that we obtain from it is 0.191127. Also, we saw that most of the angles are around 0a (Fig. 1). Hence, if we find the MSE assuming the predicted values to be 0a , we get an MSE = 0.190891. Hence, we should make sure that the MSE of our CNN model should be always less than 0.19 and should never exceed it. The ideal MSE value is 0.

3 CNN Model Earlier approaches to these problems consisted of training a model for each unique task. That means there was one model which was specifically trained to detect a road; another model was trained to detect and segregate lanes, calculate angles, and so on. Passing an image to the CNN, if we can directly get a predicted value for steering angle, we call such models as end-to-end models. Also, while training a CNN, we often use a gradient SGD or a mini batch SGD; hence, we must load the data in batches. The CNN architecture as leveraged from [4] is shown in Fig. 2. The above architecture goes end-to-end instead of breaking the whole task into subproblems. Roughly, 72 h of data has been collected to train this. NVIDIA DevBox and Torch 7 were used to train the model. An NVIDIA Dri ve T M PX along with Torch 7 was used to determine where to drive.

3.1 Model Description 1. The input being fed to the model is a colored RGB image whos values for each segment lies from 0 to 255. 2. They are then normalized after which the values lie from 0 to 1.

Steering Wheel Angle Prediction from Dashboard Data …

397

Fig. 2 The CNN architecture

3. After normalization, five convolutions are performed. Initially we have fewer kernels, which are increased as we move ahead in the network. 4. After that we perform a flattening operation on that. It is worth noting that there are no max-pooling and no batch normalization. The researchers might have tried a lot of combinations of hyperparameters to arrive at this. 5. Finally, after flattening, we have three fully connected networks, but without any dropouts. 6. To get the final output, we can apply any function on top of it which is monotonically increasing like linear or tan-1, but not sigmoid. That is because we are solving a regression task, not classification. After training the CNN, this model was trained on a real car (off-course with a driver in it) where the car moved perfectly around 98% of the time.

398

M. K. Rath et al.

Fig. 3 a View in a curvy local road. b View of the car driving in rain Fig. 4 Driving the car in snow

3.2 Driving Views A few in-vehicle and out-vehicle views have been shown from the official blog of NVIDIA in Fig. 3a, b. The blog can be found out in link https://developer.nvidia. com/blog/deep-learning-self-driving-cars/ Figure 4 shows the car being driven in snow. Figure 5a shows the input image for the NN, followed by the first and second-layer feature maps when the car moves on a road with no lane markings. Figure 5b shows the same when car is driving on no road. In the figure above, we can see that the CNNs are not able to recognize any useful feature as such.

4 Implementation The model has been implemented with arctan as the activation function in the output. Batch size of 100 is chosen, and the model is trained for 30 epochs, after which the training loss reduced from 12.9164 to 0.280036. Adam optimizer was used with a learning rate of 0.0001. It took around 6 h to train the model in a system with 8 GB of

Steering Wheel Angle Prediction from Dashboard Data …

399

Fig. 5 a Driving on a road with no lane markings. b Driving the car without a road

RAM and no GPU. NVIDIA has used inverse of the turning radius, and the steering wheel angle is proportional to the inverse of turning radius. Hence, the steering wheel angle (in radians) can be used as output.

5 Result and Discussion As discussed earlier, the metric to evaluate the performance of our regression model is the MSE. In the test data, the steering angle angles were predicted for around 9000 images in our model. The MSE between actual values and the predicted values came out to be 0.0312. This means that our model performed better than the baseline model as discussed in Sect. 2.3. Using the image of a steering wheel, a visualization

Fig. 6 a Car turning left with steering angle of −0.96. b Car moving straight with steering angle of −0.087 Fig. 7 Car turning right, with steering angle of 0.314

400

M. K. Rath et al.

was made where against each predicted value of the steering angle, the image was also rotated by that amount. The input image along with its corresponding rotated steering wheel has been shown in Figs. 6a, b and 7. All angles are in radians.

6 Conclusion and Future Work After implementing the discussed method, we could achieve an MSE value as low as 3.12 % . If there are more sensory inputs available, we can train an even better end-to-end model for our self-driving car, which will be financially expensive for implementation. Also, we might need good computation power and high-end GPUs for computing all the data within a reasonable amount of time. Using a simplistic model as implemented above, we can say that the authors in [4], using appropriate hyperparameters, have created a great model without even adding max-pooling, batch normalization, and dropouts, and we got to have a good model for our regression task.

References 1. J. Jhung, I. Bae, J. Moon, T. Kim, J. Kim , S. Kim, End-to-End steering controller with CNNbased closed-loop feedback for autonomous vehicles, in 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 617–622. https://doi.org/10.1109/IVS.2018.8500440 2. T.D. Phan, H.H.N. Nguyen, N.H.D. Le, T.S. Nguyen, M.T. Duong, M.H. Le, Steering angle estimation for self-driving car based on enhanced semantic segmentation, in 2021 International Conference on System Science and Engineering (ICSSE), pp. 32–37. https://doi.org/10.1109/ ICSSE52999.2021.9538460 3. A. Khanum, C.Y. Lee, C.S. Yang, End-to-end deep learning model for steering angle control of autonomous vehicles, in 2020 International Symposium on Computer, Consumer and Control (IS3C), pp. 189–192. https://doi.org/10.1109/IS3C50286.2020.00056 4. M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L.D. Jackel, M. Monfort, U. Muller, J. Zhang, X. Zhang, J. Zhao, K. Zieba, End to end learning for self driving cars, arXiv:1604.07316v1 [cs.CV] 5. Y. Bicer, A. Alizadeh, N. K. Ure, A. Erdogan, O. Kizilirmak, Sample efficient interactive End-toEnd deep learning for self-driving cars with selective multi class safe dataset aggregation, in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2629–2634. https://doi.org/10.1109/IROS40897.2019.8967948 6. P. Viswanath, S. Nagori, M. Mody, M. Mathew, P. Swami, End to end learning based self driving using jacintoNet, in 2018 IEEE 8th International Conference on Consumer Electronics—Berlin (ICCE-Berlin), pp. 1–4. https://doi.org/10.1109/ICCE-Berlin.2018.8576190 7. Z. Chen, X. Huang, End-to-end learning for lane keeping of self-driving cars, in 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1856–1860, https://doi.org/10.1109/IVS.2017.7995975

Steering Wheel Angle Prediction from Dashboard Data …

401

8. T. Onishi, T. Motoyoshi, Y. Suga, H. Mori, T. Ogata, End-to-end learning method for self driving cars with trajectory recovery using a path-following function, in 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. https://doi.org/10.1109/IJCNN.2019. 8852322 9. H. Jiang, L. Chang, Q. Li, D. Chen, Deep transfer learning enable end-to-end steering angles prediction for self driving car, in 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 405–412, https://doi.org/10.1109/IV47402.2020.9304611

Implementation of a Multi-Disciplinary Smart Warehouse Project with Applications Ngoc-Huan Le, Manh-Kha Kieu, Vu-Anh-Tram Nguyen, Tran-Thuy-Duong Ninh, Xuan-Hung Nguyen, Duc-Canh Nguyen, Narayan C. Debnath, and Ngoc-Bich Le Abstract In this study, a multi-disciplinary cooperative project was implemented with the desire to close the gap between university education and industry, equip students with “work-ready” competencies, and the ability to work in the global and emerging technologies environment. Consequently, a project entitled smart warehouse with the participation of students from three-related schools was proposed and implemented. Through the project, the students developed a variety of competencies, N.-H. Le · X.-H. Nguyen · D.-C. Nguyen Mechanical and Mechatronics Department, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] X.-H. Nguyen e-mail: [email protected] D.-C. Nguyen e-mail: [email protected] M.-K. Kieu (B) · V.-A.-T. Nguyen · T.-T.-D. Ninh Becamex Business School, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] V.-A.-T. Nguyen e-mail: [email protected] T.-T.-D. Ninh e-mail: [email protected] N. C. Debnath School of Computing and Information Technology, Eastern International University, Binh Duong Province, Vietnam e-mail: [email protected] M.-K. Kieu School of Business and Management, RMIT University, Ho Chi Minh City, Vietnam N.-B. Le (B) School of Biomedical Engineering, International University, Ho Chi Minh City, Vietnam e-mail: [email protected] Vietnam National University, Ho Chi Minh City, Thu Duc City, Ho Chi Minh City, Vietnam © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_34

403

404

N.-H. Le et al.

including literature review, brainstorming, concept sketching and design, teamwork, peer/cross-evaluation, hand-on skills, interdisciplinary knowledge, problem-solving, project planning, modular design concept, design of experiment, etc. Specifically, through the survey, the average improvement in the Level of Confidence is 1.35 out of 4, and that of the Level of Knowledge or Skill is 1.29. Generally, the overall average improvement levitates from 2.20 to 3.51. Keywords Engineering education · Multi-disciplinary project · Global engineer · Work-ready · Employability · Emerging technologies

1 Introduction Vietnam is an emerging market experiencing fast changes in the last two decades. Having a sufficient skilled labor force is one of the key drivers behind this success. However, to sustain the growing economy, Vietnam must face the challenge of providing more skilled labor, satisfying various demands from different employers. Universities are under massive pressure to provide students with more than just discipline-based skills. According to [1], graduates now are in short of both technical know-how and generic skills. For many employers, the academic knowledge is not as important as the ability to handle complex information and good communication skills [2]. Universities have started to acknowledge the need for graduates to “develop a range of personal and intellectual skills beyond specific expertise in an academic discipline” [3]. With the primary purpose of enhancing work-ready ability, collaboration ability, and teamwork spirit, many multi-disciplinary projects with the combination of students from many disciplines, schools, and countries were introduced [4–6]. The main purpose of these studies is to make students work-ready and create opportunities for students to have good working relationships with domestic and foreign companies as well as other universities. Eastern international university (EIU) has established a group of lecturers from three separate academic schools, namely the Becamex Business School (BBS), School of Engineering (SoE), and School of Computing and Information Technology (CIT) to plan and implement the smart warehouse project. While participating in the first stage, students from three schools are expected to have valuable and practical experiences learned from the project. This research paper describes a detailed work plan and the work content that students of each of the three disciplines have undertaken as well as the results and experiences obtained after the prototype version of the project was completed.

Implementation of a Multi-Disciplinary Smart Warehouse Project …

405

2 Project Description Expected outcomes The expected outcomes for students of each discipline are as following: (1) SOE. For SOE students, the project builds a foundation for mechatronics students to apply theoretical models to practice, carry out research, and development projects for automation products; (2) CIT: To CIT students provide a hands-on environment to develop IoT applications in a real smart warehouse testbed and develop optimization solutions based on AI algorithms to solve problems; (3) BBS: For BBS students, provide a unique experience through an environment close to reality so that students can study and conduct research on the operation process of the warehouse system. Description of the EIU smart warehouse project The EIU smart warehouse project aims at teaching and learning, research, and the links between universities and industry. In terms of teaching and learning objectives, this project will enhance the quality of education in three schools at EIU. With regard to the connection between the university and industry, this project is used as a test center for enterprises to test and implement new technologies or solutions before putting them into action. Thus, companies can optimize the production process while saving money and avoiding risks. The EIU smart warehouse is a constantly developing environment where research and teaching may be smoothly linked, allowing bachelor and master students to gain hands-on experience while offering a realistic testbed for researchers. The proposed system is a test center using a 1:10 miniature physical model with functions acting as a real system. The hardware solutions include: racks packages, AGVs, circulation conveyor systems, sensor systems (RFID, photosensor, etc.), and industrial control systems (ICS). Specifically, (1) Storage rack system includes seven racks that can store up to 1372 packages, (2) the packaged weight of three models with matching colors is 0.5 kg (green), 1 kg (yellow), and 1.5 kg (red). Package size: 12 × 12 × 20 (cm3 ), (3) AGVs are used to travel between storage racks for storing and retrieving the packages, (4) circulation conveyor system is used to boost transaction count data to enable AI applications. Pallet size: 12 × 12 (cm2 ). (5) radio-frequency identification (RFID) and sensors are used to identify and trace the packages, and (6) industrial control system (ICS) includes PLCs, drivers, motors, and so on. The software solution is warehouse management software (WMS) comprising (1) solutions to connect the physical controller system and warehouse management software, (2) WMS solutions (features, interfaces, storage, and scalability), (3) solutions to optimize operating efficiency, and (4) key performance indicator (KPIs) system is used to measure the warehouse performance.

406

N.-H. Le et al.

Fig. 1 Project Gantt chart. Note: SoE School of engineering, CIT School of computer information technology, BBS Becamex business school

3 Project Planning Figure 1 tabulates the project Gantt chart. As can be seen, the job content is related to all three major groups of Mechatronic Engineering (SOE-MEE), Software Engineering (CIT-SE), and Supply Chain Management Bachelor (BBS-SCMB). Therefore, the work content is allocated to the specialized groups accordingly. The project was planned and implemented in 12 months. This is due first to the large project workload and second to the fact that students at EIU undertake several projects in their program.

4 Implementation and Results Collaboration to build project content Due to the project’s multi-disciplinary and highly practical nature, groups of students from the three schools had to work together from the ideation stage, pre-feasibility survey, and literature review to develop the project content. Through this process, there will be mutual learning and understanding of each other’s work. Consequently, collaborating to build multi-disciplinary project content helps students develop various competencies, including literature review, life-long learning, presentation, brainstorming, concept sketching and design, presentation, listening, peer/crossevaluation, and convincing. Hand-on skill development When participating in the project, the students of each major can interact with reality and develop hands-on skills. Specifically, mechatronic engineering students have the opportunity to choose and operate on industrial standard equipment, participate in machining, installation, testing, and actual operation. Software engineering students have the chance to develop networks and programs for practical requirements. Supply chain management students can directly work with suppliers to exchange, bargain, order equipment, and participate in building algorithms to operate smart warehouses.

Implementation of a Multi-Disciplinary Smart Warehouse Project …

407

Table 1 Related engineering courses results No

Name of course

School

No. of students

Score

GPA

Status

1

Mechanical transmission system design project

SOE

2

4.0

2.92 and 3.64

Finished

2

Mechatronic systems project

SOE

1

3.2

2.77

Finished

3

PLC project

SOE

3

4.0

2.92, 3.64, and 2.77

Finished

4

Capstone project 1

SOE

3

4.0

2.92, 3.64, and 2.77

Finished

5

Capstone project 2

SOE

3

N/A

N/A

Doing

6

Capstone project 3

SOE

3

N/A

N/A

Doing

Table 2 Related business courses’ results Project course Mean

Non-project course

Weekly reports

Final report

Total

Weekly reports

Final report

Total

83.5

87.2

84.62

79.58

81.50

80.13

SD

2.55

1.46

5.02

3.33

4.43

Min

80

84.75

82.3

72.71

77.50

74.40

Max

87.5

89

87.7

86.44

87

86.6

2.01

In particular, all student groups learn a lot when working together with real-world suppliers. The representative results of the courses related to this project were highly encouraging, as given in Tables 1 and 2. The results were intentionally separated into two tables due to the distinctive characteristics of the two schools. These results show the students’ interest in the new approach compared to the traditional ones and the effectiveness of the new approach. As given in Table 2, the results of SoE’s project-related subjects are assessed compared to the students’ GPA to show the diversity. The results demonstrate the superiority of the project-based learning courses. It is noted that the project scores are the average score assessed by a committee of 2–4 lecturers depending on the subject. They must present their performance and answer the committee’s questions to receive the above assessment results. The quality of the project-based learning course at BBS is evaluated by comparing the course without actual project with the one integrating the smart warehouse project. The course called integrated supply chain which is the capstone course of supply chain management concentration. The students who learn this course must pass all courses of SCM concentration and have essential knowledge and skills related to operation, purchasing, production planning, transportation, quality management, and project management. The assessments are designed into summative assessment and formative assessment to evaluate both the progress and final results of learners.

408

N.-H. Le et al.

0.2

0.5

0

0 70

80

90

75

80

85

Project course

Project course

Non Project course

Non-Project course

(a)

90

(b)

Fig. 2 a Weekly and b final results of project and non-project courses

There are five groups in the project course and six groups in the non-project course. Summative assessment is the final report that accounts for 30%, and formative assessment is weekly reports representing 70% of the total with seven reports weekly. Apparently, the results in Fig. 2 and Table 2 illustrate the differences between the two classes. The mean grade is higher than the non-project class in summative and formative assessment in the project class. The project is an effective method for both lecturers and students to enhance the quality of teaching and learning. It could be derived from the students’ improvement in knowledge and skills and the professor’s teaching quality enhancement. The lower variation (SD) of the project class shows the small gap between the groups of students. Students could be active in finding the solutions for an actual project and do not follow the stereotype. By observing both classes, lecturers realize that the project could trigger problem-solving skills, creation of students in various ways. The liberal education through project-based learning method impulses the students’ diverse talents. The rise of 4.4% in project class compared with 2.4% in non-project one also emphasizes the project’s advantage in learning and teaching, especially in integrated supply management course. Interdisciplinary knowledge and competencies development Equipping interdisciplinary knowledge and experience is one of the crucial goals of implementing the multi-disciplinary cooperative capstone project. Understanding the multi-disciplinary knowledge and requirements from majors helps a lot for students when implementing their work. Furthermore, interaction, support, and understanding between groups is an excellent practice to help groups develop interdisciplinary knowledge and skills. Real project development One of the recent issues attracting the attention of universities is bridging the gap between university training and the industrial environment. One of the solutions is to create conditions for students to interact with reality and carry out real projects. Therefore, the implementation process, requirements, standards, equipment, etc.,

Implementation of a Multi-Disciplinary Smart Warehouse Project …

409

are all according to actual industrial conditions. Specifically, the industrial standardbased devices were utilized in the project, and students were given a chance to access a real system development. Enhancement of students’ competencies There are 12 survey questions allocated to two groups, including (1) Level of Confidence and (2) Level of Knowledge or Skill. A total of 45 students participated in the project, including 30 supply chain management students, ten mechatronics engineering students, and five software engineering students. The survey results are shown in Fig. 3. The results demonstrate that the average improvement in the Level of Confidence is 1.35 out of 4, and that of the Level of Knowledge or Skill is 1.29. In general, the overall average improvement levitates from 2.20 to 3.51. Project’s products The results of system design and development are presented in Fig. 4. Specifically, Fig. 4a, b depicts the overall design of the whole system with twin rack systems, AGVs, and conveyor systems that help circulate pallets between output and input. Figure 4c shows the installation results of AGV controller components, including industry-standard equipment such as servo motors and drivers, stepper motors and drivers, programmable logic controller (PLC), etc.

Fig. 3 Results of capstone project outcome survey

410

N.-H. Le et al.

a)

b)

c)

Fig. 4 Smart warehouse project’s products: a the overall design, b AGV design, and c AGV controller components

5 Conclusions and Future Work The EIU smart warehouse project was implemented in collaboration between lecturers and students in three academic schools. The results that students from the three schools benefited from the project can be summarized as follows: (1) Students from each major have had valuable practical experience in the major they are studying, (2) students have experienced real-world jobs with essential skills such as problem-solving skills, teamwork skills, hands-on skills, practical communication skills, etc., (3) students used the results from their work to report in their respective courses and projects. The students’ grades of the project-related courses are significantly higher than those of BBS students who have not participated in the project. For SOE students, the grades they get are much better than their GPAs, and (4) specifically, through the survey on the effectiveness of 45 participating students in the project, with 12 questions asked, the overall average improvement increased from 2.20/4.0 time before participating to 3.51/4.0 time after completing the prototype phase. After finishing the prototype phase, a group of 10 lecturers engaged directly in the project had a meeting and agreed on the advantages and disadvantages of implementing this multi-discipline project as follows: • Students benefit most from being directly involved in the project’s implementation phase and conducting experiences when the entire project is completed. • Lecturers are under more pressure to constantly deal with real-world problems that would be difficult to overcome for passive teachers who rarely do scientific research projects and practical projects. Finally, this project deserves to be replicated with the benefits of the project to students, lecturers, the university, and the business community. In the next phase, when the system is completed, students will be assessed based on programming skills, WMS solutions, solutions to optimize operating efficiency and KPIs system used to measure the warehouse performance. Acknowledgements This research is financially supported by Eastern International University, Binh Duong Province, Vietnam.

Implementation of a Multi-Disciplinary Smart Warehouse Project …

411

References 1. K. Khir, in Training Employable Graduates: Innovation in Training Methodology. National Conference on Continuing Technical Education and Training. The Katerina Hotel, Batu Pahat Johor (2006), pp. 34–52 2. P. Knight, M. Yorke, Skills plus: tuning the undergraduate curriculum. Skills Plus Project Report, Teaching in Higher Education 8(1), 3–16 (2002) 3. A. Shah, K. Pell, P. Brooke, Beyond first destinations. Act. Learn. High. Educ. 5(1), 9–26 (2004) 4. M. Krishnan, M. Paulik, N. Rayess, in A Multi-Disciplinary and Multi-Cultural CompetitionBased Capstone Design Program. 2007 37th Annual Frontiers in Education Conference—Global Engineering: Knowledge Without Borders, Opportunities Without Passports. Milwaukee, WI, USA (2007), pp.18–23. https://doi.org/10.1109/FIE.2007.4417956 5. P. Lago, J. Schalken, H. van Vliet, in Designing a Multi-Disciplinary Software Engineering Project. 2009 22nd Conference on Software Engineering Education and Training. Hyderabad, India (2009), pp.77–84. https://doi.org/10.1109/CSEET.2009.42 6. K. Tan, W. Goh, in Designing a Multi-Disciplinary Group Project for Computer Science and Engineering Students. 2019 IEEE Global Engineering Education Conference (EDUCON) (2019), pp. 51–57. https://doi.org/10.1109/EDUCON.2019.8725147

Machine Learning Approach Based on Fuzzy Logic for Industrial Temperature Regulation Sunanda Gupta, Jyoti Verma, Shaveta Thakral, and Pratima Manhas

Abstract In recent times, Fuzzy logic has entered as a superior control methodology for processes that are mathematically difficult to model. Fuzzy-based system focuses on systems that use knowledge-based techniques to support human decision-making, learning and action. In this paper, a Fuzzy knowledge-based controller (FKBC) has been proposed based on Fuzzy logic knowledge using linguistic variables representation and inference formalism. The proposed FKBC is a unique closed loop Fuzzy logic controller (FLC) structure having a small rule base with an efficient realization and can be easily implemented in existing industrial temperature controllers. This includes dealing with unknown and probably variable time delays, operating at very different temperature set points without returning. Keywords Fuzzy logic controller (FLC) · Proportional integral derivative (PID) · Temperature regulation · Membership function (MF)

1 Introduction Machine learning is a wide spread current application of artificial intelligence. Intelligent control is considered as a new approach in which an attempt is made to

S. Gupta Model Business Services Pvt. Ltd, Faridabad, India J. Verma (B) Manav Rachna International Institute of Research and Studies, Faridabad, India e-mail: [email protected] S. Thakral ECE, Zeal College of Engineering and Research, Pune, India e-mail: [email protected] P. Manhas Manav Rachna International Institute of Research & Studies, Faridabad, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_35

413

414

S. Gupta et al.

emulate the human intelligence through different control methods. Human intelligence depicts human behavior of learning and adaptation, also planning and dealing with large data under great uncertainty. Hence, intelligent control is considered an approach to incorporate everything of real world that is not characterized as conventional control. The ideas shaped in human brain for perceiving recognition and categorizing natural phenomena are frequently Fuzzy concepts [1, 2]. In general, industrial equipments are operated over a wide temperature range (140°–500°). To control the processes, which include these equipments, require different values of gain at lower as well at higher end of temperature range [3– 5]. This kind of control not only eliminates overshoot, but also oscillations in the system. It is essential, as sometimes a small value of overshoot in the temperature may cause a false alarming and implication could be costly shutdown of the process being controlled. While implementing conventional approach shown in Fig. 1, at first, based on the understanding of physical system plant, a mathematical model is derived. This model represents the modeling for system, its different sensors and actuators. Then PID parameters are calculated by determining simplified version of controller theory. Then an algorithm is developed for this. Finally, this model is simulated which not only include the effects of nonlinearity but also variable time delays and different parameter variations [6, 7]. In this paper, concentration is on Fuzzy-based inference system (one kind of the intelligent control techniques) as a substitute control strategy to the conventional proportional integral derivative (PID) method widely used in industries. The controller designed, based on a self-tuning algorithm, is simpler to understand and

Fig. 1 Basic approach of a conventional controller

Machine Learning Approach Based on Fuzzy Logic for Industrial …

415

Fig. 2 A generic heat exchanger industrial system

implement. A generic heat exchanger temperature control application is shown in Fig. 2. The main aim is to regulate the heat exchange temperature over wide range of temperature (i.e. output controlled variable) as close as to the set point as possible. It is measured with suitable measuring means, e.g. resistance temperature detectors (RTDs), thermistors, etc. The purpose of measuring sensor is to convert the output temperature into an equivalent electrical quantity, which is used to manipulate the parameters of temperature controller. The controller is provided with a target value, i.e. set point at which output temperature of the plant is required to be maintained. The output of controller actuates the control element. Control element is a device which in turn control the input, i.e. manipulated variable of the plant and consequently final output variable, i.e. temperature is regulated at desired value. These existing systems become more difficult to handle when these systems are compounded with variable time delay. There could be many reasons which introduce variable time delay in system, e.g. placement of RTD sensor at different location every time put a physical constraints, new development in existing products, and variation in manufacturing. Classical PID controller used in industrial temperature regulation, for tuning proportional, integral and derivative constants are time-consuming as well as costly process. So it is a demand of time to investigate new intelligent techniques such as Fuzzy logic, which not only address the problem of variable time delays but also the nonlinearity of the system and manual tuning of procedures.

2 State of Art It is difficult to control precisely and accurately the various nonlinear processes existing in nature. These systems are well known for their unpredictable changes in the controlled variable. These systems are also difficult to derive mathematical model. A paper presented in 1999 based on temperature regulation for air heat plant. It also showed that convention PID controller is insufficient to adapt with change in process environment and different variable parameters of the plant [8, 9].

416

S. Gupta et al.

Furthermore, research on PID controllers was done to obtain more accurate results from conventional methods. In this regard, a paper in 2012 by Ghadimi et al. suggested an approach for optimal PID controller based on simulated annealing algorithm to control the fuel cell voltage with varying load [10–12]. Later Xu et al. in 2008 showed in their paper that there is great nonlinearity, large time delays and long constant time in generic temperature control systems. These systems are also undetermined systems [13]. At present, PID controllers are being used in industries for temperature control. But the controlling and tuning of PID controller is a complex task. Also, the output variable generated is with long time constants as well as big overshoots. Fuzzy inference systems have the ability to acquire the human knowledge in terms of a linguistic variable for imprecise and undefined processes. While neural networks are adaptive in nature and can be trained for a particular set of data, neural system can be used to adopt for a new set of data by generating the acquired knowledge. But in both cases, Fuzzy inference system as well as in neural system, the extraction and acquisition of data and knowledge are difficult [13]. Many papers have been presented based on Fuzzy inference system for different industrial application. In 2014, Kiran et al. presented a paper to gain accuracy in step response power system stabilizer using Fuzzy logic System [14]. Farah et al in 2018 also presented one paper on Takagi–Sugeno Fuzzy inference approach for induction motor drives [15]. In this context, Goswami et al. in 2009 presented a paper on temperature control using microcontroller. It also showed the monitoring of light intensity. It also provided not only hardware which control the device when specific conditions are met, but also have the arrangement of acquisition of data in the future [16]. During the evolution of Fuzzy logic controller, many researchers worked on the hybrid approach of Fuzzy logic and PID convention controller. In Dec 2018, Baharudin and Ayob in 2015 developed to present this hybrid approach to control the speed and brushless DC motor [17, 18]. There are further advancements being done in this field. Artificial intelligence has been introduced, So that there is minimum human interference in the process. Monte and Lokhande in 2012 proposed a methodology for water temperature control system using adaptive neuro Fuzzy inference system. It showed the results of varying set point changes and different operating conditions [19]. Later PID controller integrated with neural network approach has been introduced. In 2018, Nafea et al. showed the designing for optimum PID controller parameter using neural network parameter on redial function and metamodel basis [20, 21].

3 Fuzzy Inference Based Control Design In this paper, knowledge-based Fuzzy logic controller (FLC) is proposed for a closed loop temperature control application shown in Fig. 3. Where G(s) measures the plant to be controlled, C(s) shows the controller and e−τs gives the delay in the system. It is

Machine Learning Approach Based on Fuzzy Logic for Industrial …

417

Fig. 3 Closed loop temperature control system

well known that stability of a system is significantly affected by large and probably varied time delays. Throughout the years, prediction techniques were developed to compensate time delay loss [22, 23]. It shows that for an appropriate procedure model the framework performs well, although due to error in the process parameters and time delay execution gets corrupted. Clearly Smith predictive remuneration is never a suitable method for an unknown or variable time delay. Figure 4 shows structure of any Fuzzy logic controller. FLC interacts with real world through fuzzification and defuzzification interface. It includes designing of membership functions for input and output universe of discourse depending upon the knowledge base provided by experienced and practicing engineer. Use of linguistic variables or Fuzzy variables is the main feature of FLC. FLC is designed in MATLAB and implemented in discrete form with zero-order hold circuit as shown in Fig. 5. The proposed FLC is designed for two input variables and one output variable. The input variables are taken as error (e) and rate of change of error (∆e), respectively. Error term shows the deviation of output variable, i.e.

Fig. 4 Structure of generic Fuzzy inference system

418

S. Gupta et al.

temperature from set point, while ‘∆e’ term provides gradual and cursive control for the output variable close to target value. Membership functions designed for ‘e’ and ‘∆e’ are depicted in Figs. 6a, b. While the membership function for output variable, which maps the universe of discourse for output variable, also provides the interface between Fuzzy values to real world is shown in Fig. 6c. This process is commonly known as defuzzification. Here the choice of initial values of membership functions and number of membership functions depends purely on intuition and process knowledge provided by practicing engineer covering the full operating regions of plant.

Fig. 5 Closed loop FLC system

(a)

(b)

(C) Fig. 6 Fuzzy membership functions

Machine Learning Approach Based on Fuzzy Logic for Industrial …

419

Control action produced by FLC depends upon the present input, error ‘e(k)’ and rate of change of error ‘∆e(k)’. Output membership function quantifies the contribution of rules relative to output universe of discourse. Contribution of each Fuzzy rule is considered and combined. It is further defuzzified using weighted average defuzzification method to generate final control Fuzzy action.

4 Research Methodology The purposed FLC has been implemented for heat exchanger temperature regulation shown in Fig. 7. Fluid temperature of heat exchanger has to be maintained by manipulating the flow of steam through heat exchanger tube. Intelligent sensor is used to measure the output temperature. This sensor converts the temperature into equivalent current and also amplifies it acceptable to controller. Controller output controls the position of the steam valve actuator, which in turn controls the flow of steam through heat exchanger. Thus steam flow is a function of the position of the valve. Further, the process time constant is a function of process residence time (total fluid volume/volumetric flow rate). Temperature of fluid also depends upon the inlet temperature and in turn changes pressure within the tube, consequently results in change in temperature. So the time delay of system is a varying quantity and alters with change in fluid flow rate and inlet temperature. A transfer function of the feedback system must be found to check the performance of the system analytically. The proposed methodology is shown in Fig. 7b. The simulated PID and FLC results of aforesaid control system are shown in Fig. 8 and response plots for PID and FLC with rules are shown in Fig. 9.

5 Results and Analysis The proposed knowledge base Fuzzy logic controller is described in Fig. 8. The simulations results are obtained using FLC rules given in Table 1 to provide the control strategy. Also, the time delay from the material is assumed constant during simulation. The variable time delay aspects of this system are investigated in the simulations. Simulations are performed for knowledge-based Fuzzy logic controller as well as for conventional PID controller for varying time delays. Settling time and overshoot are taken as performance metrics. Table 2 gives the values of obtained settling time and overshoot of both controllers tuned for different time delays. Figure 9a shows the response of system in case of both controllers. Though knowledge base Fuzzy logic controller is showing slower rise time, but the performance improves in terms of overshoot and settling time ultimately. Subsequent Figs. 9b, c shows the reaction of controller tuned to larger values of time delays than 10 s. Response of the system is

420

Fig. 7 a Industrial plant (heat exchanger plant) and b proposed methodology

S. Gupta et al.

Machine Learning Approach Based on Fuzzy Logic for Industrial …

421

Fig. 8 Simulation with PID controller and Fuzzy logic controller

Fig. 9 Response of PID controller and FLC, a time delay 10 s, b time delay 25 s and c time delay 40 s

422

S. Gupta et al.

Table 1 FLC rule base

Shaded areas represent zero control action

Table 2 Settling time and overshoot for different time delays

Controller

Delay time

PID

10 s

FLC PID

25 s

FLC PID FLC

40 s

Settling time

% Overshoot

350 s

115%

410 s

28.7%

450 s

133%

460 s

40%

460 s

120%

470 s

50%

also tested for different parameters of ‘P’, ‘I’, and ‘D’ of conventional PID controller and is shown in Fig. 10a, e. In this proposed work, Fuzzy logic controller adapts faster to longer time delays. Conventional PID controller pushes the system unstable in presence of unknown and varying time delay. Figure 11 shows PID and FLC analysis. From the reproductions, it is obviously observed that within the sight of an obscure or potentially differing time delay, the proposed FLC demonstrates a critical improvement in keeping up execution over standard strategies. True to form, FLC gives great execution regarding motions and overshoot without a forecast instrument. Simulation results show that the proposed FLC shows a significant performance improvement in oscillations and overshoot in the presence of varying time delay.

6 Conclusion In this paper, Fuzzy logic controller with lesser rule base is proposed for temperature regulation for industrial application. Results are proven for variable time delays. The proposed work shows desirable overall performance with variable time delay over wide range of operating conditions. Also, there is significant improvement in the system performance over conventional PID controller.

Machine Learning Approach Based on Fuzzy Logic for Industrial …

423

Fig. 10 Response of PID controller and FLC, a p = 7.5, I = 0.4, D = 0, b p = 7.5, I = 0.5, D = 0, c p = 7.5, I = 0.3, D = 0, and d p = 7.5, I = 0.3; D = 4; e p = 7.5, I = 0.3, D = 6

424

S. Gupta et al.

Fig. 11 PID and FLC analysis

References 1. N. Dhayagude, Z. Gao, F. Mrad, Fuzzy logic control of automated screw fastening. Robot. Comput.-Integr. Manuf. 12(3), 235–242 (1996) 2. J.A. Bernard, Use of a rule-based system for process control. IEEE Control Syst. Mag. 8(5), 3–13 (1988) 3. J.G. Dawson, Z. Gao, in Fuzzy Logic Control of Variable Time Delay Systems with a Stability Safe Guard. Proceedings of International Conference on Control Applications. IEEE (1995), pp. 347–353 4. L. Foulloy, S. Galichet, Fuzzy control with fuzzy inputs. IEEE Trans. Fuzzy Syst. 11(4), 437–449 (2003) 5. R.C. Berkan, S. Trubatch, Fuzzy System Design Principles (Wiley-IEEE Press, 1997) 6. P.J. King, E.H. Mamdani, The application of fuzzy control systems to industrial processes. Automatica 13(3), 235–242 (1977) 7. Z. Gao, T.A. Trautzsch, J.G. Dawson, A stable self-tuning fuzzy logic control system for industrial temperature regulation. IEEE Trans. Ind. Appl. 38(2), 414–424 (2002) 8. T. Thyagarajan, J. Shanmugam, M. Ponnavaikko, P.G. Rao, in Advanced Control Schemes for Temperature Regulation of Air Heat Plant. FUZZ-IEEE’99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No. 99CH36315), vol. 2. IEEE (1999), pp. 767–772 9. W.T. Sung, J.H. Chen, S.J. Hsiao, in Fish Pond Culture Via Fuzzy and Self-Adaptive Data Fusion Application. 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE (2017), pp. 2986–2991 10. N. Ghadimi, M. Marefati, R. Ghadimi, Adjusting parameters of lead lag controller using simulated annealing to control fuel cell voltage. Res. J. Inf. Technol. 4(1), 23–26 (2012) 11. S.J. Seyed-Shenava, O. Khezri, Optimal PID controller designing for voltage control of Fuel Cell. Bull. Electr. Eng. Inf. 3(4), 229–238 (2014) 12. O.M. Ahtiwash, M.Z. Abdulmuin, in An Adaptive Neuro-Fuzzy Approach for Modeling and Control of Nonlinear Systems. International Conference on Computational Science (Springer, Berlin, Heidelberg, 2001), pp. 198–207

Machine Learning Approach Based on Fuzzy Logic for Industrial …

425

13. K. Xu, X. Qiu, X. Li, Y. Xu, in A Dynamic Neuro-Fuzzy Controller for Gas-Fired Water Heater. 2008 Fourth International Conference on Natural Computation, vol. 7, IEEE (2008), pp. 240–244 14. N. Iran, M.S. Kumar, M.N. Raju, Improved step response of power system stabilizer using fuzzy logic controller. Bull. Electr. Eng. Inf. 3(3), 187–194 (2014) 15. N. Farah, M.H.N. Talib, Z. Ibrahim, J.M. Lazi, M. Azri, Self-tuning fuzzy logic controller based on Takagi-Sugeno applied to induction motor drives. Int. J. Power Electron. Drive Syst. 9(4), 1967 (2018) 16. A. Goswami, T. Bezboruah, K.C. Sarma, Design of an embedded system for monitoring and controlling temperature and light. Int. J. Electron. Eng. Res. 1(1), 27–36 (2009) 17. N.N. Baharudin, S.M. Ayob, in Brushless DC Motor Drive Control Using Single Input Fuzzy PI Controller (SIFPIC). 2015 IEEE Conference on Energy Conversion (CENCON). IEEE (2015), pp. 13–18 18. J.A. Vieira, F.M. Dias, A.M. Mota, in Hybrid Neuro-Fuzzy Network-Priori Knowledge Model in Temperature Control of a Gas Water Heater System. Fifth International Conference on Hybrid Intelligent Systems (HIS’05). IEEE (2005), p. 6 19. T.P. Mote, S.D. Lokhande, Temperature control system using ANFIS. Int. J. Soft Comput. Eng. (IJSCE) 2(1), 2231–2307 (2012) 20. M. Nafea, A.R.M. Ali, J. Baliah, M.S.M. Ali, Metamodel-based optimization of a PID controller parameters for a coupled-tank system. Telkomnika 16(4)( 2018) 21. E. Rakhman, Distributed control system applied in temperature control by coordinating multiloop controllers. Telkomnika, 16(4) (2018) 22. Q.W. Brone, S.L. Harris, in Varying Time Delay Estimation and Self-Tuning Control. 1991 American Control Conference. IEEE (1991), pp. 1740–1741 23. G.P. Liu, H. Wang, in An Adaptive Controller for Continuous-Time Systems with Unknown Varying Time-Delay. International Conference on Control 1991. Control’91. IET (1991), pp. 1084–1088

Predictive Analytics of Logistic Income Classification Using Machine Learning S. Beski Prabaharan and M. N. Nachappa

Abstract Accurate income data is one of the hardest piece of data to obtain across the world. Subsidy Inc. company delivers subsidies to individuals based on their income. We wish to develop an income classifier system for individuals. The income prediction model is designed using logistic regression classifier. The logistic regression model is a machine learning classification approach for predicting the likelihood of a categorical dependent variable. In this work, we also made an experiment to compare machine learning algorithms logistic regression with K-nearest neighbor on a dataset with dimension 473,421 rows and with 15 different number of columns or attributes. This experiment results show that logistic regression classifier performs better with 90% accuracy where as KNN gives an accuracy of 87%. Keywords Predictive analytics · Logistic application · Income classification · Machine learning · Dataset

1 Introduction Machine learning (ML) systems used in wide range of applications, such as computer vision, speech recognition, comprehension of the natural language, psychology, health care, and the Internet of Things [1]. Machine learning (ML) applications are numerous; predictive data mining is the most essential of these. In any dataset utilized by ML concept, each instance is mentioned by the similar collection. The characteristics can be continuous, categorical, or binary in nature. When examples are presented with the correct outputs, the learning is referred to as supervised, as opposed to unsupervised learning, where instances are left unlabeled. Many machine learning applications [2] necessitate tasks that may be set up in a supervised manner. S. Beski Prabaharan (B) School of Computer Science and Engineering, Galgotias University, Greater Noida, U.P, India M. N. Nachappa Department of Computer Science and IT, Jain Deemed-to-be-University, Bangalore, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_36

427

428

S. Beski Prabaharan and M. N. Nachappa

One of the most common functions performed by artificial intelligence is supervised categorization, in which artificial intelligence (logical approach) and statisticsbased techniques are used to generate a wide variety of techniques. The purpose of supervised algorithm is to develop a descriptive model [3]. Machine learning within computer science is a fairly new discipline that offers a range of techniques for data analysis. Many of these approaches are based on wellstatistical principles (e.g., structural regression and analysis of key components) whereas many others are not. Machine learning may provide a wider class of more versatile alternative methods of research better suited to modern data sources. It is crucial for statistical agencies to explore the potential use of machine learning techniques to decide if these techniques could better serve their future needs than conventional ones [4]. Predictive analytics uses different techniques, including knowledge mining, analysis, simulation, machine learning, and artificial intelligence. This involves data extraction from information and is used to predict trends and activity trends [5].

2 Related Works The prevalent strategy in ML is to pick the coordinates in different samples using a model based on data, and it is categorized as context and associated classification study. Classification is a learning strategy for categorizing data examples into the classes and labels that exist as a data. Han [6] described about the two stages of classification, the first stage is the learning stage and the second stage is the classification stage [7].

2.1 Classification Algorithms The classification algorithms include a result or dependent variables that are used to predict a given set of independent variables or predictors using a function and generating inputs/outputs, with the training cycle continuing until the model achieves the desired level of accuracy over the training dataset. Regression, decision trees, random forests, KNNs, logistic regressions, and other supervised learning techniques are employed. Classification is a data processing procedure that assigns distinct groups to a collection of data and is used to make accurate predictions over the dataset being evaluated [8, 9]. The different classification algorithms are proposed with the learning systems to make predictions in the machine learning process. In order to predict individual income, we concentrate and summarize in the following subsections a range of explained and most general classification techniques applicable to our final region.

Predictive Analytics of Logistic Income …

429

KNN is a simplest classification method in machine learning. Using KNN classification methodology in their analyzes focused on machine learning when forecasting the income of individual employees [10]. The Naive Bayes classification approach is one of the most widely used data mining classification algorithms (NB). A Naive Bayes classifier is a statistical method for determining the likelihood of belonging to a given class. It is also easy to manage missing values of attributes by simply eliminating the appropriate probabilities for those attributes, because it determines the chance of membership for each class. In the Naive Bayes classifier, class conditional independence refers to the fact that an attribute’s influence on a specific class is usually independent of other attributes’ influence. This classifier assesses the probability of correctly classifying or predicting a class in a dataset [11, 12]. A statistical model for solving classification difficulties is logistic regression (LR) [13]. In most cases, a logistic equation, also known as a sigmoid equation, is used to determine the probabilities in logistic regression. The logistic regression hypothesis restricts function to a range of 0 to 1. This classifier examines the relationship between one or more independent variables and the category dependent variable for a particular dataset. The target class, which we must predict, is the dependent variable. The qualities or contextual cues are the independent variables, which we will utilize to forecast the target class [14]. All qualities or features can be discrete in the decision tree baseline algorithm. We have used the separation privacy model that executes the task at hand by choosing the apt perturbation parameters depending on the input dataset and their characteristics [15].

3 Logistic Income Classifier 3.1 Dataset Here, we have used income dataset with the 473,421 rows and 15 columns. The dataset contains various types of variables or few with categorical datatype and few are of type integer. This dataset is best described with 13 columns such as age, jobtype, maritalstatus, occupation, relationship, race, gender, capitalloss, hoursperweek, nativecountry, and salarystatus. The training dataset is now presented to the machine learning algorithm; the algorithm is trained on the basis of this dataset (Fig. 2). The architecture of logistic income classifier system is shown in Fig. 1.

3.2 Implementation Procedure Steps for Implementation i.

We need to identify if the data is clean

430

S. Beski Prabaharan and M. N. Nachappa

Fig. 1 Architecture of logistic income classifier system

Fig. 2 Income prediction methodology

ii.

Look for missing values by removing samples which have incomplete data or we can fill it with missing data based on some assumptions that we make. This depends on what % of data is missing based on that we can decide. iii. Identify input variables that influencing salary status and look for possible relationships between variables. Correlation, chi-square test, box plot, scatter plot, etc., we are using iv. Identifies if categories can be combined

Predictive Analytics of Logistic Income …

431

v.

Build a model with reduced number of variables to classify the individual salary status to plan subsidy outlay, monitor, and prevent misuse vi. Here, we are taking three different algorithms, i.e., logistic regression, random forest, K-nearest neighbors we apply on the same dataset. And we decide the best technique based on the performance. vii. Evaluate performance metrics, if assumptions are satisfied and solutions are acceptable, then we decide model is good Firstly, we have imported relevant packages, i.e., Numpy, Pandas, Seaborn in to the IDE(Spyder) using Python language and then reading the income dataset into the Spyder environment. Here, as part of preprocessing, it is necessary to eliminate the missing values and contradictions (noise and outliers), so that there is no overlap or unwanted repetition and there are only specific values. For accurate outcomes, the missing values need updating with correct values or dropping [4]. In the final step, the data is split into train and test data and build a model using the train data and then test this model based on the test data, i.e., left over data. If the performance of the model is quiet good, then we conclude that it can also work effectively in the future also. Under supervised machine learning classification algorithms [16], logistic regression classification technique we have used to construct the model and predict the income classification of employees.

4 Experimental Results The values of the statistical criteria that are compared to the classification algorithms are calculated using a confusion matrix (Fig. 3). Fig. 3 Confusion matrix

432

S. Beski Prabaharan and M. N. Nachappa

F1 =

1 recall

precision × recall 2 −2· 1 precision + recall + precision

where F1 score is a harmonic mean of precision and recall. Figure 4 shows the result of 92% (78,172) of the capital gain is 0 (Fig. 5).

Salary Status Vs Employee count

Histogram of Age

Box plot for Age Vs Salary Satus

Education Type Vs Employee count

JoBtype Vs SalaryStatus

Capital Gain

Fig. 4 Predictive analysis of various classification results

(1)

Predictive Analytics of Logistic Income …

433

Fig. 5 Predictive comparison result

Data Visualization tool has been used to show bar plot, box, and whisker plots for clear data analysis. Experimental analytics of logistic dataset using Spyder is shown in Fig. 6. From Fig. 4, it is clearly understood that 75% of employees salary status is ≤50,000 and 25% of peoples salary status is >50,000, the employees who are working with age group 20–45 age are high in frequency. People with 35–50 age are more likely to earn >50,000 and people with 25–35 age are more likely to earn ≤50,000 USD. Employee who is working with education type is HS-grade which is more than other education type of employees. Employees who are earning less than or equal to 50,000 USD are private employees and also who are earning >50,000 USD are also private employees. 92% (78,172) of the capital gain is 0. From the plot, it is clearly visible that those who make more than 50,000 USD per year are more likely to spend 40–50 h per week. Spyder working environment and also we came to know that who make more than 50,000 USD per year are more likely to work as managers and professionals, hence an important variable in avoiding the misuse of subsidies.

Fig. 6 Experimental analytics of logistic dataset using Spyder

434

S. Beski Prabaharan and M. N. Nachappa

5 Conclusion This paper includes a survey of machine learning techniques on addressing employee’s income classification problem is presented. In this case study, we have applied two techniques, i.e., (1) logistic regression and (2) KNN machine learning algorithms on the income dataset to build a model for predicting income classifier. Both the algorithms give almost the same results but in terms of overall performance when we looked at accuracy and number of miss classification samples logistic regression classifier performs better with 90% accuracy where as KNN gives an accuracy of 87%. With the help of the confusion matrix, we compare the above two algorithms and concluded. Further, we may extend this paper by adding few more variables or columns to the existing dataset analyze more accurately by considering more number of variables or attributes for the dataset.

References 1. K. Sree Divya1, P. Bhargavi, S. Jyothi, Machine learning algorithms in big data analytics. Int. J. Comput. Sci. Eng. (IJCSE) 6(1) (2018). E-ISSN: 2347-2693 2. P. Sundsøy, J. Bjelland, B.-A. Reme, A.M. Iqbal E. Jahani, in Deep Learning Applied to Mobile Phone Data for Individual Income. ICAITA 2016 International Conference on Artificial Intelligence: Technologies and Applications 3. S. Beski Prabaharan, R. Ponnusamy, Trust based random energy efficient routing in mobile adhoc networks. Int. J. Appl. Eng. Res. 11(1), 448–455 (2016) 4. S. Beski Prabaharan et al., Method for providing efficient real-time multimedia communication using VoIP over communication channels. Int. J. Adv. Sci. Technol. 28(16), 1832–1843 (2019). ISBN: 22783075 5. S. Nagaparameshwara Chary, B Rama, 3-A research travelogue on classification algorithms using R programming. Int. J. Recent Technol. Eng. (IJRTE), 8(4) (2019). ISSN: 2277-3878 6. I.H. Witten, E. Frank, M.A. Hall, Data mining: practical machine learning tools and techniques, 3rd edn. (Morgan Kaufmann, 2011) 7. B.S. Kumar, K. Sudhakar, Performance evaluation of 10 MW grid connected solar photovoltaic power plant in India. Energy Rep. 1, 184–192 (2015) 8. S. Beski Prabaharan, N. Kaur, A Study on Cloud Storages and its types. A research paper has been published in Journal of Xi’an Shiyou University, Natural Science Edition, ISSN: 1673-064X.https://www.xisdxjxsu.asia/viewarticle.php?aid=355 9. G.H. John, P. Langley, in Estimating Continuous Distributions in Bayesian Classifiers. Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (1995) 10. J.M. Chambers, Computational methods for data analysis. Appl. Stat. 1(2):1–10, 1077 (Wiley) 11. K. Arun, G. Ishan, K. Sanmeet, Loan approval prediction based on machine learning approach. IOSR J. Comput. Eng. (IOSR-JCE) 18(3), 79–81 e-ISSN: 2278-0661, p-ISSN: 2278-8727, Ver. I (May-Jun. 2016). www.iosrjournals.org 12. B. Çı˘gs¸ar, D. Ünal, Comparison of data mining classification algorithms determining the default risk. J. Hindawi Sci. Program. 2019, 8 pages. Article ID 8706505. https://doi.org/10.1155/2019/ 8706505 13. N. Chakrabarty, S. Biswas, in A Statistical Approach to Adult Census Income Level Prediction. International Conference in 2018,Economics, Computer Science, Mathematics on IEEE 14. N. Mduma, K. Kalegele, D. Machuve, A survey of machine learning approaches and techniques for student dropout prediction. Data Sci. J. 18(14), 1–10 (2019). https://doi.org/10.5334/dsj2019-014

Predictive Analytics of Logistic Income …

435

15. H. Wang, S. Smys, Big data analysis and perturbation using data mining algorithm. J. Soft Comput. Paradigm (JSCP) 3(01), 19–28 (2021) 16. S. Dutta, S. Ghatak, R. Dey et al., Attribute selection for improving spam classification in online social networks: a rough set theory-based approach. Soc. Netw. Anal. Min. 8, 7 (2018). https://doi.org/10.1007/s13278-017-0484-8

Improvement of Speaker Verification Using Deep Learning Techniques Kshirod Sarmah

Abstract Deep learning (DL) has been used to solve a range of real-time artificial intelligence (AI) challenges with great success. This is a cutting-edge area of machine learning (ML) that has been rapidly evolving. As a result, deep learning is quickly becoming one of the most popular and well-defined machine learning techniques, with applications in a wide range of fields, including image processing, computer vision, speech and speaker recognition, emotion recognition, natural language processing, hand-written character recognition, cyber-security, and many others. Over other prevalent methods, DL approaches have demonstrated superior performance in speech processing areas like as voice recognition and speaker recognition. We describe an experimental setup for speaker verification (SV) utilizing DL techniques, and discuss its performance and findings, as well as how it outperformed established approaches such as HMM, GMM-UBM, and SVM. In this research works, we analyse and review deep neural network (DNN) approaches employed in SV systems. With a 1.51% equal error rate (EER), the final result is the best performance of the SV systems of restricted Boltzmann machine (RBM)-based DNN. Keywords Speaker verification · Machine learning · Deep learning · Deep neural network · MFCC

1 Introduction Speaker verification (SV) can be defined as the process of determining the speaker’s identification based on the speech or utterance he or she delivers. SV system works on the principle that every speaker’s utterance is unique like finger prints; thus, it can be used to identify the speaker or authenticate his/her claim. These systems in general analyse the characteristics or features in the speech signals which are different among speakers and are used in applications for speaker authentication, surveillance, and forensics. Depending upon the applications, speaker recognitions (SR) may be K. Sarmah (B) Pandit Deendayal Upadhyaya Adarsha Mahavidyalaya, Amjonga, Goalpara, Assam, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_37

437

438

K. Sarmah

broadly categorized into three types, namely speaker identification (SI) [1], speaker verification (SV) [2], and speaker diarization (SD) [3]. Machine learning (ML) is one of the most important and state-of-the-art area belonging to artificial intelligence (AI). Deep learning (DL) has recently emerged as one of the most prominent subsets of ML, with notable success in practically every application space. The main concept of DL comes from the concepts of deep neural networks (DNN). Actually, DL learns features spontaneously from large amount raw data by a structural form of multi-layer which is also known by the term deep neural networks (DNN). In order to learn the feature and classify its diverse patterns, DL can be built by mixing several layers between the input and output layers, allowing numerous stages of information processing units which are not linear [4, 5]. According to the study of current literatures, DL-based learning interprets about the interactions of a hierarchy of characteristics, such that high-level interpretations can be defined from low-level ones and vice versa. Deep learning has been referred to as a universal learning strategy in several state-of-the-art research evaluations since it has the ability to address a large number of real-time issues in a variety of fields in state-of-the-art concepts. The generalization and scalability power of the DL technique are other essential features. DL approaches can be classified into four types based on how they are trained: supervised, semi-supervised, unsupervised, and a hybrid that combines the supervised and unsupervised groups [6]. The necessity for labelled data for training, which offers discriminative power for pattern classification, lies at the heart of supervised approaches. Convolutional neural networks (CNN), recurrent neural networks (RNN), and deep neural networks (DNN) are examples of well-known approaches in this first group. Partially labelled datasets are used in semisupervised learning. Deep reinforcement learning (DRL) and generative adversarial networks (GAN) are two semi-supervised learning approaches that can be used in particular situations. Unsupervised techniques benefit from a large amount of unlabelled data that can be easily accessible in order to capture high-order correlations in data. We can add the deep belief network (DBN) and RBM as unsupervised groups in this situation. Hybrid approaches have been found to be more successful and efficient since they utilize both labelled and unlabelled data. A DBN-DNN is a well-known hybrid group architecture. The primary task of RBM is trained for learning about the speaker and its backdrop GMM supervectors have different session variability. The RBM, also known as the Universal RBM (URBM), is a technique for transforming invisible supervectors into low-dimensional vectors. The concatenation of i-vectors with GMM-RBM vectors at the score level improves the performance of the SV system. The most remarkable and well-known GMM-UBM [1] and i-vector model are some examples of generative models that utilize unsupervised learning [6]. The discriminative model like support vector machine (SVM) to boost the discriminant for speakers for the GMM-UBM approach [7] and the probabilistic linear discriminant analysis (PLDA) model for the i-vector approach [8]. DNN-based i-vector method, a supervised learning approach, is utilized to create a more accurate picture of acoustic space. The most popular mel-frequency cepstral coefficients (MFCC) feature has

Improvement of Speaker Verification …

439

been utilized by all these model-based methods. i-vector-based approach has shown considerable better improvement in speaker verification task [9]. DBNs, on the other hand, are generative models whose fundamental advantage is that they may be trained completely unsupervised technique. Now, in order to build a more powerful and efficient model, it has been discovered that combining DBNs with DNNs yields numerous benefits from both large amounts of unlabelled data and little amounts of labelled data [10]. Another type of generative model, RBMs, has been trained using the stochastic gradient descent approach and a maximum likelihood criterion [11]. In this research, we are observing at how DL approaches may be used in speaker verification and compare them to established classifiers like GMM-UBM and SVM.

2 ML-Based Classifiers The primary aim of a classifier is to produce a well-trained model that predicts the target value or score of the test sample given only the test data features. In this study, we have used MATLAB, Python, and TensorFlow library to implement different DL-based classification techniques as programming tools.

2.1 GMM-UBM-Based System In this type of traditional classifier, a UBM has been built by utilizing a huge amount of feature vectors namely MFCCs. A UBM is GMM trained on entire pool of large numbers of speaker’s speech data. On the other hand, GMM is a parametric probability density function with Gaussian component densities. Typically, GMMs are used for distribution models and speaker specific models are derived by using MAP estimation with the UBM acting as the prior model. The UBM is a large independent GMM (1024 mixtures) model that has been trained to represent the speaker-independent distribution of features.

2.2 Support Vector Machine (SVM) SVM is a powerful discriminative binary classifier that has recently been used to the task of speaker verification. One of the most appealing features is that it may be used with spectral [6, 12], prosodic [13], and high-level [14] features. SVM is one of the most resilient classifiers in a state-of-the-art speaker verification system, and it has also been effectively integrated with GMM to boost accuracy due to its high generalization ability to categorise unknown data [6, 12]. In the SV system, the SVM optimizer computes a distinct hyperplane that maximizes the margin of separation

440

K. Sarmah

between two classes: target speaker training vectors (designated as +1) and training vectors from an imposter population (labelled as –1).

3 DNN-Based Classifiers DNN is an extension of the ANN of machine learning (ML) techniques. Here, DNN is trained as a background model using supervised learning from the frame level MFCC features. Speaker-specific features are extracted by DNN, and speaker model is computed by taking average, per frame of last layer’s output of DNN. In SV system, DL strategies can also be applied in various ways like in the frontend, backend, and finally end-to ends system. On the other hand, PLDA is observed as the most effective and efficient backend techniques for i-vectors, which performs better scoring along with good results in the session variability compensation [15]. The third one is end-to-ends DL technique. The multiple phases of speech signal processing are done with the help of a unified DL architecture.

3.1 GMM-Based DNN A conventional i-vector extraction strategy has been applied to train the model as well as it replaces the Baum–Welch statistics estimation algorithm. Basically in this method, firstly, an UBM is trained from the raw data, and after that, UBM is finally utilized for the estimation of posterior probability. In another approach, after obtaining the trained UBM from the development dataset, the statistics of Baum–Welch are constructed by utilizing the Gaussian component in order to train DNN for easy prediction. Here, DNN has been trained by utilizing multiple frames of speech features so that it incorporates proper context and make easy prediction to make robustness to minor variations in the cepstral domain.

3.2 RBM-Based DNN Here, in this approach, the baseline of SV system is trained for UBM with the help of a DBN which is constructed by the stacked of RBM. As this approach is unsupervised therefore it does not requiring the prior text content knowledge. All hidden layers use a sigmoid activation function. The main advantage of DBNs is that it does not require of labelled data for training. The model of DBNs with unsupervised training is called universal DBN, and it uses i-vectors extracted from speakers and fed into DBN. Every node in an RBM’s hidden layer has been observed attempting to learn a unique higher level feature from the data collected from the previous layer. In a trained

Improvement of Speaker Verification …

441

DBN, hidden output nodes have different levels of activation for each input feature. In this study, we one node is chosen specially as a label for DNN discriminative training in this strategy. Weights borrowed from the DBN that have been pre-trained are used to train a DNN in a supervised manner using the labels collected from the DBN. The DNN now has a new output layer that comprises of softmax function nodes, which is commonly utilized better way in the classification tasks. This layer takes the place of the DBN’s top layer, from which the labels were retrieved.

4 Experimental Setup Here, we aim to find the performance of SV system by implementing two MLbased namely GMM-UBM and SVM as well as other two DL-based classifiers namely GMM-based DNN and RBM-based DNN. In the baseline system, features are extracted using MATLAB and Python, whereas the classification task has been done using Python and TensorFlow library. The Arunachali Language Voice Database (ALS-DB), a multilingual speech database, recently acquired in Arunachal Pradesh has been used to test the experiments [16, 17]. Mel-frequency cepstral features (MFCC) with first derivative and second derivatives coefficients of 39-dimensional total size of feature vectors are used. MFCC characteristics have been discovered as one of the powerful inputs to the DNN by concatenating both single-frame and multi-frame data. It has been noticed that for single-frame vectors, unsupervised training in pre-phase is not necessary, but generative pre-training is required for optimal DNN training, and the input vectors are scaled at 252 dimensions and created by concatenating seven frames. The dataset was used to pre-train the DNN with stacked denoising autoencoders (SDA). For the pre-training phase, stacked DBN was also used. In this study, a 1024-mixture UBM and a DNN with 1024 output nodes are used to test the proposed technique. Table 1 displays EER and MinDCF values of different ML and DL-based classifiers in speaker. Verification tasks. DET curves for the different ML and DL-based SV system are shown in Fig. 1. Table 1 EER and MinDCF values of different ML and DL-based classifiers in speaker verification tasks Speaker modelling techniques (classifiers)

EER%

MinDCF

Verification rate (%)

GMM-UBM

9.25

0.1587

90.75

SVM

4.73

0.0848

95.27

GMM-based DNN

2.46

0.0453

97.54

RBM-based DNN

1.51

0.0250

98.49

442

K. Sarmah 90 RBM Based DNN, EER = 1.51, MinDCF = 0.0250

80

GMM Based DNN,EER = 2.4621,MinDCF = 0.0453 SVM,EER = 4.7348,MinDCF = 0.0848 GMM-UBM , EER = 9.25, MinDCF = 0.1587

Miss probability (in %)

60

40

20 10 5 2 1 0.5 0.2 0.1 0.1 0.2 0.5

1

2

5

10

20

40

60

80

90

False Alarm probability (in %)

Fig. 1 DET curves for the different ML and DL-based SV system

5 Conclusions This study examines a variety of ML and DL-based techniques for SV tasks in depth. For certain level of ML challenges, DL-based strategies have produced improved results, generating increased interest in the research community in the near future. The state-of-the-art performances of DL-based approaches have been observed far better than that of traditional approaches. Equal error rates (EER) of 9.25%, 4.73%, 2.46%, and 1.51% have been obtained in a set of trials for GMM-UBM, SVM, GMMbased DNN, and RBM-based DNN, respectively. Finally, the DNN-based algorithms outperformed the GMM-UBM and SVM-based algorithms in the speaker verification challenge. The baseline RBM-based DNN system has the best performance of 1.51% of EER.

References 1. D.A. Reynolds, T.F. Quatieri, R.B. Dunn, Speaker verification using adapted Gaussian mixture models. Digital Signal Proc 10, 19–41 (2000) 2. D.A. Reynolds, R.C. Rose, Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Trans. Speech Audio Process. 3, 72–83 (1995) 3. S. Tranter, D.A. Reynolds, An overview of automatic speaker diarisation systems. IEEE Trans. Audio Speech Lang. Process. 14(5), 1557–1565 (2006) 4. J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

Improvement of Speaker Verification …

443

5. Y. LeCun, G. Hinton, Deep learning. Nature 521, 436–444 (2015) 6. L. Deng, D. Yu, Deep learning: methods and applications. Found. Trends R Signal Process. 7(3–4), 197–387 (2014) 7. N. Dehak, P.J. Kenny, R. Dehak, P. Dumouchel, P. Ouellet, Front-end factor analysis for speaker verification. IEEE Trans. Audio Speech Lang. Process. 19(4), 788–798 (2011) 8. W. Campbell, D. Sturim, D.A. Reynolds, Support vector machines using gmm supervectors for speaker verification. Signal Process. Lett. IEEE 13(5), 308–311 (2006) 9. P. Kenny, V. Gupta, T. Stafylakis, P. Ouellet, J. Alam, Deep neural networks for extracting baum-welch statistics for speaker recognition. Odyssey (2014) 10. Y. Lei, N. Scheffer, L. Ferrer, M. McLaren, in A Novel Scheme for Speaker Recognition Using a Phonetically-Aware Deep Neural Network. Acoustics, Speech and Signal Processing (ICASSP), IEEE International Conference on. IEEE (2014), pp. 1695–1699 11. G. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527 12. L. Ferrer, Y. Lei, M. McLaren, N. Scheffer, Study of Senone-based deep neural network approaches for spoken language recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 24(1), 105–116 January (2016) 13. E. Singer, P. Torres-Carrasquillo, T. Gleason, W. Campbell, D.A. Reynolds, in Acoustic, Phonetic and Discriminative Approaches to Automatic Language Identification. Proc. European Conference on Speech Communication and Technology (Eurospeech), Geneva, Switzerland, September (2003), pp. 1345–1348 14. B. Jyotsna, A. Murthy Hema, T. Nagarajan, Language identification from short segment of speech. Proc. ICSLP-2000 III, 1033–1036 (2000) 15. P. Kenny, in Bayesian Speaker Verification with Heavy Tailed Priors. Proc. Odyssey (2010) 16. U. Bhattacharjee, K. Sarmah, in A Multilingual Speech Database for Speaker Recognition. Proc. IEEE, ISPCC, March (2012) 17. U. Bhattacharjee, K. Sarmah, Development of a speech corpus for speaker verification research in multilingual environment. Int. J. Soft Comput. Eng. (IJSCE) 2(6), 443–446 January (2013). ISSN: 2231–2307

Hybrid Texture-Based Feature Extraction Model for Brain Tumour Classification Using Machine Learning Ishfaq Hussain Rather, Sonajharia Minz, and Sushil Kumar

Abstract The effort of detecting brain tumours by radiologists or clinical experts is arduous and time-consuming, and their accuracy is dependent on their level of knowledge. Medical scans, such as magnetic resonance imaging (MRI), provide a wealth of data that can be exploited to overcome these constraints by creating advanced methodologies and approaches for tumour detection. These approaches can assist radiologists in offering a second opinion when predicting tumours, hence reducing the human aspect in the process. In this context, the paper proposes a hybrid texture-based feature extraction (HTFE) technique by employing Grey level co-occurrence matrix (GLCM) and Gabor Filters for identifying brain tumours. Specially, the proposed HTFE technique assists the classifiers Gradient Boosting (GB), Random Forest (RF), and Decision Tree (DT) in predicting Glioma, Meningioma, and Pituitary brain tumours from T1-weighted contrast-enhanced MRI (T1CEMRI) dataset. To train and evaluate the classifiers, the HTFE technique extracts a total of seventy-two second order texture features from T1-CEMRI. In terms of accuracy, the suggested HTFE approach beats state-of-the-art techniques. Keywords Grey level co-occurrence matrix · Brain tumour · Random forest · Decision Tree · Gradient boosting · Gabor filters

1 Introduction The human body comprises trillions of cells [1]. Only when the old cells need to be replaced are new cells produced. A tumour forms when the body produces new cells I. H. Rather (B) · S. Minz · S. Kumar Jawaharlal Nehru University, New Delhi, India e-mail: [email protected] S. Minz e-mail: [email protected] S. Kumar e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_38

445

446

I. H. Rather et al.

despite the fact that there is no need for them. Tumour in brain is an unnecessary blob of cells that develops with in brain or central spinal canal. Correct and timely detection of a brain tumour can improve the patient’s chances of survival [2]. As a result, early detection of a brain tumour is critical. A technique based on radiations such as MRI is used to visualize the internal structure of the body organ. The most common application of MRI is for cancer detection. The contrast in images plays an important role in the analysis because the information about the brain is obtained from the variations in the intensity of the MRI signal [3]. MRI image evaluation and analysis is a time-consuming and error-prone operation for radiologists [4]. Manual segmentation of brain MRI has long been regarded as the “gold standard” for locating a lesion. For automatic disease detection, medical image pre-processing, computer-aided diagnosis, image segmentation, and classification ML algorithms have been used [5–7]. To train and evaluate the ML-based classifiers, a number of feature extraction approaches are employed. These feature extraction approaches, on the other hand, are computationally expensive. Furthermore, the accuracy of deep learning algorithms utilized for brain tumour classification [8, 9] is reliant on the magnitude of the data collected, and their performance is also computationally costly. We address the problems of accuracy and prediction time in brain tumour diagnosis as a result of the aforementioned research. To solve the aforesaid difficulties, we suggest a new HTFE approach. In this work, the HTFE approach extracts texture features using a combination of the GLCM and Gabor Filters. The GLCM model recovers the MRI’s global neighbourhood in the spatial domain, whilst the Gabor Filters gathers local information that would be multi-scale and multi-directional in the frequency domain. Finally, to detect Glioma, Meningioma, and Pituitary Brain Tumours, the feature vector comprises properties that are passed to an RD, DT, and GB classifier. The following are the paper’s key contributions: • Firstly, the proposed HTFE technique is presented in detail for the extraction of features. • Secondly, the process of training, testing, and classification of brain tumours using RF, DT, and GB classifiers are explained. • Finally, the suggested model’s classification accuracy was compared to that of existing state-of-the-art deep learning and ML algorithms using an empirical method.

2 Literature Review In [8] brain tumour was predicted by applying two feature extraction algorithms derived from the local binary pattern (LBP). The two techniques are nLBP and αLBP. These approaches use distance from the neighbouring pixels and the pixel values based on the angle. [9] used a three-step technique, such as skull stripping, “Firefly-assisted fuzzy entropy-based multi-thresholding”, and distance regularized level set-based segmentation. Then Minkowski distance measure was employed to analyse the size of a cancer mass after tumour region is extracted. GLCM and Haar

Hybrid Texture-Based Feature Extraction Model …

447

wavelet is utilized for feature extraction. Authors in [10] discuss Slantlet transform (SLT) combined with neutrosophy (NS) which is an upgraded variant to extract features by using the discrete wavelet transform (DWT). The malignant brain tumour is identified using the statistical characteristics provided by NS-SLT. The features extracted are reduced by using one-way ANOVA. Then four neural network classifiers are used to classify the brain tumours. In [11] authors presented a Self-Organizing Maps (SOM) clustering to conduct brain MR image segmentation. The characteristics were retrieved through Histogram Equalization and GLCM. For the feature selection, Principle Component Analysis (PCA) method was utilized to enhance the accuracy. To automatically detect the tumour from the brain MRI Proximal Support Vector Machines (PSVM) were employed. In [12] the characteristics LBP (texture), HOG (Shape), and SFTA are extracted and merged in serial-based approaches. Then the Boltzmann entropy technique is used to choose the characteristics. These features are applied to various classifiers. The Dice score of 0.99 was attained with the BRATS dataset. Reference [13] used fully connected and CNN approach to classify MRI data into three different cancer categories. The average five-fold cross-validation resulted in an accuracy of 91.43%. Reference [14] reported ischemic stroke detection method for diffusionweighted image (DWI) series of MRI. Expectation-maximization (EM) was utilized for segmentation then “Fractional-order Darwinian particle swarm optimization” (FODPSO) was employed to increase the classification accuracy of SVM and RF. Reference [15] presented a classification approach based on GLCM and CNN for classifying MRI images into three classes viz; meningioma, glioma, and pituitary cancers. Reference [16] “Adaptive Regularized Kernel-based Fuzzy C-Means Clustering” (ARKFCM) approach was employed for the process of segmentation. Then two classification algorithms SVM and ANN were utilized for tumour classification.

3 Hybrid Texture-Based Feature Extraction (HTFE) Technique This section provides the details of hybrid texture-based feature extraction technique. It focusses on the architecture of HTFE model and its associated feature extraction methods.

3.1 Architecture of HTFE Model The proposed architecture of HTFE consists of four main components (Fig. 1): T1 weighted CEMRI dataset, GLCM, and Gabor filter methods for feature extraction of brain tumours from MRI images, the concatenation of the features extracted from GLCM and Gabor filters, classifiers like RF, DT, and GB to classify MRI slices with

448

I. H. Rather et al.

Fig. 1 Architecture of the proposed HTFE model

the most prevalent tumours like: Glioma, Meningioma, and Pituitary brain tumours. The detailed working of each components is presented in the preceding sections.

3.2 Grey Level Co-occurrence Matrix (GLCM) A matrix of various co-occurring pixel values defined on an image is referred to as GLCM. A set of second order texture characteristics are retrieved here. The features that evaluate the relationship between groups of two pixels in the source image are known as second order texture features as can be seen in Fig. 2. GLCM has the same number of rows and columns as the number of grey levels ‘G’ in the image. The distance ‘d’ and the positional angle ‘θ ’ between the two pixels (i, j) and (m, n) are the two parameters that affect the computation of GLCM. ‘θ ’ represent four different directions as shown in Fig. 2c. The co-occurrence values in the GLCM are normalized as shown in Eq. (1). fi j Pi j =  N  N i=0

j=0

fi j

(1)

i, j represents grey levels along vertical and horizontal directions. Pi j is the matrix after normalization, f i j is the frequency of grey levels within a window. A total of eight different GLCM matrices are calculated from each image by keeping pixel distance ‘d’ first as 1 and then 2, After this the value of orientation ‘θ ’ is changed between four angles 135°, 90°, 45°, and 0°. A single value is calculated to

Hybrid Texture-Based Feature Extraction Model … 0

1

1

2

2

0

2

2

2

2

1

1

3

3

3

1

0

3

3

1

1

2

2

2

0

0

1

1

1

1

2

1

1

1

0

6

0

0

0

0

3

449

a

b

c

Fig. 2 GLCM construction a shows Grey level image, b shows the generation of co-occurrence matrix, c image texture variation with distance and orientation

summarize the GLCM matrix. These values are defined from Eqs. 2 to 7, which are second order texture values. Thus, a total of forty-eight texture features is extracted. (a) Contrast: This measure uses weights related to the distance from the diagonal elements of GLCM. The contrast at diagonal element is always 0. It increases exponentially as we go away from the diagonal. Contrast = 0 in all directions means an image is entirely uniform. Contrast =

N 

P(i, j)

(2)

i, j=1

(b) Energy: Energy will be high when the image is very orderly i.e. it will give value equal to one when all pixels are identical. It will increase with the increase in the order of the image. Energy =

N 

P(i, j )2

(3)

i, j

(c) Correlation: It assesses the linear relationship between grey levels and those of neighbouring pixels. High correlation indicates high predictability of relationship amongst pixel values.   N  (i − μi ) j − μ j ∗ P(i, j ) Correlation = σi σ j i, j

(4)

450

I. H. Rather et al.

(d) Homogeneity: The weights are valued based on the inverse of distance from the diagonal. The weights will decrease exponentially as we go away from the diagonal. Homogeniety =

 i

j

Pi j 1 + |i − j|

(5)

(a) Dissimilarity: Dissimilarity can be defined as follows. The weights in the dissimilarity increase linearly. Dissimilarity =

N −1 

Pi j |i − j|

(6)

i, j=0

(b) Entropy: Entropy refers to the variability in the image. More variability means more information. Entropy will be high when a complex variability is present in an image. Entropy is considered as a first order degree. The entropy calculated from the GLCM matrix is the Shannon entropy defined as follows. Entropy = −

N 

P(i, j)(log p(i, j ))

(7)

ij

where P (i, j) is the probability of i and j, μi and μj are the GLCM matrix’s mean in ‘x’ and ‘y’ directions, respectively, and σ i and σ j are the GLCM matrix’s standard deviation in ‘x’ and ‘y’ directions, respectively.

3.3 Gabor Filter Gabor filters are a type of convolutional filter that uses a Gaussian Kernel function modulated by a sinusoidal plane wave to represent a Gaussian Kernel function. The weights are provided by the Gaussian component, whilst the directionality is provided by the sine component as shown in Eq. (8). The analysis of images by using Gabor filters has been considered similar to human perception by many authors like [17]. We can extract a whole bunch of useful features by varying the parameters like ‘θ ’,’f ’, ‘σ ’     2 x x + γ 2 y 2 exp i 2λ + ϕ (8) g(x, y; σ, λ, γ , θ, ϕ) = exp − 2σ 2 λ where x  = x cos cos θ + y sin sin θ and y  = −x sin sin θ + y cos cos θ

Hybrid Texture-Based Feature Extraction Model …

451

‘σ ’ represents the standard deviation of the Gaussian envelope, ‘λ’ represents wavelength of the sine component, ‘γ ’ represents spatial aspect ratio, ‘θ ’ is the filter orientation, ‘ϕ’ represents phase offset. Following parameters are used for generating filters. The value of ‘θ ’ used are 0°, 45°, 90°, and 135°. Three different spatial frequency values are used 1/4, 1/6, 1/8 (‘λ’ = 4, 6, 8). A total of twelve filtered images are obtained and from which energy and entropy features are extracted. Thus, a total of twenty-four features are extracted. All the features extracted by GLCM and Gabor filters are concatenated or stacked horizontally making it a total of 72 features to be used to train and test our classifiers.

3.4 Classification The classification process was carried out utilizing the DT, RF, and GB algorithms. There are several applications of DT classifiers in the literature, ranging from remote sensing images to medical diagnostics, some of which are mentioned in [18]. RF is made up of DTs, and it combines the simplicity of DT with the flexibility of RF, resulting in a significant increase in accuracy [19]. Each step in RF results in a wide variety of DTs. RFs are more effective than individual DTs because of their variability. GB algorithms can be highly customized to the particular needs of an application. They have shown great success in a wide range of applications [20].

4 Experimental Results The performance of the proposed HTFE technique is carried out by conducting the experiment and the results obtained are compared with state-of-the-art models to show its effectiveness. The model was implemented by writing our scripts in python 3.6. It also made use of the matplotlib library for visualizing metrics and sklearn for plotting the confusion matrix, accuracy score, and classification report.

4.1 Dataset Description The data for this study was gathered between 2005 and 2010 from “Nanfang Hospital, Guangzhou, China, and Tianjin Medical University General Hospital.” [21]. The collection contains 2D slices of 1426 gliomas, 930 pituitary, and 708 meningiomas, representing three tumour types. Figure 3 shows graphs plotted for the number of images in each tumour class. The dataset was divided into a training set (85%) and a testing set (15%). Since the dataset is imbalanced a stratified sampling technique

452

I. H. Rather et al.

Glioma

Pituitary

Meningioma

1500 1426

1000 500 0

908

708

Glioma 300 200 100 0

Types of Brain Tumours

Pituitary

106

Meningioma

214

140

Brain tumour test dataset with equal distribution of classes

a

b

Fig. 3 Shows the distribution of classes a shows the distribution of classes in the dataset and b shows the equal distribution of classes in the test set

is used to split the data. Figure 3b shows the proportion of different classes in the testing dataset after stratified sampling.

4.2 Training, Testing, and Evaluation Metrics The selected features are stacked horizontally and passed to the classifiers RF, GB, and DT for training and testing. The commonly used performance evaluation measures like accuracy, precision, recall, and F-measure are used to validate our model as shown in Table 1. Figure 4 shows the confusion matrices for this setting. Accuracy (A) is defined as the number of right predictions (true positives and false negatives) divided by the total number of predictions as shown in Eq. 9. Precision (P) can be defined as the proportion of all positive predictions that were right to the total number of actual positives see Eq. 10. Recall (R) of a model can be defined as the total True Positives identified correctly as defined in Eq. 11. It is important to consider precision and recall too in addition to accuracy metrics for evaluating the model. In situations where both precision and recall are important one more measure called F1-score is very important. It is the harmonic mean of precision and recall. Acc =

TP + FN TP + TN + FP + FN

(9)

Table 1 Shows precision recall and F1 score Classes

Random forest (P)

(R)

Gradient boosting (F1)

(P)

(R)

Decision tree (F1)

(P)

(R)

(F1)

Class 0

0.85

0.92

0.88

0.81

0.91

0.86

0.77

0.84

0.83

Class 1

0.96

0.94

0.95

0.97

0.92

0.94

0.91

0.90

0.90

Class 2

0.98

0.95

0.96

0.96

0.93

0.94

0.95

0.91

0.93

Average

0.93

0.94

0.93

0.91

0.92

0.91

0.94

0.88

0.87

Hybrid Texture-Based Feature Extraction Model …

453

Fig. 4 shows Confusion matrices a Confusion matrix for random forest. b Confusion matrix for gradient boosting, and c Confusion matrix for decision tree classifier

P=

TP TP + FP

(10)

R=

TP TP + FN

(11)

F1 - score =

2∗ P ∗ R P+R

(12)

4.3 Comparison with the Existing Approaches Figure 5 compares the proposed HTFE model with RF classifier to state-of-the-art models in terms of accuracy. It is clearly observed that the proposed HTFE technique achieves the accuracy of 93.91% with RF classifier where as the model proposed by Paul et al. [13] achieves 90%, Widhiarso et al. [15] achieves 93%, Bhat [16] achieves 91.4%, and Subudh et al. [14] achieves 88%. The HTFE model with RF classifier beats the state-of-the-art models in terms of accuracy. This is due to the fact that the MRI variances are accurately represented by the features retrieved using HTFE algorithms. The evaluation metric used for this comparison is classification accuracy (A) defined in Eq. 9.

5 Conclusion In this paper, Our Hybrid Texture-Based Feature Extraction (HTFE) model for feature extraction from MRI data is presented. The model focusses on HTFE architectures, GLCM and Gabor Filters, feature concatenation, and classification. To forecast three

454

95 94 93 92 91 90 89 88 87 86 85

I. H. Rather et al.

Paul et al., 2017 Widhiarso et al., 2018 Subudhi et al., 2020 P et al., 2019

93.91 93 91.4

90 88

Paul et al., 2017 Widhiarso et al., 2018

Subudhi et al., 2020

P et al., 2019

HTFE with RF

Fig. 5 Shows the results of comparisons with earlier methodologies based on the classification accuracy parameter

tumour classes Glioma, Meningioma, and Pituitary Tumour, seventy-two second order texture characteristics are derived from eight GLCM and six distinct Gabor filtered images. RF, GB, and DT classification methods were utilized for the process of classification. RF had the best classification accuracy of 93.91%. The feature extraction-based classification model proposed in this study outperformed numerous traditional feature extraction-based and deep learning models when compared with the state-of-the-art methods. Because of its simplicity, low computing requirements, and ease of application, the suggested approach may be utilized by radiologists to make diagnosis.

References 1. B.P. Jena, D.L. Gatti, S. Arslanturk, S. Pernal, D.J. Taatjes, Human skeletal muscle cell atlas: Unraveling cellular secrets utilizing ‘muscle-on-a-chip’, differential expansion microscopy, mass spectrometry, nanothermometry and machine learning. Micron 117, 55–59 (2019). https:// doi.org/10.1016/j.micron.2018.11.002 2. S. Siuly, Y. Zhang, Medical big data: neurological diseases diagnosis through medical data analysis. Data Sci. Eng. 1(2), 54–64 (Jun. 01, 2016). https://doi.org/10.1007/s41019-0160011-3 3. J. Taranda, S. Turcan, 3d whole-brain imaging approaches to study brain tumors. Cancers 13(8). MDPI AG (Apr. 02, 2021). https://doi.org/10.3390/cancers13081897 4. O. Commowick, F. Cervenansky, F. Cotton, M. Dojat, MSSEG-2 challenge proceedings: multiple sclerosis new lesions segmentation challenge using a data management and processing infrastructure (2021). [Online]. http://portal.fli-iam.irisa.fr/msseg-2/ 5. S. Hussain, S.M. Anwar, M. Majid, Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282, 248–261 (2018). https://doi.org/10.1016/j.neu com.2017.12.032

Hybrid Texture-Based Feature Extraction Model …

455

6. S.M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, M.K. Khan, Medical image analysis using convolutional neural networks: a review. J. Med. Syst. 42(11) (Nov. 01, 2018). https://doi.org/10.1007/s10916-018-1088-1 7. M.W. Nadeem et al., Brain tumor analysis empowered with deep learning: a review, taxonomy, and future challenges. Brain Sci. 10(2). MDPI AG (Feb. 01, 2020). https://doi.org/10.3390/bra insci10020118 8. K. Kaplan, Y. Kaya, M. Kuncan, H.M. Ertunç, Brain tumor classification using modified local binary patterns (LBP) feature extraction methods. Med. Hypotheses 139 (Jun. 2020). https:// doi.org/10.1016/j.mehy.2020.109696 9. V. Rajinikanth, N. Sri, M. Raja, K. Kamalanand, Firefly algorithm assisted segmentation of tumor from brain MRI using Tsallis function and Markov Random Field (2017). [Online]. https://www.researchgate.net/publication/320431856 10. S.H. Wady, R.Z. Yousif, H.R. Hasan, A novel intelligent system for brain tumor diagnosis based on a composite neutrosophic-slantlet transform domain for statistical texture feature extraction. BioMed Res. Int. (2020). https://doi.org/10.1155/2020/8125392 11. K.B. Vaishnavee, K. Amshakala, An automated MRI brain image segmentation and tumor detection using SOM-clustering and Proximal Support Vector Machine classifier (Sep. 2015). https://doi.org/10.1109/ICETECH.2015.7275030 12. Jouf University and Institute of Electrical and Electronics Engineers, 2019 International Conference on Computer and Information Sciences (ICCIS) : Jouf University-Aljouf-kingdom of Saudi Arabia, 03–04 April 2019 13. J.S. Paul, A.J. Plassard, B.A. Landman, D. Fabbri, in Deep Learning for Brain Tumor Classification, Medical Imaging 2017: Biomedical Applications in Molecular, Structural, and Functional Imaging, Mar. 2017, vol. 10137, p. 1013710. https://doi.org/10.1117/12.2254195 14. A. Subudhi, M. Dash, S. Sabut, Automated segmentation and classification of brain stroke using expectation-maximization and random forest classifier. Biocybern. Biomed. Eng. 40(1), 277–289 (2020). https://doi.org/10.1016/j.bbe.2019.04.004 15. W. Widhiarso, Y. Yohannes, C. Prakarsah, Brain tumor classification using gray level cooccurrence matrix and convolutional neural network. IJEIS (Indonesian J. Electron. Instrum. Syst.) 8(2), 179 (2018). https://doi.org/10.22146/ijeis.34713 16. T.P.B. Bhat, K. Prakash, Detection and classification of tumour in brain MRI. Int. J. Eng. Manuf. 9(1) (11–20 Jan. 2019). https://doi.org/10.5815/ijem.2019.01.02 17. E. Vazquez-Fernandez, A. Dacal-Nieto, F. Martin, S. Torres-Guijarro, Entropy of Gabor Filtering for Image Quality Assessment 18. S.R. Safavian, D. Landgrebe, A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991). https://doi.org/10.1109/21.97458 19. J. Ali, R. Khan, N. Ahmad, I. Maqsood, Random Forests and Decision Trees (2012). [Online]. www.IJCSI.org 20. A. Natekin, A. Knoll, Gradient boosting machines, a tutorial. Front. Neurorobotics 7 (no. DEC, 2013). https://doi.org/10.3389/fnbot.2013.00021 21. J. Cheng et al., Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLoS ONE 10(10) (Oct. 2015). https://doi.org/10.1371/journal.pone.014 0381

Intensity and Visibility of PSO-Based Waveguide Arrays with an Overview of Existing Schemes and Technologies for Multi-beam Combination Simarpreet Kaur , Mohit Srivastava , and Kamaljit Singh Bhatia

Abstract Spectro-interferometry is a sophisticated astronomy technology that produces high-resolution pictures of celestial objects and allows researchers to investigate their morphological characteristics. In this paper, a waveguide array of 2 × 2 waveguides has been investigated with more than one waveguide excited simultaneously. Each and every waveguide available operates as a beam combiner, whose results determine the waveguide intensity, which means, the output intensity is determined by the waveguides used for stimulation. Hence, it is very important to select the appropriate waveguide in order to increase the intensity and visibility of outputs. The primary goal of this study is to determine which metaheuristic technique can fix the issues of waveguide selection. To achieve this, a basic variant of the Particle Swarm Optimization algorithm has been implemented and its software simulation has been done. An analysis is also carried out by calculating the performance matrix concerning magnification, intensity, visibility, and 1/Signal to Noise Ratio. Simulation results are compared with the existing models, which demonstrate an improvement of the proposed system by achieving high intensity and visibility. Keywords Interferometers · MZI · Waveguides array · Waveguide selection · PSO · Intensity · Visibility · SNR

1 Introduction Interferometry is a technique that is utilized during the optical astronomy to integrate waves generated from multiple telescopes so that higher-resolution images can be S. Kaur (B) IKG Punjab Technical University, Jalandhar, Punjab, India e-mail: [email protected] M. Srivastava Chandigarh Engineering College, Landran, Punjab, India K. S. Bhatia G.B. Pant Institute of Engineering and Technology, Pauri Garhwal, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_39

457

458

S. Kaur et al.

attained. This technology serves as the foundation for optical astronomical interferometer arrays that can detect very small celestial objects by distributing telescopes over a large region. When a huge proportion of telescopes are utilized, an image which has quality of a solo telescope along with the size of the total range of telescopes can be formed. However, when four or more telescopes were combined, the limits of planar combined optics became apparent as complexity increases when couplers are utilized in beam combination [1]. The basic structure of an optical interferometer telescope is shown in Fig. 1. After analyzing the figure, it is clear that the delay lines and two utilized telescopes can be adjusted in order to equalize the path for optical length in two arms of interferometer, and detector including beam combiner. A discrete beam combiner (DBC) is used to overcome the limitations of the planer integrated optics, according to which if N telescopes are integrated, a standard 2D array of N × N temporarily coupled waveguides are utilized [2]. Due to the temporary coupling, a light wave is injected in the solo waveguide that will stimulate nearby waveguides of the array. Each and every waveguide will transmit a small proportion of light as input with magnitude and phase determined solely by the location of input waveguide and length of the array. When additional waveguides are excited at the same time, the propagating fields from each input waveguide add up constructively in the array’s waveguides, resulting in a discretized interference pattern [3]. The key difficulty is determining the best waveguides from the available huge number of waveguides, which, if accomplished, can lead to high intensity outputs [4].

Fig. 1 Optical interferometry telescope [1]

Intensity and Visibility of PSO-Based Waveguide …

459

1.1 Metaheuristic Algorithms However, there are large number of optimization algorithms like Genetic Algorithm (GA), Ant Colony Optimization (ACO), Artificial Bee Colony (ABC), Grey Wolf Optimization (GWO), Particle Swarm Optimization (PSO) applied to many fields. GA is a famous algorithm, but due to random crossover and mutation processes, its performance gets affected that results in slow convergence rate [5]. Ant Colony Optimization (ACO) algorithm is a population-based computational optimization algorithm that mimics the behavior of ants in nature. However, the major drawback of ACO is its slower convergence rate, and the time it takes to reach convergence is also unpredictable during big search space problems which degrades its performance [6]. Particle Swarm Optimization (PSO) is a non-linear approach that comes under the umbrella of evolutionary algorithms. It is a class of algorithms that search for the optimal solution by replicating the behavior and flocking of birds. The method operates by scattering a flock of birds around the search area at random, each bird being referred to as a particle. It is simple and robust to control different factors with high computational efficiency [7]. There are several search algorithms that can be considered for the proposed problem solution, but PSO algorithm has been selected for this research after reviewing its simplicity and ability to find optimum solutions in complicated search spaces.

1.2 Particle Swarm Optimization (PSO) PSO is a basically a form of swarm intelligence algorithm that mimics natural biological predation patterns. PSO employs grouping intelligence, which stems from collaboration and competition among various particle groups. The ‘swarm’ is initiated in PSO with the population of the random solutions. Each particle in the swarm represents a unique set of anonymous variables to be adjusted [7]. Each particle, which represents a point in state space, changes the flying experience, and conveys social information among particles. The main goal is to efficiently seek for a resolution by pairing the particle to the best-fitting way out, as determined by the process in previous iterations, and eventually concentrate on a unique minimum-error solution.

2 Proposed Model Let us discuss the technique presented using PSO algorithm. The brief overview on how the proposed system works is explained below; Step 1: The very first step in the process is to initialize various PSO parameters like population size which include velocity of particle, waveguide’s set, and so on.

460

S. Kaur et al.

Step 2: The next step that is followed in the proposed system is to calculate the fitness function of the given initial population. In the proposed work, we have calculated the fitness value of population by summing up the intensities as per the equations given in Eqs. 1 and 2; I =

N N  

< > ∗ ∗ αm, f ( j) αm, f (k) A j Ak

(1)

j=1 k=1

fitness =

n 

Ij

(2)

j=1

where n is the number of waveguides to be selected. By using Eq. 1, the intensity of waveguides can be calculated. This process is repeated for every waveguide and then the fitness is evaluated by considering the intensities of individual waveguides as per Eq. 2. Step 3: Once the initial fitness is calculated, the next step is to select the best waveguide. The best waveguide set is selected with great intensity values that are saved in pbest. In addition to this, and a waveguide set whose fitness value came out to be optimum will be saved in gbest. Step 4: This gbest represents the best-fitted waveguide set for current iteration and will be compared with the remaining iterations that need to be performed. Step 5: Next step is to update the initial waveguide numbers or population by using velocity updating Eq. (3a) given below: ( ( ) ) v(t + 1) = W (t) ∗ v(t) + c1r1 pi,d − xi,d (t) + c2 r2 pbest − xi,d (t)

(3a)

Step 6: Once the velocity attribute is updated, new number of waveguides will be achieved by using: xi.d (t + 1) = xi,d (t) + v(t + 1)

(3b)

where xi,d (t) is the population or waveguide numbers of iteration number t. Step 7: After performing the given number of iterations, the best gbest values are obtained which represent the selected waveguide number.

2.1 PSO Parameters Selection PSO algorithm includes some internal parameters for tuning of the algorithm. These parameters are inertia weight, population size, accelerate constant c1 and c2, and number of iterations. In this simulation, these factors are selected as per the literature, as well as, the experiments performed with the proposed model.

Intensity and Visibility of PSO-Based Waveguide …

461

(a) Selection of Inertia weight This factor gives idea about the effect of previous iteration’s velocity on the current iteration’s velocity. Controlling this, generally affects the local and global search ability of the particles. w = 0 means that the particle velocity be governed by its present personal position (pbest) and the global best position (gbest). There is no memory of velocity. w /= 0 means that the particle can explore new space, and the flying velocity of the particle is directly proportional to w. In this model, the inertia weight has been selected by using the equation given in [7], ( w = wmax −

wmax − wmin t Itertaions

) (4)

where wmax and wmin are maximum and minimum inertia weights considered in this model as 0.7 and 0.2, respectively. “t” represents the number of current iteration. (b) Selection of accelerate constants The two constants c1 and c2 represent the particle’s acceleration weight toward personal and global best. Its small value can result in premature convergence and the greater value can result in slow convergence or no convergence at all. The values considered in the proposed model are c1 = c2 = 2, finalized by experiments done on the proposed model design and by referring [8]. (c) Selection of Particle Numbers/Population Count It is observed that when the particle count is small, i.e., less than 50, its impact on PSO performance is very high and vice-versa. Moreover, it increases the computation time, and reliability. So, for the proposed scheme, the value of particles has been decided to be 20. For this, 2 × 2 array has been considered. Moreover, various important initialization factors are also taken into consideration, whose values are recorded in Table 1. The selected waveguides are then evaluated and simulated in terms of their intensity, visibility, and 1/SNR values. Table 1 Optimization system and waveguide Setup

S.No

Parameters

Values

1

Waveguide arrays

2× 2

2

Simulated wavelengths

3.1–3.6 µm

3

Reflectivity

0.2

4

Iterations

20

5

Population size

20

6

Inertia factor minimum (W min )

0.2

7

Inertia factor maximum(W max )

0.7

8

Accelerate constants C1, C2

2

462

S. Kaur et al.

1. Intensity The waveguides intensity should always be high to achieve an effective result. The peak intensity at the output of an mth waveguide of length z = L can be represented as: |2 \ /| N N N  | |  < > | | ∗ ∗ Im = | αm, f (k) Ak | = αm, f ( j) αm, f (k) A j Ak | | k=1

(5)

j=1 k=1

where f (k) depicts the function, whose values range from k = 1…N to the M waveguide where fields of Ak are coupled. αm, f (k) coefficients illustrate the mode amplitudes at the end of waveguide’s (m) end when unit power field is injected to site f (k) at z = 0 [9]. 2. Visibility In addition to the waveguide intensity, the visibility must also be high for attaining the positive output. The visibility of synchronized and normalized fields can be calculated by the equation given in 6, ⌈ |) / \(2 ) / \(2 | + I Ai A∗j | R Ai A∗j / \/ \ Vi j = | i /= j | Ai A∗j Ai A∗j

(6)

where R and I are the real and imaginary parts of the complex field product [10]. 3. 1/SNR Another critical factor that is considered for evaluating the performance is 1/SNR value. This value should be low for determining the effective output in system whose values can be calculated as per the equation given in 7; / 1 = SNR

N −1 2I0

(7)

where N depicts the number of telescopes and I0 represents the intensity [11].

3 Results and Discussion In this section, results obtained after implementing the proposed work are represented and discussed. The comparative analysis is performed among proposed PSO-based system and traditional systems [12–14] for 2 × 2 and 3 × 3 waveguides as per their magnification, normalized intensity, visibility, and 1/SNR values. The results obtained in terms of all the considered parameters are shown graphically as below.

Intensity and Visibility of PSO-Based Waveguide …

463

4 FOR 2 × 2 Waveguide Array Firstly, the simulation has been performed for 2 × 2 waveguide array, in terms of a two-telescope combiner with magnification and normalized intensity by considering different wavelength ranges. Figure 2 shows the magnification of the two-telescope combiner and Fig. 3 shows the normalized intensity obtained for the 2 × 2 waveguide. The normalized intensity of the 2 × 2 waveguide array is also compared with conventional Bar and Cross models [12] at different wavelength from 3.1 to 3.6 µm, as shown in Fig. 4. Both of these have a lower intensity as compared to the intensity of the proposed PSO-model which is 0.8. In addition to this, the numerical comparison of intensity is also performed with the 2-telescope model presented in [13]. Figure 5 represents the convergence graph for PSO algorithm which presents the relation between the fitness in terms of intensity summation w.r.t iterations at wavelength 3.1 µm. The analysis of minimum and maximum 1/SNR of the proposed system at 105 detected photons [12] for 2 × 2 waveguides is performed and the values recorded from the obtained results are delineated in Table 2.

Fig. 2 Two-telescope combiner with magnification

464

S. Kaur et al.

Fig. 3 Normalized Intensity for 2 × 2 waveguide array

Fig. 4 Comparison of normalized intensity for 2 × 2 waveguide with traditional [14] approach

Intensity and Visibility of PSO-Based Waveguide …

465

Fig. 5 Convergence graph of PSO for 2 × 2 waveguide model

Table 2 Minimum and Maximum 1/SNR for 2 × 2 waveguide array

Wavelength (um)

Minimum (1/SNR)

Maximum (1/SNR)

3.1

0.11417

0.22942

3.15

0.10361

0.22908

3.2

0.12218

0.22994

3.25

0.11845

0.25044

3.3

0.11477

0.22494

3.5

0.11207

0.21282

5 Conclusion In this paper, the interferometer-waveguide system based on PSO-approach is proposed for selecting the optimum waveguide from the available waveguides. To prove the efficacy of the suggested approach, its simulation has been performed with 2 × 2 waveguides and their performance is evaluated as per their magnification, normalized intensity, visibility, and 1/SNR. The proposed system is also compared with the traditional systems. The simulated results verify that metaheuristic algorithms can achieve an automated waveguide selection scheme to achieve high intensity output. Thus, for future work, variants of PSO or other metaheuristic algorithms can be implemented and analyzed on higher order waveguide arrays. Acknowledgements Simarpreet Kaur would like to thank Dean RIC, I.K. Gujral Punjab Technical University Jalandhar, Kapurthala for making required resources available throughout the completion of this research work.

466

S. Kaur et al.

References 1. E. Pedretti, S. Piacentini, G. Corrielli, R. Osellame, S. Minardi, in A Six-Apertures Discrete Beam Combiner for J-band Interferometry. Proc. SPIE 10701 (2018) 2. S. Kaur, M. Srivastava, K.S. Bhatia, in Optical Stellar Interferometry—A Review. 4th International Multi-Track Conference on Sciences, Engineering & Technical Innovations. IMTC-18, pp. 13–17. Jalandhar (2018) 3. S. Minardi, A. Saviauk, F. Dreiso, S. Nolte, T. Pertsch, in 3D-Integrated Beam Combiner for Optical Spectro-Interferometry. Proc. SPIE 9146 (2014) 4. S. Kaur, M. Srivastava, K.S. Bhatia, Simulation and analysis of telescopic photonic beam combiner for stellar interferometry in labview. Int. J. Innov. Technol. Exploring Eng. (IJITEE) 8(9S), 230–236 (2019) 5. S. Katoch, S.S. Chauhan, V. Kumar, A review on genetic algorithm: past, present, and future. Multimed. Tools Appl. 80, 8091–8126 (2021) 6. W. Deng, J. Xu, H. Zhao, in An Improved Ant Colony Optimization Algorithm Based on Hybrid Strategies for Scheduling Problem. IEEE Access, vol. 7, pp. 20281–20292 (2019) 7. M. Zemzami, N.E. Hami, M. Itmi, N. Hmina, A comparative study of three new parallel models based on the PSO algorithm. Int. J. Simul. Multidisc. Des. Optim. 11(5), 1–10 (2020) 8. Y. He, Y. Ma, J.P. Zhang, in The Parameters Selection of PSO Algorithm Influencing On performance of Fault Diagnosis. MATEC Web of Conferences, vol. 63, pp. 1–5 (2016) 9. S. Minardi, Photonic lattices for astronomical interferometry. Mon. Not. R. Astron. Soc. 422(3), 2656–2660 (2012) 10. S. Minardi, F. Dreisow, S. Nolte, T. Pertsch, in Discrete Beam Combiners: Exploring the Potential of 3D Photonics for Interferometry. Proc. SPIE—The International Society for Optical Engineering (2012) 11. R. Errmann, S.Minardi, in 6 and 8 Telescope Discrete Beam Combiners. Optical and Infrared Interferometry V, Proc. SPIE 9907 p. 990733 (2016) 12. L. Labadie, S. Minardi, J. Tepper, R. Diener, B. Muthusubramanian, J.U. Pott, S. Nolte, S. Gross, A. Arriola, M. Withford, in Photonics-based Mid-Infrared Interferometry: 4-year Results of the ALSI Project and Future Prospects. Proc. SPIE 10701, Optical and Infrared Interferometry and Imaging VI, 107011R (2018) 13. J. Tepper, L. Labadie, R. Diener, S. Minardi, J.U. Pott, R. Thomson, S. Nolte, Integrated optics prototype beam combiner for long baseline interferometry in the L and M bands. Astron. Astrophys. 602, A66 (2018) 14. S. Minardi, A. Saviauk, F. Dreisow, S. Nolte, T. Pertsch, in 3D-Integrated Beam Combiner for Optical Spectro-Interferometry. Optical and Infrared Interferometry IV, Proc. SPIE 9146, Montreal (CA), p. 91461D (2014)

Traffic Sign Recognition Using CNN Kavita Sheoran, Chirag, Kashish Chhabra, and Aman Kumar Sagar

Abstract With the revolutionary advancements in technologies such as machine learning and artificial intelligence, many big companies such as Google, Tesla, and Uber are working on creating autonomous vehicles or self-driving cars. Traffic sign recognition (TSR) plays a really important role in this as it is essential for vehicles to understand and follow all traffic rules to assure the safety of the passengers as well as other drivers and pedestrians on the road. In this paper, we study how traffic signs recognition can be done using machine learning. The dataset that we have used is taken from Kaggle and will contain around 50,000 images divided into various different classes. This dataset will be used for testing as well as training our model. We will be trying out two major approaches used in traffic sign recognition. Our approach is based on convolution neural network (CNN) in which we vary some parameters and prepare a comparative study of how these factors affect the accuracy of our model. After comparing the accuracy of our models, we have implemented the best performing model in a web application using Flask. Also, we have included the text-to-speech feature which speaks the result to the user and makes our project more accessible. Keywords Traffic signs · CNN · TSR · NumPy · Keras

1 Introduction In recent times, there has been such a revolutionary growth in the technology industry. The challenging task of developing autonomous vehicles such as self-driving cars is being taken up by big MNCs such as Tesla, Uber, Google, and Ford. One of the most important processes in achieving high levels of autonomy is traffic sign recognition K. Sheoran · Chirag (B) · K. Chhabra · A. K. Sagar Maharaja Surajmal Institute of Technology, Janakpuri, New Delhi 110058, India e-mail: [email protected] K. Sheoran e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_40

467

468

K. Sheoran et al.

(TSR). It is important for the car to be aware of and adhere to all driving laws in order to accomplishing higher precision in this innovation. It is essential for the safety of both the driver and other people on the road. Now, traffic signs are mainly separated among three sections: regulatory signs, warning signs, and guide signs. The dataset that we have chosen contains around 50,000 images divided into 43 classes where each class contains multiple images of a traffic sign. All three types of traffic signs have been included in the database. We have taken images of different variations such as blur images and low light images which will help in improving the performance of our model. In this paper, we present a comparative study of three models which will be based on the convolutional neural network approach. The approach is based on CNN, i.e. convolutional neural network, this enables us to retrieve the greater levels of depiction for the image content. CNN uses an unprocessed image pixel data, trains the model using it and then retrieves the image information for better classification. We will be designing a model and varying various factors such as the number of layers, activation functions, and comparing the results. The major objective of this proposed model is to recognize the traffic sign and caution the driver to react appropriately in order to prevent mishaps or accidents on the road. Sometimes, due to different issues, drivers get distracted and become less mindful towards the traffic signs which lead to fatal accidents causing injuries and even loss of lives in some cases. This becomes even more important in autonomous vehicles as the driver has very little or no control over the decision-making and vehicle handling, making it even more prone to accidents. Thus, TSR plays an important role in ensuring safety of driver as well as other vehicles and pedestrians on the road. The following is a summary of the structure of the paper. Section 2 summarizes present traffic sign detection and classification research. Both of the approaches that we have examined and analysed are shown in Sect. 3. The results obtained from both the approaches have been presented in Sect. 4, and finally, in Sect. 5, conclusion and future scope have been drawn from the obtained result.

2 Related Work There have been several studies dealing with the TSR problem in the past years. According to [1, 2], the first work on traffic sign recognition was published and presented in Japan in 1984. Over the years, various researchers presented numerous strategies for developing an efficient system which could solve the problems faced previously. The TSR system is divided mainly into the following phases: preprocessing of the dataset, detection, tracking, and recognition. Firstly, in the preprocessing phase, the main aim is to improve the visual appearance of the images. Various approaches are used to reduce the effect of environment on the images based on colour and shape features. There have been multiple efficient approaches for shape detection such as distance transform matching, similarity detection, and Hough transformation. A monitoring step is used in certain TSR techniques to help reduce the number of candidates passed towards the classification phase by identifying and

Traffic Sign Recognition Using CNN

469

following an object. The last step in any TSR system is recognition. Greenhalgh and Mirmehdi [3] studied for important classifiers: SVM, MLP, HOG based and decision trees. The decision tree approach was found to have the highest accuracy rate and was found to be the fastest [4]. When we discuss about recognition algorithms, CNN is one of the most popular approaches used for image classification. After the use of CNN for image classification, various researchers once again started adapting CNN approaches for object detection and recognition. The most popular are: Hough transformation [5, 6], similarity detection [7], distance transform matching [8], and edges with Haar-like features [9].

3 CNN-Based Approach Traffic sign recognition is based on current dataset assets and use strong identification calculations to correctly interpret recognized traffic signs and provide feedback to smart vehicles. CNN legally eliminates and includes from the information identification image and generates characterization findings using a predefined classifier based on image highlights. This circumstance demonstrates CNN’s excellent realistic acknowledgement execution. CNN also does not have to physically separate the participants. Through a forward learning and criticism system, the concrete psychological cycle of human brains may be re-enacted in a very real way, gradually enhancing the ability of traffic sign layout and recognition. The shortcomings of the previous style organize model are analysed in this section, and the model is greatly enhanced to also expand the exceptional focal points of CNN in design recognition. We will create a deep neural network model which helps categorize traffic signals in an image into many categories. We can read and understand traffic signs using our model, which is a critical duty for all autonomous vehicles. Vehicles must be able to comprehend traffic signs and make appropriate decisions in order for this technology to be accurate. Speed limits [10], prohibited entry, traffic signals, turn left or right, children crossing, no passing of big trucks, and so on are all examples of traffic signs. The method of deciding which class a traffic sign belongs to is known as traffic sign classification. Experiments are conducted on three types of traffic signs: warning signs, restriction signs, and required signs. We will use a CNN model to categorize the photos into their appropriate groups. For image categorization, CNN is the best alternative. Firstly, we start with a model with two convolution layers in which we have kept the kernel size as 5 × 5 and used the ReLU activation function. After running our model over the training data, we note down the model accuracy and loss values for the dataset. Next, we increase the number of convolution layers in our model to three and also change the kernel size to 3 × 3. Again, we note down the model accuracy and loss values for the same dataset as used earlier. Lastly, we increase the number of convolution layers to four and also keep the kernel size as 5 × 5 for first two layers and 3 × 3 for the next two layers. After running the model over the same dataset, we note down the accuracy and loss values and compare it

470

K. Sheoran et al.

Fig. 1 Proposed CNN model

with the previous values. The architecture of our model with four convolution layers is shown in Fig. 1: 1. Explore GTSRB dataset Each of the 43 folders in our train folder represents a different class (Fig. 2). The folder’s size ranges from 0 to 42. We iterate over all of the classes using the OS module, appending images and their labels to the data and labels list. To open picture content into an array, the PIL library is utilized. Finally, we have organized all of the images and labels into lists (data and labels). To feed the model, we must turn the list into NumPy arrays. The data has the shape (39,209, 30, 30, 3), indicating that there are 39,209 images of 30 × 30 pixels and that the last three indicate that the data comprises coloured images (RGB value). The train test split () method is used to split training and testing data by using sklearn package. 2. Train and Validation of the model

Fig. 2 Traffic signs classes of the GTRSB

Traffic Sign Recognition Using CNN

471

After constructing the model architecture, we use model.fit() to train the model. With 64 batch size, our model fared better. The accuracy was stable after 20 epochs. On the training dataset, our model had a 93% accuracy rate. Then, we plot the graph for accuracy and loss using matplotlib. 3. Test the model with test dataset A test.csv file in our dataset contains the details related to the image path and their corresponding class labels. We retrieve the image path and labels using pandas. Then, in forecast the model, we must resize our images to 30 × 30 pixels and put most of the image data into a NumPy array. We used sklearn.metrics() to get the accuracy score and looked at how our model predicted the real labels. Finally, we will use the keras.model() method to store the model that we have trained. 4. Traffic Sign Classification There are two steps to the classification and interpretation of traffic signs, namely the system preparation and testing stages. The preparation set examples of is used as information in the system preparation stage. Limits, for example, coordinate loads and balances, and are continuously modified depending on forward training and backward fomenting tools until the unfortunate work is limited to the basis, describing and predicting future traffic indications by performing a large number of system focuses. Lastly, the test set images are used for testing the finalized model and understanding how the model will perform in real time (Fig. 3). 5. Text-to-speech converter We have also included a text-to-speech feature in our project. We have used the pyttsx3 library which is a text-to-speech conversion library in Python and converts the entered text into speech. After comparing the results obtained from the three models, we implement the text-to-speech feature by feeding the result obtained from the model into the say() function available in the pyttsx3 library.

Fig. 3 Recognition results of traffic sign test images

472

K. Sheoran et al.

Table 1 CNNs trained in C-CNN approach summary Model

Kernel size

Epochs

Dataset image size

Test accuracy

CNN1

5×5

20

51,839

5

CNN2

3×3

20

51,839

99

CNN3

5 × 5 and 3 × 3

20

51,839

98

4 TSR Experimental Results This section details the results of a series of tests used to validate the proposed methodologies recognizing road signs. We have designed three models and trained them on the same dataset. In this section, we will compile and compare the results obtained and also conclude model which performs the best. First, we will go over the datasets we used in our trials, as well as the recognition outcomes. For the implementation, we used the Python programming language. Keras with a Tensorflow backend was utilized to train the CNN models. Below is a tabular comparison (Table 1) of the three models that we have tested for the dataset: Validation and training accuracy graph plot and validation and training loss graph plot are shown in Figs. 4 and 5, respectively.

5 Conclusion and Future Scope Out of the three models that we have designed, CNN2, i.e. the one with three convolution layers performed the best. In this model, we were able to attain a validation accuracy of 99% accuracy. This was the best model up until now. The model with two convolution layers, i.e. CNN1, clearly failed as the capacity of the model was not enough for the dataset, and thus, we got an accuracy of only 5%. Further, if we increase the number of layers after three in model CNN3, we see diminishing results as the accuracy starts decreasing. This is evident as the validation accuracy of the model with four layers was 98%. Compared to other calculations, the recommended computation is more respectable in terms of accuracy, improved long-term performance, more solid speculating abilities, and higher preparation effectiveness. Both the accurate acknowledgement rate and overall average processing duration were greatly improved. The suggested traffic sign recognition and estimate features surprising points of interest from the standpoint of traffic sign exactness and calculation tediousness. It is beneficial to improve the security of smart vehicles in realworld driving settings and to fulfil the on-going goal requirements of smart vehicles. Furthermore, the continuous advancement of smart car driving assistance is supported by a strong specialist assurance. To enhance the entire presentation of the calculation, both comprehensiveness and errors identification of the traffic signal identification computations may be improved and reinforced in future.

Traffic Sign Recognition Using CNN

Fig. 4 Validation and training accuracy graph plot

473

474

K. Sheoran et al.

Fig. 5 Validation and training loss graph plot

References 1. P. Pacl´ık, J. Novoviˇcov´a, R.P.W. Duin, Building roadsign classifiers using a trainable similarity measure. IEEE Trans. Intell. Transp. Syst. 7(3), 309–321 (2006) 2. A. Bouti, M.A. Mahraz, J. Riffi, H. Tairi, A robust system for road sign detection and classification using LeNet architecture based on convolutional neural network. Soft Comput. 1–13 (2019) 3. J. Greenhalgh, M. Mirmehdi, in Traffic Sign Recognition Using MSER and Random Forests, Proceedings of the 20th European Signal Processing Conference (EUSIPCO’12), pp. 1935– 1939, Aug 2012 4. J. Jin, K. Fu, C. Zhang, Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE Trans. Intell. Transp. Syst. 15(5), 1991–2000 (2014) 5. G. Overett, L. Petersson, in Large Scale Sign Detection Using HOG Feature Variants. Proceedings of the IEEE Intelligent Vehicles Symposium (IV ’11), pp. 326–331, Baden-Baden, Germany, June 2011 6. M.M. Møgelmose, M. Trivedi, T.B. Moeslund, Visionbased traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans. Intell. Transp. Syst. 13(4), 1484–1497 (2012) 7. S. Vitabile, G. Pollaccia, G. Pilato, E. Sorbello, in Road Signs Recognition Using a Dynamic Pixel Aggregation Technique in the HSV Color Space. Proceedings of the 11th International

Traffic Sign Recognition Using CNN

475

Conference on Image Analysis and Processing (ICIAP ’01), pp. 572–577, Palermo, Italy, Sept 2001 8. D.M. Gavrila, in Traffic Sign Recognition Revisited. Mustererkennung 1999: 21. DAGMSymposium Bonn, 15–17. September 1999, pp. 86–93, Springer, Berlin, Germany (1999) 9. B. H¨o4ferlin, K. Zimmermann, in Towards Reliable Traffic Sign Recognition. Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 324–329, Xi’an, China, June 2009 10. R. Mandal et al., in City Traffic Speed Characterization Based on City Road Surface Quality, ed. by J.M.R.S. Tavares, S. Dutta, S. Dutta, D. Samanta, Cyber Intelligence and Information Retrieval. Lecture Notes in Networks and Systems, vol. 291 (Springer, Singapore). https://doi. org/10.1007/978-981-16-4284-5_45

Motion Controlled Robot Using Multisensor and Powered by AI Engine M. K. Mariam Bee and B. Bhanusri

Abstract The main objective of our research is to design a motion controlled robot namely a wind mill robotic model to produce electricity in highways by movement of heavy vehicles. The electric energy consumption rate is increased day by day. Among the various renewable energy sources, wind and solar energy are vital energies. Wind energy is mainly used in generating power due to its generation cost. The term wind energy is a form of conventional, and it is available in affluence. Electricity is obtained with the help of a vertical axis wind turbine. Wind energy is the method where the wind is used to generate electricity. In this process, wind turbines convert the kinetic energy present in the wind into mechanical power into electricity. In this, the windmills are specially designed, and they are placed in between the divider on highways. The turbines are designed and fabricated based on the speculations the blades used in the turbines are semi-circular shape, and they are connected to the disk that is then connected to the shaft. The generated energy is stored on the battery for further usage. Keywords Wind energy · Turbines · Battery · Shaft · Electric energy

1 Introduction Wind energy has many advantages. It is a renewable and clean source of energy, efficient use of land space, but the only disadvantage in obtaining wind energy is the setup of windmills is very high. Electricity plays a significant role in the improvement of the country. So the production of electricity is significant. An average of about 68% of the electricity production is based on the thermal power plant, where coals, fossils, and diesel are utilized for the generation of the power, which is available in a small amount. Also, this fuel produces global warming, pollution, the greenhouse effect, etc. The power generation produced with the help of non-conventional resources such M. K. Mariam Bee (B) · B. Bhanusri Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Thandalam, Chennai, Tamilnadu 602105, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_41

477

478

M. K. Mariam Bee and B. Bhanusri

as wind increasing day by day, and this type of power generation is considered clean and safe. The wind turbines are of two types: horizontal axis wind turbines (HAWT) and vertical axis wind turbines (VAWT). The first type of HAWT has successfully evolved in the process of making electricity from the wind. Anyway, working on VAWT has an advantage over HAWT, such as it dint need yaw machines as it can produce power independent of wind directions. VAWT can be generated at low cost than HAWT and also affordable cost maintenance. VAWT is further classified as Savonius vertical axis wind turbines, Darrien’s vertical axis wind turbines, and Giro mills. The article represents the use of wind energy in effective manner for better elelctricity performance. The shaft has then emerged with the pulley with the help of bearing; then, the pulley is connected to the alternator that produces the power. The power that is developed is stored in the battery, and after that, it is used in street lights, signals, or tolls. This project aims for the best outcome with the minimum cost. The remaining part of the paper is categorized into various sections. Chapter two explains the reviews of various author’s proposed devices. Chapter three explains the block diagram of the proposed system. Chapter four demonstrates the implementation process of the proposed system, and finally, chapter five concludes the proposed system.

2 Literature Review Somnath et al. (2019) focus research to collect wind energy from highways with the help of vertical axis wind turbine (VAWT). The automobiles passed on the roads make a considerable volume of energy because the vehicles passed on the roads at high speed. The wind touches on the VAWT blade, and it rotation the turbine in a particular direction. The solar energy system is also used to make electrical type energy, and it is placed in the path, and it redirects the wind toward VAWT. The output energy of the VAWT and solar system is saved in the battery system. The saved energy is used for the lights fix on the streets. VAWT is helping to decrease the difference between the energy demand and the power supply. It is the combination of two types of energy systems like solar energy and wind energy. If anyone the system fails, the other system will generate the power continuously. Based upon the number of blades in the VAWT, the amount of generated energy is also changed. The proposed system is suitable for all areas; the area features also consider for the better result. It is also the best option for managing global warming [1]. The need for electric energy is higher than its production. Patil et al. [2] developed a new system to generate electric energy with less amount and suitable for current environmental conditions. Here, electric energy has been produced with the help of working vehicles on the highway paths. A large amount of electrical energy is generated by moving automobiles on the road. The authors propose a new system for producing electric energy at less cost. They use the perpendicular blade setup for

Motion Controlled Robot Using Multisensor …

479

making energy. The developed model is placed on the intermediate portion of the road. The generated energy is helpful for lighting, traffic signals, and electric cars, etc. The energy is stored on the battery for further usages [2]. Venkata Subbaiah et al. [3] say that wind energy is also playing a significant part in managing the association between human and the requirement of the energy. Compare with other resources, the wind is one of the fee resources, and it is received quickly. This new system is mainly focused on using energy from wind by VAWT. It is collecting energy from the moving objects on the highway roads. The main aim of the proposed work is to design a VAWT with low weight and high stiffness. The developed VAWT is fixed in the divider on the road. So, the air is collected from the vehicles during their moving condition. It makes the utmost energy. The generated energy is stored on the battery, and it is used for various devices. If this device is placed on all the highway roads, it generated a massive volume of energy. Based on wind energy, this system will generate power [3]. Sayais et al. [4] use wind energy from the highway roads by VAWT. A high amount of wind is collected when automobiles pass on the road. The wind strikes on the VAWT blades. Due to the force of the wind, the machine is rotated and electrical energy is generated. The generated energy is stored on the battery. Here, the author uses the solar system also to make the energy. Compared with other energy generating systems, this is eco-friendly. It is the efficient and best way to generate power [4]. Kulkarni et al. [5] developed a system for creating electrical energy from using wind energy. The main objective of this proposed project is to attain the highest energy. The developed project is placed on the highway path divider. It collects the wind energy from the moving vehicles on both sides. The system is developed with semi-circular type blades, docs, and all devices connected with the centralized shaft. It is one of the best ways to produce a large amount of natural kind energy with less cost [5]. Naki et al. [6] aim to create electrical energy using the wind speed in highways. In this system, the blades are developed vertical track and placed at the divider center portion on the highway roads. The wind force is high in the middle of the road than the other portion of the road. The force of the blade is rotated and mounted with a generator that generates electricity. In this system, the extra generator is fixed to improve effectiveness. The generated amount of energy is changed on the system configuration and speed of the wind. Rotator features also play a vital role in attaining the performance of the system [6]. Wind energy is one of the precise energy sources among the various types of energy. It is also a rapidly rising energy resource. On highways, a massive amount of energy is created due to the vast number of heavy vehicles. Malave et al. [7] designed a new system for generating electrical energy from wind energy, and it is placed on the divider in the highway path. Moreover, the force of the wind is changed, a saving system used to maintain and manage the fixed power source. Here, helical kind turbine is used and collects the wind energy from all directions. Noise level and placed position are also essential for this project [7]. Shah et al. [8] examined the shapes of turbine blades, improved a math algorithm, and instituted techno-economic performance. Four unique Savonius rotor types of

480

M. K. Mariam Bee and B. Bhanusri

Fig. 1 Block diagram of proposed VAWT

blades are analyzed for the performance of the rotation. The overall objective of this type of research work is modeling and designing VAWT on a small scale that can compensate for the power demand. Two different shapes of Savonius rotor blades are analyzed based on their rotational characters over the curved blades. MATLAB does the stimulation to improve the math model that includes the ratio of tip speed, coefficient of power, mechanical, and electrical subcomponents. The results of the improved turbines are utilized for the validity of the model. The result from analyzing the rotation is recorded in the lab and utilized for the validation model [8].

3 Proposed System In the current modern world, the requirement for electricity is higher than its production level. Energy is essential in every one day to day activates. But the energy generation resources are significantly less. VAWT is the easiest way to implement the developed areas at the top of the buildings or the flat ground. The improvement of the design of VAWT will offer new chances for the acceptance of the machines. Figure 1 illustrates the block diagram of the proposed VAWT system.

4 Implementation Energy from the solar source is accessible from the beginning of the day. But the wind energy is accessible based upon the speed of the wind. On highways, wind energy is created based on the speed of the vehicles. The proposed system is designed with the help of various hardware components like bearings, blades, shaft, gears, battery, generator, etc. The storage facility is used to manage and share out the power constantly with the electric devices. VAWT turbine is a globally used device for distributing electric power. Figure 2 shows the VAWT system with three blades; Fig. 3 shows the generator, and finally, Fig. 4 shows the shaft which is attached to generator.

Motion Controlled Robot Using Multisensor …

481

Fig. 2 VAWT with three blades

Fig. 3 VAWT generator

Various hardware components are involved in the development of VAWT. Few components are explained in the following section. Now, 68% of the electrical energy is generated by the thermal power plant and the next 22% by hydropower plant, nuclear power plant, gas power plant, then the fossils are finished by one day. Both wind and solar energy are renewable energy sources. Solar energy is available when the day starts over, but wind energy is maximum on the highway due to the vehicle’s speed.

482

M. K. Mariam Bee and B. Bhanusri

Fig. 4 VAWT shaft

Dynamo: It is a generator that creases the direct current (DC) power using the electromagnetism technique. Dynamos are the initial energy creators able to deliver power to the industries. Rectifier: It is also one of the electrical components that change alternating current (AC), which constantly reverses the route to DC, which transfers only one way. The procedure is called rectification. Battery: It is a self-sufficient, chemical power bundle that generates a restricted quantity of electrical energy and provides it to the required devices. Battery released their power over a time of years, months or days. Inverter: It is a simple DC to AC inverter that issues 220VAC when a 12VDC power basis is offered. It is used to provide power to very few load devices such as night lights and telephones. But this device is changed into a better inverter by adding additional MOSFETs. Air Velocity Sensor: We have used four air velocity sensors which will be placed along the corners of blades. This sensor will help in measuring the velocity of air flow when vehicle passes and according to that the blade will start rotating. When the blade rotates the shaft also rotates which is connected to the generator which produces electricity.

Motion Controlled Robot Using Multisensor …

483

5 Conclusion Today in this modern world, we require electricity is much greater than the generation of electrical energy. One of the main issues ever since the natural resource is going to be over one day. Fossil fuels play a vital role in the generation of global warming effect and greenhouse gas production. The main aim of this proposed system is used to produce electrical energy using wind energy. Compare with the other energy sources, wind energy is the cheapest one. The main component of the turbine is the blade. Several blades and forces of wind influence the amount of energy generation. The proposed model is placed on the divider in the highways. Due to the speed of automobiles, the turbine blades are rotated and generate electrical energy—the generated energy stored on the battery for future usage. The saved energy is helpful for electric vehicles, street lamps, or lights on the traffic signals.

References 1. S.M. Somnath, G.R. Abhishek, S. Channabasavana Gouda, B.M. Kavya, J. Kruthi, Power generation on highway using vertical axis wind turbine and solar energy. Int. J. Eng. Sci. Res. Technol. 8(6), 232–240 (2017). ISSN: 2277-9655 2. S.A. Patil, M.L.wagh, S.G. Gavali, Working design of verticle axis wind turbine with road power generation. Int. J. Adv. Res. Sci. Eng. 6(2), 395–403 (2017) 3. N. Venkata Subbaiah, M.L.S. Deva Kumar, Power generation by using highway vertical axis wind mill. Int. J. Creative Res. Thoughts 5(4), 1755–1764 (2017) 4. S.Y. Sayais, G.P. Salunkhe, P.G. Patil, M.F. Khatik, Power generation on highway by using vertical axis wind turbine & solar system. Int. Res. J. Eng. Technol. (IRJET)05(03), 2133–2137 (2018). e-ISSN: 2395-0056 5. S.A. Kulkarni, M.R. Birajdar, Vertical axis wind turbine for highway application. Imperial J. Multidisc. Res. 2(10), 1543–1546 (2018) 6. K. Nakil, A. Tekale, P. Sambhus, R.S. Patil, Analysis of vertical axis windmill turbine for electricity generation on highways. Int. J. Curr. Eng. Technol. Special Issue 113–119 (2016) 7. S.N. Malave, S.P. Bhosal, Highway wind turbine (Quite Revolution Turbine). Int. J. Eng. Res. Technol. 6(6), 789–794 (2013). ISSN 0974-3154 8. S.R. Shah, R. Kumar, K. Raahemifara, Design, modeling and economic performance of a vertical axis wind turbine“, ELSEVIER. Energy Rep. 4, 619–623 (2018)

Pattern Recognition

Smart Health Care System for Elders’ Home to Monitor Physical and Mental Health in a Controlled Environment Abhilash Krishan, Chinthaka Ashoda, Dilini Madhumali, and Gayan Pradeep

Abstract Nowadays, with the busyness of lifestyle, the focus on parents has diminished. At the same time, the need for nursing homes has increased. As the number of elderly people in nursing homes increases, the number of elderly people living there cannot be considered individually. And nowadays, people are not willing to work in such places. Because they have to work without pay or at low wages. About 95% of seniors in nursing homes have poor health. And their behavior patterns are very different. For these reasons, the authority to manage nursing homes faces a major challenge. They have a big role to play in giving each adult proper medication and healing their mental level. As a solution to this problem, authors hope to compare it with technology and provide a solution. Authors decided to create the relevant items and software for this. This is done using an adult friendly handrail and camera system in the proposed methods. Advanced ML, image processing and hardware parts are used in this technology world. This allows you to focus on each adult 24 h throughout the day. One of the main benefits of this is that it pays more attention with less staff. According to our survey, this has proved to be an essential factor for nursing homes. By creating this, it will be possible to provide healthier and more productive services to the elderly. Keywords Accelerometer sensor · Pulse oximeter · Humidity sensor · GPS locater · Image processing · Machine learning · id3 algorithm

A. Krishan (B) · C. Ashoda · D. Madhumali · G. Pradeep Sri Lanka Institute of Information Technology, Colombo, Sri Lanka e-mail: [email protected] C. Ashoda e-mail: [email protected] D. Madhumali e-mail: [email protected] G. Pradeep e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_42

487

488

A. Krishan et al.

1 Introduction In today’s competitive world, the number of elderly people entering nursing homes is increasing day by day, which is why a great deal of effort must be put into caring for the elderly. Due to the busy schedule of many jobs, people who are engaged in such jobs are often unable to pay attention to their relatives and their elderly parents are temporarily housed in nursing homes and it is well known that the elderly living in nursing homes suffer from many physical and mental illnesses. Was able to take an extensive study of nursing homes identified major illnesses such as heart disease, stress, pneumonia and dementia. It was also found that the number of adults who died due to the inability to diagnose sudden cardiovascular diseases was very high, and the number of adults who died due to pneumonia due to excessive mucus due to intolerance to very small weather changes was not insignificant. It was also revealed that the elderly who reluctantly live in nursing homes due to separation from their relatives are suffering from high levels of stress and a significant number of them have died on the streets due to wandering and leaving the nursing home due to unconsciousness. Adults with physical or mental illness are also more likely to be involved in accidents due to physical disabilities or sudden heart attacks during active activities such as walking. In such a case, there are many instances where other people associated with the old age home are not noticed. The collection of data on nursing homes using a survey confirmed the urgent need for some methodology to quickly detect and prevent these hazards that are commonly encountered by adults. Also, when asked about the creation of a new system by the people in charge of the old age homes and the elders, about 99% were very keen. As a solution to these problems, authors have developed a smart device and a web application. Using this device in case of an emergency heart attack, falling due to sudden illness, and informing adults of adverse environmental conditions such as pneumonia while happening if an adult with a mental illness leaves the nursing home without notice such instances are identified. In order to maintain the physical and mental health of the elderly, the web application is used to analyze the data obtained through the device and predict future illnesses. That is, the prognosis of heart disease, behavior and facial expressions are analyzed to predict the potential for mental illness. The mental level is also detected using a camera system using facial expressions. The web application also analyzes all this data and gives recommendations to adults to avoid this situation.

2 Related Work New technologies like Internet, location-based services and IoT have make this easy. GPS Smart Sole is a commercial device which was designed by GTX Corp with the objective of making a wearable GPS tracking solution for those who wander specialty of this is, it is water resistance. It uses patients GPS location data and

Smart Health Care System for Elders’ Home …

489

through a satellite they are sent to servers of GTX Corp and then though their system responsible persons are notified though SMS and email [1–3]. They use 2G for connectivity purposes. There are even several drawbacks of this as they are using 2G as their connectivity medium. Since cellular signals might be affected by the building’s structures, and geography, and even may be 2G cellular coverage is not available everywhere, GPS Smart Sole® may not work in some areas. And these cost 299$ that means Rs. 58,135 in Sri Lanka [4, 5]. There is a research article on accelerometry analysis of physical and sedentary activities of elderly people. At first elder’s physical activity and sedentary behavior patterns of older adults and identified cut-points used to classify intensity of activities. Second, we used this information to analyze an accelerometry sample of older women’s activity patterns over 7 days to illustrate the effect of changing the different reported cut-points. Issue in this is there is no any way to notify sudden abnormalities and no action taken to avoid any incident that can occur on their health. This just analyzes data and shows difference [6, 7]. Real-time monitoring and detection of “heart attack” using wireless sensor networks is another research done on this field to develop a wireless sensor network system that can continuously monitor and detect cardiovascular disease experienced in patients at remote areas. A wearable wireless sensor system (WWSS) is designed to continuously capture and transmit the ECG signals to the patient’s mobile phone, doctors and CDC. The following system is integrated with a dynamic data collection algorithm that collects the ECG signals at regular intervals, according to the health risk perceived in each patient. Implementing this system will contribute to reducing heart diseases, leading to death of a patient. Problem in this is cost this system costs so much so normal middle-class people will not be able to purchase this. So, it will not do much good for elderly people who are abandoned in elderly homes [8, 9].

3 Problem Issues and Proposed Solution Though there are several existing systems which help to track elders’ movements, their physical health conditions, and their mental health conditions, there is not any system which do all at once. And also, all the available systems are highly cost even they are performed only single task. But the proposed system is capable of providing positive effects on both care takers and innocent elders. The proposed smart health tracker system can do following all in same time. • Track and monitor patients in a controlled environment and suggest necessary recommendations. • Track and analyze the heart rate and suggest recommendations to avoid and reduce the risk of heart related accidents. • Analyze the behaviors to determine if an elder has a behavioral change or illness and suggest recommendations to avoid that.

490

A. Krishan et al.

Table 1 Features Features

A

B

C

Proposed Solution

Connectivity.

WIFI

2G

3G

WIFI, 3G

GPS tracking

V

V

V

V

Alerting beep sound to notify patient

X

X

X

V

Temperature sensor

X

X

X

V

SOS alert service

SMS

SMS, Mail

Push notifications

Push notifications, SMS

Accelerator sensor

X

X

V

V

Cost effective

X

X

X

V

Doing recommendations

X

X

X

V

Heart rate monitor

X

X

X

V

• Analyze the face reactions to determine if an elder has a change in mood and suggest recommendations to avoid that. Table 1 shows that comparing features of the proposed solution with past research.

4 Methodology The proposed system is consisted of both web application and an IoT device. Several sensors were used when implementing the device. Humidity sensor, GPS locater, accelerometer and heart rate sensor were used and programed using C language. The web application was developed using Java Spring Boot, and frontend was developed using Thymeleaf framework with separate fragments. The smart device which is developed by the authors detects heart rate and emergency heart attack, fallen detection, predict diseases, monitoring location and predict mental illness. System notifies it and send it to care taker through short message service (SMS) and email. Furthermore, web application gives recommendations to avoid and handle the emergency heart attack. The flowing Fig. 1 shows that data flow in the system.

4.1 Keeping Elders Safe and Constant Monitoring Dementia and asthma are two main problems most of the elders suffer. Elders with this dementia conditions wander and face accidents. So, keeping track of the elders is an essential task of a caretaker in an elder’s home. And also elders who suffer from asthma face severe health conditions in wet seasons when they wander without

Smart Health Care System for Elders’ Home …

491

Fig. 1 Data flow in the system

any idea and humidity percentage is responsible for that. The proposed solution is consisting of a smart device and a web application. The smart device is use to collect data of locations and humidity percentages while web application assesses them and do the notifying and recommendations to do. Here, a NEO-6M GPS module is used to take the location data of the person. This GPS module was chosen as it is the best from both sides’ low price and high accuracy and fits the objectives. A signal antenna is also used to enhance the signal strength. Communication is happened through serial communication. Device is implemented using C language. After data is collected from device, data is sent to web application and at their database check for diseases and notify the caretaker if an elder person with dementia situation exits the elder home premises. The way system identifies

492

A. Krishan et al.

if a certain person has left the premise is that, the static location coordinates are given to the system when implementing and when the subject moves he sends new coordinates once a minute and system cross check the radius or the distance between new coordinates and static coordinates. And if this radius is above the predefined distance given, system will get this as a trigger. Also, it uses DHT 11 sensor for checking humidity percentage. It is accurate to 20–80% humidity. There are API routes to communicate web application and device. It uses low power also for operating. And also, it is ultra-low cost. We could have use the DHT 22 but it will increase the cost. DHT 11’s size is also small. Here, also when data is sent to application it checks with database and notify the caretaker if an elder person with asthma situation is in a high humidity location. As found from the readings humidity level more than 80% is considered as a trigger point. Notification is happening in few ways. It sends an email, a notification to web application and also a short message (SMS) to the caretaker’s mobile phone. And the system is capable of generating sudden reports with details of elders in case of an emergency. All the services are implemented using Java Spring Boot in web application and objectoriented relational mapping under the hibernate JPA. Frontend is developed using Thymeleaf framework with separate fragments. The system is designed to receive signal only after a minute to save power. And also, as taking signal every second is not necessary because the elders cannot move at much speed so they will not move much far with in a minute.

4.2 Real-Time Predicting and Detection of “Heart Attack” with Recommendations Iterative dichotomiser 3 algorithm (ID3 algorithm) is used to predict heart attack. ID3 is a decision tree algorithm that follows a greedy approach of building a decision tree by selecting a best attribute that yields maximum information gain (IG) or minimum entropy (H).

4.3 Prediction Procedure • Select the dataset for which the test to be retrieved. • By using the ID3 algorithm classify the datasets based on the heart rate, oxygen saturation level (SpO2), humidity and temperature. • Then pre-process the fields of dataset. • The paper focuses, the retrieval of dataset, based on the ID3 algorithm that result in the specific dataset fields retrieval. ID3 works mainly on three things, firstly the entropy of each attribute, second information gain and third, entropy of whole dataset, using these three, it picks a

Smart Health Care System for Elders’ Home …

493

Fig. 2 Select the best attributes from 9973 data

root node. The condition for selection of root node is that, the attribute with lower value of entropy (or higher value of information gain) becomes the root node. Figure 2 shows that how select the best attributes from 9973 data. Figure 3 shows that classify the datasets based on the heart rate oxygen saturation level (SpO2), humidity and temperature. Below are the two reasons for using the ID3 decision tree algorithm: • It can work with both categorical variables and continuous variables. • ID3 algorithm selects the best feature at each step while building a decision tree System solution (as the prototype) is completed using MAX30102 pulse oximeter and nodemcu esp8266 to ease the design-in process for detecting heart rate and emergency heart attack. Communication is done through I2C standard compliant interface through the sensors. Authors implemented all functions using C language and MAX30102 library to handle high speed readout of the IR data. API routes use to communicate the web application and device.

4.4 Fall Detection and Predicting Diseases by Analyzing Behaviors Fall is detected when the acceleration exceeds a critical threshold. ID3 decision tree algorithm is used to create model, and it classifies chance of being a disease based on average step counts, humidity and temperature. Figure 4

494

A. Krishan et al.

Fig. 3 Classify the datasets based on the heart rate, oxygen saturation level (SpO2), humidity and temperature

shows that data processing, using data mining algorithm and trained so as to predict the disease based on the input data. 1. 2. 3. 4. 5.

Data preparation Data transformation Feature extraction Implementation of ID3 algorithm Model

Figure 5 shows that how select the best attributes from 9000 data. Figure 6 shows that classify the datasets based on the step counts, humidity and temperature. System solution (prototype) is developed using MPU6050 accelerometer sensor for getting step counts. MPU6050 library is used when implementing the device. Communication is done through I2C standard compliant interface.

Smart Health Care System for Elders’ Home …

Fig. 4 Block diagram for general disease prediction

Fig. 5 Select the best attributes from 9000 data

495

496

A. Krishan et al.

Fig. 6 Classify the datasets based on the step counts, humidity and temperature

4.5 Face Identify and Stress Calculation and Security Adults in nursing homes first collected their data through a web system. The web application was created using the Java language and used a real-time database. A camera system used to identify facial expressions and video technology to study their behavior patterns. . The images will be identified and processed using image processing technology. Here, we used to train the photos of the elders in the nursing homes and the photos obtained through the Internet. Images were trained with the help of tenseflow. Random images taken by cameras are analyzed using CNN algorithms. The end result is that each adult’s condition is identified by facial expressions. They maintain a separate database that includes all their data and even pre-existing diseases when registering adults. The images taken by the cameras are analyzed, and then, they are collected separately in a database. The next step is to make the recommendation and monthly report section. Different foods, different activities, medical advice, etc., will be notified to care takers by notification through the web system. The dashboard of the web application can easily access the information of each adult. In addition, when sending a recommendation, the information coming from the band is also taken into consideration. Analyzing is done using Python, ML.

Smart Health Care System for Elders’ Home …

497

They also collect their data once a month using the PSS method. All of these can be included in the monthly report of each adult. In addition, depending on the situation, it is mandatory for adults to wear face masks. Visitors are also required to wear face masks. Image processing can be used to identify whether or not a facemask is present. This can be easily monitored by inserting it into the CCTV system or by senior home security guards. The video can be used to guide adults and inform emergency care takers of the sudden fall stage. This is indicated by the ringing of a small trumpet.

5 Result and Discussion Here, data signals obtain from the devices are the inputs to the system. The humidity sensor takes the humidity percentage and sent to system application, at the system checks for the diseases list of elder to determine whether this humidity percentage affects health of elder. This sensor has accuracy level of 80%. The GPS locater also work as same it sends data to system and at their it checks for the elder’s health condition and proximity of elder home and notify if there is a trouble. This system can work with both devices at same time. It can work low power. Its operating voltage is 3–5 V. The system sends notification to caretaker in three ways as discussed. He should be notified immediately in some times to take actions. And also, the emergency report is also can be issued with all the details from personal details to medical details and also the last known location. It is small in size also, easy to wear if it is designed to work with batteries. Here, device was designed and implemented with wires and direct current. As the micro modules which can work with small batteries were not able to access due to prevailed situation of the country but this can be upgraded as a future development when implementing as a commercialized level. Then, several new methodologies also can be in traduced to save power like deep sleep modes and only to work when receive a trigger. Neo 6 g sensor also can work with less power to give high success rate, and here an antenna also used to boost the signal. It can provide these facilities with low cost also. The sensor DHT 11 can take readings every second. But it is programmed to take reading after a minute so we can save power. Heart attack and disease prediction is challenging altogether, with id3 and this research could benefit more from, having a complex dataset. Model accuracy rate is flattering, being above 94%. Authors compare with other techniques and among the three techniques (Naïve Bayes, k-nearest neighbors—K-NN), ID3 shows high accuracy and takes the least amount of time. It is shown in Table 2. Figure 7 shows that results of disease prediction. It is clearly classified diseases and get predictions. Figure 8 shows that system notification when an emergency situation. Figures 9 and 10 show that email and SMS notification. The main thing is to verify their identity through photographs taken and to identify them without mixing. In addition to analyzing data by computer, it analyzes their world-renowned PSS method. The accuracy of the data obtained by the algorithm is

498 Table 2 Classification accuracy

A. Krishan et al. Algorithm

Accuracy (%)

Time taken (ms)

ID3

94.24

609

Naïve Bayes

87.12

719

K-NN

71.87

1000

Fig. 7 Results of disease prediction

about 88.33% The world recognized PSS method has an accuracy of about 90. If a comparison is made between the two methods, then the correct data can be retrieved from the algorithm [10, 11]. Figure 11 shows sample face detection. Questionnaire Outcome Authors conducted two different surveys among the elder homes’ caretakers and elders to find the difficulties; they are facing with prevailing procedures within the

Smart Health Care System for Elders’ Home …

499

Fig. 8 System notification

Fig. 9 Email notification

elder homes. Each and every caretaker answered positively and asked for a system like this. Figure 12 shows how much they were willing to recommend the system to other users. According to survey results, the final system achieves a high satisfactory level from the users, and 0% of the users would recommend system for further use.

500

Fig. 10 SMS notification

A. Krishan et al.

Smart Health Care System for Elders’ Home …

501

Fig. 11 Face detection

Fig. 12 How much they were satisfied with the outcome

6 Conclusion Smart device with a multi-functional web application which is developed using machine learning, Java Spring Boot, Thymeleaf and MYSQL. Smart device is implemented in C language, and it is equipped with mini heart pulse sensor, mini temperature sensor, mini GPS sensor and two mini accelerometer sensors. Overall application consists in object-oriented relational mapping with Hibernate JPA implementation based on MVCS architecture. There are web routes, API end points and data storage sessions for data communication. In summary, the device and application use for elders’ home to track and detect elders’ physical and mental diseases and monitor elders’ location and environment. As a result, there is a notification system when detect the heart attack, fallen-down event, out of premise and elder who is suffering

502

A. Krishan et al.

from pneumonia is in an unfavorable environment. There is a process for predicting heart attack situations and mental illness by analyzing heart rate, behaviors and facial expressions. Furthermore, web application provides recommendations to avoid diseases. According to the survey, most of elders are suffering from heart attacks, mental illness and pneumonia. Hence, this application and device is most important for elders’ home to avoid sudden accidents. As future work, authors will implement mobile application to make it easier to use and for stability. Acknowledgements Authors are grateful for all lectures in SLIIT, colleagues and for the family members for their encouragement provided throughout.

References 1. Elderly Care Facilities in Sri Lanka, YAMU, 2021. [Online]. https://www.yamu.lk/blog/eld erly-care-facilities-for-your-aging-parents. [Accessed: 26-Feb-2021] 2. is cold reason for wheeze disease—Google Search, Google.com, 2021. [Online]. https:// www.google.com/search?q=is+cold+reason+forwheeze+disease&oq=is+&aqs=chrome.0. 69i59l3j69i57j35i39j69i60l2j69i61.1964j0j1&sourceid=chrome&ie=UTF-8. [Accessed: 26-Feb-2021] 3. P. Ray, D. Dash, D. De, A systematic review and implementation of iot-based pervasive sensorenabled tracking system for dementia patients. J. Med. Syst. 43(9) (2019) 4. GPS SmartSole® GPS SmartSole® Used in Wandering Prediction Trial, Gpssmartsole.com, 2021. [Online]. https://gpssmartsole.com/gpssmartsole/gps-smartsole-used-in-wandering-pre diction-trial/. [Accessed: 26-Feb-2021] 5. i. Plan, iTraq Nano (Global) + 1-Month Reporting Plan, iTraq, Inc., 2021. [Online]. https:// www.itraq.com/products/itraq-nano. [Accessed: 26-Feb-2021] 6. 2021. [Online]. Available: https://link.springer.com/article/https://doi.org/10.1007/s11556013-0132-x. [Accessed: 05-Nov-2021] 7. A. Steinert, M. Haesner, E. Steinhagen-Thiessen, Activity-tracking devices for older adults: comparison and preferences. Univ. Access Inf. Soc. 17(2), 411–419 (2017) 8. 2021. [Online]. https://ieeexplore.ieee.org/document/5558100. [Accessed: 05-Nov-2021]. 9. E. Laskowski, What’s a normal resting heart rate? Http com/health/heart-rate/AN01906://www. mayoclinic, 2012 (Accessed 15 01 2016) 10. 2021. [online]. https://www.researchgate.net/publication/342162879_Identity_Image_and_ Brand [Accessed 19 Sept 2021] 11. 2021. [online]. https://www.researchgate.net/figure/Steps-to-select-design-methodology-fora-PSS-problem_fig2_283965846 [Accessed 19 Sept 2021]

Privacy Conserving Using Fuzzy Approach and Blowfish Algorithm for Malicious Personal Identification Sharmila S. More , B. T. Jadhav, and Bhawna Narain

Abstract As we know that when we exchange data through the network, system needs more security and approval. Generally, crisp algorithms are used for extraction and identification of data. When we use crisp logic, it is very complex and tedious work, so we recommend a Fuzzy approach for data extraction. Security is the main part of data exchange. In this article, we have to discuss how Fuzzy logic is used in Blowfish algorithm to secure communication. Multimodal images use the rules of the Fuzzy logic Fuzzy inference system to design a 64-bit Blowfish algorithm that increases security and improves performance. This algorithm helps protect data from unauthorized access and runs faster. Our proposed algorithm is designed using MATLAB R2017a. We also discussed the pros and cons of Fuzzy and Blowfish. Keywords Blowfish · Fuzzy inference system · Fuzzy logic · Cryptography

1 Introduction 1. Biometrics It is known that biometrics or biometric authentication is the process used in information technology to identify the authorized user using their characteristics [1]. It is also used to identify the authorized person in groups. In the modern era, identifying people using their physiological and behavioral characteristics is the emerging trend. The main functions of biometric systems are the identification of people using their S. S. More (B) · B. T. Jadhav Yashvantrao Chavan Institute of Science (Autonomous), Satara, India e-mail: [email protected] B. Narain MATS University, Raipur, Chhattisgarh, India e-mail: [email protected] B. T. Jadhav Principal, Yashvantrao Chavan Institute of Science, Satara, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_43

503

504

S. S. More et al.

measurable, unique and permanent characteristics such as fingerprint, iris, face, palm print, vein, DNA, handwriting, hand geometry and many more [2–4]. We know that biometric concepts were first developed and used by Joao De Barros in China in the fourteenth century, i.e. Chinese merchants imprinted palm prints and child footprints on paper with paper ink, to differentiate young children from each other [1]. In the late 1800s, identification was largely based on “photographic memory” and in 1890, an anthropologist and Paris police employee used the term biometrics to identify convicted criminals. Today, we use different types of biometric devices for verification and identification in all fields [5, 6]. Figure 1 shows a block diagram of the biometric system modes with enrollment and verification/identification process. The registration process is also referred to as the registration process [7, 8]. The registration or registration process means that the user has registered his data for the first time. Initially, the user uses biometric data obtained from biometric sensors, and then we process the data and then store it in the form template. This template will be used for a later authentication process. The figure above contains biometric data, template, feature extraction, storage and matching [9, 10]. 2. Multimodal biometrics As we know, multimodal biometrics is actually an amalgamation of unimodal biometrics designed to overcome unimodality issues. This captured multimodal biometrics is based on the combination of several types of modalities or biometric qualities, namely face, iris, fingerprint and palm print. As we know, the fusion of

Fig. 1 Enrollment and identification process

Privacy Conserving Using Fuzzy Approach …

505

Fig. 2 Multimodal images of our dataset

multimodal biometric system is done at four levels such as sensor level, feature level, match score level and decision level [11, 12]. The final classification is based on merging the output of the different features of the captured images [1]. For our dataset, we collected over 5000 multimodal face; iris, fingerprint and palm print image samples from school, college and village (Fig. 2). The figure above shows the facial, fingerprint, iris and palm print images of P1, P2 and P48, respectively. An approach to address security issues in personal identification has been given [13]. 3. Fuzzy logic As known, Fuzzy logic contains True or False statements, i.e. 1 or 0, but it is not always possible to store accurate information using only 1 or 0. Fuzzy logic is the control system’s very good approach to problem solving [14]. It is an approach to computing based on “partial truth” or “degrees of truth”. As we know, the concept of Fuzzy logic was first introduced by Dr. Lofty Zadeh from University of California, Berkeley in 1960 [1]. The Fuzzy system requires auxiliary memory to store the sets of bits. Fuzzy logic is a form of multi-valued logic instead of fixed logic and extraction process [15]. Fuzzy logic is used to shorten the work of the system designer and calculate more accurate and faster results [16]. Fuzzy association rules used to find an abstract co-relation between different security features. These Fuzzy association rules are easy to write and understand, therefore, Fuzzy designer to effectively describe the Fuzzy system. It is very difficult to identify the extracted data; this can be minimized by using Fuzzy logic [17]. A Fuzzy method requires less memory to store information and minimal time for model comparisons compared to clear logic. 4. Fuzzy and multimodal logic As we know Fuzzy logic works on member function. In digital image processing, the membership function belongs to gray level transformation. Multimodal has various digital images of biometric traits. Fuzzy logic is used to calculate pixel intensity. Multimodal is a combination of unimodal biometric traits. We tested the accuracy of

506

S. S. More et al.

multimodals by evaluating by the values of various parameters. Fuzzy logic is used in the multimodal evaluation process. 5. Blowfish Algorithm As we know, Blowfish is an encryption technique developed by Bruce Schneier in 1993 [18] as an alternative to DES encryption technique. This algorithm is already developed, and we used these algorithmic steps to evaluate our dataset. Later on, we compare these datasets with our new modified algorithm. These algorithms are significantly faster and provide a good encryption rate without any effective cryptanalysis techniques found so far. The Blowfish algorithm is one of the first secure blocked encryption algorithms that is patent-free and, therefore, freely available to everyone. The Blowfish algorithm is primarily used to secure email encryption tools, backup software, password management tools and against hackers and cybercriminals [19]. This Blowfish algorithm has a block size of 64 bits, and the key size is variable from 32 to 448 bits with 18 subkeys [table P] and 16 round numbers and 4 substitution boxes (each having 512 entries of 32 bits each). Here are the algorithmic steps of the Blowfish algorithm. Step 1: Start. Step 2: We generated the subkeys, i.e. 18 subkeys {P[0]…P[18]}. Step 3: After that, we initialized {S[0]…S[14]} substitution boxes for the encryption and decryption process. Step 4: The encryption process is done using 16 rounds, and for these rounds, it will take input in the form of face, iris, finger and palm print image. Step 5: In the postprocessing step, the output after the 16 rounds is processed. Step 6: Finish. Biometric characteristics for iris, fingerprint, face and palm print are collected separately based on age and gender in the future. Then, we apply the Blowfish algorithm on these multimodal images. We ran the above algorithm against our dataset for comparative analysis.

2 Fuzzy Inference System Fuzzy association rules used to find an abstract co-relation between different security features. These Fuzzy association rules are easy to write and understand; therefore, Fuzzy designer to effectively describe the Fuzzy system. It is very difficult to identify the extracted data; this can be minimized by using Fuzzy logic [17]. A Fuzzy method requires less memory to store information and minimal time for model comparisons compared to clear logic. Figure 3 shows the Fuzzy inference system (FIS) design of our new modified algorithm. Fuzzy association rules used to find an abstract co-relation between different security features. Fuzzy association rules are easy to write and understand. It is up to the Fuzzy designer to adequately describe the Fuzzy system.

Privacy Conserving Using Fuzzy Approach …

507

Fig. 3 Fuzzy inference system

3 Methodology, Results and Analysis As we know that fuzziness operates between 0 and 1, these membership function values are different for each fuzziness-based parameter, i.e. MSE, PSNR, SSIM, quality and entropy. The following table (Table 1) gives: 1. For the values of the MSE parameters, we have defined the membership function mf1 between 0 and 0.5 and the membership function mf2 between 0.4 and 1. The median values between mf1 and mf2 are, respectively, between 0 0.25 and 0.7. 2. Values of PSNR parameters—we set membership function mf1 between 0 and 35 and membership function mf2 between 35 and 100. The average values between mf1 and mf2 are between 17 and 65, respectively. 3. Values of SSIM parameters—we set membership function mf1 between 0 and 0.5 and membership function mf2 between 0.4 and 1. Average values between mf1 and mf2 are between 0.25, respectively, and 0.7. Table 1 Fuzzy logic parameters membership functions values Fuzzy parameter

mf1

mf2

Mid values

Result

Output

MSE

0–0.5

0.4–1

0.25–0.7

If values near to zero

Better

PSNR

0–35

35–100

17–65

If greater than 35

Better

SSIM

0–0.5

0.4–1

0.25–0.7

If greater than 1

Better

Quality

0–0.5

0.4–1

0.25–0.7

If greater than 1

Better

Entropy

0–0.5

0.4–1

0.25–0.7

If values near to zero

Better

508

S. S. More et al.

4. Values of quality parameters—we have defined the membership function mf1 between 0 and 0.5 and the membership function mf2 between 0.4 and 1. The average values between mf1 and mf2 are, respectively, between 0, 25 and 0.7. 5. Values of entropy parameters—we set membership function mf1 between 0 and 0.5 and membership function mf2 between 0.4 and 1. Average values between mf1 and mf2 are between 0, respectively, 0.25 and 0.7. Diagrammatic Fuzzy Rules for MSE, PSNR, SSIM, Quality and Entropy Parameters In the given Fig. 4, we have shown diagrammatic representations of 19 Fuzzy rules for new modified algorithm. These membership function will work according to these rules. Table 2 gives the accuracy tests of the Blowfish algorithm. We need to classify our dataset into two categories using age and gender, namely Male and Female dataset.

Fig. 4 Diagrammatic representation of Fuzzy rules for new modified algorithm

Table 2 Accuracy testing of new modified Blowfish algorithm Accuracy Sensitivity Specificity Precision False F1-score (TPR and (TNR and (PPV) discovery Recall) Selectivity) rate (FDR) 0.65

0.068966

0.7

0.2

3

FPR

FNR Multimodal trait

0.102564 0.088889 2.7

FACE

0.72

0.153846

0.755556

0.4

5

0.222222 0.066667 2.2

IRIS

0.68

–0.05

0.766667

–0.1

0

–0.06667 0.122222 2.1

PALM PRINT

0.71

0

0.788889

0

1

0

FINGER

0.111111 1.9

Privacy Conserving Using Fuzzy Approach …

509

To test the accuracy of the algorithms, we used various parameters such as sensitivity, specificity (TNR and selectivity) (TPR and recall), FALSE DISCOVERY RATE (FDR), F1-SCORE, FPR and FNR. These parameters are used in our multimodals such as face, iris, finger and Palm print. This dataset is used in Blowfish algorithms for accuracy and performance evaluation. In the Blowfish algorithm, the precision of the iris is high and the precision of the face is low, for the new modified Blowfish algorithm.

4 Conclusion In these articles, we have discussed the Blowfish algorithm related to the security issues of multimodal using Fuzzy logic which has been studied in detail. The new methodology of the modified Blowfish algorithm, and the analysis of the results was also discussed. In these new algorithms, we found greater encryption accuracy. In this algorithm, we used a Fuzzy approach to increase the personal identification selection optimization. For all these processes, we selected six parameters, namely—MSE, PSNR, SSIM, quality, entropy and time complexity. The accuracy of identifying people using our algorithm was high.

References 1. S.S. More, B.T. Jadhav, An overview on technologies used in biometric system. Int. J. Innov. Res. Comput. Commun. Eng. 4(2) (February 2016). ISSN (Print): 2320-9798, ISSN (Online) : 2320-9801, 5.618 2. A. Kumar, C. Wu, Automated human identification using ear imaging. www.elsevier.com/loc ate/pr (Received 3 September 2010 Received in revised form 5 May 2011 Accepted 24 June 2011 3. A. Kumar, B. Wang, Recovering and matching minutiae patterns from finger knuckle images. Pattern Recogn Lett (December 2015) 4. A. Jain, L. Hong, Y. Kulkarni, A Multimodal Biometric System Using Fingerprint (2007) 5. H.B. Kekre, T. Sarode, P. Natu, Performance comparison of Walsh wavelet, Kekre Wavelet and Slant Wavelet transform in image compression. Int. J. Adv. Res. Comput. Commun. Eng. 2(10) (October 2013), Copyright to IJARCCE www.ijarcce.com 3834. ISSN (Print): 2319-5940, ISSN (Online) : 2278-1021 6. K. Bhargavi, S. Jyoth, An efficient Fuzzy logic based edge detection algorithm. Int. J. Tech. Res. Appl. 4(3), 194–196 (May-June, 2016). e-ISSN: 2320-8163, www.ijtra.com 7. S.S. More, B. Narain, B.T. Jadhav, in Role of Modified Gabor Filter Algorithm in Multimodal Biometric Images. 2019 6th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 2019, pp. 1–5 8. S.S. More, B.T. Jadhav, in Overview of Biometric Systems. National Conference Reflection: Emerging Drift, in Wathar College (27th Dec-2015). ISBN: 978-93-85665-02-8 9. S.S. More, B. Narain, B.T. Jadhav, in Advanced Encryption Standard Algorithm in Multimodal Biometric Image, ed. by A.A. Rizvanov, B.K. Singh, P. Ganasala, Advances in Biomedical Engineering and Technology. Lecture Notes in Bioengineering (Springer, Singapore, 2021). https://doi.org/10.1007/978-981-15-6329-4_7

510

S. S. More et al.

10. S. Jaywant Shinde, J.G. Shinde, M.M. Kharade, H. Dhanashree, Biometrics: Ooverview and potential use for E-Governance Services. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 4(6) (June 2014) 11. C.-T. Hsieh, C.-S. Hu, A fingerprint identification system based on Fuzzy encoder and neural network. Tamkang J. Sci. Eng. 11(4) (2008) 12. V. Kanhangad, A. Kumar, A unified framework for contactless hand verification. IEEE Trans. Inf. Forensics Secur. 6(3) (September 2011) 13. S.S. More, B.T. Jadhav, in Fuzzy Logic Algorithms For Extracting Biometric Data. National Conference on Modern Approach for Green Electronics and Computing (2014). ISBN 97881-928732-2-0 Page 200 14. B. Narain, P. Shah, M. Nayak, in Impact of emotions to analyze gender through speech. 2017 4th International Conference on Signal Processing, Computing and Control (ISPCC), Solan, India, 2017, pp. 31–34. https://doi.org/10.1109/ISPCC.2017.8269645 15. S.S. More, B.T. Jadhav, in Fuzzy Logic Approach For Extracting Biometric Data. International Conference on Functional Materials@ Nanoscale:Concerns and challenges, (March 9–11) 2015, ISBN 978-81-930740-0-8 16. F. Robert, Fuzzy logic and neural nets in intelligent systems (1999) 17. A. Piegat, in Fuzzy Models. Fuzzy Modeling and Control. Studies in Fuzziness and Soft Computing, vol. 69 (Physica, Heidelberg, 2001). https://doi.org/10.1007/978-3-7908-1824-6_5 18. S. Singh, R.K. Bansal Savina Bansal, Comparative study and implementation of image processing techniques using MATLAB. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2(3) (March 2012). ISSN: 2277 128X 19. S. Goyal, R. Wahla, A survey on image fusion. Int. J. Innov. Res. Comput. Commun. Eng, 3(8) (August 2015). (An ISO 3297: 2007 Certified Organization), ISSN(Online): 2320-9801, ISSN (Print): 2320-9798

Career Advisor Using AI Harish Natarajan, Dereck Jos, Omkar Mahadik, and Yogesh Shahare

Abstract Nowadays, students have difficulty choosing the right career choice for their future. Students need to know their abilities and interest before choosing their careers for a bright future. The system will help the students opt for the apt career choice based on their performance in the aptitude test. The system uses AI to predict the career path of students. Once the student clears the aptitude, a certificate of achievement is issued to motivate the student, along with a dashboard consisting of career trends for the next five years. The dashboard will help students in choosing a career in that field. The model learns continuously from student feedback that latest trend could be captured. Using such a system would prove beneficial for educational institutions in shaping the future. Keywords Career prediction · Continuous learning · Random forest classifier · Optical character recognition · Image processing

1 Introduction Career Advisor is a Web-based application that guides 10th standard students to select the right career field. As we know that selecting the right career field is a big challenge for 10th students, and we aim to help them to choose the best for their future. H. Natarajan · D. Jos (B) · O. Mahadik · Y. Shahare Department of Information Technology, M.G.M. College of Engineering and Technology, Kamothe, Maharashtra, India e-mail: [email protected] H. Natarajan e-mail: [email protected] O. Mahadik e-mail: [email protected] Y. Shahare e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_44

511

512

H. Natarajan et al.

Our app uses an AI system that will understand the strengths and weaknesses of the students and suggest appropriate fields for them. Our app takes board exam results as input and predicts the best possible fields. To give more accurate predictions, our app consists of an aptitude test [1] from where we get to know the actual strengths of the students and recommend them the best possible career fields. We are also providing top trending fields dashboard [2] which will help them to know the best career fields in the future. The dashboard consists of a bar chart, pie chart and line chart which will be very easy for students to understand the trending fields. Our app is providing OCR-based validation so that the access is restricted to only 10th standard students. We are also providing a 24 × 7 chatbot service [3] so that students can ask their queries at any time.

2 Literature Review As we know choosing the right career choice is very difficult these days. The number of careers is growing day by day. If the students choose any wrong career field in which they are not interested, then they may not learn it well. The main aim of our system is to know the skills and interests of students and then guide them in choosing a career path using AI. There are a lot of technologies that use AI to counsel students. Let us see some of them. Career Bot [3]. It is a chatbot specifically designed to guide students in choosing the right career. The bot would take the student’s query as input in either text or speech. Based on these inputs, the bot would analyze and then predict a career. This would initialize a human–machine interaction through Fuzzy logic expert system [4, 5]. These systems take inputs from students and assign a value between 0 and 1 based on skill and aptitude. Based on these values for the individual student, the expert system guides a career. Following section is mentioned as per this study paper. Section 3 represents as motivation and scope, and Sect. 4 mentioned a problem statement. Section 5 is proposed system, and Sect. 6 is implementation. There are some drawback of current system, First is various data mining and statistical models are used which makes the system difficult to maintain. Second is the system which is not generic as it only works in certain geographical regions.

3 Motivation and Scope This project will be helpful to the 10th students to select the right career fields. This app can be useful for schools and colleges to guide their students. This app can be used by book publishing companies to sell their products. This app can be useful for online educators as they can easily teach students through this app.

Career Advisor Using AI

513

4 Problem Statement According to the recent survey of 8000 students, based on GTI media Asia, over three-fourth of students are selecting career fields that their parents compel and some students decide their career based on the influence of their peers. Since the students are making wrong career choices, the unemployment rate is increasing in India. As per research, the unemployment rate is 47% in 2021 in India. Most of the unemployed population is educated. These problems arise due to poor career counseling. Also guiding students in the rural area are difficult since they do not have enough facilities. We plan to develop an AI-based software that will enable students to choose the right career choice for their bright future.

5 Proposed System Career Advisor is a Web-based application to guide 10th standard students to select the right career field for their future. It also aims to reduce human’s efforts from suggesting career fields and completely giving this job to the AI system. Our system takes board exam results as input and predicts top trending fields as output. For predicting career fields, our system uses random forest classifier algorithm. This system also provides aptitude tests where it can give personalized recommendations of career fields based on the test result. The aptitude test is completely dynamic so every time students will see a new set of questions. To motivate students, our system generates certificates if they score above 75% in every subject. We are also providing a top trending career dashboard where students will get an idea about the trending fields from the current date to the next 5 years. The dashboard consists of visualizations, which students can easily understand. We are providing a chatbot so that students can easily interact with us and ask their queries. Our bot is available 24 × 7 so students can ask their queries at any time. If the students have any complaints regarding our Web app, they can post them on our feedback form. For storing data, we are using firebase. We are storing aptitude questions in firebase’s real-time database, and for storing images, we are using firebase storage. We are using firebase authentication in our login and signup form. We are using OCR-based validation on the signup page to ensure that only 10th standard students get access to our Web app. Our Web app also consists of an admin panel from where the admin can add questions one by one or can add an excel file containing multiple questions. Similarly, the admin can also add top trending career fields.

514

H. Natarajan et al.

6 Implementations 1. Best Career Path This component helps to predict career fields based on the 10th board result. For science field prediction, we are using Science and Mathematics score, for predicting commerce field, we are using Commercial Application, Economics and Mathematics score, and for predicting arts field, we are using English, Social Science and Economics score. Random forest classifier algorithm is used for predicting career fields. 2. Aptitude This component is used to test the skill of students on a particular subject. The test would be dynamic, and random questions will be generated each time. The test is divided into sub-topics, e.g. in the science field, sub-topics are physics, chemistry, biology and mathematics. The test will be timer-based. To avoid unfair means, we have disabled the mouse right-click event. If the user scores above 75% in each subtopic, then the certificate is issued. If the user scores below 75% in any sub-topic, then they are listed. Students cannot give tests more than 3 times a month. Students need to clear the aptitude within 3 attempts in a month or else the quiz will be locked for a month. This component will give students exposure to giving tests on real-world coding platforms. a. Admin Dashboard To ensure ease of admin, we have included excel uploading feature. Admin can upload questions one by one via our dashboard or use above mentioned excel uploader feature. Data is stored in firebase DB. Similarly, admin can upload data for visualization as well. b. Student Validator To ensure that, only 10th standard students access our Web app, and we are using optical character recognition (OCR) technique. The students need to upload their mark sheets while registering. We perform OCR using the cloud. OCR uses convolutional recurrent neural network (CRNN) architecture. c. CRNN Model: The network starts with the traditional 2D convolutional neural network followed by batch normalization, ELU activation, max-pooling and dropout with a dropout rate of 50%. Three such convolution layers are placed in a sequential manner with their corresponding activations. The convolutional layers are followed by permute and the reshape layer which is very necessary for CRNN as the shape of the feature vector differs from CNN to RNN. The convolutional layers are developed on 3-dimensional feature vectors, whereas the recurrent neural networks are developed on 2-dimensional feature vectors. The permute layers change the direction of the axes of the feature vectors, which is followed by the reshape layers, which convert the feature vector to a 2-dimensional feature vector. The RNN is compatible with the 2-dimensional feature vectors. The proposed network consists of two bidirectional GRU layers with “n” no. of GRU cells in

Career Advisor Using AI

515

each layer where “n” depends on the no. of classes of the classification performed using the corresponding network. The bidirectional gated recurrent unit (GRU) is used instead of the unidirectional RNN layers because the bidirectional layers take into account not only the future timestamps but also the future timestamp representations as well. Incorporating two-dimensional representations from both the timestamps allows incorporating the time dimensional features in a very optimal manner. Finally, the output of the bidirectional layers is fed to the time distributed dense layers followed by the fully connected layer (Fig. 1). d. Chatbot To resolve students’ query, we are providing a 24 × 7 chatbot service. To create a chatbots, we are using Google Dialogflow messenger. Google Dialogflow messenger is a cloud-based chatbot as a service solution that would make it easier

Fig. 1 CRNN architecture

516

H. Natarajan et al.

Fig. 2 Architecture diagram of proposed system

for developing and deploying enterprise-ready chatbots within a few minutes. It makes it easier for integrating chatbots with custom Web sites. e. Top trending career Dashboard This dashboard helps students to identify top trending career fields. Students will come to know trending careers from current date to 5 years ahead. This dashboard is created using Dash. Dash is a user-interface library for creating analytical Web applications. Those who use Python for data analysis, data exploration, visualization, modelling, instrument control and reporting will find immediate use for Dash. f. Student Dashboard This dashboard helps students to identify top trending career fields. Students will come to know trending careers from the current date to 5 years ahead. This dashboard is created using Dash. Dash is a user-interface library for creating analytical Web application. Those who use Python for data analysis, data exploration, visualization, modeling, instrument control and reporting will find immediate use for Dash. g. Architecture Diagram The Architecture diagram of proposed system is shown in Fig. 2.

7 Conclusion Career Advisor is a unique system. This software provides the best career options for students based on their skills. Since the number of career opportunities increases daily, the need for career counseling has also increased. This research paper proposes a career guidance system using AI. The system uses AI to predict the best career field

Career Advisor Using AI

517

based on the student’s score in aptitude and skills. It also uses a chatbot to provide 24 × 7 guidance to students. Currently, this system provides career trends only in AI field. The aptitude system can be made much more robust by locking users from opening new tab while giving test. In future, more data can be used to increase the accuracy of the system. We conclude this research paper by stating that this AI-based “Career Advisor” can be a boon for today and the future world.

References 1. D.M. Bhnushali, P. Itankar,Online career counsellor system based on artificial intelligence: an approach. Int. Organ. Res. Dev. 8(2), 4 (2021) 2. C.M. Chang, in New Organizational Design to Promote Creativity: Virtual Teams with Anonymity and Structured Interactions. Proceedings of IAMOT (International Association for Management of Technology) Conference, Miami Beach, Florida (April 10–14) (2011) 3. J. Chai, J. Lin, The role of natural language conversational interface in online sales: a case study. Int. J. Speech Technol. 4, 285295 (2001) 4. T.R. Razak,M.A. Hashim, N.F.F. Mohammad Noor, in Career Path Recommendation System for UiTM Perlis Students using Fuzzy Logic. Intelligent and Advances Systems (ICIAS), 2014 5th International on, pp. 1–5, IEEE (2014) 5. E. El Haji, A. Azmani, M. El Harzli, Multi-expert systemdesign for educational and career guidance: an approach based on a multi-agent system andontology. Department of Computer Science, LIST Laboratory, Faculty of Science and Technology. 3. S. Saraswathi, Design of an online expert system for career guidance. Department of Computer Science, LIST Laboratory, Faculty of Science and Technology (2014)

A Novel Blood Vessel Parameter Extraction for Diabetic Retinopathy Detection R. Geetha Ramani and J. Jeslin Shanthamalar

Abstract Diabetic retinopathy (DR) is one of the leading causes of visual loss if it is not treated at an earlier stage. Manual identification of diabetic retinopathy is a time-consuming process, and regular screening is a must for an early diagnosis. This paper presented a novel blood vessel parameter extraction method for DR identification using image processing and data mining techniques. An automatic DR diagnosis through image processing techniques, by extraction of blood vessel parameters such as vessel density, minimum and maximum thickness of blood vessels and classification through data mining techniques was proposed. Mostly diabetic retinopathy identification was done by lesion pattern identification such as exudates, microaneurysms, cotton wool spots, etc. However, this work concentrated on calculating the disease parameter through segmented blood vessel region from full fundus image, optic nerve head region and Inferior, Superior, Nasal and Temporal (ISNT) region. Evaluation of this work was performed on DRIVE and HRF datasets and achieved overall accuracy of 97.14% in terms of DR prediction. Keywords Optic nerve head · ISNT region · Diabetic retinopathy · Blood vessel

1 Introduction Retinal blood vessel segmentation is considered an important key feature because it reveals the fine changes that occurred in the artery, vein and its branches which are much important for identification of retinal diseases, namely diabetic retinopathy, glaucoma, vein occlusion, artery occlusion, etc., as well as cardio vascular diseases. In order to avoid human interference and provide better segmentation accuracy toward disease prediction, many computerized techniques were addressed by researchers in the area of blood vessel segmentation which are detailed here. Diabetes is a disorder that occurs when the pancreas does not secrete enough insulin, which is identified by retinal pathologies. However, the disease progresses R. Geetha Ramani · J. Jeslin Shanthamalar (B) Department of Information Science and Technology, Anna University, Chennai, India e-mail: [email protected]; [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_45

519

520

R. Geetha Ramani and J. Jeslin Shanthamalar

lead to retinal diseases, namely DR, diabetes mellitus, diabetic macular edema, vein or artery occlusion, etc., will cause blurred vision or vision loss. Bright lesions, such as hard or soft exudates, are considered as one of the most important retinal abnormalities to identify DR at its initial stage. Generally, severities of DR are identified based on size, number of exudates and localization of exudates but it is difficult to identify due to pathological distractions. Many existing techniques proposed for automatic diabetic retinopathy diagnosis through different approaches which are detailed further. Early detection of DR will be most helpful to overcome blindness problem, and it was developed through automatic detection of dark and bright lesion from the fundus image [1]. Lesions like microaneurysms, hemorrhages, exudates and cotton wool spots were identified using hybrid fuzzy neural classifier after blood vessel enhancement and OD segmentation. Another approach for DR classification, from the identification of exudates region, was introduced [2]. Exudates region is identified through mathematical transformations and models to extract the shape and location of anomalous features. Using this technique, DR was classified into five stages, namely normal, mild, moderate, severe and very severe. An ensemble-based diabetic retinopathy screening was developed using image-level, lesion-level and anatomical-level feature extraction [3]. Features like image quality, inhomogeneity measure, Amplitude/Frequency Modulation (AM/FM), microaneurysms, exudates, macula and OD were extracted for efficient classification of DR. Classification of nonproliferative diabetic retinopathy through SVM classification, by extracting features of blood vessel, microaneurysms and hard exudates was proposed [4]. In recent days, DR diagnosis process using a deep learning-based approach was proposed because of its higher accuracy and less complexity. An approach of imagelevel performance through six predefined CNN model for DR classification was presented using transfer learning and hyper-parameter tuning method [5]. Again, an approach to diagnose DR through exudate characterization by using adaptive neuro-Fuzzy inference system (ANFIS)-based classification was introduced [6]. This method suppresses the blood vessel region using morphological operations, OD masking and then exudates characterization through enhancement and feature extraction by using shape, color, texture and intensity information, followed by ANFIS classification. Finally, another predefined machine learning models, such as VGG-16 and VGG-19, was adopted for five-stage DR classifications and achieved 80% of sensitivity, 82% of specificity and accuracy [7]. From the related studies, it clearly states that most of the existing techniques identify DR disease through lesion pattern detections, feature extraction and deep learning-based methods. The following section presents the processes involved in DR classification.

A Novel Blood Vessel Parameter Extraction …

521

2 Materials and Methods This section details the process of diabetic retinopathy (DR) prediction which includes image preprocessing, blood vessel parameter extraction and DR classification. Figure 1 presents the proposed system architecture diagram for automatic diabetic retinopathy identification through blood vessel parameter extraction.

2.1 Dataset Description Digital retinal images for vessel extraction (DRIVE) database contains 40 retinal fundus images with ground truth of diabetic retinopathy disease was presented. This dataset contains 33 images with 565 × 584 resolution size and 33 images which do not show any signs of diabetic retinopathy and 7 images shows symptom of diabetic retinopathy. High resolution fundus image dataset (HRF) contains 45 images with Fig. 1 Architecture diagram for diabetic retinopathy identification

Diabetic Retinopathy and Healthy Images

Left / Right Eye

Optic Disc Localization Full Fundus Image

ONH segmentation

Blood Vessel Segmentation

Blood Vessel (BV) Parameter BV Density

BV Thickness

Diabetic Retinopathy

Healthy

ISNT Region

522

R. Geetha Ramani and J. Jeslin Shanthamalar

3504 × 2336 resolution size which includes 15 healthy, 15 glaucoma and 15 diabetic retinopathy images. The proposed approach was evaluated on these two datasets in terms of DR identification.

2.2 Image Pre-processing Initially, retinal fundus images are categorized into left and right eye images, which help to compute blood vessel density and thickness of Inferior, Superior, Nasal and Temporal (ISNT) region for the identification of diabetic retinopathy disease. In general, ISNT rules are used to calculate the fundus image abnormality in terms of glaucoma disease identification, whereas proposed work adopted for DR diagnosis. Normally, fundus images are labeled into left or right eye images based on OD with respect to the macula/fovea region [8]. Using this information, left and right image categorization is done manually before the segmentation of blood vessel region. This method is novel and it efficiently differentiates the diabetic retinopathy images from healthy images. After categorization of fundus images, OD region is localized through pixel density calculation method and then ONH region is cropped using a rectangular window template to extracts fine details from the cropped images which is most useful for diabetic retinopathy classification [9]. This method localized all the fundus images on DRIVE and HRF datasets with 100% accuracy. Combination of full fundus and ONH region images are taken as input for the vessel parameter extraction process.

2.3 Blood Vessel Parameter Extraction Primarily, blood vessels are segmented through CLAHE and dynamic gray level thresholding method because of its high accuracy, i.e. 95.59% on HRF and DRIVE datasets [10]. Blood vessel thickness is identified by extracting image features such as area and perimeter of identified region of interest (ROI). These parameters are used for the calculation of minimum and maximum thickness of blood vessel and blood vessel density of full and ONH cropped images. The proposed method extracted 16 blood vessel parameters from the segmented blood vessel region for DR identification is presented in Table 1.

2.4 Diabetic Retinopathy Classification Predictions of diabetic retinopathy retinal disease using blood vessel parameters through different classification algorithms and visualization trees are explored. The alternating decision tree (ADTree), J48 Graft Tree, LogitBoost ADTree (LAD Tree)

A Novel Blood Vessel Parameter Extraction …

523

Table 1 Blood Vessel parameters for diabetic retinopathy identification S. No

Blood vessel parameters

1

Blood vessel density of full fundus image

2

Minimum blood vessel thickness of full fundus image

3

Maximum blood vessel thickness of full fundus image

4

Blood vessel density of optic nerve head

5

Blood vessel density of Inferior region from optic nerve head

6

Blood vessel density of Superior region from optic nerve head

7

Blood vessel density of Nasal region from optic nerve head

8

Blood vessel density of Temporal region from optic nerve head

9

Minimum blood vessel thickness of Inferior region from optic nerve head

10

Minimum blood vessel thickness of Superior region from optic nerve head

11

Minimum blood vessel thickness of Nasal region from optic nerve head

12

Minimum blood vessel thickness of Temporal region from optic nerve head

13

Maximum blood vessel thickness of Inferior region from optic nerve head

14

Maximum blood vessel thickness of Superior region from optic nerve head

15

Maximum blood vessel thickness of Nasal region from optic nerve head

16

Maximum blood vessel thickness of Temporal region from optic nerve head

and random tree classifiers are investigated for the early prediction model. From that analysis, the smallest tree among the decision tree models with good accuracy and less complexity is chosen for disease prediction. J48Graft Tree model has small tree size and small leaves with high accuracy generated on DRIVE dataset is selected as final classifier. Visualization of decision tree model obtained through extracted blood vessel parameter is presented in tree structure, as shown Fig. 2.

3 Results and Discussion The experimental results of the proposed blood vessel parameter calculation for DR disease identification are discussed in this section. To perform overall processing “intel(R) Core(TM), 1.70 GHz” processor with 8 GB RAM is used as a personal computer configuration. Image processing techniques were implemented using MATLAB R2017b version, and the performance measures were evaluated on TANAGRA1.4.50 data mining software. The proposed method utilized traditional blood vessel parameters along with novel disease parameter calculation through ISNT region is utilized for improved accuracy. ISNT blood vessel parameter calculation method segmented blood vessel region with better accuracy on overall 70 retinal fundus images. Further, ISNT regions blood vessel density, thinness and thickness are calculated by segmenting the ONH fundus

524

R. Geetha Ramani and J. Jeslin Shanthamalar

Fig. 2 Sample decision tree of J48 Graft Tree classifier for diabetic retinopathy prediction

Minimum Blood Vessel Thickness

0

Minimum Blood Vessel Thickness (Inferior Region)

0

Blood Vessel Density (Nasal Region)

1000

Diabetic Retinopathy

image into four regions such as Inferior, Superior, Nasal and Temporal. Figures 3 and 4 shows the visualization results of blood vessel segmentation. Finally, rules from J48Graft Tree classifier is chosen for DR prediction model on DRIVE and HRF dataset. From these rules, three blood vessel parameters such as minimum blood vessel thickness of inferior region, full fundus image and blood vessel density of nasal region are considered as important for further process. These parameters are applied through image processing techniques on DRIVE and HRF dataset and achieved good prediction accuracy are noted in Table 3. From Table 2, the performance result of DRIVE dataset shows that 100% accuracy on healthy image identification, whereas HRF dataset achieved 100% accuracy on diabetic retinopathy image identification. The proposed system achieved overall accuracy of 97.92% on healthy images and 95.46% on diabetic retinopathy images were noted. Comparative analysis of proposed approach with existing technique is presented in Table 3. The proposed method achieved high overall accuracy of 97.14% among all the existing techniques represents the robustness and competency of presented approach.

A Novel Blood Vessel Parameter Extraction …

525

Fig. 3 Blood vessel segmentation on HRF dataset a Full fundus image, c ONH fundus image and b and d segmented blood vessel

Fig. 4 Blood vessel segmentation on HRF dataset for ISNT parameter computation a ONH fundus image, b segmented blood vessel, c–f Inferior, Superior, Nasal and Temporal region mask and g–j mask mapped with segmented blood vessel

526

R. Geetha Ramani and J. Jeslin Shanthamalar

Table 2 Performance results of DRIVE and HRF dataset S. No

Dataset

Image class

Total images

Correctly identified

Accuracy (%)

1

DRIVE

Healthy

33

33

100

Diabetic Retinopathy

7

6

85.71

2

HRF

Healthy

15

14

93.33

Diabetic Retinopathy

15

15

100

Table 3 Comparative analysis of proposed approach versus existing approaches

S. No

Technique

Accuracy (%)

1

Carrera et al. [4]

85.10

2

Wan et al. [5]

95.68

3

Qummar et al. [11]

80.80

4

Nguyen et al. [7]

82.00

5

Proposed method

97.14

4 Conclusion This research work proposed a novel method of DR identification through disease parameter identification from ISNT region by using blood vessel properties. In this paper, disease identification is attempted through simple blood vessel features with high accuracy. This approach adopted existing methods of blood vessel properties, along with novel methods such as calculating the blood vessel features of Inferior, Superior, Nasal and Temporal region individually. Feature extraction method also done on full fundus images as well as cropped ONH region. Hence, the proposed method could be adopted on heterogeneous images with respect to diabetic retinopathy classification. Generally, ISNT calculation are used for glaucoma disease identification, whereas proposed methodology considered blood vessel density and thickness of ISNT region for diabetic retinopathy prediction, which was unique and achieved better results. The features selected by J48Graft Tree classifier is evaluated on DRIVE and HRF datasets and achieved 97.14% accuracy will be more helpful to assist the ophthalmologist in early prediction of DR. Acknowledgements This work was supported by Centre For Research, Anna University under the Anna Centenary Research Fellowship, Anna University, Chennai, India (Reference: CFR/ACRF/2018/AR1).

A Novel Blood Vessel Parameter Extraction …

527

References 1. U.M. Akram, S.A. Khan, Automated detection of dark and bright lesions in retinal images for early detection of diabetic retinopathy. J. Med. Syst. 36, 3151–3162 (2012). https://doi.org/10. 1007/s10916-011-9802-2 2. V. Zeljkovi´c, M. Bojic, C. Tameze, V. Valev, in Classification Algorithm of Retina Images of Diabetic Patients Based on Exudates Detection. Proceedings of the 2012 International Conference on High Performance Computing and Simulation, HPCS 2012. 167–173 (2012). https:// doi.org/10.1109/HPCSim.2012.6266907 3. B. Antal, A. Hajdu, An ensemble-based system for automatic screening of diabetic retinopathy. Knowl.-Based Syst. 60, 20–27 (2014). https://doi.org/10.1016/j.knosys.2013.12.023 4. E.V. Carrera, A. Gonzalez, R. Carrera, in Automated Detection of Diabetic Retinopathy Using SVM. Proceedings of the 2017 IEEE 24th International Congress on Electronics, Electrical Engineering and Computing, INTERCON 2017 (2017). https://doi.org/10.1109/INTERCON. 2017.8079692 5. S. Wan, Y. Liang, Y. Zhang, Deep convolutional neural networks for diabetic retinopathy detection by image classification. Comput. Electr. Eng. 72, 274–282 (2018). https://doi.org/10. 1016/j.compeleceng.2018.07.042 6. R. Valarmathi, S. Saravanan, Exudate characterization to diagnose diabetic retinopathy using generalized method. J. Ambient. Intell. Humaniz. Comput. 12, 3633–3645 (2019). https://doi. org/10.1007/s12652-019-01617-3 7. Q.H. Nguyen, R. Muthuraman, L. Singh, G. Sen, A.C. Tran, B.P. Nguyen, M. Chua, in Diabetic Retinopathy Detection Using Deep Learning. ACM International Conference Proceeding Series, pp. 103–107 (2020). https://doi.org/10.1145/3380688.3380709 8. X.Lai, X. Li, R. Qian, D. Ding, J. Wu, J. Xu, Four models for automatic recognition of left and right eye in fundus images. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 11295 LNCS, pp. 507– 517 (2019). https://doi.org/10.1007/978-3-030-05710-7_42 9. R.G. Ramani, J.J. Shanthamalar, Improved image processing techniques for optic disc segmentation in retinal fundus images. Biomed. Signal Process. Control 58, 101832 (2020). https:// doi.org/10.1016/j.bspc.2019.101832 10. J. Jeslin Shanthamalar, G.R.R., in Automatic Blood Vessel Segmentation in Retinal Fundus Images Using Image Enhancement and Dynamic Gray Level Thresholding. 4th International Conference on Computational Intelligence and Data Engineering, pp. 1–11 (2021) 11. S. Qummar, F.G. Khan, S. Shah, A. Khan, S. Shamshirband, Z.U. Rehman, I.A. Khan, W. Jadoon, A deep learning ensemble approach for diabetic retinopathy detection. IEEE Access. 7, 150530–150539 (2019). https://doi.org/10.1109/ACCESS.2019.2947484

Identifying Criminal Communities in Online Networks via Non-negative Matrix Factorization-Incorporated Sentiment Analysis Shafia and Manzoor Ahmad Chachoo

Abstract Right from the Internet, which has been publicly available, users have been able to engage with one another through virtual networks and in the last decade, due to the emergence of online social networks, community identification in complex networks has received a lot of attention. As community identification task involves identifying important people and their linkages, social network analysis is one such technique to analyse complex networks such as criminal networks. Keeping in view the diversity of actors and gangs involved in crime activities, the goal is to investigate and assess their characteristics so that the essential information characterising their behaviour is extracted. The current work will employ a social network analysis-based novel approach called sentiment analysis on influential nodes (SAOIN) to attain this important goal. Our approach claims to be computationally efficient as only the influential nodes (aka leaders) of the established subnetworks (communities) are taken into consideration for further investigation rather than inquiring each and every actor of a network. This discerns our model from other already existing community identifying techniques. The proposed model generates small subnetworks that can be used to discover the list of actors and their relationships that need to be inquired further, As opposed to other already existing community detecting methods that generates larger and much complex networks. This study inquires actors of the social network like Twitter whose activities promotes criminal propaganda across diverse stages. The information dissemination among these actors directs sole insight towards their behaviours. Keywords Non-negative matrix factorization (NMF) · Degree centrality

1 Introduction Online social media platforms have become a breeding base for criminal activities, and law enforcement authorities are facing up significant challenges because Shafia · M. A. Chachoo (B) Department of Computer Science, University of Kashmir Srinagar, Jammu & Kashmir, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_46

529

530

Shafia and M. A. Chachoo

there is nope consistent and generalised framework specifically designed to analyse user misconduct and its activities on these social media platforms. Data transmitted through these platforms is a valuable source of information, but characteristics like unstructured nature, huge volumes, and data interconnectivity make it difficult for law enforcement authorities to analyse it using traditional strategies and offer it to legal realm. Despite the fact that numerous studies over digital forensics have been conducted, there has been little focus over developing suitable tools to comprehensively meet all of the requirements of criminal investigation involving information exchange, data integration, and the collection and conservation of digital evidence. To close this gap, we presented a generic and robust digital data framework called SAOIN that can assist law enforcing authorities in identifying and preventing criminal activities occurring over social media platforms. Community identification is a significant endeavour in network analysis research because it provides vital information which may be deployed to resolve real-life problems. In this research, an attempt was made to identify criminals across social networks. Because of their ubiquity, Facebook and Twitter play a big part in this setting. This paper makes a two-fold contribution. Firstly, this paper proposes the knowledge-based architecture, which enables law enforcing authorities to analyse unstructured data and discover hidden patterns and correlations among criminals with an emphasis on criminal investigation. Secondly, the paper focuses on the usability and scalability issues that complex criminal graphs face when it comes to discovering groups. Rest of the study is structured as follows: Sect. 2 discusses and summarises the criminal network analysis. Section 3 describes criminal identification by deploying sentiment analysis on influential nodes (SAOIN) methodology. This section will explain and describe the criminal graph construction along with algorithmic framework. Lastly, we have presented the conclusion along with some future directions in Sect. 4.

2 Background and Related Work Criminals are getting more technologically proficient in today’s environment, and they frequently vent their emotions across the Internet. Because of the extraordinary rise of the social media platforms, more people are expressing their ideas online. Criminals have always deployed the sophisticated technology to pursue of their goal. They have shifted to social media platforms as a result of their own perseverance [1–3], thus making the monitoring and controlling more difficult for law authorities. As a result, there is a growing interest in assisting in the detection of crime activities based on Internet and identifying online criminals, as seen by a growing number of research activities in this field. Criminals can use social media networks to gain free web hosting and post content anonymously and reliably for no cost [4]. Twitter was given special consideration. Because Twitter is one among the most prominent microblogging sites, it was chosen over other social media sites for its political worth and transparency. Furthermore, people are highly vocal regarding

Identifying Criminal Communities in Online Networks …

531

their viewpoints and beliefs, and they don’t hesitate to voice them via Twitter. As a result, this study is motivated by the evidence that the vast amounts of data available across these networks can be utilised to extract a considerable quantity of data for administration and the law enforcement, which can then be utilised to forecast criminal behaviour patterns. The majority of current criminal prediction models rely on the relative static features such as long-term verifiable data, because this data gradually changes over time, traditional models are unable to detect transient variations in criminal activity [5]. The main disadvantage in these models was that they reduce the social settings to the verifiable criminal record while ignoring information on social conduct from users of social media sites, including victims and criminals, as keeping track of social behavioural information in such a large society is hard and cumbersome task [6]. Gerber [7] is considered the first to use social media data as part of a crime prediction algorithm. Despite the fact that it is the first study to look at Twitter text, Gerber’s use of latent Dirichlet allocation is tricky because it being an unsupervised method, which means the link between the word-clusters and crimes isn’t based on past theoretical ideas. The authors of the work [8, 9] used sentiment analysis in tweets and meteorological information from KDE to estimate the location and time of theft. Their research, however, was limited to spatial data, such as meteorological information for a certain time and area. Furthermore, the key flaw in these researches was that they ignored tweet text in favour of relying solely over geolocation data. Aside from the studies mentioned above, sentiment analyses have also been used to detect and prevent crime [10]. Machine learning methods [11–13] have been used also to perform sentiment analysis on tweets. These investigations were successful, but they failed to account for semantics in capturing the tweet meanings. Nevertheless, criminals exploit the ease through which they approach social data platforms to spread their data and pull in new members. On the basis of the research survey discoursed so far within this paper, it is evident that further research is expected and needed so that to this domain could be advanced. In this research, we have attempted to overcome the shortcomings of the preceding discussed investigations.

3 Proposed Approach To gain a better understanding of criminal behaviour of diverse actors, the communities to which they belong, and the information they publish on social networks, a machine learning-based model called SAOIN was proposed. Our approach distinguished from the other approaches discussed in literature (like [14, 15]), we attempted to deploy the two significant techniques ‘matrix factorization’ and ‘sentiment analysis’ altogether so that the smarter performance is achieved. Moreover, our model affirms to be computationally efficient as only the influential nodes from the established subnets (in particular communities) have been taken into consideration for further examination instead of examining every node of a subnet. This distinguished our strategy from other baseline community identifying strategies.

532

Shafia and M. A. Chachoo

The proposed method generates small subnets which can be utilised in identifying list of influential persons along with relationships that are inquired further. Contrastingly recognising subnets of interest employing other baseline community identification approaches produce larger and much complex subnets [16, 17]. This research has chosen Twitter as a social data network from where to gather data. For the efficacious criminal community detection strategy, the following sentiment analysis on influential nodes (SAOIN) methodology might be adopted. The proposed SAOIN model (Fig. 2) would focus upon the collaborative working of non-negative matrix factorization, degree centrality along with sentiment analysis. Therefore, the proposed model would follow the three stage approach as is described below: A. Non-Negative Matrix Factorization: Here the similarity graph based on contact strength among nodes is computed first, where contact strength represents the degree of closeness among nodes of a network. Because the triangle structure may better characterise the tightness among the nodes, we employed triangles to formalise the contact strength definition. It is computed using the following formula: CSuv =

N (u) ∩ N (u) Tu

where T u represents the no. of triangles of vertex u, and the intersection between N(u) and N(v) denotes the no. of triangles common to node u and node v. So the strategy is based on the fact that strong ties play a major part in community formation and information diffusion. In the network having n nodes, the similarity of every node pair according to the above described contact strength formula is computed first, and therefore, n*n form of similarity matrix as S = {sij } is obtained, whereas an element sij indicates the closeness between I and j nodes. Then this similarity matrix is fed to the existing non-negative matrix factorization algorithm (Fig. 3) through which different communities are obtained. NMF uncovers the inherent network community composition and improves interpretability and compression because of its ‘sparse’ and ‘parts-based’ representation with the solely additive constraint or non-negativity. A. Influential Node Identification Based On The Degree Centrality: As the different community formation is done, degree centrality is applied upon these communities so as to identify the influential nodes. The influence of nodes in a network is determined by the two main factors. (i) When node is at the network’s centre, it has a lot of influence. Otherwise, when the node is at the network’s edge, its influence will be minimal. (ii) The no. of node neighbours: The more neighbours a node has, the more influence it has. Due to these factors, we have employed degree centrality measure as measurements based on centrality attempt to get the relative node importance inside a network (Fig. 1). The actor holding the maximal degree centrality within the community represents the influential node with respect to that particular community.

Identifying Criminal Communities in Online Networks …

533

Fig. 1 Schematic illustration of degree centrality

B. Sentiment Analysis: Next phase is to examine the influential node’s sentiment on pre-processed tweets. As opposed to machine learning approaches like SVM, the technique here applied would be based on lexicon sentiment analysis, as it is much intelligible and each tweet would be having 2 labels: positive and negative. Depending upon the tweets content, criminal tweets are verified as described below:

• Negative Tweets: Includes unpleasant emotions, provocative statements, scandalous news, or controversial viewpoints. • Positive Tweets: Includes positive feelings,pleasant statements, day to day routines, or funny things. These tweets are often called as tweet advertisements. Figure 1 is depicting the influential nodes of Zachary karate network where nodes 1, 33, and 34 are holding the maximal degree centrality.

4 Conclusion and Future Scope Social networks are strong tools that bring together millions of people across the world. As a result, they’re turning to social criminalism as well. A methodological approach has been devised in particular for the detection of features related to criminals. Twitter was chosen to display the proposal because it is not only the prominent

534

Fig. 2 Depicting the working of SAOIN methodology

Fig. 3 Showing the complete architecture of SAOIN model

Shafia and M. A. Chachoo

Identifying Criminal Communities in Online Networks …

535

social networks, but it is also heavily used by criminals as a means of communication to diffuse criminal information, propagate propaganda, and so on. Sentiment analysis employing other social media platforms, such as Instagram, and others, as well as non-text messages to aid the law enforcement authorities, is possible for the next study. We think that by employing strategies like SAOIN, we will be able to counteract the impact of these criminal agencies in the near future.

References 1. M. Girvan, M.E. Newman, Community structure in social and biological networks. Proc. Nat. Acad. Sci. 99(12), 7821–7826 (2002) 2. M.E. Newman, M. Girvan, Finding and evaluating community structure in networks. Phys. Rev. E 69(2), 026113 (2004) 3. D. Reeder, Jihadi social media. [Online] (2011). http://defensetech.org/2011/06/27/jihadi-soc ial-media/ 4. S. Stalinsky, Why haven’t the taliban’s twitter accounts been shut down? Retrieved from The Middle East Media Research Institute. [Online] (2011). http://www.memri.org/report/en/0/0/ 0/0/0/0/5707.htm 5. S. Sathyadevan, M.S. Devan, S.S. Gagadharan, in Crime Analysis and Prediction Using Data Mining. International Conference on Soft Computation. ICNSC (2014) 6. J. Chan, L.B. Moses, Is Big Data challenging criminology? Theor. Criminol. (2016) 7. M.S. Gerber, Predicting crime using Twitter and kernel density estimation. Dec. Support Syst. (2014) 8. T. Waskiewicz, in Friend of a Friend Influence in Terrorist Social Networks,.Proc Int. Conf. on Artificial Intelligence (2012) 9. X. Chen, Y. Cho, S.Y. Jang, in Crime Prediction Using Twitter and weather. Systems and Information Engineering Design Symposium (SIEDS), IEEE (2015) 10. N. Zainuddin, A. Selamat, R. Ibrahim, Improving Twitter aspect based sentiment analysis using hybrid approach. Intell. Inf. Database Syst. (2016) 11. B. Gokulakrishnan, P. Priyanthan, T. Ragavan, N. Prasath, As. Perera, Opinion mining and sentiment analysis on a Twitter data stream (2012) 12. S. Dutta, M. Roy, A.K. Das, S. Ghosh, in Sentiment Detection in Online Content: A WordNet Based Approach, ed. by B. Panigrahi, P. Suganthan, S. Das, Swarm, Evolutionary, and Memetic Computing. SEMCCO 2014. Lecture Notes in Computer Science, vol 8947 (Springer, Cham, 2015). https://doi.org/10.1007/978-3-319-20294-5_36 13. B. Pang, L. Lee, S. Vaithyanathan, in Thumbs Up? Sentiment Classification Using Machine Learning Techniques. Proceedings of the ACL-02 Conference on EMPIRICAL METHODS in Natural Language Processing (2002) 14. M.C. Benigni, K. Joseph, K.M. Carley, Online extremism and the communities that sustain it: detecting the ISIS supporting community on twitter. PLOS one (2017) 15. T. Junjing, Research on the application of cluster analysis in criminal community detection. J. Phys. (2018) 16. F. Calderoni, D. Brunetto, C. Piccardi, Communities in criminal networks: a case study (2016) 17. R. Al-Zaidy, B.C.M. Fung, A.M. Youssef, F. Fortin, Mining Criminal Networks from Unstructured Text Documents (Elseiver, 2012)

Fuzzy Logic-Based Disease Classification Using Similarity-Based Approach with Application to Alzheimer’s Ankur Chaurasia, Priyanka Narad, Prashant K. Gupta, Mahardhika Pratama, and Abhay Bansal

Abstract With the advent of the post genomics era, there has been a surge in the amount of medical data for healthcare applications as well as the number of novel solution methodologies. The management and analysis of this data are a tricky task, as the data is scattered over a plethora of public and private repositories. The task becomes challenging if the data exhibits diverse characteristics such as gene information. Finding the gene of interest, causing a certain disease, or understanding the case–control data set is a much daunting task for the researchers. Locating the targeted gene, responsible for causing a certain disease, will depend upon the ability of computational methods. The current software based on these methods uses various statistical tests for performing the intended function. The statistical test used performs univariate classification independently for each gene of interest. To circumvent the uncertainty of the role of each gene independently, fuzzy logic plays a crucial role. It uses similarity-based approach to analyse the data especially real-time data. Therefore, the aim of this work is to demonstrate the use of fuzzy logic for gene identification and disease classification in an efficient manner. Keywords Fuzzy logic · Disease classification · Gene identification · Mamdani model · Neurological disorder · Alzheimer’s disease

A. Chaurasia · P. Narad · A. Bansal (B) Amity University, Noida, Uttar Pradesh, India e-mail: [email protected] A. Chaurasia e-mail: [email protected] P. Narad e-mail: [email protected] P. K. Gupta DFKI, Kaiserslautern, Deutschland, Germany M. Pratama Nanyang Technological University, Singapore, Singapore e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_47

537

538

A. Chaurasia et al.

1 Introduction Neurological disorders are caused by more than 600 conditions which play a vital role in effecting our nervous system. A study states that approximately 50 million people are affected by this disorder every year in America only [1]. In recent years, scientists have learned and developed techniques to deal with such neurological disorders. Despite of this much of work, there is always a room for improvement in this area. Even after this much of work, an ample amount of neurological system is still untouched. Some of these disorders are very well known like Alzheimer’s, Dystonia, Huntington’s, Parkinson’s, Stroke, etc. [1]. Alzheimer’s disease (AD) is a neurological disorder which deteriorates the human brain. This degradation causes memory loss and lowers the ability of brain imaging. It is hard to be diagnosed in an early stage because degradation can also be caused due to age. It is, therefore, difficult to identify the root cause whether it is age or Alzheimer’s [1]. In a study, World Health Organization (WHO) has predicted that by 2050 there would be more than 100 million people suffering from Alzheimer’s [2]. In another study, it has been predicted that in next two decades there would be a significant rise in the number of AD patients and the figures may go up to 13.8 million people by 2050. [3] AD can progress slowly in three stages, mild (the initial stage), moderate (the middle stage), and severe (the late stage). An early stage diagnosis of AD plays a vital role in terms of curing the disease. Mild cognitive impairment (MCI) is an intermediate stage of AD where there is a high probability of reduction in ability of performing daily routine [3]. A study states that 20% of the people above age of 65 are having a high risk of suffering from MCI [3]. Recently, AD classification and the conversion of MCI to AD has been successfully monitored by using neuroimaging techniques like structural magnetic resonance imaging (sMRI) [4, 5], functional MRI [6, 7], diffusion tensor imaging [8], positron emission tomography (PET), and single photon emission computed tomography (SPECT) [9]. Even after this much of work, there is always a scope of improvement in early diagnosis of AD. When it comes to brain-related studies, very limited amount of work has been done using machine learning. Previous studies are based on univariate classifications and in the biological system, the disease progression is multi-variate in nature. Neurodegenerative diseases are as a result of complex interplay of multiple variables and factors. A number of algorithms have been designed for early stage identification of AD with a variable range of accuracy from 75 to 96% [10, 11]. In a study, I has been discovered that the probability maps using features, like voxels of tissues of the whole brain and volumes of interest (VOI), are used to achieve 95.6% accuracy in identifying normal controls and AD [12]. Another study used a feature selection step after selecting the regions with significant difference between groups as VOIs and considering each voxel in the VOIs as a feature. [13] This phenomenon gave 96.32% accuracy between normal controls and AD [14, 15]. All the above discussed techniques are working to identify AD during early stages with certain amount of accuracy but what if a situation of uncertainty occurs? There is no answer to that question. Handling uncertainty is also a trivial task because

Fuzzy Logic-Based Disease Classification …

539

predicting something with uncertain data is not easy with general concepts. Here fuzzy logic can help us in better way. In this work, we have applied Fuzzy logic (FL)-based approach to Affymetrix data from Alzheimer’s data set (GSE1297) [16] and predict the genes that are responsible for the occurrence of the disease. These genes are differentially expressed and can serve as biomarkers for the early prediction of Alzheimer’s.

2 Materials and Methods 1. Data set Preparation The gene expression data was retrieved from the public repository Gene Expression Omnibus (GEO) from NCBI as given in Table 1, these microarray data sets comprise both normal and disease samples across Alzheimer’s. The acquired data set had done the analysis of the hippocampal expression for 9 control and 22 diseased patients having different levels of severity using microarrays. The data set also contains the tested values of the correlation of expression intensity with MiniMental Status Examination (MMSE) and neurofibrillary tangle (NFT) scores for each patient irrespective of the severity levels. The data set was useful to predict biomarkers of significant genes that are related with the disease condition. A number of these genes also correlated with biomarkers across only control and incipient AD subjects having MMSE > 20. Detailed knowledge of microarray data sets can be accessed through https://www.ncbi.nlm.nih.gov/geo/ query/acc.cgi?acc=GSE1297. 2. Data Pre-processing and DEGs Analysis First, we downloaded GEO data set in CEL file are pre-processed using Affy package in R Language which is in-cluded in background correction, normalization, and ex-pression calculation. Chip description files (CDF) is used for pre-processing that is obtained from GEO databases. (1) Each data set is preprocessed, resulting in a normal-ized text file containing gene expression values. (2) LIMMA package [17] is used for DEGs screened where throughout the cut-off criteria is P value < 0.01 and log2 (fold change) |>2 (3). 3. Fuzzy Logic Implementation The FL is a type of reasoning that resembles the functionality of human reasoning. Every logical block has at least 2 possible outcomes and that are True (1) or False (0). Fuzzy Logic is used to look after all possible nearby values between 0 and 1. Hence, Fuzzy Logic works on the level of input values to determine the possible outcome of the logical block. Referring Fig. 1, FL system consists of five steps: fuzzification, defining membership function, formation of rule base, obtaining fuzzy value, and defuzzification. Every step has been provided a unique functionality to perform. Fuzzification converts crisp input values into fuzzy numbers. Defining membership function will be required to identify the membership function of the variable, which would be used

540

A. Chaurasia et al.

Table 1 Results of differentially expressed genes Gene ID 217961_at

logFC 0.422381

AveExpr 5.981415

t 3.081746

P value

Adj.P.val

B

0.004357

0.990746

–3.99392

207321_s_at

–0.66894

4.534049

–3.01201

0.0052

0.990746

–4.01865

207276_at

–0.92005

2.60356

–3.00914

0.005237

0.990746

–4.01967

0.005538

0.990746

–4.0275

209607_x_at

0.583902

2.486426

2.987005

0.005781

0.990746

–4.03355

220176_at

–0.48028

3.494306

–2.92762

0.006427

0.990746

–4.04847

208383_s_at

–1.01841

2.578361

–2.91877

0.00657

0.990746

–4.05159

207063_at

0.696842

5.527442

2.969902

207289_at

–0.53372

3.22624

–2.86582

0.007494

0.990746

–4.07021

207499_x_at

–0.34562

3.072051

–2.77755

0.00931

0.990746

–4.10109

216467_s_at

–0.41469

2.620331

–2.77124

0.009455

0.990746

–4.10329

0.010139

0.990746

–4.11328

220232_at

204224_s_at

–1.0798

6.502395

–2.73742

0.010265

0.990746

–4.11504

215462_at

–0.63143

3.774276

–2.73454

0.010337

0.990746

–4.11605

213087_s_at 204564_at 212851_at 201657_at 210597_x_at 206078_at 205997_at

0.63865

0.415731 –0.29228 0.659307 0.437155 –0.53421 0.658012 –0.65303

2.993762

2.710239 2.77724 5.012711 2.88603 6.356729 5.345672 4.635851

2.742511

2.679047 –2.67669 2.671927 2.666461 –2.623 2.61038 –2.579

0.01182

0.990746

–4.13525

0.011887

0.990746

–4.13606

0.012024

0.990746

–4.13771

0.012182

0.990746

–4.13959

0.013516

0.990746

–4.15453

0.013927

0.990746

–4.15885

0.015002

0.990746

–4.16958

in next step. Formation of rule step will use the membership function and will be used to design and codify all if and then blocks. Obtaining fuzzy value will convert the crisp values into fuzzy values. Finally, in defuzzification, step the fuzzified value of last step is converted into the crisp value to obtain the final result. FLS can be obtained by using two methods: a. Mamdani Fuzzy Interface System b. Takagi–Sugeno Fuzzy Model.

Fig. 1 FLS System

Fuzzy Logic-Based Disease Classification …

541

Fig. 2 Designing output rule base on 2 variable input

Any FLS is designed to obtain the fuzzy value by using IF–THEN rules. In Mamdani Fuzzy interface system, we require following steps to get the fuzzy value: Step-1 Determine all input and output variables and decide its descriptors. Step-2 Now for all input and output variables, we need to determine the membership function. Let’s take a variable VAR. The XY graph plot for the variable Var is shown in Fig. 2. Step-3 After determining the member function of all the input and output variables, we need to create rule base amongst these variables. This requires a small quotient of common sense for deciding which value is to be putted at which place. Now the membership function for the variable Var is given by: µ(VARY 0 ) =

Max of X axis(X 1) − Min of Y axis(Y 0) 0 ≤ VAR ≤ X 1 (1) Max of X axis(X 1) − Min of Y axis(0)

Similarly, µ(VARY n ) =

Max of X axis(X n ) − Min of Y axis(Yn−1 ) X n−1 ≤ VAR ≤ X n Max of X axis(X n ) − Min of X axis(X n−1 ) (2)

Step-4 Define rules based on the rule base designed in last step. Rule n: IF value of VAR1 is “X n ” and VAR2 is “Y n ”… and value of VARn is “Valuen ” THEN output is Y n . Since the output of these rules are evaluated by using AND operator, this is because, we need to put the input values and use the minimum operator to find the strength of each and every rule. After calculating the strength, replace the values in rule base by identifying the cells satisfying the given inputs.

542

A. Chaurasia et al.

Step-5 This step belongs to defuzzification of the fuzzy values received in last step. In this step, we will use Mean of Max defuzzification technique to obtain the most suitable rule for the given input. After identifying the suitable rule, compare the rule base table with the fuzzy value table and identify the output range required as the solution. 4. Results Differentially expressed genes: Through our analysis, we retrieved top 100 genes that are differentially expressed across the different stages of the disease progression. Top 20 genes are represented in Table 1 to give an idea of the output parameters. Limma package in R is a useful way of identifying differentially expressed genes from a biological sample. Results include fold changes, standard error of deviation, t-test statistic, and p-values for each of the top 100 genes. The identified probes were further subjected to gprofiler to identify the gene names from standard annotations. The detailed description of the results is given in Table 1. Exploratory Data Analysis To further extrapolate the trends in the gene expression data, we plotted the UMAP plot, mean–variance trend and the expression density plot (Fig. 3) across all samples using GEO2R. UMAP constructs a high-dimensional graph representation of the data then optimizes a low-dimensional graph to be as structurally similar as possible. In our case, it gives us a representation of the number of samples of the groups-control, incipient, moderate, and severe. The second plot is the Mean–variance trend. The average log-expression values are plotted for all the 22,283 probes present in the sample. Further, we represent a graphical representation of expression density across samples. Here we observed that the peak density of expression is seen for the severe group of patients. Mamdani Model for Fuzzy Inference The samples are pre-processed with the help of Bioconductor and then normalized by using Limma software package with R. Analysis of differentially expressed genes is performed on the training data set with the help of Limma software package. In this part, Limma package is working with default parameters and options. The output obtained after the differential expression test will produce the result in the form of “Adjusted p value”. Depending upon adjusted p value, the gene raking is determined. The output of the previous step would be generated (Table 2) under columns logFC, AveExpr, t, and P. Value, adj. P. Val, and B. The logFC column gives value of the contrast. The AveExpr column denotes the average expression level for the given gene. The output of t-test is written under t column and the output of F-test is written under P.Value column. The adj.P.Val column is the adjusted P.Value to make if usable for other type of testing. Most commonly used form of adjusting the value is “Benjamin and Hochberg’s (BH) method”. The B column signifies the log odds of the gene that is differentially expressed. With these differentially expressed genes, the Mamdani model of Fuzzy logic system is further implemented to get the

Fuzzy Logic-Based Disease Classification …

543

Fig. 3 UMAP plot, mean–variance trend, and the expression density plot

gene selection and obtain the result. The flow chart of implementation is represented below (Fig. 4): Step-1: Identify the input and output variable of the problem. Answer Input: (i) B Value (ii) P Value Output: (i) Prediction of the severity of the disease Step-2: Defining descriptors of the input and output variables. Answer Input: Input

Descriptor 1

Descriptor 2

Descriptor 3

B Value (X)

Low

Medium

High

P Value (Y)

Low

Medium

High

Output:

6.502395

3.774276

–1.0798023

220232_at

2.620331

–0.4919693

215778_x_at

7.629006

4.635851

5.345672

0.6580116

–0.6530277

206078_at

205997_at

6.356729

–0.5342078

210597_x_at

5.012711

2.88603

0.6593069

0.4371554

212851_at

2.77724

–0.2922806

204564_at

201657_at

2.710239

–0.6314311

0.4157312

215462_at

213087_s_at

2.993762

–0.4146851

0.6386504

216467_s_at

3.072051

3.22624

204224_s_at

–0.3456222

207499_x_at

2.578361

–1.0184052

–0.5337207

208383_s_at

207289_at

3.494306

–0.4802808

220176_at

5.527442

2.486426

0.5839022

0.6968422

209607_x_at

207063_at

4.534049

2.60356

–0.6689367

–0.9200544

207321_s_at

207276_at

AveExpr

5.981415

logFC

0.4223806

217961_at

Table 2 Output of the previous step t

–2.5691

–2.579

2.61038

–2.623001

2.666461

2.671927

–2.676694

2.679047

–2.734537

–2.737421

2.742511

–2.771237

–2.777545

–2.865817

–2.918773

–2.927619

2.969902

2.987005

–3.009144

–3.012005

3.081746

P.Value

0.015357123

0.015002426

0.01392749

0.013515655

0.012182335

0.01202359

0.011886687

0.011819681

0.010337467

0.010265388

0.010139322

0.009454568

0.009310122

0.007493875

0.006570156

0.00642674

0.005781127

0.005537818

0.005237268

0.005199574

0.004356539

adj.P.Val

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

B

–4.172949

–4.169576

–4.158854

–4.15453

–4.139591

–4.137707

–4.136063

–4.135251

–4.116046

–4.115044

–4.113277

–4.103285

–4.101088

–4.070211

–4.051588

–4.048471

–4.033548

–4.027501

–4.019666

–4.018653

–3.993915

Severe

Very Severe

(continued)

544 A. Chaurasia et al.

2.663268

5.079951

0.6182555

201372_s_at

7.420007

0.4483151

213555_at

3.396292

2.545052

5.482427

0.6637346

0.6334105

221288_at

206235_at

4.76275

1.0645919

220148_at

6.699953

8.148956

0.4023906

0.3602585

203615_x_at

5.863736

0.6760077

216623_x_at

218604_at

6.977398

0.6514577

0.5777196

52837_at

203105_s_at

7.988815

0.427336

1.1070534

219117_s_at

4.56147

7.321572

218951_s_at

–0.5246566

203199_s_at

4.281978

–0.4994975

–0.5197097

221563_at

221045_s_at

3.456688

0.3907233

210015_s_at

2.561637

7.995951

0.7858349

0.6646405

200628_s_at

212646_at

3.041741

5.822673

–0.5024927

1.1465609

207752_x_at

8.208467

205119_s_at

AveExpr

logFC

0.4860018

213274_s_at

Table 2 (continued)

2.395507

2.417378

2.424555

2.428366

2.431802

2.436822

2.443285

2.444168

2.451483

2.459498

2.47438

2.476294

–2.478348

–2.483017

–2.493924

2.502105

2.508378

2.525633

2.534346

–2.543221

2.556204

t

0.022975673

0.021854772

0.021497982

0.021310705

0.021143118

0.020900469

0.020591777

0.020549927

0.020206248

0.019835719

0.019164253

0.019079423

0.018988773

0.018784143

0.018314065

0.017968596

0.017707756

0.017008145

0.016664632

0.016321316

0.015830834

P.Value

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

adj.P.Val

–4.231324

–4.224057

–4.221666

–4.220396

–4.219249

–4.217574

–4.215414

–4.215119

–4.212671

–4.209986

–4.204993

–4.20435

–4.203659

–4.202089

–4.198417

–4.195659

–4.193542

–4.187708

–4.184756

–4.181747

–4.177337

B

(continued)

Moderately Severe

Fuzzy Logic-Based Disease Classification … 545

5.324099

5.871668

0.4811515

221669_s_at

3.064967

0.4170362

203630_s_at

6.756044

3.439463

3.819719

0.4434471

0.449922

200988_s_at

204079_at

4.387036

0.5347763

218582_at

8.005497

2.885683

0.6090317

0.5499036

212390_at

4.316581

0.2944673

219248_at

210650_s_at

6.485952

–0.3370232

0.5329493

203547_at

218757_s_at

3.289651

–0.3242425

0.3475458

220946_s_at

5.879929

8.275803

208032_s_at

0.562832

203158_s_at

6.715865

0.7299405

0.4029161

209315_at

215299_x_at

4.037084

–0.2829207

213850_s_at

3.40023

5.004075

0.5977643

–0.3699452

4.430555

6.338672

–0.5336035

–0.5150592

211660_at

396_f_at

209902_at

4.215197

217137_x_at

AveExpr

logFC

0.5304194

209849_s_at

Table 2 (continued)

2.239286

2.239481

2.241512

2.241908

2.24478

2.249142

2.253794

2.265898

–2.285018

2.288715

2.292023

–2.29875

2.303776

2.315222

2.336375

–2.342138

–2.34221

2.34367

–2.350367

–2.368302

2.390046

t

0.032627926

0.03261395

0.032467988

0.032439627

0.032234366

0.031924934

0.031597935

0.03076108

0.029479733

0.029237638

0.029022532

0.028589438

0.028269739

0.027553637

0.026273337

0.025934026

0.025929833

0.025844502

0.0254564

0.024442959

0.023263612

P.Value

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

adj.P.Val

–4.28239

–4.282328

–4.281674

–4.281547

–4.280621

–4.279215

–4.277715

–4.273802

–4.267602

–4.266401

–4.265325

–4.263135

–4.261497

–4.25776

–4.250833

–4.248941

–4.248917

–4.248437

–4.246235

–4.240325

-4.233134

B

Moderate

(continued)

546 A. Chaurasia et al.

5.455256

2.520872

0.7472323

200744_s_at

3.756321

0.2738136

212269_s_at

7.005381

6.494526

3.283134

0.6923501

–0.351797

206584_at

218199_s_at

6.618337

0.6487152

202655_at

7.830758

6.306337

0.678049

–0.3711128

212427_at

209559_at

5.564396

–0.2715509

219611_s_at

5.057721

3.383172

–0.6890051

0.5569738

202234_s_at

3.867447

4.296202

204671_s_at

–0.5106594

219284_at

6.827648

–0.4102359

0.4546524

201908_at

221903_s_at

6.709592

0.822957

211980_at

4.274752

–0.4453621

–0.5400242

207242_s_at

205313_at

3.735189

0.529895

–0.3646095

203209_at

220206_at

7.212279

2.802128

–0.3410927

–0.4919794

216333_x_at

5.70939

205419_at

AveExpr

logFC

0.5064093

217997_at

Table 2 (continued)

2.186694

–2.193615

2.195624

2.197832

–2.199658

2.200335

–2.202039

2.204589

–2.205933

–2.207832

2.209782

–2.215077

2.215836

–2.217842

–2.218775

2.219082

–2.219398

2.230584

–2.230949

–2.231177

2.235148

t

0.036619529

0.03607043

0.035912437

0.035739422

0.035597031

0.03554434

0.035411996

0.035214781

0.03511128

0.034965451

0.034816261

0.034414115

0.03435678

0.034205734

0.034135705

0.034112724

0.034089047

0.033260202

0.033233446

0.033216772

0.032927256

P.Value

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

adj.P.Val

–4.299219

–4.297016

–4.296376

–4.295671

–4.295089

–4.294873

–4.294329

–4.293515

–4.293086

–4.292479

–4.291855

–4.290161

–4.289918

–4.289276

–4.288977

–4.288879

–4.288777

–4.285189

–4.285071

–4.284998

–4.283722

B

Mild

(continued)

Fuzzy Logic-Based Disease Classification … 547

6.325001

2.870909

0.7146007

219478_at

6.046063

7.107029

6.329366

-0.4064687

202947_s_at

6.425217

3.704763

–0.396836

202249_s_at

6.184395

4.606173

6.71136

–0.3465851

0.5298141

-0.5240361

-0.5477338

222333_at

215108_x_at

210880_s_at

204568_at

6.262353

0.4867244

0.4761451

211952_at

213988_s_at

5.39143

1.0025776

0.5528199

204338_s_at

202733_at

2.379001

–0.3466541

–0.2724904

219039_at

206637_at

8.558984

4.48539

0.4696645

0.6064629

202333_s_at

7.030705

219515_at

AveExpr

logFC

–0.4161842

213263_s_at

Table 2 (continued)

–2.150868

–2.155475

2.155747

–2.158245

–2.158426

2.159833

2.16905

–2.170148

2.172349

2.172819

2.177288

–2.178024

–2.179029

2.180038

2.181082

–2.181101

t

0.039583309

0.039190568

0.039167525

0.038956171

0.038940844

0.038822314

0.038053512

0.037962821

0.037781649

0.037743046

0.037377881

0.037318022

0.037236482

0.037154744

0.03707033

0.037068796

P.Value

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

0.9907459

adj.P.Val

–4.310568

–4.309114

–4.309028

–4.308239

–4.308182

–4.307737

–4.30482

–4.304473

–4.303775

–4.303626

–4.302208

–4.301974

–4.301655

–4.301335

–4.301003

–4.300997

B

548 A. Chaurasia et al.

Fuzzy Logic-Based Disease Classification …

549

Fig. 4 Mamdani model of Fuzzy logic system

Output

Descriptor 1

Descriptor 2

Descriptor 3

Descriptor 4

Descriptor 5

Prediction

Mild

Moderate

Moderately Severe

Severe

Very Severe

Step-3: Fuzzification of variables using triangular membership function

550

µLow (X ) = (−2.5 − X )/(−2.5 − 4.5) = (−2.5 − X )/(−7) µMedium (X ) = (X + 4.5)/(−2.5 − 4.5) = (X + 4.5)/(−7) µMedium (X ) = (0 − X )/(0 + 2.5) = (0 − X )/2.5 µHigh (X ) = (X + 2.5)/(0 + 2.5) = (X + 2.5)/2.5 µLow (Y ) = (0.5 − Y )/(0.5 − 0) = (0.5 − Y )/(0.5) µMedium (Y ) = (Y − 0)/(0.5 − 0) = (Y − 0)/(0.5) µMedium (Y ) = (1 − Y )/(1 − 0.5) = (1 − Y )/(0.5) µHigh (Y ) = (Y − 0.5)/(1 − 0.5) = (Y − 0.5)/(0.5) µMild (Z ) = (25 − Z )/(25 − 0) = (25 − Z )/(25) µModerate (Z ) = (Z − 0)/(25 − 0) = (Z − 0)/(25) µModerate (Z ) = (50 − Z )/(50 − 25) = (50 − Z )/(25) µModerate Severe (Z ) = (Z − 25)/(50 − 25) = (Z − 25)/(25) µModerate Severe (Z ) = (75 − Z )/(75 − 50) = (75 − Z )/(25) µSevere (Z ) = (Z − 50)/(75 − 50) = (Z − 50)/(25) µSevere (Z ) = (100 − Z )/(100 − 75) = (100 − Z )/(25) µVery Severe (Z ) = (Z − 75)/(100 − 75) = (Z − 75)/(25)

A. Chaurasia et al.

−4.5 ≤ X ≤ −2.5 −4.5 ≤ X ≤ −2.5 −2.5 ≤ X ≤ 0 −2.5 ≤ X ≤ 0 0 ≤ Y ≤ 0.5 0 ≤ Y ≤ 0.5 0.5 ≤ Y ≤ 1 0.5 ≤ Y ≤ 1 00 ≤ Z ≤ 25 00 ≤ Z ≤ 25 25 ≤ Z ≤ 50 25 ≤ Z ≤ 50 50 ≤ Z ≤ 75 50 ≤ Z ≤ 75 75 ≤ Z ≤ 100 75 ≤ Z ≤ 100

Step-4: Creating the rule base:

Conclusion To summarize, we have developed a novel process to identify the essentiality of the genes responsible of a particular disease. In our case, we calculated the probability of the genes found out to be differentially altered in neurological disorders such as Alzheimer’s disease responsible for the occurrence of the disorder. Results include (log2) fold changes, standard errors, t-statistics, and p-values. The basic statistic used

Fuzzy Logic-Based Disease Classification …

551

for significance analysis is p-values, and B-statistics will normally rank genes in the same order, i.e., from the most differentially altered at the top to the least altered at the bottom. The genes were selected on the basis of differential gene expression analysis using R and Bioconductor. These genes were ranked according to their p-values and then used for our predictive model using Fuzzy logic. Our results outperform the analysis using differential expression in most of the data sets used. Through this approach, we are able to identify a protocol for establishing the essential genes for a particular disease using principles of supervised learning. This may be beneficial in principle for application to other disease data sets from public repositories for finding out the essential genes and development of novel targets for disease management. Identification of most significant gene will lead to conclude that the values of this attribute will play a vital role in decision-making. Hence, the gene values are given to the prediction model, where attribute participation is designed in form of fuzzy functions. These fuzzy functions are designed considering five stages, i.e., Mild, Moderately, Moderately Severe, Severe, and Very Severe. Using fuzzy functions, prediction model will make its prediction that in which stage patient disease condition could be.

References 1. A. Valliani, A. Soni, in Deep Residual Nets for Improved Alzheimer’s Diagnosis. ACMBCB’17, August 20–23, 2017, p. 615, Boston, MA, USA 2. World Health Organization and Alzheimer’s Disease International. Dementia: a public health priority [Online] (2012). http://www.who.int/mentalhealth/publications/dementia report 2012/en/ 3. A. Association, Alzheimer’ s Association Report 2015 Alzheimer’ s disease facts and figures, Alzheimer’s. Dement 11, 332–384 (2015). https://doi.org/10.1016/j.jalz.2015.02.003 4. D. Zhang, Y. Wang, L. Zhou, H. Yuan, D. Shen, A.D.N. Initiative et al., Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55, 856–867 (2011). https://doi.org/10.1016/j.neuroimage.2011.01.008 5. G.A. Papakostas, A. Savio, M. Graña, V.G. Kaburlasos, A lattice computing approach to Alzheimer’s disease computer assisted diagnosis based on MRI data. Neurocomputing 150, 37–42 (2015). https://doi.org/10.1016/j.neucom.2014.02.076 6. A.H. Andersen, W.S. Rayens, Y. Liu, C.D. Smith, Partial least squares for discrimination in fMRI data. Magn. Reson. Imaging 30, 446–452 (2012). https://doi.org/10.1016/j.mri.2011. 11.001 7. Y. Fan, S.M. Resnick, X. Wu, C. Davatzikos, Structural and functional biomarkers of prodromal Alzheimer’s disease: a high-dimensional pattern classification study. Neuroimage 41, 277–285 (2008) 8. L. Mesrob, DTI and structural MRI classification in Alzheimer’s disease, Adv. Mol. Imaging 2, 12 20 (2012). https://doi.org/10.4236/ami.2012.22003 9. H. Hanyu, T. Sato, K. Hirao, H. Kanetaka, T. Iwamoto, K. Koizumi, The progression of cognitive deterioration and regional cerebral blood flow patterns in Alzheimer’s disease: a longitudinal SPECT study. J. Neurol. Sci. 290, 96–101 (2010). https://doi.org/10.1016/j.jns.2009.10.022 10. Y. Benjamini, Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Stat. Soc. B 57, 289–300 (1995)

552

A. Chaurasia et al.

11. R. Casanova, C.T. Whitlow, B. Wagner, J. Williamson, S.A. Shumaker, J.A. Maldjian et al., High dimensional classification of structural MRI Alzheimer’s disease data based on large scale regularization. Front. Neuroinformatics, 5:Article 22 (2011) 12. Y. Fan, N. Batmanghelich, C.M. Clark, C> Davatzikos, Spatial patterns of brain atrophy in MCI patients, identified via high-dimensional pattern classification, predict subsequent cognitive decline. NeuroImage 39(4), 1731–1743 Epub 2007/12/07 (2008). pmid:18053747 13. S. Kloppel, C.M. Stonnington, C. Chu, B. Draganski, R.I. Scahill, J.D. Rohrer et al., Automatic classification of MR scans in Alzheimer’s disease. Brain: J. Neurol. 131(Pt 3), 681–689 (2008). Epub 2008/01/19. 14. I. Beheshti, H. Demirel, Feature-ranking-based Alzheimer’s disease classification from structural MRI. Mag. Reson. Imaging. 34(3), 252–263 (2016). Epub 2015/12/15. pmid:26657976 15. M. Chupin, E. Gerardin, R. Cuingnet, C. Boutet, L. Lemieux, S. Lehericy et al., Fully automatic hippocampus segmentation and classification in Alzheimer’s disease and mild cognitive impairment applied on data from ADNI. Hippocampus 19(6), 579–587 (2009). Epub 2009/05/14. pmid:19437497 16. E.M. Blalock, J.W. Geddes, K.C. Chen, N.M. Porter et al., Incipient Alzheimer’s disease: microarray correlation analyses reveal major transcriptional and tumor suppressor responses. Proc. Natl. Acad. Sci. U S A 101(7), 2173–2178 (2004) 17. G.K. Smyth, in Limma: Linear Models for Microarray Data. Bioinformatics and Computational Biology Solutions Using R and Bioconductor, pp. 397–420 (Springer, New York, NY, 2005)

WhyMyFace: A Novel Approach to Recognize Facial Expressions Using CNN and Data Augmentations Md Abu Rumman Refat, Soumen Sarker, Chetna Kaushal, Amandeep Kaur, and Md Khairul Islam

Abstract Aptitude, in terms of human facial recognition, cases prior one of digital image’s fundamental parts. This conveys facial parameters in many social contexts. Medical imaging, robotics, intrusion detection system with sentiment analysis, and automation and some industries use computer vision to understand human facial expressions. Studying human facial expressions using deep learning has become popular in recent years, and several efforts have been made. However, facial expression recognition remains challenging because of the wide range of persons with similar facial expressions. This paper proposed a 16-layer efficient CNN technique to categorize human facial expressions with data augmentation. Then, we evaluated our proposed approach on a well-known facial expression recognition, the FER2013 benchmark dataset. And, the proposed technique achieves state-of-the-art testing accuracy of 89.89% exceeding some prior research.

1 Introduction What it is in your Mind is on your Face!. Facial expressions are one of the vital research field in computer vision. There are three methods to exhibit in inter-personal communication, including 55 % facial expression and gesture, 38 % vocal, and 7% spoken. Whatever only 7% of global conversations are delivered in roughly 7000 languages, and rest 93 % using another mode of communication [1]; however, facial expressions have become an easy way to incorporate in the communicating world. The capacity to effectively depict emotion using facial expressions is known as M. A. R. Refat · S. Sarker · M. K. Islam (B) Information and Communication Technology, Islamic University, Kushtia 7003, Bangladesh e-mail: [email protected] C. Kaushal · A. Kaur Chitkara University Institute of Engineering and Technology, Rajpura, Punjab, India e-mail: [email protected] A. Kaur e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_48

553

554

M. A. R. Refat et al.

facial emotion recognition. It reveals information about a person’s mental condition by delving into their sensations on the inside. However, more research has been conducted on human facial expression recognition for autonomous in recent years and human–computer interaction with a broad range of applications in automation like social robot interaction [2], advertisement [3], medical application like medical diagnosis at early stage [4, 5], sentiment analysis [6, 7], psychology [8], neuroscience and medicine [9], augmented reality [10], and security monitoring [11]. Convolutional neural networks (CNNs) are a common deep learning technique for image identification, segmentation, and classification established by Bengio et al. [12]. In some previous similar work, CNN has enabled tremendous performance for improving facial expression recognition using features [13, 14]. They are preprocessing, appropriating optimization, learning, and preparing training and test datasets which all impact the performance of CNN models, which are the primary differences in this study. However, in order to increase human facial expressions recognition performance, it is necessary to be able to discover current bottlenecks in CNN-based automatic facial expression recognizer. In our previous work [15], we have freeze earlier neurons so that the later one can fit its weights with the nature of the problem. In our proposed novel, 16-layer CNN architecture recognizes the seven human facial expressions with greater validation accuracy than previous work. It is capable of recognize Anger (as well as fear and sadness), Surprise (as well as happiness), and Disgust (as well as neutral). What’s left of this paper is broken down into the four subsections as following. We present a quick overview of the prior related studies in Sect. 2. We sift through the dataset in Sect. 3. We highlight the critical technique of our proposed approach in Sect. 4. Finally, we conclude our study and discuss future research direction in Sect. 6. The experiment and results of our proposed system are summarized in Sect. 5.

2 Related Works While understanding human facial expressions which is a critical field of study, getting the information to computers is difficult. The facial expression, [16], speech, [11], gesture, and event stance may convey a lot about this work. Following AlexNet’s success [17] in early 2013, deep learning has become more popular for accurately recognizing human facial expressions in the previous decade. In this part, we outline some of the scientific work in facial expression relying on human facial expression analysis. Yadav et al. [18]suggested a convolutional neural network for facial expression recognition based on the architecture of AlexNet, [17] which was tested on the FER 2013 dataset. According to their findings, the suggested model attained 57% accuracy on this dataset. Liu et at. [16] proposed an ensemble convolutional neural network technique for face emotion assessment, with an average accuracy of 62.44% on the same dataset. A deep neural network-based CNN for human facial expression recognition was also developed by Babajee et al. [14], with an accuracy of 79.8% on the FER2013

WhyMyFace: A Novel Approach to Recognize Facial Expressions …

555

dataset, where 70% of the images were utilized for training. At the same time, the remainder were used for validation and testing. With the FER2013 and CK+ datasets, Change et al. [19] established a CNN model based on the extraction features, then implemented three distinct perceptual classification techniques (random forest, linear SVM, and softmax) on top of the model. Finally, using the FER 2013 and CK+ datasets, CNN with softmax achieved accuracies of 71.35%and 98.78%, respectively. Nguyen et al. [20] created an 18-layer deep CNN model, which is very close to the VGG model on the FER 2013 dataset, the suggested multilevel CNN model obtained 73.03% accuracy by using mid-level features. There is an 80.29% accuracy rate in the CNN model developed by Cao et al. [13]. The initial value of the convolutional kernel of the CNN model is determined by K-means clustering, and a feature from the trained CNN model is employed by SVM to identify facial expression. According to the findings of a study by Christou et al. [3], a 13-layer CNN model on FER 2013 was successfully deployed, with a validation accuracy of 91.12% on facial expressions classification 7. A CNN model, inspired by Gudi et al. [21], cites Moran et al. [22], which utilized 14524 photos for training and 9000 remaining pictures from the FER 2015 dataset to validate the model, and the network has obtained 66% accuracy on the FER 2013 dataset. Machine learning and deep learning algorithms has been extensively using in various task such as spam filtering [23], water quality indexing [24], disease early detection, automated object [15], smart health care [25], etc.

Fig. 1 Examples of images from the FER2013 dataset [26] in which images in the same column represent identical facial expressions, namely Anger, Disgust, Fear, Happiness, Sadness, Surprise, and Neutral

556

M. A. R. Refat et al.

3 Dataset Analysis To test our model’s performance, we used the large open-source dataset for human facial expression recognition released on the Kaggle ICML 2013 competition [26], the FER2013 dataset in Fig 1. This dataset contains approximately 35887 images with 28709 images used during training and 3589 images used during public tests (validation) and 3589 images used during private tests (testing) separately. Anger, Disgust, Fear, Happiness, Sadness, Surprise, and Neutral are among the basic seven facial expression categories in the gray-scale dataset, which measures 48 × 48 pixels. On this dataset, the human reliability is roughly 65.5% [26].

4 Methodology of Proposed Approach Figure 2 depicts the methodological stage (interpretation and research conducted on related facts to make research goal in the first place) of the human facial expressions recognition system. It involves the following sub-steps: Data augmentation, CNN architecture, and training are all examples of preprocessing. The stages overview are as follows.

4.1 Data Preprocessing Normalization (divide every gray pixel’s value by its most excellent pick value 255) and scaling (scaled to 48 × 48 pixels) had been performed on a dataset (FER 2013) of the gray images that allows low-resolution normalized pictures suitable for faster training without affecting recognition accuracy.

Fig. 2 Following steps of proposed methodology

WhyMyFace: A Novel Approach to Recognize Facial Expressions …

557

4.2 Data Augmentation Naturally, CNN prefers enough data samples to train a model for better output with data with variances from several sources (scrapped from Web sites, mobile app data, and cloud data) that may appear in the model during the training or in the inference phase. Rotation, scaling, shifting, and flipping are typical data augmentations techniques [27]. Here we generate image data that may not or may be on the fly by doing operations like shifting (10%) and zooming (10%) both horizontally and vertically while angles are altered in −10 to 10◦ at random.

4.3 Convolutional Neural Network Choosing a specific architecture depends on previous experience, reading projectrelated articles, and automated tests to select the best model architecture of convolutional blocks to achieve optimum performance, as this is an empirical process [28]. As there is no standard rule in our case, we proposed an approach with eight convolutional layers, four pooling levels, and a fully connected layer, and the output of one layer is the input of the next layer. In the Table 1 below, the model’s graph network has been depicted.

4.3.1

Input Layer

The preprocessed images are supplied directly into the network using the network’s input layer (2304 nodes, each node for each pixel of resolution 48 × 48). 4.3.2

Convolution Layer

Convolution kernel/filter (choosing or engineering this kernel size is also an empirical process) is a significant building block of the CNN model. Convolution performs between the input data matrix(batches of image’s tensor) passes through the convolution layer’s kernel of size (3×3) via the forward and backward propagation stage of training. The output of this operation is a feature map and is applied to the consecutive layer as an input. We got maximum accuracy in our proposed model because different kernels experimented with varied sizes at 32–128, and a combination was applied. 4.3.3

Activation Function

Activation functions (functions varied based on the gradient of distributions) threshold out some features and only allowed significant elements to be triggered on the feature map. Thus computational time is reduced. All negative values have been

558

M. A. R. Refat et al.

set to 0 using the ReLU [29] activation function, while the positive values are left unchanged. Gradient vanishing is less of a concern with ReLU training than with other methods (softmax, tanh, and Sigmoid) in the convolutional layer since ReLUs seem to be much faster [30] formulated as follows: ReLU(x) = max(0, x)

(1)

where x is the neuron’s input. 4.3.4

Pooling Layer

In the CNN forward and backward propagation, pooling layer with no trainable parameters decreases the links between the convolutional layers, speeds up CNN training, and minimizes the network’s memory space with a sliding window (size 2×2) operation over an input space with pooling of an enormous value inside that window. Stride size of two has been adjusted to alleviate the overlapping issue with the input image’s shape, and following resultant dimension has been obtained after the max pooling operation:

Table 1 Proposed CNN’s architecture Model content First convolution layer Second convolution layer First max pooling layer Third convolution layer Fourth convolution layer Second max pooling layer Fifth convolution layer Sixth convolution layer Third max pooling layer Seventh convolution layer Eighth convolution layer Fourth max cooling layer First fully connected layer Dropout layer Output layer Optimization function Learning rate Callback

Model details 32 filters of size 3 × 3, ReLU activation, and input size 48 × 48 32 filters of size 3 × 3 and ReLU activation Pooling size 2 × 2 64 filters of size 3 × 3 and ReLU activation 64 filters of size 3 × 3 and ReLU activation Pooling size 2 × 2 96 filters of size 3 × 3 and ReLU activation 96 filters of size 3 × 3 and ReLU activation Pooling size 2 × 2 128 filters of size 3 × 3 and ReLU activation 128 filters of size 3 × 3 and ReLU activation Pooling size 2 × 2 64 nodes and Swish activation Excludes 50% neurons randomly 7 nodes for 7 classes and Sigmoid Adam gradient descent 0.0001 Early stopping, ReduceLROnPlateau, and model checkpoint

WhyMyFace: A Novel Approach to Recognize Facial Expressions …

( Mout = floor

Min − F S

559

) +1

(2)

where Min , F, and S are the input image, kernel, and stride sizes, respectively.

4.3.5

Fully Connected Layer

The fully connected layer (FC) layer receives the features map produced by the last max pooling layer. The neurons in one layer are linked to the neurons in another layer in an FC layer. The FC layer works similarly to a convolution layer with a size of 1×1 filter.

4.3.6

Dropout

To avoid over-fitting issues caused by the model’s training accuracy is much higher than validation accuracy or model is low bias but a large variance in contrast to the model’s validation data. A regularization strategy, known as dropout(0.5), is incorporated to set input to zero with a certain probability randomly. During the backpropagation stage, these nodes were discarded and learned parameters for other’s node, as a result, prevents over-fitting and enhances efficiency worth generalization occurs. [31]

4.3.7

Output Layer

Output layers equal the number of neurons in the output layer to match the number of classifications. A Sigmoid function [32]returns the percentages of each class in a multi-class classification problem, with the target class having the maximum probability. The Sigmoid function is mathematically expressed as follows: F(xi ) =

1 (1 + e−xi )

(3)

where xi denotes the inputs to each softmax layer node from the preceding FC layer and K denotes the number of classes.

4.4 Training Details Keras [33] and TensorFlow [34] were used to implement our proposed model and relevant evaluation. We initiate training with a gradient descent-based Adam optimization algorithm having a learning rate of 0.001 to reduce the loss of a cost-entropy

560

M. A. R. Refat et al.

cost function [35]. The learning rate was modified by half of its value and updated every 15 epoch if our validation accuracy rate was unchanged or not improved. We also permitted early stopping to monitor the validation accuracy at every epoch, and training was stopped after 20 periods due to not increasing validation accuracy. However, our proposed model was training up to 100 epochs with a batch size of 64 and 448 steps per epoch. Our proposed model was executed on a PC NVIDIA Tesla K80 GPU and a 2.3 GHz Xeon hyper-threaded Processor, 1TB HDD, 3.98 GHz CPU, and 8 GB RAM with windows 10. Our suggested model’s interface time averaged 58 s per epoch. After the training phase, we assessed our strategy on the remaining 10% of the test dataset.

5 Result and Discussion The training and validation accuracy of our proposed 16-layer CNNs model on the FER2013 dataset are shown in Fig. 3. The Navy and Maroon lines reflect the training and validation accuracy. Our proposed model has been validated with 91.17 % validation accuracy utilizing suitable data augmentation technique that produces additional validation in the training samples to minimize over-fitting. As shown in Fig. 4, our proposed CNN model accurately recognizes each human face emotion. As a result, the performance of the recognition system might vary depending on the dataset used for both training and testing, making it difficult and illogical to properly compare them. So we chose the FER2013 dataset to compare our model’s performance against other well-known similar models [13, 14, 18] with similar architectures. Performance comparison [3, 13, 14, 16, 18–20] depicted in Table 2. The state-of-the-art accuracy(≥9%) of our proposed CNN strategy compares with

Fig. 3 Performance curve of the proposed model for recognition of facial expression

WhyMyFace: A Novel Approach to Recognize Facial Expressions …

561

Fig. 4 Confusion matrix of the classification performance on testing data with seven fundamental facial expressions, namely Anger, Disgust, Fear, Happy, Sadness, Surprise, and Neutral Table 2 Performance comparison between proposed system and existing related works Model type Accuracy [%] Model Liu et al. (2016) [16] Chang et al. (2018) [19] Nguyen et al. (2019) [20] Cao et al. (2019) [13] Yadav et al. (2020)‘[18] Babajee et al. (2020) [14] Proposed model

Ensemble network ResNet + CPC Multilevel CNN CNN + SVM Shallow CNN Deep CNN 16-layer CNN

62.44 71.35 73.03 80.29 57.0 79.8 89.89

various earlier state-of-the-art CNN approaches [13, 14, 16, 18–20] on the same dataset.

6 Conclusion In real-world application, facial expressions recognition use ranging from medical to augmented reality to security surveillance. In this paper, we implemented a 16layer convolutional neural network model for automatically recognizing human facial expressions, and we evaluated its performance using the FER2013 dataset. On the same dataset, our proposed CNN’s model achieved substantial recognition accuracy, which we compared to the performance of some current relevant work. Finally, we have proven state-of-the-art accuracy of 89.89% without using any extra datasets. In the future, we want to enhance our model’s accuracy while also evaluating it with different datasets and exploring for large datasets for tests.

562

M. A. R. Refat et al.

References 1. A. Mehrabian, Communication without words, in Communication Theory. (Routledge, UK, London, 2017), pp. 193–200 2. M. Shidujaman, H. Mi, Which country are you from? A cross-cultural study on greeting interaction design for social robots, in International Conference on Cross-Cultural Design. (Springer, Cham, 2018) 3. N. Christou, N. Kanojiya, Human facial expression recognition with convolution neural networks. in Third International Congress on Information and Communication Technology. (Springer, Singapore, 2019) 4. M. A. R. Refat, M. A. Amin, C. Kaushal, M. N. Yeasmin, M. K. Islam, A comparative analysis of early stage diabetes prediction using machine learning and deep learning approach, in 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC) (2021), pp. 654–659 5. M.M. Hassan, M.A. Mamun Billah, M.M. Rahman, S. Zaman, M.M. Hasan Shakil, J.H. Angon, Early predictive analytics in healthcare for diabetes prediction using machine learning approach, in 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT) (2021), pp. 1–5 6. C. Kaushal, M. A. R. Refat, M. A. Amin, M. K. Islam, Comparative micro blogging news analysis on the covid-19 pandemic scenario, ed. by M. Saraswat, S. Roy, C. Chowdhury, A. H. Gandomi, Proceedings of International Conference on Data Science and Applications. Lecture Notes in Networks and Systems, vol. 287 (Springer, Singapore, 2022) 7. N. I. Mahbub, M. R. Islam, M. A. Amin, M. K. Islam, B.C. Singh, M.I. H. Showrov, A. Sarkar, Sentiment analysis of microblogging dataset on coronavirus pandemic (2021 ). arXiv preprint arXiv:2111.09275 8. T. Zhang, Facial expression recognition based on deep learning: a survey. in International conference on intelligent and interactive systems and applications. (Springer, Cham, 2017) 9. J.B. Fugate, A.J. O’Hare, W.S. Emmanuel, Emotion words: facing change. J. Experimental Soc. Psychol. 79, 264–274 (2018) 10. N.T. Cao, A.H. Ton-That, H.I. Choi, An effective facial expression recognition approach for intelligent game systems. Int. J. Comput. Vis. Robot. 6, 223–234 (2016) 11. C. Clavel, I. Vasilescu, L. Devillers, G. Richard, T. Ehrette, Fear-type emotion recognition for future audio-based surveillance systems. Speech Commun. 50, 487–503 (2008) 12. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition (1998) 13. T. Cao, M. Li, Facial expression recognition algorithm based on the combination of cnn and k-means, in ICMLC ’19 (2019) 14. P. Babajee, G. Suddul, S. Armoogum, R. Foogooa, Identifying human emotions from facial expressions with deep learning, in 2020 Zooming Innovation in Consumer Technologies Conference (ZINC) (2020), pp. 36–39 15. M. K. Islam, M. N. Yeasmin, C. Kaushal, M. A. Amin, M. R. Islam, M. I. Hossain Showrov, Comparative analysis of steering angle prediction for automated object using deep neural network, in 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) (2021), pp. 1–7 16. K. Liu, M. Zhang, Z. Pan, Facial expression recognition with cnn ensemble, 2016 International Conference on Cyberworlds (CW) (2016), pp. 163–166 17. A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, ed. by F. Pereira, C. J. C. Burges, L. Bottou, K. Q. Weinberger. Advances in Neural Information Processing Systems, vol. 25 (Curran Associates, Inc., 2012), pp. 1097–1105 18. Y. Yadav, V. Kumar, V. Ranga, R.M. Rawat, Analysis of facial sentiments: a deep-learning way, in 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC) (2020), pp. 541–545 19. T.-Y. Chang, G. Wen, Y. Hu, J. Ma, Facial expression recognition based on complexity perception classification algorithm (2018). ArXiv abs/1803.00185

WhyMyFace: A Novel Approach to Recognize Facial Expressions …

563

20. H.-D. Nguyen, S. Yeom, G. Lee, H.-J. Yang, I.-S. Na, S. Kim, Facial emotion recognition using an ensemble of multi-level convolutional neural networks. Int. J. Pattern Recognit. Artif. Intell. 33, 1 940 015:1–1 940 015:17 (2019) 21. A. Gudi, H. Tasli, T. M. D. Uyl, A. Maroulis, Deep learning based facs action unit occurrence and intensity estimation, in 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 6 (2015), pp. 1–5 22. J. T. Moran, Classifying emotion using convolutional neural networks. UC Merced Undergraduate Res. J. 11 (2019) 23. M.K. Islam, M.A. Amin, M.R. Islam, M.N.I. Mahbub, M.I.H. Showrov, C. Kaushal, Spamdetection with comparative analysis and spamming words extractions, in 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) (2021), pp. 1–9 24. M.M. Hassan, M.M. Hassan, L. Akter, M.M. Rahman, S. Zaman, K.M. Hasib, N. Jahan, R.N. Smrity, J. Farhana, M. Raihan et al., Efficient Prediction of Water Quality index (WQI) Using Machine Learning Algorithms (Hum. Centric Intell. Syst., 2021). https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=4135697 25. M. K. Islam, C. Kaushal, M. AL AMIN, Smart home-healthcare skin lesions classification with IoT based data collection for https://www.techrxiv.org/articles/preprint/Smart_Homedevice (2021). Healthcare_for_Skin_Lesions_Classification_with_IoT_Based_Data_Collection_Device_/ 16870729 26. I.J. Goodfellow, D. Erhan, P.L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee, Y. Zhou, C. Ramaiah, F. Feng, R. Li, X. Wang, D. Athanasakis, J. Shawe-Taylor, M. Milakov, J. Park, R.T. Ionescu, M. Popescu, C. Grozea, J. Bergstra, J. Xie, L. Romaszko, B. Xu, C. Zhang, Y. Bengio, Challenges in representation learning: A report on three machine learning contests. Neural Netw. Official J. Int. Neural Netw. Soci. 64, 59–63 (2015) 27. X. Cui, V. Goel, B. Kingsbury, Data augmentation for deep neural network acoustic modeling. IEEE/ACM Trans Audio Speech Lang Process 23, 1469–1477 (2015) 28. C. Kaushal, A. Singla, Analysis of breast cancer for histological dataset based on different feature extraction and classification algorithms, in International Conference on Innovative Computing and Communications (Springer, Berlin, 2021), pp. 821–833 29. A.F. Agarap, Deep learning using rectified linear units (relu), in ArXiv abs/1803.08375, 2018 30. A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in CACM (2017) 31. S. Park, N. Kwak, Analysis on the dropout effect in convolutional neural networks,’ in ACCV (2016 ) 32. C. Nwankpa, W. Ijomah, A. Gachagan, S. Marshall, Activation functions: comparison of trends in practice and research for deep learning. ArXiv abs/1811.03378 (2018) 33. F. Chollet, Keras: the python deep learning library. in Astrophysics Source Code Library, ascl-1806 (2018) 34. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zhang, Tensorflow: a system for large-scale machine learning, in OSDI (2016) 35. S. Mannor, D. Peleg, R. Rubinstein, The cross entropy method for classification,’ in ICML’05 (2005)

Location Accuracy and Prediction in VANETs Using Kalman Filter Ritesh Yaduwanshi and Sushil Kumar

Abstract Many vehicular network applications such network administration, routing, and data transmission protocols require location information. If a precise prediction of the vehicle’s future move can be made, resources can be allocated optimally while vehicle travels. Will result in improving VANETs performance. For that purpose, Kalman filter is proposed for correcting and predicting vehicle’s position. The research used both real vehicle movement traces and model-driven traces. Kalman filter and neural network-based techniques are quantitatively compared. Across all scenarios proposed, model exhibits superiority than other correction and prediction schemes. Keywords VANET · Location prediction · Location accuracy · Kalman filter

1 Introduction VANETs facilitates vehicles and roadside equipment to interact with each other. Urgent messages are sent among vehicles via vehicle-to-vehicle communication to help intelligent transportation networks [1]. Most protocols, algorithms, and applications in these networks use the premise that they know the real-time position of nodes. Some VANET applications do not need localization to operate, when vehicle position information is available, they can still profit from localization and show better performance. Various types of precise localization data adapt to different applications [2, 3]. The worse the performance is, the greater the localization inaccuracy. Localization errors of 1–5 m can be tolerated by applications such as cooperative adaptive cruise control and cooperative intersection safety Table 1 lists the localization requirements for VANETs applications. R. Yaduwanshi (B) · S. Kumar School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, India e-mail: [email protected] S. Kumar e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_49

565

566

R. Yaduwanshi and S. Kumar

Table 1 Degree of localization accuracy for various VANETs applications Technique

Localization accuracy High

Automatic parking

✓

Vehicle collision warning system

✓

Vision enhancement

✓

Medium

Blind crossing

✓

Platooning

✓

Cooperative adaptive cruise control

✓

Cooperative intersection safety

✓

Low

Data dissemination

✓

Map localization

✓

Routing

✓

For location accuracy and prediction, learning automation implements linear reward penalty reinforcement technique [4]. However, the accuracy of such a model is insufficient for correcting and predicting location. When knowledge about mobility trends is unavailable, evidential reasoning [5] is utilized in location correction and prediction. Movement rules [6] based on the node’s previous movement patterns were also tested, albeit with low predictability. The authors of Ref. [7] present a gray theory-based prediction model. Nodes and their neighbors collaborate to update an auto-regression-based mobility model [8] in order to better correctly anticipate their own future position. In cellular mobile networks, Markov mobility models [9–11] leverage the movement history. Adaptive resonance theory [12] tests GPS traces using state-full prediction model. The shortcoming of this technique is that it requires a lot of storage space. Furthermore, because the model in Ref. [12] has slow response to change, it is unable to adjust quickly to previously unforeseen movement behavior. After that, short-memory adaptive location predictor is presented that can predict and correct vehicles mobility in the absence of large previous mobility data. Their location prediction and correction based on local linear regression model, with a fuzzy controller providing adaptive capability [13]. For MANET and cellular mobile networks, the aforementioned prediction models were applied. Despite the fact that VANETs are a subset of MANETs, there are substantial differences between the two. Because vehicles move quickly, the network topology in VANETs changes more quickly. Because of the fast fluctuation, pinpointing the exact location of the vehicle may be challenging. In this paper, real traffic traces are used generated from Vehicular Ad hoc Networks (VANETs) collected under various network environments to investigate the capabilities of the Kalman filter. A comparison of the Kalman filter and artificial neural networks (ANN) for one-step ahead prediction is also discussed. Our key contribution is to demonstrate that a Kalman filter can be used to successfully anticipate the future location information of a moving vehicle using GPS

Location Accuracy and Prediction in VANETs Using Kalman Filter

567

Table 2 Nomenclatures At

State transition matrix at time t

Ht

Output matrix relating the node’s state to the measurement Zt

Kt

Kalman gain

M

Total number of the predicted values

Pt|t-1

Covariance matrix of the apriori error

Pt|t

Covariance matrix of the aposteriori error

Q

Covariance matrix of the model noise

R

Covariance matrix of the measurement noise

T

Time variable

T

Transpose operator

Zt

Measurement vector at time t

Ut

The measurement noise is Gaussian with a mean of zero and a covariance matrix R

in a VANET. Many applications, such as data dissemination protocols, road congestion, routing protocols, and map localization, can benefit from the proposed location correction and prediction. The following is a breakdown of the paper’s structure: The Kalman filter model is described in Sect. 2. The proposed mobility prediction and correction is described in Sect. 3. The dataset, performance indicators, and effectiveness of the suggested mobility prediction and correction are all shown in Sect. 4. The symbols and notations used in the rest of this paper are given in Table 2.

2 Kalman Filter It is basically a fast recursive filter that uses a sequence of noisy observations to estimate the state of a linear dynamic system [14]. The filter is a hybrid of state space models and mathematical equations that creates an effective predictor–corrector estimator that minimizes estimated error covariance. It can be applied to both stationary and nonstationary processes (Fig. 1). The state and measurement vectors are vital in a Kalman filter. The state vector X t is the smallest set of data that can be used to explain the system’s dynamic behavior. To put it another way, the state is the smallest quantity of information about the system’s previous behavior that is required to forecast its future behavior. A measurement at time t represents measurement vector. The process equation used to forecast the system’s condition at a given point in time is defined as follows: X t+1 = At X t + wt

(1)

where W t represents the process noise that is Gaussian with zero mean and a covariance matrix Q. The measurement equation is:

568

R. Yaduwanshi and S. Kumar

Fig. 1 Kalman filter

Z t = Ht X t + u t

(2)

The Kalman filter is a recursive estimation method that uses measurement results Z t up to time t to estimate an unknown state X t . We have an estimate of state X t projected at time t 1 at time t. The priori estimate of X t is called X t|t1 , and the posteriori estimate of X t is called X t|t . The posteriori estimate is written as [15].   Xˆ t|t = Xˆ t|t−1 + K t Z t − H Xˆ t|t−1

(3)

  K t = Pt|t−1 H Tt Ht Pt|t−1 H Tt + R − 1

(4)

The posterior error covariance matrix is Pt|t = (I − K t Ht )Pt|t−1

(5)

The one-step-ahead estimate is Xˆ t+1|t = At Xˆ t|t

(6)

The one-step-ahead error covariance matrix is Pt+1|t = At Pt|t ATt + Q

(7)

Recursive procedures for creating one-step location correction and prediction are updated from time to t + 1 based on Eqs. (3)–(7).

Location Accuracy and Prediction in VANETs Using Kalman Filter

569

2.1 Filter Parameters Selection There are two popular methods for generating Q and R in Kalman filters required to analyze GPS location data are. • Empirical investigation of the system and measurement errors are two traditional ways for getting the values for these two matrices [16]: Q and R must be symmetric and positive semi-definite because they are covariance matrices. Measuring Q is more difficult than measuring R. Making Q large enough to match the effects of the measurement noise R is a rough concept. Generally tweaking the filter parameters Q and R can often result in better filter performance. Offline tuning is the most common method. • A vast number of researchers are working on developing better approaches. Mehra [17] initially estimated Q and R using the auto-correlation functions of the innovation sequence. To obtain R and a scale factor for Q [18], or to estimate Q and R indirectly [19], adaptive techniques are applied. The expectation–maximization (EM) technique and direct gradient-based optimization approaches are used to create the filter parameters R and Q [16]. Empirical analysis is used to determine the values for these two matrices [19].

2.2 The Location Correction and Prediction Algorithm In order to correct and predict vehicles future location, Kalman filter is used. Speed and location are contained in state vector of vehicles. As a result, let X t = (x t , vxt , yt , vyt ) be a 4 × 1 state vector, with yt and x t representing the vehicle’s y and x coordinates, and vyt and vxt representing the vehicle’s speed along y-axis and x-axis. As the state vector do not change much inside ∆t, vyt and vxt represent average speed. As a result, the next time step’s new coordinates are xt = xt−∆t + vxt ∆t, yt = yt−∆t + v yt ∆t

(8)

Then, we obtain the 4 × 4 transitional matrix At : ⎡

1 ⎢0 At = ⎢ ⎣0 0

∆t 1 0 0

0 0 1 0

⎤ 0 0 ⎥ ⎥ ∆t ⎦ 1

(9)

Here ∆t is the sampling interval that represents time interval we assumed ∆t = 1. Based on the measurement values Z t , we change the estimation of the unknown state X t in measurement update stage. Z t (t = 1, 2,…) is the 2 × 1 observation vector that reflects the historical position data sequence (x t , yt ). Therefore, the matrix H t is

570

R. Yaduwanshi and S. Kumar

defined as Ht =

1000 0010

(10)

There are no prior measurements in the estimate problem. The starting state is frequently set to the unconditional expectation of X, resulting in Xˆ 0|1 = 0. For P0|-1 matrices, the initial values are determined as follows: ⎡

P0|−1

10000 ⎢0 =⎢ ⎣0 0

0 10000 0 0

0 0 10000 0

⎤ 0 ⎥ 0 ⎥ ⎦ 0 10000

(11)

The initial values for the other matrices are determined as follows ⎡

0.001 ⎢0 Q=⎢ ⎣0 0

0 0 0.001 0 0 0.001 0 0

10 R= 01

⎤ 0 ⎥ 0 ⎥ ⎦ 0 0.001

(12)

(13)

Location accuracy and prediction algorithm [15] are given: (1) Initial conditions, i.e., Xˆ 0|−1 , Pˆ0|−1 , Aˆ 0 , Hˆ 0 , Q, R have to be known to start recursive steps. (2) Recursive procedures for creating a one-step location accuracy and prediction from time t to time t + 1 are updated based on Eqs. (3)–(7).

3 Performance Evaluation We built a location correction and prediction algorithm in MATLAB. The performance is evaluated using one model-driven trace and three real vehicle mobility traces. The real movement traces of mobile users were gathered from Delhi, Mumbai, and Bangalore. The First Batch of Traffic: The information includes an automobile time, velocity, identification, position, and so forth. The GPS position of a moving automobile was sampled each and every second. Universal Transverse Mercator (UTM) coordinate (32nd) format was used to save GPS coordinates. This trace contains fine-grained location date [20].

Location Accuracy and Prediction in VANETs Using Kalman Filter

Description

571

Equation

Kalman Gain Update Estimate Update Covariance One Step Ahead Error Covariance And Prediction

Fig. 2 Kalman filter recursive algorithm

The Taxi GPS System: Cubic splines were used to interpolate the data points in order to extract fine-grained (1 s) while maintaining that two curves obtained from either side are same. The Gauss–Kruger projection is used in MATLAB to transform the latitude and longitude measurements to planar rectangular coordinates. The testing included a Garmin 72 GPS receiver, computers running Red Hat Linux, and ORINOCO 802.11b gold card mounted on the vehicle’s roof with a 2.5 dB omnidirectional antenna. The GPS broadcasts vehicle’s speed, longitude, latitude, and heading every two seconds. Global Positioning System (GPS) provides location data with 5–7-m precision. To collect position data each second, interpolation was utilized (Fig. 2).

3.1 Comparison with ANN ANN is a non-parametric, data-driven, and nonlinear modeling method. The capacity to adjust to poor input, nonlinearity, and arbitration function mapping is the main advantage of ANN. Furthermore, a neural network is better at recognizing high-level properties like serial correlation in a training set. Moreover, neural network does not require prior knowledge of the noise distribution. The network can be directly trained with the actual coordinate locations using noisy distance measurements. The neural network can identify and adjust for noise in order to acquire an accurate

572

R. Yaduwanshi and S. Kumar

position. ANN can outperform moving average models, auto-regressive, and exponential smoothing even with a large number of random errors and limited training set. As a result, for performance comparisons, a well-known ANN model is used to predict location information for mobile nodes. Reference [21] describes the details of ANN-based correction and prediction (Fig. 3). • Note how little the NMSE of two prediction and correction methods is. That is to say, both neural network-based approaches and Kalman filter is effective in predicting and correcting node position in VANETs. Except for NMSEx for Delhi trace, all NMSE values for ANN are higher than Kalman-based prediction. • For each mobility trace, the average distance error value for viewing ANN is larger than that of Kalman-based prediction. These findings suggest that the Kalman-based prediction and correction outperforms the ANN method in terms of localization. The Kalman-based prediction is better than the ANN for both Bangalore and Mumbai traces. The ANN estimates a higher MSEy for the Delhi trace than a Kalman-based prediction. The MSEx of Kalman-based prediction and correction, on the other hand, is 4314.8080, whereas the MSEx of ANN is 2090.7093, reflecting that the MSEx for Kalman-based prediction and correction is higher than for perceiving ANN. The superiority of Kalman-based prediction is unaffected by this 4000

5000 4000 ANN based correction

3000

MSEy(m2)

MSEx(m2)

Kalman-filter based correction

3000 2000

2000

1000

1000 0

0 1

2

3

4

1

2

trace

average distance error(m)

8

0.03 0.025

NMSEy

NMSEx

6 0.02 0.015

4

0.01 2 0.005 0

0 1

2

3

trace

4

3

4

3

4

trace 80

60

40

20

0 1

2

3

4

trace

Fig. 3 Kalman filter-based correction versus ANN-based correction

1

2

trace

Location Accuracy and Prediction in VANETs Using Kalman Filter

573

discovery. Because the average distance error is 34.6728 and 31.8796, respectively. The Kalman-based location correction and prediction has a smaller distance error value than the ANN-based forecast. In VANETs applications, the distance error is more critical than the NMSE and MSE of the x- and y-coordinate time series as distance among vehicles has a direct impact on communication between them. To conclude, it is safe to say that Kalman-based location correction and prediction outperforms neural network-based location correction and prediction.

3.2 Computational Complexity Comparison To calculate the Kalman gain K, the Kalman-based prediction and correction presented in Fig. 2 requires numerous matrix multiplications along with inverse operation. Because these two processes have a complexity of O(K 3 ), the Kalman filter has a complexity of O(K 3 ) as well, where k represents total parameters in state. The network is presumed to be completely linked. The variables n, o, and I, respectively, reflect the number of hidden nodes, number of output nodes, and input nodes. For training a single epoch, the complexity is O (i * n * o + n * o) [22]. The prediction and correction method is run in MATLAB in our experiment. The Kalman-based correction and prediction is much more effective than the ANN-based as it takes less time. Correction and prediction. But comparing the Kalman filter to ANN [23] presents some challenges. This is due to the fact that these two families of localizing procedures have no traits in common. Another reason why a wide comparison between the Kalman filter and the ANN is problematic where the ANN’s scalability is unknown [23].

4 Conclusions Kalman filter is used to correct and predict the location of mobile nodes in Vehicular Ad hoc Networks (VANETs), allowing for the combination of basic location analysis and technological analysis. Using both model-driven traces and real vehicle mobility traces, we examined the performance of position correction and prediction. Extensive testing revealed that the proposed prediction method might attain greater levels of location accuracy. It has a wide range of applications, including data transmission protocols, routing protocols, map localization, and road congestion among others. The final goal of prediction and correction model is to improve the performance of Vehicular Ad hoc Networks (VANETs). This prediction and correction will be applied to a variety of applications in the future including data dissemination protocols, routing protocols and so on.

574

R. Yaduwanshi and S. Kumar

References 1. R. Kasana, S. Kumar, O. Kaiwartya, W. Yan, Y. Cao, A.H. Abdullah, Location error resilient geographical routing for vehicular ad-hoc networks. IET Intell. Trans. Syst. 11(8), 450–458 (2017) 2. O. Kaiwartya, S. Kumar, D.K. Lobiyal, A.H. Abdullah, A.N. Hassan, Performance improvement in geographic routing for vehicular Ad Hoc networks. Sensors 14(12), 22342–22371 (2014) 3. O. Kaiwartya, S. Kumar, Geocasting in vehicular adhoc networks using particle swarm optimization, in Proceedings of the international conference on information systems and design of communication, pp. 62–66 (2014) 4. O. Kaiwartya, S. Kumar, Guaranteed geocast routing protocol for vehicular adhoc networks in highway traffic environment. Wirel. Pers. Commun. 83(4), 2657–2682 (2015) 5. O. Kaiwartya, S. Kumar, Enhanced caching for geocast routing in vehicular Ad Hoc network, in Intelligent computing, networking, and informatics, pp. 213–220 (Springer, New Delhi, 2014) 6. DK. Sheet, O. Kaiwartya, A.H. Abdullah, Y. Cao, A.N. Hassan, S. Kumar, Location information verification using transferable belief model for geographic routing in vehicular ad hoc networks. IET Intell. Trans. Syst. 11(2), 53–60 (2017) 7. P. Bai, Energy efficient communication protocol at network layer for internet of things, in 2018 5th International conference on signal processing and integrated networks (SPIN), pp. 148–153 (IEEE, 2018) 8. X. Li, N. Mitton, D. Simplot-Ryl, Mobility prediction based neighborhood discovery in mobile ad hoc networks, in International conference on research in networking, pp. 241–253 (Springer, Berlin, Heidelberg, 2011) 9. J. Capka, R. Boutaba, Mobility prediction in wireless networks using neural networks, in IFIP/IEEE International conference on management of multimedia networks and services, pp. 320–333 (Springer, Berlin, Heidelberg, 2004) 10. P. Fülöp, S. Imre, S. Szabó, T. Szálka, The accuracy of location prediction algorithms based on markovian mobility models. Int. J. Mob. Comput. Multimedia Commun. (IJMCMC) 1(2), 1–21 (2009) 11. H. Kaaniche, F. Kamoun, Mobility prediction in wireless ad hoc networks using neural networks. arXiv preprint arXiv:1004.4610 (2010) 12. T. Anagnostopoulos, C. Anagnostopoulos, S. Hadjiefthymiades, An online adaptive model for location prediction, in International conference on autonomic computing and communications systems, pp. 64–78 (Springer, Berlin, Heidelberg, 2009) 13. T. Anagnostopoulos, C. Anagnostopoulos, S. Hadjiefthymiades, An adaptive location prediction model based on fuzzy control. Comput. Commun. 34(7), 816–834 (2011) 14. Kalman, Rudolph Emil. “A new approach to linear filtering and prediction problems.“ (1960): 35–45. 15. S.K. Yang, T.S. Liu, State estimation for predictive maintenance using Kalman filter. Reliab. Eng. Syst. Saf. 66(1), 29–39 (1999) 16. V.A. Bavdekar, A.P. Deshpande, S.C. Patwardhan, Identification of process and measurement noise covariance for state and parameter estimation using extended Kalman filter. J. Process Control 21(4), 585–601 (2011) 17. R. Mehra, On the identification of variances and adaptive Kalman filtering. IEEE Trans. Autom. Control 15(2), 175–184 (1970) 18. W. Ding, J. Wang, C. Rizos, D. Kinlyside, Improving adaptive Kalman estimation in GPS/INS integration. J. Navig. 60(3), 517–529 (2007) 19. A.H. Mohamed, K.P. Schwarz, Adaptive Kalman filtering for INS/GPS. J. Geodesy 73(4), 193–203 (1999) 20. C.S. Jensen, H. Lahrmann, S. Pakalnis, J. Runge, The INFATI data. arXiv preprint cs/0410001 (2004)

Location Accuracy and Prediction in VANETs Using Kalman Filter

575

21. H. Feng, M. Ma, Traffic prediction over wireless networks, in: Wireless network traffic and quality of service support: trends and standards, pp. 87–112 (IGI Global, 2010) 22. E Istook, T. Martinez, Improved backpropagation learning in neural networks with windowed momentum. Int. J. Neural Syst. 12(03n04), 303–318 (2002) 23. A. Shareef, Y. Zhu, M. Musavi, B. Shen, Comparison of MLP neural network and kalman filter for localization in wireless sensor networks, in: Proceedings of the 19th IASTED international conference on parallel and distributed computing and systems, pp. 323–330 (2007)

Early Detection of Breast Cancer Using CNN S. Gayathri, K. Jeyapiriya, V. A. Velvizhi, M. Anbarasan, and S. Rajesh

Abstract High death rate in women is caused mainly due to breast cancer. Image processing, as well as data mining and machine learning approaches, has all aided in the development of autonomous cancer detection systems. One employed machine learning classification algorithms to distinguish between benign and malignant tumors. Digital mammography is found to be the successful tool for detecting cancer in women who have no symptoms and diagnosing cancer in women who have symptoms such as discomfort in a lump or nipple discharge. As a result, one can use convolutional neural networks (CNN) in the suggested system, and OpenCV was performed on a data set from the UCI repository (Laurance in Breast Cancer Cases Rise 80% since Seventies; BREAST CANCER, The Independent, London, 2006). The accuracy, precision, sensitivity, specificity, and false positive rate of each algorithm are analyzed and evaluated. Keywords Breast cancer · CNN · Early detection · UCI · CAD

S. Gayathri · K. Jeyapiriya (B) · V. A. Velvizhi Department of Electronics and Communication Engineering, Sri SaiRam Engineering College, Chennai 600044, India e-mail: [email protected] S. Gayathri e-mail: [email protected] V. A. Velvizhi e-mail: [email protected] M. Anbarasan Department of Mechanical Engineering, University College of Engineering, Tindivanam, India e-mail: [email protected] S. Rajesh Department of Mechanical Engineering, Kalasalingam University, Krishnankoil, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_50

577

578

S. Gayathri et al.

1 Introduction Breast cancer happens quite frequently among female, and the exact cause cannot be predicted. Diagnosis is critical for breast cancer control because the origin of the disease is uncertain. Early detection can improve treatment success, save lives, and save money. Ultrasound imaging is one of the most common diagnostic tools for detecting and classifying breast abnormalities. A computer-aided diagnosis (CAD) system is a valuable and beneficial technique for breast cancer detection and categorization since it eliminates operator dependency and improves diagnostic accuracy. The goal of the study is to create an image processing-based machine learning technique for early diagnosis of breast cancer. Ultrasound imaging is one of the robust techniques to detect the breast lesions. Aforesaid technique is widely used all over the world. Though it is widely used the accuracy and reliability of the test results depend on the operator. Using a deep convolutional neural network, one can construct a computer-aided detection system for masses in ultrasound images. To the best of the knowledge, one can plan to develop a low-cost system that integrates both software and hardware consoles and communicates via UART.

2 Methodology and Implementation Recently doctors advised to use X-ray mammography (Fig. 1) technique to detect the breast cancer owing to its accuracy. The breast is compressed between two plates in this procedure. The X-rays are then sent through the breast and recorded on film. Ionizing radiation is used in X-ray mammography, which is dangerous. The breast is crushed between plastic plates, which the patient claims is painful. In case of magnetic resonance imaging (MRI), images are obtained by non-invasive technique. The images are generated using radio waves with the assistance of strong magnets. It can tell the difference between soft and brittle tissue. The MRI system operates at high frequencies, and it is in a protected and insulated chamber to avoid any interference from the environment. The patient who is being examined must lie down on a table. Small scanners or gadgets are also positioned around the breast to check and improve the quality of the yet-to-be-formed image. To perform a single test, many photos are normally necessary, which takes hours and is time demanding. During operating, the machine makes loud thumping and buzzing noises. Patients are given ear caps to help with noise reduction [2]. Automatic detection of lung cancer nodules in computerized tomography images is one of the technologies implemented to detect the cancer cells in CT images [3].

Early Detection of Breast Cancer Using CNN

579

Fig. 1 Breast cancer detection on mammography

3 Design Implementation For evaluating the patient input photos, one employs classification procedures to Convolutional Neural Network (CNN). OpenCV is used to extract the necessary features from the selected ROI areas. Modified contourlet transform-based effective image compression framework is studied for reducing the obtained image for the design implementation [4]. When deep learning algorithms are used to classify benign and malignant tumors, accurate results are obtained with less analysis time. The proposed algorithm’s efficiency is promising in connection with precision, accuracy, sensitivity, specificity, and false positive rate. As a result, the suggested methodology can obtain patient input ultrasound images and use image processing and machine learning techniques to forecast the outcome. As a result, the suggested method eliminates the need for manual inspection and assists clinicians in making quick decisions. As a result of the model’s integration of deep learning techniques, it has a high level of accuracy [5]. This is the first time it is considered combining hardware with a software-based image processing technique. The result, the forecast result, would be sent to the hardware via UART and shown on the LCD (Fig. 2).

580

S. Gayathri et al.

Fig. 2 Basic block diagram

4 Methodology 4.1 Data Set The Mini-MIAS database provided the data for the experiments. Break Hist data set is a data set with four directories, each indicating a different magnification of the photos, such as 100X, 200X, 400X, and 40X. There are 7858 occurrences in total in the collection, which are grouped into four magnification directories. Each magnification directory is divided into two sections, one for benign and one for malignant tumors.

Early Detection of Breast Cancer Using CNN

581

4.2 Preprocessing 4.2.1

Feature Selection

In a machine learning model, the relevance of feature selection is unavoidable. It removes all ambiguity from the data and minimizes the data’s complexity. It also reduces the number of data and enhances the quality of the training process with least computational effort. It prevents data from being over fit. It could be possible to obtain the best possible feature from all the feature and thus improves accuracy in the later part of the modeling [6]. Wrapper methods filter methods, and embedded methods are examples of feature selection methods. In fourth-order diffusion modelbased edge map extraction of infrared breast images, features are extracted using fourth-order diffusion model-based edge map for infrared images [7].

4.2.2

Recursive Feature Elimination

RFE is one of the feature selection algorithms, and it uses wrapper methods to select the most important features. Wrapper methods used as core technique, to evaluate the machine learning algorithm, to facilitate and choose features. Whereas in filter-based method, feature is selected based on the score either it may be highest or lowest score. Former is a wrapper-style method but internally uses filter-based feature selection method, and it does the job by exploring the subset of features in the training data set, starting with all of them and successfully deleting them till the target number remains. This is achieved by re-fitting the model employing the supplied machine learning method. Re-fitting is done by selecting the relevant features, ignoring the least valuable features, and fitting the model again. This procedure is done until only a certain number of features are left.

4.2.3

Segmentation

Segmentation is a splitting technique accomplished on images in 2 × 2, 3 × 3, and 10 × 10 patches. One can train the algorithm to distinguish close regions of importance that are critical for detecting the BC during this segmentation phase. It is simple to identify the tumor as early as feasible by removing unrelated data from the image. K-mean method is used as clustering algorithm for grouping the data, which means that related objects are grouped together. It is used in segmentation operations for better results, and it produces improved results when there are related objects in one group. It works quickly when compared to scattered data.

582

S. Gayathri et al.

4.3 Classifiers Machine to Support Vectors (SVM) algorithm’s aims to locate a hyperplane in the N-dimensional space (N-the number of characteristics) that sorts the data points undoubtedly. There are several hyperplanes from which it splits the two kinds of data points. The aim is to locate a plane with the best margin or the maximum distance between data points from the two data points. Hyperplanes are used as decision boundaries to classify the data points and subsequently enable the classification process. Various classes could be possible to assign to locate the data points on either side of the hyperplane. Reference [8] SVM is the successful and best classifier of all since it functions perfectly with obvious margins of separation and sufficient dimensional data. However, it is not ideal for huge dimensional data sets owing to its training time, and it also performs relatively poor if the data set contains more noise.

4.3.1

K-Nearest Neighbor (KNN)

The K-nearest neighbor algorithm is based on the supervised learning technique and is one of the most basic machine learning algorithms. The KNN algorithm works by locating points in the data that are close to the new point entered into the machine. The programme then sorts it by distance frame arrival point to separate the closest points. Different approaches are used to estimate this distance in a point; however, Euclidian distance is the most utilized by specialists. In the following phase, select a subset of points with a smaller distance between them and sort them into distinct categories. In KNN, points are selected in the structure of an odd number, such as two classes, and the peak point is classified independently as a new data point [9]. Although the KNN is straightforward tool and efficient to handle large data sets, the computation cost is high due to the need to calculate the distance between data points for the entire training samples, as well as the constant essential to find the value of K, which may add to the algorithm complexity.

4.3.2

Random Forest

Random forest is a learning algorithm that is supervised. It consists of a set of decision trees. Nodes express certain requirements on a certain collection of features, and branches split the decision toward the leaf nodes. The class labels are determined by leaf. Recursive partitioning or conditional inference tree can be used to create a decision tree. Recursive partitioning is a step-by-step procedure for creating a decision tree by splitting or not splitting each node. The tree is learned by subdividing the source set into subgroups based on attribute value tests. When all of the subsets at a node have the same value of the target variable, the recursion is complete.

Early Detection of Breast Cancer Using CNN

583

5 Proposed Methodology The system employs a convolution neural network (CNN) (Fig. 3) in the create model function, but simply the basic CNN. Every NumPy array that contains digitalized forms of images data sets receives zero padding in the first phase. CNN is divided into three tiers. Input layer, output layer, Maxpool layer, or concealed layer are all examples of layers. The activation function utilized in the maxpool layer is ReLu. CNN employs the sigmoid activation function. Samples are fed one after the other, and the accuracy of each model is evaluated. When the best model is discovered, it is saved and utilized to calculate the results. [10]As the number of samples provided to the model grows, the model’s accuracy improves. With the help of NodeMCQ and the ATmega 328, the LCD screen is synchronized with the programmed.

5.1 Layer 1 5.1.1

Convolutional Layer

The process of obtaining valuable characteristics from an image begins with convolution. Various filters present in the convolutional layers execute convolutional operations. Every image is viewed as a pixel value matrix. CNN’s first convolutional layer is just a regular image filter with ReLU added. Its goal is to extract basic information from a raw image. The so-called mid-level characteristics are extracted by layers in the network’s middle, while the high-level features are extracted by the final layers [11].

Fig. 3 Structure of CNN

584

5.1.2

S. Gayathri et al.

ReLU Layer

The rectified linear unit is abbreviated as ReLU. After the feature maps have been removed, they must be moved to a ReLU layer. ReLU performs an element-byelement procedure, setting all negative pixels to zero. It causes the network to become nonlinear, and the result is a rectified feature map. For finding the features, the original image is scanned with numerous convolutions and ReLU layers [12].

5.2 Layer 2 5.2.1

Pooling Layer

Another component of CNN is the pooling layer. As a result, rather than precisely positioned features created by the convolution layer, following actions are conducted on summarized features. As a result, the model is more resistant to changes in the position of features in the input image. Layer of convolution as a result, rather than precisely positioned features created by the convolution layer, following actions are conducted on summarized features. As a result, the model is more resistant to changes in the position of features in the input image.

5.3 Layer 3 Layer with a dropout: The dropout layer is used to make a few randomly selected nodes’ weights zero. As a result, a few nodes are dropped, allowing the CNN model to learn the parameters from the new distributed collection. It assists of the model in overcoming the issue of overfitting [13].

5.4 Layer 4 Layer should be flattened. Flattening is a technique for converting all of the pooled feature maps’ two-dimensional arrays into a single long continuous linear vector. This flattening step is critical for using the fully linked network after the convolutional or maxpool layers. It also incorporates all of the previous layers’ local features. To identify the image, the flattened matrix is provided as an input to the fully connected layer.

Early Detection of Breast Cancer Using CNN

585

5.5 Layer 5 Layer of density: The regular deeply linked neural network layer is the dense layer. It is essentially CNN’s outer layer, which can be viewed as a fully connected layer. To produce the final output, the pooled feature map is flattened and supplied to a fully linked layer.

6 Algorithm 1. 2.

Gather data samples from the UCI repository. Increase the number of samples available for analysis by supplementing the data. 3. Mix all sorts of samples, shuffle the data sets, and then split the training and test data sets. 4. For fair and error-free deep learning, convert every sample into equal pixels and remove any undesirable noise. 5. To give the data to the machine, convert each training sample into a NumPy list. 6. After each set of data learning, evaluate the efficiency and choose the most effective model. 7. Open a Jupyter Notebook and run Anaconda as administrator. 8. Put the address of the sample that will detect from the test data sets in the Jupyter note. Align an ATmega controller and a NodeMCU with a 16 by 2 LED display. Connect the system to this setup. 9. Now, using the sample address from the test set, run the application in the Jupyter Notebook and observe the results on the LED screen and the computer screen. 10. It can determine the results of the samples with nearly 75% accuracy using the machine learning programme, and this accuracy can be enhanced by feeding the computer a larger number of samples to develop more effective models.

7 Results Every NumPy array that contains digitalized forms of images data sets receives zero padding in the first phase. CNN is divided into three tiers. Input layer, output layer, maxpool layer, or concealed layers are all examples of layers. The activation function utilized in the maxpool layer is ReLu. CNN employs the sigmoid activation function. Samples are fed one after the other, and the accuracy of each model is evaluated (Figs. 4 and 5). When discovering the best model, one can save it and utilize it to calculate the results. As the number of samples provided to the model grows, the model’s accuracy improves. With the help of NodeMCQ and the ATmega 328, the LCD screen is synchronized with the programme.

586

S. Gayathri et al.

Fig. 4 Patient not affected

Fig. 5 Patient affected

8 Future Scope and Conclusion This work proposes to create and develop a system for evaluating and inspecting breast cancer using CT images from a data set. For extracting relevant characteristics from photos, one can present a convolutional neural network model with principal component analysis in this research. The outcomes appear to be positive. The entire experiment was carried out in Google Colab using the Python programming language and a Jupyter Notebook. As a result, the suggested technology eliminates the need for manual detection and enables ordinary people to identify breast cancer anywhere and at any time. This proposed system can also be developed to deliver drugs and recommendations using various embedded-based devices. Here’s a quick rundown

Early Detection of Breast Cancer Using CNN

587

of the actions one can follow to reach goals: Model development with more data sets, Module for evaluating image quality and Experiment with a variety of different deep learning architectures.

References 1. J. Laurance, Breast cancer cases rise 80% since seventies; Breast Cancer (The Independent, London, 2006) 2. J. Ferlay, I. Soerjomataram, R. Dikshit, S. Eser, C. Mathers, M. Rebelo et al., Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int. J. Cancer 136(5), E359–E386 (2015) 3. D. Jose, A.N. Chithara, P.N. Kumar, H. Kareemulla, Automatic detection of lung cancer nodules in computerized tomography images. Nat. Acad. Sci. Lett. 40(3), 161–166 4. S. Dutta, S. Ghatak, A. Sarkar, R. Pal, R. Pal, R. Roy, Cancer prediction based on fuzzy inference system, in Smart Innovations in Communication and Computational Sciences, ed by S. Tiwari, M. Trivedi, K. Mishra, A. Misra, K. Kumar. Advances in Intelligent Systems and Computing, vol 851 (Springer, Singapore, 2019). https://doi.org/10.1007/978-981-13-2414-7_13 5. K. Ganesan, U.R. Acharya, C.K. Chua, L.C. Min, K.T. Abraham, K.H. Ng, Computer-aided breast cancer detection using mammograms: a review. IEEE Rev. Biomed. Eng. 6, 77–98 (2013) 6. C. Woolston, Breast cancer: 4 big questions. Nature 527(7578), S120–S120 (2015) 7. J. Thamil Selvi, G. Kavitha, C.M. Sujatha, Fourth order diffusion model based edge map extraction of infrared breast images. J. Comput. Methods Sci. Eng. 19(2), 499–506 8. X.U. Juan, Q. Wang, M.A. Hong-Min, J.H. Xia, Primary efficacy of physical examination combined with ultragraphy and complemented with mammography for breast cancer screening. Chin. J. Cancer Prev. Treat. 20(17), 1295–1299 (2013) 9. C.E. Jacobi, G.H. de Bock, B. Siegerink, C.J. van Asperen, Differences and similarities in breast cancer risk assessment models in clinical practice: which model to choose? Breast Cancer Res. Treat. 115(2), 381–390 (2009) 10. M.H. Gail, L.A. Brinton, D.P. Byar, D.K. Corle, S.B. Green, C. Schairer et al., Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J. Nat. Cancer Inst. 81(24), 1879–1886 (1989) 11. F. Wang, Z.G. Yu, Current status of breast cancer prevention in China. Chronic Dis. Transl. Med. 1(1), 2–8 (2015) 12. L. Liu, A pilot study on risk factors and risk assessment score screening model for high-risk population of breast cancer. M.S thsis, School of Public Health, Shandong University, Jinan, Shandong (2010) 13. J.P. Costantino, M.H. Gail, D. Pee, S. Anderson, C.K. Redmond, J. Benichou et al., Validation studies for models projecting the risk of invasive and total breast cancer incidence. J. Nat. Cancer Inst. 91(18), 1541–1548 (1999)

Analysis of Misogynistic and Aggressive Text in Social Media with Multilayer Perceptron Sayani Ghosal and Amita Jain

Abstract Cyber-aggression and misogynistic aggression is a form of abusive text that has been risen on various social media platforms. Any type of cyber-aggression is harmful to society and social media users, and it also provokes hate crimes. Misogynistic aggression shows hatred toward women. Automatic detection models for hateful tweets can combat this problem. Contextual analysis of tweets can improve existing detection models. We propose a novel syntactic and semantic feature-based classification model where paragraph embedding enriches the contextual analysis of aggressive tweets and TF-IDF identifies the importance of each term in tweet and corpus. This study considers Trac-2 shared task English language dataset for multitasking classification. This novel research shows achievable performance against a baseline model and various classifiers. Among the various machine learning and deep learning classifiers, multilayer perceptron (MLP) portrays the preeminent performance. Keywords Text classification · Misogyny · Aggression · Multilayer perceptron

1 Introduction Abusive speech is malicious, offensive, and aggressive language where hate speech is also abusive language. Hate speech targets one person or group based on race, sex, color, religion, and disability [1]. Nowadays, growing social media like YouTube, Twitter, and Facebook provides platforms to share negative views or opinions and post various hostile and malicious contents [2]. Negative posts on social media create psychological effects to the users and provoke hate crimes. These types of content S. Ghosal (B) · A. Jain Netaji Subhas University of Technology, East Campus (Erstwhile A.I.A.C.T.R.), Guru Gobind Singh Indraprastha University, Delhi, India e-mail: [email protected] A. Jain Netaji Subhas University of Technology, Delhi, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_51

589

590

S. Ghosal and A. Jain

are generally aggressive text. One of the most target hate speeches is women or gender-based aggressive posts. Gender-oriented hate speech that targets women is called misogyny [3]. Numerous aggression and misogyny contents flow on the Web that requires automatic detection algorithms to combat inappropriate content. For example, “He should have also killed that bitch” is an overtly aggressive tweet as well as a misogynistic tweet, and it contains the slang word “bitch”. Problems of offensive content on the social Web have been recognized by various researchers, and some researchers classify various abusive behaviors with NLP like hate speech [4], aggressive [5], sexism [6], and misogyny content [7] detection. The availability of various shared task datasets encourages researchers to improve state of the art and create a better system to detect aggressive and misogynistic content. Some authors state that aggressive classes overlap with each other, and implicit aggressive tweets require more contextual research [8]. Overcome with the above limitations, we contribute a novel aggressive and misogynistic detection research. The vital contributions for this proposed research are as follows—first, analyze and develop an aggressive and misogynistic content classification model with a combination of features. We consider Trac-2 shared task English language dataset for our research. Two, comparing the proposed model with various machine learning classifiers for aggressive content classification and misogynistic content classification. This study is organized as follows. Section 2 presents previous studies for aggressive content and misogynistic content research. The proposed method with all features and classification model is described in Sect. 3. The implementation with results and analysis describes in Sect. 4. Finally, we conclude in Sect. 5.

2 Related Work Hate speech, cyber-stalking, cyberbullying, and offensive text are various forms of cyber-aggression that comes under trolling and abusive language detection [4, 8, 9]. All these terms are related to each other and synonyms of hate speech. Various terminologies are applied by the researchers to detect aggression from the text. Detection of abusive content against women or particular gender called sexism and misogyny [7, 10] is also part of our research. Methods applied to the above researches as supervised, deep learning, unsupervised and semi-supervised approaches. Most of the aggression detection studies used SVM [11, 12], LSTM [5], CNN [5], MLP [8], RNN [11], Bert [12], and many ensembles classifiers. One recent study [5] has applied a CNN-BiLSTM classifier with the Word2Vec model to detect overtly, covertly, and no aggression tweets. This research applied pseudolabeling to enhance model performance, whereas the model faced challenges to detect covertly aggression class. Another research [8] considers MLP classifier with deep neural network to detect cyber aggressive and non-cyber aggressive with 92% accuracy. Along with the above deep learning-based model, one study [11] applied a combination of SVM and RNN classifier with sentiment, irony, and

Analysis of Misogynistic and Aggressive Text …

591

word embedding features. This study also represents an improved performance of aggression dataset for multiple classifications—overtly, covertly, and no aggression. Author Nikhil et al. [13] presents an attention-based LSTM model for the same dataset but the proposed model over-fit for training data. The RoBERTa, BERT, and SVM classifiers [12] have been applied to English, Bengali, and Hindi language aggression dataset. Whereas above research needs further attention regarding contextual analysis of aggression detection. Contextual analysis and importance identification for each term of tweets required more research that can enhance the model accuracy of the aggression dataset. Along with aggression detection, sexism or gender-based aggression categorization [3, 6, 7] and misogynistic aggression [7, 10] detection have also applied various classifiers like BERT [6], CNN [6], and LSTM [3] ensemble approach of neural models [7]. One recent research [6] considers psychological scales to detect various dimensions of overt sexism. This research has applied CNN and BERT models to detect sexist comments. Another research [10] detects misogynistic tweets with attention layer-based BERT model, and the model shows that classification of tweets is difficult for the Hindi language. Author Parikh et al. [7] proposed multi-label sexism and misogyny detection with a concatenation of various embedding models—Glove, Ling, Elmo, and FastText. In this research, LSTM-based approach for misogyny detection performs well. One former multilingual automatic misogyny detection [3] research applied on three datasets—English, Spanish, and Italian. This study also has shown the relation between misogyny and abusive content with linear SVC and LSTM model. The above research also concludes that misogyny is different from sexism. Above abusive and aggression-based studies show that misogyny detection research in social media is in an early stage and it requires more analysis. Along with misogyny identification, aggressive content identification in social media is also a challenging task that requires more contextual analysis to detect accurately. This proposed research focuses on contextual analysis for aggressive content.

3 Proposed Method This section describes the proposed model with three main layers—preprocessing, feature extractions, and classifier. Figure 1 represents the architecture of the proposed model. Detail description for each step describes below.

3.1 Preprocessing Prior to studying the text data, this research employed several preprocessing steps to clean tweets in proper form. Various preprocessing steps help to improve model accuracy. This study removes various unwanted data and clean tweets—(1) Remove

592

S. Ghosal and A. Jain

Fig. 1 Architecture of proposed aggressive and misogynistic tweet detection model

emoji (2) Remove user names from tweet (3) Remove URLs (4) Clean punctuation and symbols (5) Clean blank and less than two words tweets (6) Convert all break to single space (7) Remove all email id from tweets, and (8) Remove English stopwords using NLTK library [14].

3.2 Feature Extractions and Classification This proposed model used three feature extractions for offensive text and misogynistic text detection. These three feature extractions are POS tagging, TF-IDF, and paragraph embedding. • POS Tagging: This model has applied the syntactic feature to analyze the tweets. Parts of speech tagging is an important feature to understand the grammatical categories. POS tags consider various tags like nouns, verbs, adjectives, adverbs, etc. Each tag in a tweet represents the significance of this tag. This research analyzes the importance of each tag in the total corpus [15]. • TF-IDF vectorization: Term frequency-inverse document frequency (TF-IDF) model follows the concept of the bag of word model. This model represents the importance and relevance of terms in a context [16]. This feature is not able to consider the semantic meaning of terms. It is only responsible to capture the significance of a particular term. Aggression identification and gender-based abusive content detection highly require significance of terms in tweets. • Paragraph Embedding: Paragraph vector aims to learn semantic similarity of proposed aggression and misogynistic model. This model presents low dimension vectors that jointly learn token vector representations with distributed memory

Analysis of Misogynistic and Aggressive Text …

593

model. This embedding model mapped each sentence to a unique vector and the same for each word. This research considers PV-DM vectors or distributed memory vector that remember missing terms in the context of tweets [17]. This paragraph vector model is implemented with the Genism Python library. • Multilayer Perceptron: The MLP [18] provides a nonlinear mapping of input and output vectors. This classification model is based on feed-forward artificial neural network (ANN). It consists of three layers, namely input, weight, and output layers, and the model utilizes activation function in each layer computation. The MLP considers an important method to adjust weights of the model in the output layer, called the backpropagation algorithm. Proposed aggression and misogynistic aggression detection research concatenate with all three feature extraction approaches. This study implemented the MLP classifier and successfully outperformed other selected machine learning classification models.

4 Experimentation, Results, and Analysis This section provides implementation details with dataset, evaluation metrics, and result analysis. • Dataset: This TRAC-2 [19] aggression and misogyny annotated dataset considers from social media that contains 4263 English tweets for the training dataset and 1200 for the testing dataset. The aggression dataset labeled with overtly aggressive (OAG), covertly aggressive (CAG), and non-aggressive (NAG) for subtask-A and subtask-B refers to misogynistic aggression dataset that classified as gender (GEN) and non-gender (NGEN). This study only considers the English language dataset for aggression and misogynistic aggression detection. Both the tasks are related to each other. • Evaluation Metrics: This research considers three evaluation metrics for analysis of proposed research—precision, recall, and weighted F1-score. This research only considers the weighted F1-score for the best analysis of proposed research. • Result and Analysis: This section is divided into two parts—the first part shows the performance of the proposed model for aggression detection, and the second part shows misogyny identification performance with the same proposed model. In Table 1 for aggression detection, we have compared the proposed model with three other machine learning classifiers—SVM, logistic regression, and random forest algorithm. It is observed that the MLP and the random forest algorithm perform better compared to SVM and logistic regression. All classification models applied concatenation of three features. Precision is higher for the random forest model, but recall and weighted F1 are higher for MLP. As we consider the weighted F1 score as the main evaluation metric, the proposed aggression detection model with MLP classifier improves 0.08% compared to random forest classifiers. Along with that, MLP classifier also outperformed the baseline model [20] LSTM classifier and FastText feature.

594

S. Ghosal and A. Jain

Table 1 Experiment results for aggression detection and misogyny detection Aggression detection

Misogyny detection

P

R

F1

P

R

F1

SVM

0.67

0.78

0.72

0.86

0.90

0.89

Logistic regression

0.75

0.80

0.74

0.90

0.93

0.90

Random forest

0.78

0.77

0.70

0.94

0.93

0.90

LSTM [20]

0.73

0.76

0.73

0.90

0.92

0.91

Multilayer perceptron (MLP)

0.75

0.78

0.76

0.91

0.93

0.92

Classification algorithms

Misogyny detection in Table 1, we have compared the proposed model for misogyny detection with three same machine learning classifiers, namely SVM, logistic regression, and random forest algorithm. Along with that, MLP classifier also outperformed the LSTM classifier and FastText feature [20]. It is observed that both the random forest and MLP classifiers’ overall performance are better with concatenations of all three feature sets. Precision and recall are higher for the random forest model, but weighted F1 is better for MLP, and it improves 0.02% compared to the random forest model. We have plotted two heatmap classification reports for aggression and misogyny classes. In Fig. 2, it is observed that most of the tweets classifier for NAG classes and F1-score is 0.88 for NAG class. In the heatmap of the English Trac-2 dataset, OAG and CAG class shows that OAG is higher compared to the CAG class. The misogyny class considers the GEN class as a misogynistic tweet class, and NGEN is vice-versa. It is observed in Fig. 2 that non-misogynistic aggression posts are much higher than misogynistic tweets. The weighted F1-score for NGEN class is 0.96. Both aggression and misogyny identification show noteworthy performance with our MLP-based proposed model.

Fig. 2 Classification report for aggression and misogyny class with heatmap

Analysis of Misogynistic and Aggressive Text …

595

5 Conclusion and Future Work In this research, proposed model can efficiently detect aggression and misogynistic tweets. Syntactic and semantic features are examined with POS tagging to analyze the contextual aggressive contents. Several machine learning classifiers are applied, but overall improvement is observed for deep learning classifier MLP. We considered Trac-2 shared task English dataset for this multitask classification, first subtask for aggression and second for misogyny classification. Aggressive tweet classification with MLP demonstrates significant improvement with 0.76 weighted F1 score, and binary classification for misogyny tweets reported 0.92 weighted F1-score. The performance of the proposed model can be enhanced by adding domain-specific lexicon features. In the future, we can advance the proposed research by analyzing misogyny tweets in granular levels with different languages.

References 1. F.M. Plaza-Del-Arco, M.D. Molina-González, L.A. Ureña-López, M.T. Martín-Valdivia, Detecting misogyny and xenophobia in Spanish tweets using language technologies. ACM Trans. Internet Technol. (TOIT) 20(2), 1–19 (2020) 2. R. Kumar, B. Lahiri, A.K. Ojha, Aggressive and offensive language identification in hindi, bangla, and english: A comparative study. SN Comput. Sci. 2(1), 1–20 (2021) 3. E.W. Pamungkas, V. Basile, V. Patti, Misogyny detection in twitter: a multilingual and crossdomain study. Inf. Process. Manage. 57(6), 102360 (2020) 4. S. Ghosal, A. Jain, Research journey of hate content detection from cyberspace, in Natural language processing for global and local business, pp. 200–225 (IGI Global, 2021) 5. S.T. Aroyehun, A. Gelbukh, Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling, in Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 90–97 (2018) 6. M. Samory, I. Sen, J. Kohne, F. Flöck, C. Wagner, Call me sexist, but…: Revisiting sexism detection using psychological scales and adversarial samples, in International AAAI Conference on Web and Social Media, pp. 573–584 (2021, May) 7. P. Parikh, H. Abburi, N. Chhaya, M. Gupta, V. Varma, Categorizing sexism and misogyny through neural approaches. ACM Trans. Web (TWEB) 15(4), 1–31 (2021) 8. S. Sadiq, A. Mehmood, S. Ullah, M. Ahmad, G.S. Choi, B.W. On, Aggression detection through deep neural model on twitter. Futur. Gener. Comput. Syst. 114, 120–129 (2021) 9. M. Mladenovi´c, V. Ošmjanski, S.V. Stankovi´c, Cyber-aggression, cyberbullying, and cybergrooming: a survey and research challenges. ACM Comput. Surv. (CSUR) 54(1), 1–42 (2021) 10. N.S. Samghabadi, P. Patwa, P.Y.K.L. Srinivas, P. Mukherjee, A. Das, T. Solorio, Aggression and misogyny detection using BERT: a multi-task approach, in Proceedings of the second workshop on trolling, aggression and cyberbullying, pp. 126–131 (2020, May) 11. A. Tommasel, J.M. Rodriguez, D. Godoy, Textual aggression detection through deep learning, in Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018), pp. 177–187 (2018, August) 12. A. Baruah, K. Das, F. Barbhuiya, K. Dey, Aggression identification in english, hindi and bangla text using bert, roberta and svm, in Proceedings of the second workshop on trolling, aggression and cyberbullying, pp. 76–82 (2020, May) 13. N. Nikhil, R. Pahwa, M.K. Nirala, R. Khilnani, Lstms with attention for aggression detection, in Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018), pp. 52–57 (2018, August)

596

S. Ghosal and A. Jain

14. E. Loper, S. Bird, Nltk: The natural language toolkit. arXiv preprint cs/0205028 (2002) 15. D. Cutting, J. Kupiec, J. Pedersen, P. Sibun, A practical part-of-speech tagger, in Third conference on applied natural language processing, pp. 133–140 (1992, March) 16. G. Salton, C.S. Yang, On the specification of term values in automatic indexing. J. Doc. (1973) 17. T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in Advances in neural information processing systems, pp. 3111–3119 (2013) 18. E.B. Baum, On the capabilities of multilayer perceptrons. J. Complex. 4(3), 193–215 (1988) 19. S. Bhattacharya, S. Singh, R. Kumar, A. Bansal, A. Bhagat, Y. Dawer, ... & A.K. Ojha, Developing a multilingual annotated corpus of misogyny and aggression. arXiv preprint arXiv:2003. 07428 (2020) 20. K. Kumari, J.P. Singh, AI_ML_NIT_Patna@ TRAC-2: Deep learning approach for multilingual aggression identification, in Proceedings of the second workshop on trolling, aggression and cyberbullying, pp. 113–119 (2020, May)

Information Retrieval

Development of Android Applications and Its Security Hridya Dham, Tushar Dubey, Kunal Khandelwal, Kritika Soni, and Pronika Chawla

Abstract Smartphones have gotten to be a major portion of a human’s life. These days’ versatile applications are playing a major part in numerous zones such as keeping money, social organizing, money-related apps, amusement, and so on. For each desktop or Web application, a substitute portable app is accessible. With fair single press number of versatile apps are accessible from Google’s play advertise. With this tremendous number of applications, security is a critical issue. This investigative article talks about the security of the applications and the malevolent apps that will influence or spill delicate information such as Worldwide Portable gear Character Number (IMEI) of gadget, credit or charge card data, area data, and so on. As the Android showcase is developing, security hazard has expanded, and, in this way, a center ought to be given to security. Keywords Android architecture · Automated · Multimedia · Security · Self-signed · Testing

1 Introduction Profession openings for portable application engineers keep on developing. Toward the finish of 2010 “Dhanjani” (2011) announced that more than 100 million iPhones were being used, 15 million iPads, and more than 10 billion portable applications were downloaded. Portable application stage applications are created utilizing customary programming dialects, which have been altered for the prerequisites and limitations of versatile stages. Portable stages license one dynamic application, H. Dham · T. Dubey · K. Khandelwal (B) · K. Soni · P. Chawla Manav Rachna International Institute of Research and Studies, Faridabad, Haryana, India e-mail: [email protected] K. Soni e-mail: [email protected] P. Chawla e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_52

599

600

H. Dham et al.

one application window, no client security, restricted consoles and show sizes, and restricted equipment assets. Because of these limits, stage explicit, specific portable systems have changed conventional programming dialects, e.g., “Java, C/C++/C#, and VB.NET”. For instance, a portable programming language is called Android, not Java. While portable application programming dialects depend on customary programming dialects, there are contrasts in the strategies that a versatile application is improvement, tried, sent, and got. In any event, when portable stages depend on comparative programming dialects, versatile stages fluctuate in the security model used to shield cell phones from malware and other security assaults. This paper analyzes “Android v 2x/3x, iOS 4x, Windows Phone 7, and Blackberry v6/7” application improvement and execution according to a security point of view.

2 Proposed Android Architecture The Android architecture is divided into four layers as shown in Fig. 1: the Linux kernel, the libraries and the Android runtime, the application framework, and the applications. While giving a call interface to the higher encapsulation, each layer of the lower encapsulation gets the response from higher encapsulation [1]. 1. Application The Android application will come pre-introduced with an assortment of key projects like a “customer, SMS program, schedule, maps, program, contacts, and

Fig. 1 Android architecture

Development of Android Applications and Its Security

601

others.” These application applications are written in Java. Just the Web-based adaptation is accessible. This file’s creation of a book is ILLEGAL. International journal of multimedia and ubiquitous engineering. 2. Application Framework The designer approaches the whole API structure of the center projects. The application structure makes it simpler to reuse its parts. Some other applications can make accessible their useful parts, and any remaining applications can access and utilize them (yet need to follow the security of the structure). Additionally, clients can utilize this reuse system to supplant programming parts [2, 4]. 3. Libraries and Android Runtime The library is divided into two parts: “Android Runtime” and “Android Library”. Android runtime comprises a Java Center Library and a “Dalvik virtual machine”. Center Library furnishes the principal Java library with the majority of its usefulness. Explicit updates for cell phones are provided. The Android system library upholds the application framework, and it is additionally a significant association between the application structure and the Linux part. This system library has been created in the C or C++ language. These libraries can moreover be utilized by the different parts of the Android system. They offer administrations to engineers through the application structure. 4. Linux Part The part contraption supplier outfitted with the guide of utilizing Android inside atomic layer is basically founded absolutely on Linux 2.6 portion. Operations like internal stockpiling, strategy the executives, net convention, base power, and distinctive center supplier are on the whole essentially dependent on Linux bit. It is the dream of the Android stage. Utilizing a Linux bit licenses Android to take gain of key wellbeing capacities and grants the apparatus fabricates to make equipment drivers for a renowned part [3, 5].

3 Android App Development Tools and Applications 3.1 Android SDK Android is an open-source versatile working framework dependent on Linux portion created by Google of OHA_ (Open Handset Alliance) to foster Android versatile applications. To foster Android portable applications, a bunch of instruments that are remembered for the Android SDK is required. There are API libraries and engineer devices important to assemble, test, and investigate applications for Android in the Android SDK. For the most part, we use Eclipse IDE with Android SDK and related instruments for creating Android applications. The ADT Bundle gives all that you want to start creating applications. The fundamental stages for creating applications are four stages. Creating applications incorporates the accompanying four stages:

602

H. Dham et al.

• Arrangement: During the setup stage, we introduce and set up our improvement climate. We likewise make Android virtual devices (AVDs) and associate equipment gadgets, on which we can introduce our applications. • Improvement: During the development stage, we get ready and foster our Android project, which contains the entirety of the source code and asset records for our application [8]. • Testing and verifying: During this stage, we incorporate our venture into a.apk bundle, which is introduced and running on the emulator for troubleshooting. • Distributing: During this stage, we plan and construct our application for delivery and appropriate delivery rendition of our application to clients.

3.2 MIT App Inventor MIT App Inventor [6] is an online, drag-drop visual advancement climate where individuals can rapidly make Android versatile applications by stopping together program blocks with a graphical UI. At the point when App Inventor was reported in July 2009 as a trial instructing and learning apparatus, the designer device was mostly pointed toward giving understudies and understudies simple admittance to programming overall and cell phones specifically. Application Inventor that is a visual squares programming language for making versatile applications is a creating climate for designers ailing in programming experience.

3.3 Android Applications Android applications are the applications which are designed on android platform. Some famous applications are shown in Table 1.

4 Advantages of Android Operating System Android OS is most popular and widely used operating system because of its advantages. Advantages of Android OS are shown in Table 2.

5 Android Security 5.1 Android’s Five Key Security Features • Security at the working framework level through the Linux bit.

Development of Android Applications and Its Security

603

Table 1 Android applications Applications About applications [6] AIRBNB

Airbnb should be your go-to app for finding rooms, apartments, homes, and other unique accommodations when traveling if you do not want to go with a traditional hotel. The length of your stay need not be short-term either; some locations let you book a stay for several months, which may appeal to people working temporary jobs in new places. This travel app also highlights other experiences and restaurants, so you can find out about everything a locale offers

DROPBOX

Dropbox pioneered the personal cloud service, where all your stuff would be available no matter what device you were using. On Android, it holds its own—even against the highly integrated Google Drive. Dropbox can also act as a seamless backup for your images, automatically uploading every photo to the cloud. It even includes some light image editing tools. If you are the type of person with lots of files already stored in Dropbox, this app is a must-have

DUOLINGO If you are looking to learn another language, Duolingo gamifies language learning with bite-sized lessons and a friendly interface. Starting with simple vocabulary and building from there, Duolingo is your guide to learning a new language or brushing up on one you already know. The more you use the app, the more you unlock and—with practice—the more you learn. This free app currently supports Danish, Dutch, French, German, Irish, Italian, Portuguese, Spanish, and Swedish. Esperanto and Klingon are more practical choices available. AMAZON

Amazon is the Internet’s marketplace, the one place where you can buy just about anything—and it is cheap too! On Android, two of our favorite features are the integrated Alexa voice commands and the photo search, which makes it easy to surreptitiously compare Amazon’s prices to those of the brick-and-mortar stores it is killing. Amazon video-related functionality [7] has been moved over to a dedicated app, but all of its other consumer services, including Fresh and Restaurants, make an appearance. Prime members rejoice; there is never been a better way to stay connected with your Amazon lifestyle

Table 2 Advantages of Android OS Low Input & High ROI

Android offers a low barrier to entry in comparison with other platforms. Android makes its Software Application (SDK) available to the developer community for free, reducing development time

Open source

Take use of the open-source benefit of licensing, royalty-free, and the greatest technological foundation provided by the Android community [9]. The Android SDK architecture is open-source, which means you may connect with the community for future Android development expansions

Ease of integration

The platform as a whole is ready for modification. You may integrate and customize the smartphone application to meet your specific business requirements

Various sales channels

Unlike other mobile devices, it may be used in a variety of ways. You are not required to distribute your apps through a single market. You can utilize a third-party application market (particularly Google Android Market), but you may also create your own distribution and sales channel: apps for vertical markets, new application stores, and placing it on your Web site

604

• • • •

H. Dham et al.

Application sandboxing is required. Guaranteeing secure entomb process correspondence. Application signature. Consents set up by the application and given by the client.

5.2 Android System Security In the default setup, no program has the approval to lead any activities that may adversely affect other applications, the working framework, or the client. Security is a major part of any Android device. All of the procedures are generally apparent in two regions: Android framework security and information security. Network protection is significant on the grounds that various bodies work with the assistance of safety itself and, then, at that point, the job of digital protection becomes possibly the most important factor in the Android framework [10]. The Linux piece [11] incorporates various security highlights for Android. It gives the working situation client-based opportunities to demonstrate, prepare parcels, a secured framework for IPC, and the choice to dispense with any superfluous or conceivably powerless parcel parts. It furthermore capacities to keep different system clients from getting to and depleting each other’s resources. Because of its client-based security, Android may create an “Application Sandbox.” Each Android app has its own client ID and operates as a distinct interaction. As a result, each program is permitted at the method level by the Linux component, which prevents programs from interacting with one another and grants them only restricted access to the Android operating system. This provides the client authorization-based access control, and he/she is offered a rundown of the activities that the Android program will lead and what it should try to accomplish before the application is downloaded. The same is true for document situation privileges—each program (or client) has its own records, and unless an engineer specifically opens documents to another Android app, documents given by one cannot be read or edited by another.

5.3 Android Application Security Scans When developing and testing the security of Android applications, designers should adhere to Android security best practices and keep the following in mind: • • • • • • •

Inbound SMS members of the audience (control and order) Dangerous creating a document Improper data set stockpiling Dangerous exploitation of common impulses The capacity of sensitive data on the bulk stockpiling device SQL injection from the content supplier APN or intermediate modification.

Development of Android Applications and Its Security

605

The more mind-boggling security features of Android are intended to be visibly exposed to the client, implying that they may be altered via the UI. Using a secret key or pin, putting your phone to lock after a time of dormancy, merely engaging distant affiliations that you employ, and presenting Android programs that you trust and have long verified are all basic ways of dealing with the security of Android devices. Google moreover permits affirmed and registered safe Android applications with its business place, diminishing the customer’s chances of presenting a malevolent application. Also, the Android security system requests the customer to allow a program to be presented, making it hard to remotely present and execute an application. Customers may moreover ensure the security of their Android cell by applying system reports reliably.

5.4 Android System Security Protection Android system security inherited Linux planning inside the design ideology. Each Android program, in actuality, operates in its own process. Each program in the OS has its own system identity. The permission mechanism handles the majority of the security functions. Permission is frequently limited to certain process operations. Android is divided into privileges. The software signature method is primarily responsible for data security. It makes use of the AndroidManifest.xml file. When certain software services are invoked, the system initially checks this file. To access the device’s protected features, one must provide one or more tags stating the permission in Android Manifest.xml.

5.5 Android Anti-Theft Security A definitive assurance for your Android gadget in the event that it is at any point lost or taken. This capacity enjoys the benefits of the exact following, encoding, spy camera actuation, and gadget lockdown [6, 12]. It likewise checks freedoms for sending “SMS messages”, equipment controls, taking photographs and recordings, your area, fine (GPS) position, getting “SMS”, understanding “SMS or MMS”, altering “SMS or MMS”, full Web access, perusing contact information, and composing contact information.

6 Security Issues Related to Android Platform A large number of safety strategies are utilized to keep the Android stage secure. Each program may be of the working system. At the point when you introduce an application, the working framework makes another client profile that is connected

606

H. Dham et al.

with that program. Each program works as a distinctive client, complete with its own private documents, a record framework, the client ID, as well as protected workspace. A program runs with its own cycle, with its own “Dalvik VM” case and client ID on the working framework [13].

6.1 Explicitly Characterized Application Authorizations At the point when an Android application requires expressly expressed authorizations within the appearing record, Android applications register for the specific freedoms they need to get to normal framework assets. The required authorizations ought to be given in the Android show document while building the application. On the off chance that the phone vibration value is required, it should be given in the Android show document. The rundown of assets that the program will get to is shown all through the establishment cycle. A portion of these freedoms permits the program to settle on telephone decisions, interface with the organization, and work on camera and other equipment sensors. Admittance to shared information involving private and individual data like client inclinations, area, and telephone numbers is additionally needed for applications. Applications may likewise force their own licenses by proclaiming them for use by different applications. For more administration over the program, the application can announce quite a few unmistakable consent types, for example, read-just or read-compose freedoms.

6.2 Limited AD-HOC Consents Substance providers might choose to donate on-the-fly privileges to other applications for specific information they wish to distribute uninhibitedly [14]. Usually developed by the uncommonly designated giving and renouncement of induction to demonstrated resources utilizing Uniform Asset Identifiers (URIs). URIs insinuate unequivocal data assets on the system, like media store, contacts, call log, etc. Here is an illustration of a URI that profits the telephone quantities, all things considered.

6.3 Practices in Android App Testing There are sure factors to be thought of while putting together a technique for portable application testing as displayed in Fig. 2. (1) Device Selection This is one of the most basic strides prior to beginning the Android application testing process.

Development of Android Applications and Its Security

607

Fig. 2 Practices in Android app testing

• Figure out which gadgets will be utilized in the testing method. • The choice ought to be made so that the quantity of target customers is expanded. • In the determination interaction, standards like OS variant, screen goals, and structure factors [tablet or shrewd phone] are basic. • If essential, the utilization of emulators can be thought of. Be that as it may, emulators ought not to be utilized instead of actual gadget testing. Gadget emulators are reasonable and helpful during the underlying advancement stage. In any case, real gadgets are needed to assess true situations. To accomplish the best outcomes, the two emulators and real gadgets should be used in a reasonable way. (2) Beta Testing of the Application Beta testing is profoundly effective in testing with genuine individuals, genuine gadgets, genuine organizations, and genuine applications conveyed across a huge geographic region. This gives a reasonable perspective on network thickness, network contrasts [Wi-Fi, 4G, 3G, and 2G], and the impact on the application. Genuine beta testing is unique and cannot be replicated in a controlled setting [15, 16]. (3) Connectivity Android applications are frequently connected to the Web for an assortment of reasons. One might say that if one somehow happened to comprehend the significance of IOT frameworks in the advanced time, which shape the digital world and the actual world together, then, at that point, it will always change the manner in which people communicate with keen objects.

608

H. Dham et al.

The association of numerous gadgets is basic in fostering the arrangement. During testing, most availability is overseen by recreation programming, which assists with managing network speed, inactivity, and confined networks. For ongoing outcomes/information, it is proposed that testing with real organization associations is constantly suggested [17]. (4) Manual or Automated Testing: While automated testing consumes a large chunk of the day on the underlying run, it comes invaluable when the testing should be rehashed. This likewise diminishes the all-out stretch of time of testing during the different phases of advancement. Android automation ought to be utilized with manual testing when the repeat of experiments is successive during the execution stage, similarity testing for similar applications on different OS renditions, potential execution achievements, etc. [18].

7 Future Scope and Conclusion The versatile Web application period has recently started, as well there is far to go. Advancement of portable Web applications will be centered around the accompanying highlights: More sensors will be added to the cell phones, and in this manner new APIs to utilize those capacities will bring spic and span one’s applications for clients. Mltmda capacities will be overhauled, and the motor will support many kinds of Mltmda like blaze and SVG. “Integrated Development Environment” (IDE) will be created to speed up the improvement of utilization. Perception programming and “JavaScript” investigating will be the main elements of the IDE. The Android application gives a full diagram of how to save the data inside the application, alongside different tasks such as arranging, looking, evolving subjects, and so on. The UI of the application is made appealing to supply the customer with an incredibly easy contribution. Gadgets and different features such as voice courses and searching for the customer are accessible. From the over it is inferred that to be sure regardless of the way that android gives incredible security yet at the same time, there can be weaknesses and security issues show up because of safety flaws and despicable headway of the applications. Hence, a fitting security part is needed to avoid the security risks and recognize the malicious applications for the security of sensitive data.

Development of Android Applications and Its Security

609

References 1. L. Ma, L. Gu, J. Wang, Research and development of mobile application for android platform. Int. J. Mltmda Ubiquitous Eng. 9(4), 187–198 (2014) 2. A. Barbosa, J. Goncalves, A.N. Ribeiro, A. Costa, Integration of SIP protocol in android media framework, EUROCON, pp. 1–4 (2011) 3. S. Jin, H. Li, Y. Liu, Research on media player based on Android, FSKD, pp. 2326–2329 (2012) 4. S. Holla, M.M. Katti, Android based mobile application development and its security. Int. J. Comput. Trends Technol. 3(3), 486–490 (2012) 5. O. Bach, Mobile malware threats in 2015: Fraudsters are still two steps ahead, July 13, 2015 6. Korea Internet & Security Agency, Mobile public service security vulnerability check guide, Ministry of Public Administration and Security (2016) 7. Transcript of analysis and research of system security based on android analysis and research of system security based on android by Raghunath 8. M. Willocx, J. Vossaert, V. Naessens, A quantitative assessment of performance in mobile app development tools, in: 2015 IEEE international conference on mobile services, pp. 454–461 (IEEE, 2015, June) 9. R. Surendra, B.S. Reddy, P. Chawla, K. Soni, Cyber security in digital sector, in 2021 International conference on artificial intelligence and smart systems (ICAIS), pp. 703–710 (IEEE, 2021, March) 10. H. Bing, Analysis and research of system security based on android, in 2012 Fifth international conference on intelligent computation technology and automation, pp. 581–584 (IEEE, 2012, January) 11. S. Chatterjee, K. Paul, R. Roy, A. Nath, A comprehensive study on security issues in android mobile phone—scope and challenges. Int. J. Innov. Res. Adv. Eng. 3(3), 62–72 (2016) 12. H.W. Kim, A study on the mobile application security threats and vulnerability analysis cases. Int. J. Internet Broadcast. Commun. 12(4), 180–187 (2020) 13. M. Linares-Vásquez, C. Bernal-Cárdenas, K. Moran, D. Poshyvanyk, How do developers test android applications? in 2017 IEEE international conference on software maintenance and evolution (ICSME), pp. 613–622 (IEEE, 2017, September) 14. M. Mutreja, K. Khandelwal, H. Dham, P. Chawla, Perception of IOT: application and challenges, in 2021 6th international conference on communication and electronics systems (ICCES), pp. 597–603 (IEEE, 2021, July) 15. K. Soni, S. Kumar, Security issues in cloud computing 16. K. Jha, S. Anumotu, K. Soni, Security issues and architecture of IOT, in 2021 International conference on artificial intelligence and smart systems (ICAIS), pp. 1381–1385 (IEEE, 2021, March) 17. S. Mehta, A. Sharma, P. Chawla, K. Soni, The urgency of cyber security in secure networks, in 2021 5th International conference on intelligent computing and control systems (ICICCS), pp. 315–322 (IEEE, 2021, May) 18. S.S. Tyagi, Enhancing security of cloud data through encryption with AES and Fernet algorithm through convolutional-neural-networks (CNN). Int. J. Comput. Netw. Appl. 8(4), 288–299 (2021)

Smart COVID Simulation App for Tracking of Area-Wise COVID Patients: Aware of Number of Patients While Moving on the Places S. Janardhan Rao, S. Hrushikesava Raju, K. Yogeswara Rao, S. Adinarayna, and Shaik Jemulesha Abstract In modern days, people are eager to see everything on their mobiles through the usage of apps. In the present situation, the user supposed would like to check the count of COVID patients while the gadget in which the app is installed is moving on. There are certain specific option user has to choose in order to see the count details over the app. As the place changes, the details of the place which is in the circle of the cell tower would update the count of COVID patients. This kind of work is done with the help of an online cloud as well as an in-built service GPS module. Dynamically, the user moves, and the switching of place is also done and also updates the counting of COVID patient details. The aggregate functions are performed and are customizable. The results are produced based on the region level chosen by the user. In the future, any unknown disease would be entered, and this kind of app would display the count and statistics of that infected. Keywords Health data · Online cloud storage · COVID patients · GPS · Timely updating · Selection of the region · Extraction · Reporting

S. Janardhan Rao · S. Hrushikesava Raju (B) Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Green Fields, Vaddeswaram, Guntur 522502, India e-mail: [email protected] S. Janardhan Rao e-mail: [email protected] K. Yogeswara Rao Department of CSE, GITAM (Deemed to be University), GITAM Institute of Technology, Visakhapatnam, India e-mail: [email protected] S. Adinarayna Department of CSE, Raghu Institute of Technology, Visakhapatnam, India S. Jemulesha Department of CSE, Annamacharya Institute of Technology and Science, Tirupathi, Chittoor District 517501, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_53

611

612

S. Janardhan Rao et al.

1 Introduction Since less than 2 years, the world is covered with violent pandemic in which not only a country, many countries affected. Irrespective of the ages, the people were faced the breathing problems, and COVID is considered a dangerous spreading virus. To know the number of COVID-19 patients of a particular area or region could be traced by asking from the hospitals history of persons. One is manual approach where calling to each health center of that area, and noting on a paper with sub-area of that area, summing up the details of total infected persons. The other approach is cloud server where data of every hospital is dumped, and it became now easy to extract from that to determine the count of affected patients. From these two approaches, in order to know the number of covid-19 patients that could be extracted from the hospital database. The second approach should be very much enhanced in order to get our projected proposal. How it could be enhanced in the sense depends on the advancement of the technology. This could be a mobile app title “Statistics of covid-19 patient data: Showing the covid-19 patients through virtual objects as well as statistics of them in the specific range”. In the intended system, as the app is installed in the gadget and the gadget is moving toward the certain direction, the infected COVID-19 patients count in the specific range is shown along with the visual entities in the loaded app. In the proposed application where the areas are automatically updated by getting the information from the nearest located cell tower and are connected to the cloud server where covid-19 patients are available and are dumped from hospital online datasets. The showing of visual entities over the app is best viewed by taking the addresses stored along with patient details, and the addresses are tracked using GPS built-in service. The GPS service helps to update the patient details as the location is changing from one place to another place. A similar kind of app is there that is online FM playing where the user chooses a location, FM of that location is to be played. It is simulated, and it is dumped with many FM office online addresses. But, our intended system is a real-time application where the updates of COVID-19 patients from the online cloud server are extracted as the mobile gadget is moving on and is shown as number of patients in the area under monitoring. The satellite technology is updated and is to be useful in switching from one place to other place. The intended system is used to display the infected patients in the app along with visual diagram, while the user with gadget is moving on. The order of activities that take place in the system is as below: (1) (2) (3) (4)

Connect the GPS. Connect the every hospital data though online and is to be available in the cloud. Design the statistics of COVID-19 patient data. As mobile gadget is moving on, the area switching is done and extracts the patient data. (5) Statistics is to be displayed dynamically over the app.

Smart COVID Simulation App for Tracking …

613

2 Literature Review There are a variety of studies which explore the information on COVID-19 patient statistics, COVID affected regions, prediction of infected objects in the near future, etc., would be helpful in dealing with present scenarios. But the proposed model, as we planned, would connect the hospital data especially COVID-19 over a cloud, and the location tracking using GPS would fetch each patient address till they are to be recovered. As per aspect of Ref. [1], the designed mask would able to identify the infected footprints while moving in any specific environment and alert the infected information so that the user could take precautions against the COVID-19. It consists of modules such as integration of required sensors, mask module, and communication module, where each one would do their responsible job. The mask module consists of sensors, which interact to each other to process and make decisions for producing the information. In the view of content from Ref. [2], one of major industry that was affected by COVID-19 pandemic is construction. This was impacted severely and cause many facilities to be fallen down like time, cost, labor, payments, etc. The detailed statistics were discussed in the published article. In regard of Ref. [3], the analysis is the demonstration over a large hospital about the characteristics of admitted patients and usage of emergency for them during past 2 years as well as 2020 year, they reported after analysis is usage of emergency department is fallen 52% than previous years due to pandemic. As per direction from Ref. [4], this discovers the spray that consists of carrageenan over the nose and works as a key role in preventing the virus attachment efficiently. It is proved against previous pandemic victims, and it worked in the best form during the first COVID-19 experience. In the gathered set from Ref. [5], there are many reasons to determine complications of neuropsychiatric, and COVID-19 affects the neuronal and glial functions as well as damage significant neurons, synapses, brain connections and related options. This analysis discusses the impact of COVID-19 over the neuro and psychiatric functionaries. From the point of view from Ref. [6], the impact of COVID-19 is over the respiratory system and causes failure of it. It leads to many kinds of neurological diseases such as cerebravascular conditions, ageusia and specific neuropathies. What adversary affects to be raised would be listed in this article. With respect to the mentioned data from Ref. [7], the covering of mask over eye near surface and usage of this mask for long hours as well as usage of digital devices result in dry eye syndrome during this pandemic time. To void such issue, analyze the therapies and follow such things from time to time with safety measures. As per statistics from Ref. [8], the Covid-19 over diabetic patients effect severely and leads to cardio related problems. To prevent such cases and protect the patients, the GLP1-RA is proposed to consume that reduces cardiovascular diseases and support to lose weights. In the view of Ref. [9], the drug prescription and the plan is recommended after the post covid syndrome and acute kidney diseases, and other side-effects raised during covid. From the observation of Ref. [10], a case study over a person is analyzed and recommended to continue oxygen supplying phenomenon after a post-COVID syndrome. The recommendation is given based on symptoms like throat, anosmia, fever, etc.

614

S. Janardhan Rao et al.

With the orientation from Ref. [11, 12] the effects of earlier pandemics were analyzed and mentioned how many victims for such case were. The novel COVID-19 also caused many lives to be ruined and also changed the lifestyle, and it made to learn lesson such as modern improvement in technology, medical equipment and nurses trained to shield from COVID. In the point of view from Ref. [13], there are specific tests in a single center that are proposed for variety of outcomes expected. From AKI based on 2019 COVID, the case study from African country Nigeria is analyzed and reported output of it that belongs to a specific center, from Nigeria. In the view of data from Ref. [14], the people living in SIDS would be experiencing the food insecurity that lacks nutrition portions, which lead to noncommunicative diseases of certain age groups. Hence, the approaches are adapted to make a systematic discussion on food security and also on climate change. With regard of Ref. [15], the health organization termed 2020 as international year of nurse and midwife because shortage of nurses was found. The nurses are key in that year, and they would involve in health care, follow-up practices, timely actions to be done, etc. As per analyzation from Ref. [16], the reasons were determined that caused for death among the COVID patients and are listed with respect to cluster wise symptoms. From perspective of Ref. [17], the COVID patients deceased in the Latin America and United States and are analyzed based on six indicators. The report stated that weak follow-up of medical plan and the treatment yielded that higher fatality in these regions. With the discussion from Ref. [18], the scenario of few drinkers from Latin America and Caribbean is taken during pandemic time and analyzed such that those men were affected for certain parameters and ladies are less affected. As per information from Ref. [19], the scenario in the Brazil up to April 2021 according to WHO stated that it is second in the cases in world and 158 members lost lives per 1,00,000 cases. From the aspect of Ref. [20], the vaccines were analyzed in certain countries like Canada and informed the side effects caused like blood-cots in the circulatory system at some point, and by considering this effect, other countries like Denmark were stopped using vaccines. From the resources specified in [21–24], the usage of map for displaying of spots according to ranking and reviews, ordering of spectacle using GPS by the nearest shop and eye sight discovery report with remote doctor, prediction of health for future days based on consumption of food and previous health bulletin, and catching of weighted objects using GPS and safely float that object on the ground using IoT. All the studies which are mentioned are classified into cluster where one cluster covers determination of reasons for the disease, other cluster covers identifying the foot prints of the infections and alerting, another cluster covers the review of specific region, other cluster covers the miscellaneous traits that caused infection, etc.

3 Proposed System In this, the entire work is broken into modules for simplicity. Each module is aimed for achieving intended job. The working phenomenon of tracking of area-wise patient count is depicted in Fig. 1 and is consisting of entities, attributes of each entity, and

Smart COVID Simulation App for Tracking …

615

communication among those. Here, the identification of the modules, the architecture, pseudo procedure of significant modules and flowchart of the proposed system. The modules identified in this proposed work are as follows: (A) Storing on Cloud and Mapping of address over map: The entered patient data is automatically to be stored over a cloud. The details are provided by the hospitals, agents, medical officers and service officers. Each address automatically gets a foot stamp over a map, and the map is visible in the designed app. The map would show many addresses at a time. (B) Updating the details and selection of regions: As the patient record became inactive means that address over the map is muted. The range of 10 km is considered by default and is customized. The number of patients showing over the app and that is updated automatically when the mobile is moving.

Fig. 1 Architecture of smart COVID simulation app

616

S. Janardhan Rao et al.

(C) Computation of Statistics and Reporting: It analyzes the count of patients w.r.to the area or locality, then merging of two areas as one bigger locality, and continue processing in the given range. From these results, the user could understand the area is affected more, would contain a greater number of cases so that user would be alerted and will take safety precautions. It displays the count of cases with respect to the area that is updated while the gadget is moving dynamically. For task simplicity, the first two modules such as cloud storage as well as mapping of addresses over map are bundled as one module, third module is updating details and is now as second module, and the fourth and fifth modules such as computing and reporting are merged as one module (Fig. 2). (A) Pseudo_Procedure Cloud_Storage_Cum_Spotting_addresses_mapp (hospital_covid_data[][]): Input: Cloud storage permission, Covid_people_dataset. Output: Projection of every contact over a map using addresses. Step 1: Subscription of specific vendor cloud for online storage of COVID patient details, name that cloud as Covid_Cloud. Step 2: When report came +ve for a person, that person addresses, and details are uploaded into the Covid_Cloud. The hospital in which that infected person joined is also tracked till recovered or further critical awareness. Step 3: The addresses of every infected person are brought on to the Google map, which is a built-in service. Step 4: The addresses are added on to the map as and when new entry appears into the covid_cloud.

Fig. 2 Flowchart of smart COVID simulation app

Smart COVID Simulation App for Tracking …

617

Step 5: Shows each address as a point on the map and visually to be displayed, every area is considered as a cluster. The counts of regions are to be shown based on the option the user would choose among constitution, municipality, mandal, village, ward, etc. These counts are computed in next module and displayed based on customer option. (B) Pseudo_Procedure Updation_Cum_Selection_regions(Covid_Cloud[][]) Input: Covid_people_dataset. Output: Making points over map for each patient detection, removing point for each patient recovery or discharge. Step 1: If a person is infected through tests, it is uploaded onto the covid_cloud. Step 2: The Google map should give permission to this app, for adding of point or removing of point over the map. Step 3: When patient is recovered, that point is to be removed and that detail is added to the history. Step 4: Choose the option by the user regarding region type state, constitution, municipality, mandal, village and ward. (C) Pseudo_Procedure Computation_Cum_Reporting_statistics (Covid_Cloud[][], map[][]) Input: Covid_people_dataset, google_map_service. Output: Statistics such as accumulation based on option, the user would choose. Step 1: Choose the option regarding region type state, constitution, municipality, mandal, village and ward. Step 2: If the region is ward, automatic fetching of ward from the in-built service Google map is activated. The points belonging to that ward are available on the map. Apply aggregate function sum over those points within the ward boundary, and display the sum. Step 3: If the region is village, automatic fetching of wards of a village from the in-built service Google map is activated. Apply aggregate function sum over those wards within the village boundary, and display the sum. Step 4: If the region is municipality, automatic fetching of wards of a municipality from the in-built service Google map is activated. Apply aggregate function sum over those wards within the municipality boundary, and display the sum. Step 5: If the region is Constitution, automatic fetching of villages and municipalities from the in-built service Google map is activated. Apply aggregate function sum over those villages as well as municipalities within the constitution boundary, and display the sum. Step 6: If the region is state, automatic fetching of constitutions of a state from the in-built service Google map is activated. Apply aggregate function sum over those constitutions within the state boundary, and display the sum.

618

S. Janardhan Rao et al.

Step 7: The statistics may vary based on updating of the points on the map, which is nothing but COVID infected persons.

4 Results There should be a need to showcase the output from the order of processing each activity from the beginning of the task. The following Fig. 3 shows the order of activities from beginning of task till output the count. The screenshots of the smart COVID simulation app are demonstrated in Fig. 4 that deal with checking of authorized user or not. Figure 5 deals with COVID infected persons details with addresses, and Fig. 6 deals with selection of area level such as ward, village, mandal, municipality, constitution as well as state. The performance and accuracy of the proposed approach are demonstrated in Figs. 7, 8 and 9. Here, the automated system is built-up with respect to the requirements and is Internet-enabled always. As the data grows over time, the performance of the system is measured against traditional, semi-automated approaches in the perspective of time consumed is defined as follows in the table as well as in the graph.

Fig. 3 Order of activities in the smart COVID simulation app

Fig. 4 Home page of smart COVID simulation app

Smart COVID Simulation App for Tracking …

619

Fig. 5 Sample COVID patient details on one drive (cloud)

Fig. 6 Map with patients as points and display of count of patients to be updated as user is moving toward one end to other end of the city

620

S. Janardhan Rao et al.

Fig. 7 Updating the count of patients when switch the place

Fig. 8 Performance of smart COVID simulation versus traditional and semi-automated approaches

Fig. 9 Accuracy of smart COVID simulation versus traditional and semi-automated approaches

Smart COVID Simulation App for Tracking …

621

In Fig. 8, the numerical percentiles are specified in Table I, and they are drawn in a graph where X-axis denotes kind of approach, and Y-axis denotes performance in %. In Fig. 9, the numerical percentiles are specified in Table II, and they are drawn in a graph where X-axis denotes kind of approach, and Y-axis denotes accuracy in %.

5 Conclusion The objective of this work is to showcase the count of COVID patients with respect to the regions. The dataset is provided by the hospital authorities over a cloud with access and permitted to translate each entry as a point over Google map, and hence, the COVID patients’ data is available on the COVID_Cloud dataset. The operations that could be supported are selection of regions and compute aggregation based on option chosen as low level or high level. Based on level, if it is lower, accumulate all points in the boundary of that region, otherwise, accumulate counts of sub-regions that fall within the selected, and display the counts dynamically. The dataset is dynamic in the sense it could grow for each new entry as well as shrink for each recovery based on severity of COVID. The visual displaying of the count of COVID patients over a map would help the strangers or users in taking the decision to delay or turn to visit. This would also be an information guide to authorized users of the app. In future, the kind of similar pandemic is supposed to be raised, and the studying of this work may help a lot to the mankind.

References 1. P. Tumuluru, S. Hrushikesava Raju, C.H.M.H. Sai Baba, S. Dorababu, B. Venkateswarlu, ECO friendly mask guide for corona prevention, in IOP conference series materials science and engineering, 981(2). https://doi.org/10.1088/1757-899X/981/2/022047 2. C. Jayalath, K.K.G.P. Somarathna, Covid-19 and informal labour in construction: A narrative analysis of webinar discussions, in 9th World Construction Symposium, WCS 2021 (2021), pp. 221–230 3. J.L. Souza, V.D. Teich, A.C.B. Dantas, D.T. Malheiro, M.A. Oliveira, E.S. Mello, M. Cendoroglo Neto, Impact of the COVID-19 pandemic on emergency department visits: experience of a Brazilian reference center. Einstein 19, eAO6467 (2021) 4. H. László, Mechanism of mucosal defence and options to reduce virus invasion during the COVID pandemic. Lege Artis Med. 31(7), 251–258 (2021) 5. E.D.E. Frecska, B. Petra, Neuropsychiatric complications of COVID-19 infection. Lege Artis Med. 31(7), 267–273 (2021) 6. L. Iván, D. Levente, B. Dániel, K. Tibor, Neurological complications of COVID-19. Lege Artis Med. 31(7), 259–264 (2021) 7. S. Nicolette, Face mask associated dry eye syndrome during the COVID-19 pandemic. Lege Artis Med. 31(7), 281–285 (2021)

622

S. Janardhan Rao et al.

8. K.J. Tibor, G. Andrea, S. László, Potential benefits of using glp1-receptor agonists during COVID-19 epidemic. Lege Artis Med. 31(7), 275–279 (2021) 9. V. Edit, Treatment options for localized and widespread post-covid pain. Lege Artis Med. 31(7), 287–293 (2021) 10. C.S. Karmakar, M.A. Hannan, M.S. Islam, D.K. Bhowmik, A.K.M. Akhtaruzzaman, Pulmonary hypertension following severe COVID-19: a case report. Anaesth. Pain Intensive Care 25(4), 539–543 (2021) 11. F. Mansoor, The impact of corona pandemic on critical care medicine. Anaesth. Pain Intensive Care 25(4), 420–423 (2021) 12. O.R. Ibrahim, T. Oloyede, H. Gbadamosi, Y. Musa, R. Aliu, S.O. Bello, et al., Acute kidney injury in COVID-19: A single-center experience in Nigeria. Anaesth. Pain Intensive Care 25(4), 470–477 (2021) 13. H.M.R. Karim, T.H. Khan, The COVID-19 babies are even more blue and lethal. Anaesth. Pain Intensive Care 25(4), 424–427 (2021) 14. Murphy, et al., A COVID-19 opportunity: applying a systems approach to food security and noncommunicable diseases (2020). https://doi.org/10.26633/RPSP.2020.84 15. Cassiani, et al., Lessons learned from the COVID-19 pandemic: why we need to invest in advanced practice nurses (2021). https://doi.org/10.26633/RPSP.2021.92 16. Fernández-Niño, et al., Multimorbidity patterns among COVID-19 deaths: proposal for the construction of etiological models (2020). https://doi.org/10.26633/RPSP.2020.166 17. Fantin, et al., COVID-19 deaths: distribution by age and universal medical coverage in 22 countries (2021). https://doi.org/10.26633/RPSP.2021.42 18. Garcia-Cerde, et al., Alcohol use during the COVID-19 pandemic in Latin America and the Caribbean (2021). https://doi.org/10.26633/RPSP.2021.52 19. Sallas, et al., Genomic surveillance of SARS-CoV-2 in response to the pandemic from COVID19 in Brazil (2021). https://doi.org/10.26633/RPSP.2021.75 20. Ramírez, et al., Dealing with perceptions related to thrombosis and COVID-19 vaccines (2021). https://doi.org/10.26633/RPSP.2021.45 21. S. Hrushikesava Raju, L.R. Burra, A. Koujalagi, S.F. Waris, Tourism enhancer app: Userfriendliness of a map with relevant features, in IOP Conference Series, Materials Science and Engineering, vol. 981, no. 2. https://doi.org/10.1088/1757-899X/981/2/022067 22. S. Hrushikesava Raju, L.R. Burra, S.F. Waris, S. Kavitha, S. Dorababu, Smart eye testing. Adv. Intell. Syst. Comput. (2021), ISCDA 2020, 1312 AISC, 173–181. https://doi.org/10.1007/978981-33-6176-8_19 23. S. Hrushikesava Raju, L.R. Burra, S.F. Waris, S. Kavitha, IoT as a health guide tool, in IOP conference series, materials science and engineering, vol. 981, no. 4. https://doi.org/10.1088/ 1757-899X/981/4/042015 24. R. Mothukuri, S. Hrushikesava Raju, S. Dorababu, S.F. Waris, Smart catcher of weighted objects smart catcher of weighted objects, in IOP conference series materials science and engineering, vol. 981, no. 2. https://doi.org/10.1088/1757-899X/981/2/022002

Automatic CAD System for Brain Diseases Classification Using CNN-LSTM Model Deipali Vikram Gore, Ashish Kumar Sinha, and Vivek Deshpande

Abstract Development portrayal is a troublesome investigation issue in the space of therapeutic science. The frontal cortex development is fundamentally fundamental and envisions a threat to life. We propose a novel and totally automated brain development portrayal CAD system using significant learning estimations. The implemented model involves steps like pre-planning, division, features planning, and portrayal. In division, we revolve around discovering the ROI of tainted regions using dynamic thresholding. We dealt with the pre-taken care of MR image to the generous CNN model for modified features extraction using the pre-arranged ResNet50 model. The isolated components are, moreover, lessened using the principal component analysis (PCA). We plan the long short-term memory (LSTM) classifier to vanquish the dissipating point issue. The vanishing point issue of neural association classifiers prompts request botches. Keywords Brain tumor · Convolutional neural network · Features extraction · Segmentation · Soft computing · LSTM

1 Introduction The frontal cortex development is perhaps the most unavoidable brain problem, so its assessment and medication have become huge [1]. Hurtful glioma is one of the ordinary kinds of fundamental brain malignancy, with a yearly event of approximately five cases for each 1,00,000 people every year [2]. Early area of the ailment before it starts to spread can be refined by chipping away at the capability of the biomedical imaging modalities used and growing prosperity care among people to see early

D. V. Gore (B) · A. K. Sinha Kalinga University, Raipur, India e-mail: [email protected] V. Deshpande Vishwakarma Institute of Information Technology, Pune, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_54

623

624

D. V. Gore et al.

contamination signs [3]. Cases of malignancy advancement risk factors are receptiveness to disease-causing substances genuinely, for instance, brilliant radiation in light and inborn and genetic effects [4]. Different circumstances incorporate natural and artificial ionizing radiations, exposure to infections, viruses, allergens, cellular telephones, radio frequency (RF) electromagnetic fields, urban air pollution, chemical dyes, etc. [5]. Models are magnetic resonance imaging MRI, positron emission tomography (PET), singlephoton emission computed tomography (SPECT), magnetic resonance spectroscopy (MRS), ultrasound, computed tomography (CT) [6]. Due to the benefits of MRI over other demonstrative imaging, most of investigation in picture characterization relate to utilization of MRI pictures [7–11]. Delicate and dependable methods of surveying the viability of different treatments in cerebrum cancer patients are significant for directing therapy choices in individual patients, deciding ideal treatment for explicit patient gatherings, and for assessing new treatments [12, 13]. The raw MR images are first pre-processed using effective and lightweight median filtering, and then we applied a thresholding-based ROI extraction algorithm on the pre-processed MR image. CNN extracts the automatic features from the ROI image using the ResNet50 deep learning model. These features extracted by the CNN model are of high dimension along with irrelevant features. We need relevant features from each input MR image by applying the PCA for features selection. LSTM as the classifier, performs the estimation of the probabilities for each class of training data and classifies the predicted brain tumor disease accurately. Section 2 presents the study of the related works. Section 3 presents the methodology of the implemented model. Section 4 presents the simulation results, and Sect. 5 presents the conclusion.

2 Related Works The emergence of automatic CAD systems using deep learning algorithms has received vital research interests in the recent past. The automatic techniques for feature extraction, costs, classification errors, and improves classification accuracy. This section presents a brief study of such methods. A. State-of-Art Authors have implemented a fully automatic system for brain tumor segmentation and detection of diseased tissue from the input MRI and fluid-attenuated inversion recovery (FLAIR) images in Ref. [14]. Another automatic brain tumor detection framework that used the novel threshold-based image segmentation algorithm is implemented in Ref. [15]. Authors have focused on conducting the review on various deep learning-based image segmentation techniques using brain MRI in Ref. [16]. In Ref. [17], the completely automated CAD system for brain disease detection using deep learning was implemented. An automated deep learning system to classify brain tumors is implemented in Ref. [18]. In

Automatic CAD System for Brain Diseases …

625

Ref. [19], authors have introduced another recent deep learning-based for the automatic brain tumor classification problem. In Ref. [20], authors have implemented the pre-processing mechanism to focus on small lesions of the MR image than a complete image. In Ref. [21], authors have implemented a multilevel features extraction technique followed by features concatenation for early brain tumor detection. In Ref. [22], authors have implemented automatic brain tumor detection and classification using deep learning features and different classifiers. The novel deep neural network (DNN) designed in Ref. [23] is called LU-Net. The review study on automatic brain tumor classification using different deep learning methods is presented in Refs. [24, 25]. The DCNN-fusion-support vector machine (DCNN-F-SVM) model is implemented in Ref. [26] for brain tumor classification. The deep wavelet autoencoder (DWA) had implemented in Ref. [27] for automatic brain tumor classification. The novel CNN-based multigrade CAD system implemented in Ref. [28] for brain tumor classification. In Ref. [29], the author focused on brain tumors multi-classification for the early detection using CNN. B. Motivation and Contributions However, deep learning-based ROI extraction leads to a high computational overhead and a lack of effective features for accurate disease grading. The features scaling and vanishing gradient problem is not discussed in any of the existing literature. By considering the above research problems, we implemented a novel framework for automatic brain tumor classification using computer vision techniques and integrated deep learning algorithms. The contributions of the implemented CAD model are: • The raw brain MR images are first pre-processed using the combination of contrast limited adaptive histogram equalization (CLAHE) and median filtering to enhance the quality. After this, the dynamic threshold-based image segmentation is performed to locate the ROI. • For automatic features engineering, we have designed the fast CNN model through the four layers to estimate the features from the segmented image. The model has further extended by applying the PCA for features selection and dimension reduction. • The LSTM applied on the test and train models for the automatic classification of brain diseases, it proves effective against the vanishing gradient problem. • We present the performance analysis of the implemented model with different classifiers and related work using the publicly available research dataset.

3 Implemented Methodology Figure 1 demonstrates the methodology of representing these contributions for automatic brain disease classification. In the pre-processing step, we performed the

626

D. V. Gore et al.

image quality improvement using combined filtering. In the adaptive ROI extraction phase, we aimed to accurately locate and extract the tumor-specific regions using the dynamic thresholding mechanism. Then we have designed deep learning model robust CNN with PCA for automatic features engineering from the segmented MR image. We delegate the task of feature extraction and selection to the CNN + PCA model to suppress the problems related to misclassification and vanishing gradient. For the automatic classification of brain tumor diseases, we have designed the LSTM layers that take the input from the CNN + PCA block and perform the classification. A. Image Pre-processing Let image I is test raw MR image as the input to the implemented system, we are first required to enhance the quality of the image. Input MR images are represented in low-contrast form, we applied the contrast limited adaptive histogram equalization (CLAHE). The CLAHE improves the contrast according to its dynamic nature of histogram equalization as: I 1 ← CLAHE (I )

(1)

Improving the contrast of raw MR images further introduces the components while balancing the image contrasts. To suppress such artifacts and noises, we applied the 2D median filtering technique using the optimal window size w (3

Fig. 1 Implemented automatic brain tumor classification model

Automatic CAD System for Brain Diseases …

627

Fig. 2 Original input brain MR image

× 3) as: I 2 ← MEDFILT(I 1, w)

(2)

Figures 2 and 3 are showing the original MR image and pre-processed MR image to demonstrate the effects of applying the above pre-processing operations. It shows that by using CLAHE and median filtering, we can improve the quality of MR images while suppressing the artifacts and noises. B. Image Segmentation For accurate tumor segmentation, we have exploited the adaptive thresholdbased image segmentation algorithm in this paper. In this paper, we divide the image segmentation and features extraction steps. For image segmentation, we have re-designed Otsu’s method where the image threshold value is dynamically generated first. The adaptive threshold for each image is computed as: T ←

(max(I 2) + mean(I 2)) γ

(3)

where γ is the scaling factor that is used to estimate the threshold value in range of 0–255. We set value of this factor to 2.5 according to the accuracy obtained by fine tuning. After discovering the threshold value for input image, we must segment the image into ROI and non-ROI parts as:

628

D. V. Gore et al.

Fig. 3 Pre-processed brain MR image

 I3 ←

I 2(i, j ), I 2(i, j ) ≥ T 0, Otherwise

(4)

where I 3 represents the ROI image that shows the actual tumor regions and the rest of all parts represented as black. The sample outcome of this segmentation approach for the input MR image show in Fig. 2 is demonstrated in Fig. 4. C. Automatic Features Engineering We aim to automatically extract the features of the ROI image with minimum computational requirement. We designed the robust CNN layers to load the preprocessed ROI image and automatically learn its features using the pre-trained ResNet50 model. The CNN layers consist of layers such as the input layer, convolutional layer (Conv), batch normalization layer (BNL), rectified linear unit (ReLU) layer, and max pooling layer (MPL). The layers of a total of four produced the features for the input image. We start with the eight filters of size 15 × 15 and end up with 64 filters of size 15 × 15 as showing in Table 1. Each layer in the implemented CNN consists of sublayers such as the: • • • • •

Input image layer that loads the image of size 250 × 250 Conv(.) layer: with 8–64 filters of size 15 × 15. Batch Normalization Layer (BNL), Rectified Linear Unit (ReLU) layer, and Max Pooling Layer (MPL).

Automatic CAD System for Brain Diseases …

629

Fig. 4 Segmented MR image

To address the parameter explosion problem, we have applied batch normalization and ReLU after the convolutional filter. The third max-pooling layer generates the automated features vector for further proceedings. The implemented CNN takes input I3 and perform the consolidated one squashing function as per design of each layer as:       l F jl = tanh poolmax ReLU x l−1 (5) j (I 3) ∗ ki j + b j i

where F jl is outcome of CNN features extraction model as features set using convolutional layer l of jth input, x l−1 represents the previous convolutional layer features j maps of I 3, ki j represents ith trained convolutional kernels, and blj represents the additive bias. The function tanh(.) represents the activation function, poolmax (.) represents the operation of max pooling for features extraction, and ReLU(.) represents the operation of ReLU layer. As the extracted features are high dimensional and redundant, we applied the PCA to get the reduced CNN features as: F PCA ← PCA(F)

(6)

D. LSTM Classification The LSTM classification layer is applied to that features for automatic classification into one of the classes of the pre-trained dataset. The LSTM takes input F PCA features and performs the classification. The architecture of LSTM consists of input

630

D. V. Gore et al.

Table 1 CNN Layers for features extraction Number

Layer parameters

Significance

[250, 250, 1]

Input image layer (CNN) of size 250 × 250

Conv ([15 15], 8, ‘Padding’, 1)

8 filters of 15 × 15 size convolutions filters with stride [0 0] and padding [1 1]

batchNormalizationLayer

Mini-batch normalization layer

ReLU

ReLU layer operations

maxpool([8,8], ‘stride’ 2)

8 × 8 max pooling with stride [2 2] and padding [0 0]

Conv ([15 15], 16, ‘Padding’, 1)

16 filters of 15 × 15 size convolutions filters with stride [0 0] and padding [1 1]

batchNormalizationLayer

Mini-batch normalization layer

ReLU

ReLU layer operations

maxpool([8,8], ‘stride’ 2)

8 × 8 max pooling with stride [2 2] and padding [0 0]

Conv ([15 15], 32, ‘Padding’, 1)

32 filters of 15 × 15 size convolutions filters with stride [0 0] and padding [1 1]

batchNormalizationLayer

Mini-batch normalization layer

ReLU

ReLU layer operations

maxpool([8,8], ‘stride’ 2)

8 × 8 max pooling with stride [2 2] and padding [0 0]

Conv ([15 15], 64, ‘Padding’, 1)

64 filters of 15 × 15 size convolutions filters with stride [0 0] and padding [1 1]

batchNormalizationLayer

Mini-batch normalization layer

ReLU

ReLU layer operations

maxpool([8,8], ‘stride’ 2)

8 × 8 max pooling with stride [2 2] and padding [0 0]

gate i, output gate o, and forget gate f , and a memory cell c. For every time t, LSTM computes its gate’s activations {i t , f t } and updates its memory cell from ct−1 to ct , it then computes the output gate activation ot , and finally outputs a hidden representation h t . The hidden representation from the previous time step is ht − 1. Following equations applied in LSTM for update functions:   i t = σ Wi F PCA + Ui h t−1 + Vi ct−1 + bi

(7)

  f t = σ W f F PCA + U f h t−1 + Vi ct−1 + b f

(8)

  ct = f t ct−1 + i t tanh Wc F PCA + Uc h t−1 + Uc h t−1

(9)

  ot = σ Wo F PCA + Uo h t−1 + Vi ct−1 + bo

(10)

Automatic CAD System for Brain Diseases …

631

h t = ot  tanh(ct )

(11)

where  an element-wise product and σ is the logistic function and tanh are applied elementwise. W∗ , V∗ , U∗ , and b∗ are the parameters, further the weight matrices V∗ are diagonal. For performance analysis, the dataset divided into 70% (training) and 30% (testing and validation) for each machine learning classifier. Performance evaluations of the implemented model using each classifier, we computed the metrics such as precision, recall, accuracy, specificity, and F1-score. Precision = Recall = Accuracy =

tp tp + f p

tp tp + f n

(12) (13)

t p + tn t p + tn + f p + f n

(14)

tn tn + f p

(15)

Specificity = F1 - score = 2 ×

Precision × Recall Precision + Recall

(16)

where t p stands for true positive, f p stands for false positive, f n stands for false negative, and f p false positive.

4 Simulation Results We have implemented the implemented model using the MATALB tool under the I5 processor, 4 GB RAM, and Windows 10 OS. For performance analysis, we have used the MRI brain tumor dataset mentioned in [30]. This dataset is scalable and having total 3064 samples from 233 subjects. It consists of three types of brain tumors such as glioma (1426), meningioma (708), and pituitary tumor (930). A. Analysis of Classifiers Table 2 demonstrates the outcomes of brain tumor disease classification in terms of accuracy, precision, recall, specificity, and F1-score parameters, respectively. The CNN-PCA has combined with different classifiers such as ANN, SVM, KNN, and LSTM. LSTM achieved the higher classification accuracy, precision, recall, specificity, and F1-score. LSTM suppress the challenges related to vanishing gradient and minimum error rates during classification. On another side, the impact of applying

632

D. V. Gore et al.

Table 2 Comparative result based on classifiers and feature selection technique Performance Metrics

Artificial Neural Network (ANN)

Support Vector Machine (SVM)

K Nearest Neighbor (KNN)

Long Short –Term Memory (LSTM)

CNN + PCA

CNN

CNN + PCA

CNN

CNN + PCA

CNN

CNN + PCA

Accuracy (%) 90.17

92.52

91.86

93.65

86.78

88.94

94.77

98.11

Precision (%)

86.34

89.85

80.22

90.56

83.78

86.82

95.21

98.12

Recall (%)

91.79

93.47

89.46

88.98

89.04

91.49

94.97

96.68

Specificity (%)

88.93

91.78

88.86

92.99

84.78

87.98

94.50

97.50

F1-Score (%)

88.98

91.62

84.59

91.97

86.33

89.09

95.08

97.39

CNN

PCA on the CNN features proved the performance improvement over the highdimensional CNN features. Applying PCA not only discards the irrelevant features but also reduces the classification/training time. B. State-of-art Analysis This section presents the comparative analysis of an implemented model for brain tumor disease classification with state-of-art techniques. We have selected the different deep learning-based similar methods such as Rai et al. [23], Wu et al. [26], Mallick et al. [27], Sajjad et al. [28], and Irmak et al. [29]. We have estimated the average accuracy and average classification time using these methods via 20 executions. Table 3 demonstrates the comparative results. The implemented model has improved the overall brain tumor classification accuracy compared to all the state-of-art techniques and reduced the classification time as well. The major reason behind this improvement is separating the image segmentation from the features extraction using deep learning in the implemented CAD system. Table 3 State-of-art methods analysis

Methods

Accuracy (%)

Classification Time (Seconds)

Rai et al. [28]

96.23

1438

Wu et al. [31]

95.82

1357

Mallick et al. [32]

94.88

1561

Sajjad et al. [33]

94.57

1478

Irmak et al. [34]

96.65

1677

Implemented Model

97.11

1213

Automatic CAD System for Brain Diseases …

633

5 Conclusion and Future Work The outcome of adaptive segmentation was the segmented ROI of the tumor image. The ROI image is fed as input to the implemented CNN model for automatic features extraction followed by PCA for features reduction. For the classification, we applied the LSTM classifier. The simulation result shows that the implemented model has improved the performance of classification accuracy using the LSTM as classifier and CNN as the features extractor. CNN-PCA with LSTM outperformed existing methods in terms of accuracy and classification time. In the future, we plan to extend this model by applying the other deep learning classifier.

References 1. E.J. Rushing, WHO classification of tumors of the nervous system: preview of the upcoming 5th edition. Memo 14, 188–191 (2021). https://doi.org/10.1007/s12254-021-00680-x 2. D.N. Louis, A. Perry, G. Reifenberger et al., The 2016 world health organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131, 803–820 (2016). https://doi.org/10.1007/s00401-016-1545-1 3. Ö. Yapıcıer, Pathology, epidemiology, and WHO classification of brain tumors, in Atlas of clinical cases on brain tumor imaging, ed. by Y. Özsunar, U. Senol ¸ (Springer, Cham, 2020). https://doi.org/10.1007/978-3-030-23273-3_1 4. R. Banan, C. Hartmann, The new WHO 2016 classification of brain tumors—what neurosurgeons need to know. Acta Neurochir. 159, 403–418 (2017). https://doi.org/10.1007/s00701016-3062-3 5. B. Menze et al., The Multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015) 6. N. Gordillo, E. Montseny, P. Sobrevilla, State of the art survey on MRI brain tumor segmentation. Magn. Reson. Imaging 31(8), 1426–1438 (2013) 7. D. Bhattacharyya, T. Kim, Brain tumor detection using MRI image analysis, in Ubiquitous computing and multimedia applications. UCMA 2011, ed. by T. Kim, H. Adeli, R.J. Robles, M. Balitanas. Communications in Computer and Information Science, vol. 151 (Springer, Berlin, Heidelberg, 2011). https://doi.org/10.1007/978-3-642-20998-7_38 8. M.A.A. Hamid, N.A. Khan, Investigation and classification of MRI brain tumors using feature extraction technique. J. Med. Biol. Eng. 40, 307–317 (2020). https://doi.org/10.1007/s40846020-00510-1 9. N. Varuna Shree, T.N.R. Kumar, Identification and classification of brain tumor MRI images with feature extraction using DWT and probabilistic neural network. Brain Inf. 5, 23–30 (2018). https://doi.org/10.1007/s40708-017-0075-5 10. H. Jalab, A. Hasan, Magnetic resonance imaging segmentation techniques of brain tumors: a review. J. Neurosci. (2019) 11. K. Venu, P. Natesan, N. Sasipriyaa, S. Poorani, Review on brain tumor segmentation methods using convolution neural network for MRI images, in: 2018 International conference on intelligent computing and communication for smart world (I2C2SW) (2018). https://doi.org/10.1109/ i2c2sw45816.2018.8997387 12. R. Parvataneni, M.Y. Polley, T. Freeman et al., Identifying the needs of brain tumor patients and their caregivers. J. Neurooncol. 104, 737–744 (2011). https://doi.org/10.1007/s11060-0110534-4 13. M. Rahman, I. Parney, Journal of neuro oncology: immunotherapy for brain tumors. J. Neurooncol. 151, 1 (2021). https://doi.org/10.1007/s11060-020-03589-3

634

D. V. Gore et al.

14. M. Soltaninejad, G. Yang, T. Lambrou et al., Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI. Int. J. CARS 12, 183–203 (2017). https://doi.org/10.1007/s11548-016-1483-3 15. A. Edalati-rad, M. Mosleh, Improving brain tumor diagnosis using MRI segmentation based on collaboration of beta mixture model and learning automata. Arab J. Sci. Eng. 44, 2945–2957 (2019). https://doi.org/10.1007/s13369-018-3320-1 16. Z. Akkus, A. Galimzianova, A. Hoogi et al., Deep learning for brain MRI segmentation: state of the art and future directions. J. Digit. Imaging 30, 449–459 (2017). https://doi.org/10.1007/ s10278-017-9983-4 17. R.A. Zeineldin, M.E. Karar, J. Coburger et al., DeepSeg: deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images. Int. J. CARS 15, 909–920 (2020). https://doi.org/10.1007/s11548-020-02186-z 18. M.I. Sharif, M.A. Khan, M. Alhussein et al., A decision support system for multimodal brain tumor classification using deep learning. Complex Intell. Syst. (2021). https://doi.org/10.1007/ s40747-021-00321-0 19. S. Kokkalla, J. Kakarla, I.B. Venkateswarlu et al., Three-class brain tumor classification using deep dense inception residual network. Soft. Comput. 25, 8721–8729 (2021). https://doi.org/ 10.1007/s00500-021-05748-8 20. R. Ranjbarzadeh, A. Bagherian Kasgari, S. Jafarzadeh Ghoushchi et al., Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images. Sci. Rep. 11, 10930 (2021). https://doi.org/10.1038/s41598-021-90428-8 21. N. Noreen, S. Palaniappan, A. Qayyum, I. Ahmad, M. Imran, M. Shoaib, A deep learning model based on concatenation approach for the diagnosis of brain tumor. IEEE Access, 1–1 (2020) https://doi.org/10.1109/access.2020.2978629 22. J. Kang, Z. Ullah, J. Gwak, MRI-based brain tumor classification using ensemble of deep features and machine learning classifiers. Sensors 21(6), 2222 (2021). https://doi.org/10.3390/ s21062222 23. H.M. Rai, K. Chatterjee, Detection of brain abnormality by a novel Lu-Net deep neural CNN model from MR images. Mach. Learn. Appl. 2, 100004 (2020). https://doi.org/10.1016/j.mlwa. 2020.100004 24. A. Zadeh Shirazi, E. Fornaciari, M.D. McDonnell, M. Yaghoobi, Y. Cevallos, L. Tello-Oquendo, D. Inca, G.A. Gomez, The application of deep convolutional neural networks to brain cancer images: a survey. J. Pers. Med. 10(4), 224 (2020). https://doi.org/10.3390/jpm10040224 25. S.A.Y. Al-Galal, I.F.T. Alshaikhli, M. Abdulrazzaq, MRI brain tumor medical images analysis using deep learning techniques: a systematic review. Health Technol. 11, 267–282 (2021). https://doi.org/10.1007/s12553-020-00514-6 26. W. Wu, D. Li, J. Du, X. Gao, W. Gu, F. Zhao, … H. Yan, An intelligent diagnosis method of brain MRI tumor segmentation using deep convolutional neural network and SVM algorithm. Comput. Math. Methods Med. 2020, 1–10 (2020).https://doi.org/10.1155/2020/6789306 27. P.K. Mallick, S.H. Ryu, S.K. Satapathy, S. Mishra, N.G. Nguyen, P. Tiwari, Brain MRI image classification for cancer detection using deep wavelet autoencoder based deep neural network. IEEE Access, 1–1 (2019).https://doi.org/10.1109/access.2019.2902252 28. M. Sajjad, S. Khan, K. Muhammad, W. Wu, A. Ullah, S.W. Baik, Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J. Comput. Sci. (2018). https:// doi.org/10.1016/j.jocs.2018.12.003 29. E. Irmak, Multi-classification of brain tumor MRI images using deep convolutional neural network with fully optimized framework. Iran. J. Sci. Technol. Trans. Electr. Eng. 45(3), 1015– 1036 (2021). https://doi.org/10.1007/s40998-021-00426-9 30. J. Cheng, W. Huang, S. Cao, R. Yang, W. Yang, Z. Yun, … Q. Feng, Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLOS ONE, 10(10), e0140381 (2015). https://doi.org/10.1371/journal.pone.0140381

Implementation of E2EE Using Python Suprit Dusane, Md Iliyaz, and A. Suresh

Abstract The need for cyber security is felt now more than ever. In an era of constant mass surveillance, illegal spying and cyber-attacks, it is extremely important to have a secure means of communication which cannot be intercepted. Several new protocols were introduced such as HTTPS to encrypt the connection between the client and the server. This made sure that no third person can read the data being transmitted to and from the client. This model of encryption had one major flaw: the server itself. Every message that was encrypted by the sender was decrypted at the server, encrypted again and sent to the receiver. Thus the server can read all the messages. The users of such chatting services had to trust the owners of the services with their privacy. Even if the owners were not involved in shady data deals, they still had the risk of their servers getting hacked or being pressured by the government to reveal the data of their users. All these issues paved the way for a new type of implementation of encryption known as end-to-end encryption often abbreviated as E2EE. The message to be sent is encrypted by the sender and is sent to the server which relays it to the receiver as it is. Since the keys used to encrypt and decrypt the data are available only to the users, the server cannot read the messages sent through it. This model quickly gained popularity and was implemented by many messaging applications, the most notable being WhatsApp, Signal, Telegram, and Wire. In this project, we are going to implement E2EE using Python. For encryption, we intend to use the AES algorithm. AES stands for advanced encryption standard which was introduced in 2001 by the NITS (U.S.A.). It was developed by Vincent Rijmen and Joan Daemen in response to the earlier broken algorithm DES. AES is a symmetric key encryption algorithm meaning that the same key is used for encryption as well as decryption of the messages. This algorithm has three key lengths—128,192 and 256 bits, whereas S. Dusane · M. Iliyaz · A. Suresh (B) Department of Networking and Communications, School of Computing, SRM Institute of Science and Technology, SRM Nagar, Kattankulathur, Chengalpattu, Tamil Nadu 603203, India e-mail: [email protected]; [email protected] S. Dusane e-mail: [email protected] M. Iliyaz e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_55

635

636

S. Dusane et al.

a single block size of 128 bits. The version we are going to implement is 128 bits key size. Keywords End-to-end encryption · AES · Python · Libraries · DES

1 Introduction During the last 20 years or so, there has been much research and development in the field of secured communication, and many applications have been built on the concept. Many of them are proprietary software meaning that the source code isn’t made public and thus the user cannot evaluate the software. Thus the users have to trust the corporations to safeguard their privacy which doesn’t help build confidence. We aim to create an end-to-end encrypted messaging application that would be free and open source software (FOSS). Thus, we have created the code from the very basics of python programming. To have a better idea about how AES implementation actually works and how secure it is, several research papers have been reviewed. The project aims to create an end-to-end encryption software coded in Python in an intuitive way and to explain the working in such a way that anyone with basic knowledge of programming can understand the code. Any devices running on Linux, Windows or Mac can be used to test the code. To run the code, some software need to be preinstalled on the both the client as well as server such as Python, any Python IDE (preferably Pycharm), Python libraries such as OS, time, cryptography, tkinter, socket, and threading. Use of virtual machines or local host is recommended for testing on local networks. This project can further be improved or forked to add new features and can be distributed freely.

2 Literature Review Several research papers and other sources of information have been reviewed and analyzed for a thorough understanding of the relevant topics and concepts. The most notable papers have been discussed below: The general public is not much informed about the topic. When educated about the topic they realized it was useful and it changed their perspective about encryption and privacy. But to create awareness about the topics, various steps need to be taken. Thus we decided that we would write a code on the implementation of E2EE and explain its working in an easy manner which would encourage people to appreciate the concept and learn more about it [1]. Many of the so-called end-to-end encrypted messaging apps don’t really show how the data is actually being handled. Thus the users cannot verify if the implementation is being done correctly or if the encryption is even done in the first place. The proprietary nature of these apps do not instill confidence among the users. Thus it would be beneficial for us to build our app from scratch [2]. The most famous

Implementation of E2EE Using Python

637

end-to-end encrypted chatting app available out there across multiple platforms is the WhatsApp Messenger. Due to recent revelations by some captured criminals that the use of E2EE is widespread among criminal networks, various governments have been worried. They have long pressured WhatsApp to create a backdoor in its app solely for the governments. While this may seem intuitive, this proposal poses far bigger threats to individual privacy [3]. While the concept and the mathematics behind end-to-end encryption is quite strong, such apps do have a vulnerability. The weakness lies not in the algorithm but in its implementation. Faulty implementation leads to metadata leaks which is of great concern as it does reveal a lot about the user, like the type of message, frequency, IP, location, etc. Thus, it is crucial to have a fool-proof implementation [4]. One of the most popular forms of communication is short messaging service (SMS) which is a service provided by the mobile service provider. This service unfortunately does not use any type of encryption. SMS is used by the banking sector as well as other services that send text-based OTPs. Thus, it is advisable to avoid using SMS until they become encrypted [5]. Email is the oldest form of digital communication. Email was initially used without any encryption but nowadays many services like ProtonMail offer the encrypted version. Since emails are used in all fields, it is always a good idea to opt for the more secure one. But according to the survey conducted, not many people know about the existence of these technologies [6]. Since various encryption algorithms are available, and due to the broken (cryptographically) nature of some like the data encryption standard (DES) and the famous MD5, many alternatives were developed and analyzed such has Blowfish, 3DES, AES, and AES-RSA hybrid which proved to be the best among the given alternatives [7, 8]. Since the awareness for encryption has increased so has the use of https protocol. Many chatting Web sites have started using https to secure the connection between the server and the client. But this isn’t what E2EE is. This paved the way for modern chatting apps like WhatsApp Messenger, Signal, Threema, Wire, and Telegram [7]. Since most of the E2EE messaging apps are proprietary, their security is not analyzed by independent researchers, and this has led to the widespread use of Signal because it is an open source app [9]. Since the adoption of E2EE, the government has always been against it under the pretext of criminals abusing the technology. While this may be partially true, under no circumstances should the government be able to break the encryption. To counter hoaxes and other viral fake news, WhatsApp has made a feature to show if the exact same message is being forwarded too many times. This accomplishes both the goals of countering hoaxes as well as safeguarding the privacy of its users [10].

638

S. Dusane et al.

3 Methodology To implement and analyze the working of AES encryption in Python, we have adopted experimental methodology. We have designed a code entirely written in Python which consists of various modules, and it’s shown in the Fig. 1. The entire project is divided into two parts, namely the ‘server side’ and the ‘client side’. The entire code is explained in the following sections.

3.1 Server Side The server is set up by creating a socket with the IP address of the server and a port number of choice. A limit for the maximum number of users that can connect to the server is set. An empty list of clients is created. This function sends the message it passed to all the clients present in the list of clients. This module receives the message from the client and passes it to the broadcast function. It also attempts to display the message that is being sent which is in encrypted form and thus of no use. This demonstrates that the project is indeed a correct implementation of end-to-end encryption since the server is unable to retrieve the actual message that the users are exchanging. This module is the exception handling part which is very much important but often neglected. If the server is unable to communicate with a client, it immediately terminates the connection and removes the client from the list to avoid getting the same error again. This prevents the server from crashing or any undesirable consequences. This final part of the code adds clients to the list created earlier and starts a process to handle the client simultaneously with other clients in a list. The concept of threading is used here which works very well in python. The server also displays the IP and port numbers of the clients connected as well as acknowledges successful connections to the client.

Fig. 1 Data flow diagram

Implementation of E2EE Using Python

639

3.2 Client Side All the important libraries are imported: Socket—creating connections with a machine with a specified IP and port number. Threading—to run various processes simultaneously. Time—to keep track of time on the machine. Tkinter—this forms the GUI for the project. Cryptography—this is the main library that contains Fernet which is an implementation of 128bit AES encryption in Python. This is the GUI part of the code which has a window that consists of the following elements: IP field: An input box to type the IP address of the server. Port field: An input box to type the port number of the server. Name field: An input box to type the name of the user. Send button: Passes the above arguments to the ‘connect’ function. Text box: An output box where all the messages sent and received are displayed. Send field: An input box where the message to be sent is typed. Send button: A button that triggers the ‘send’ function with the above argument. This function encrypts the message to be sent with the key the user already has and then sends it to the server with appropriate encoding (ASCII or utf-8), utf-8 in this case, since no encoding is specified. This function connects the client to the server as per the credentials provided by the user. It waits for the server to acknowledge a successful connection and shows it to the user. The credentials boxes (IP, port number, and name) are locked to avoid unnecessary modifications. A threading process is started which receives and decrypts the messages received and displays it to the user in the output box. The output box is generally kept locked after each update to avoid editing the chat history. The system sleeps for a second to avoid rapid interactions that might cause the system to crash.

3.3 Key Generator The following code is executed to generate a 128 bit AES key and store it in a file. The same key should be used by every user and should never be stored on the server as it would defeat the whole point of E2EE.

4 Results Both the codes work perfectly fine and the messages are received without any distortion. The encryption and decryption processes work seamlessly. The server just relays the messages in the encrypted form and cannot read the actual messages due to lack

640

S. Dusane et al.

of key used by the users. The names of the users are visible only to the user and not to the server. The only data that is visible to the server is the IP address of the users and the port by which they connect. This data can easily be hidden by using TOR or VPN. Connections of more clients and server console shown in the Figs. 2, 3, and 4. Client1: See Fig. 2. Client2: See Fig. 3.

Fig. 2 Connection of Client 1

Fig. 3 Connection of Client 2

Implementation of E2EE Using Python

641

Fig. 4 Server console

Server: See Fig. 4. Note how the messages ‘hey there’ and ‘hello’ look the encrypted form. The key used in the above example QNe8yUUFa6InzoAAwk6errGkadb85P_VQhjiTg2vXns=

in is:

5 Conclusion Thus, we infer several conclusions from the literature review as well as the implementation done by us. The concept of end-to-end encryption is an important one but relatively less people are aware about it. The more people understand the concept and review FOSS, it improves the quality of such apps, one of them being this code. AES encryption is fairly easy to implement on Python due to the ‘cryptography’ library and the ‘Fernet’ module. Python remains one of the most versatile languages due to its integration with various other syntaxes and concepts and a huge library of functions. AES key can easily be generated using the Fernet module. Due to the self-sufficient nature of the modules, the project can easily be forked and developed without the need to reinvent the wheel.

References 1. W. Bai, M. Pearson, P. G. Kelley and M. L. Mazurek,Improving non-experts’ understanding of end-to-end encryption: an exploratory study, in 2020 IEEE european symposium on security and privacy workshops (EuroS&PW), pp. 210–219 (2020).https://doi.org/10.1109/EuroSPW51 379.2020.00036

642

S. Dusane et al.

2. F. Schillinger, C. Schindelhauer, End-to-End encryption schemes for online social networks, in Security, privacy, and anonymity in computation, communication, and storage. SpaCCS 2019, ed. by G. Wang, J. Feng, M. Bhuiyan, R. Lu. Lecture Notes in Computer Science, vol. 11611 (Springer, Cham, 2019). https://doi.org/10.1007/978-3-030-24907-6_11 3. R.E. Endeley, End-to-End encryption in messaging services and national security-case of whatsapp messenger. J. Inf. Secur. 9, 95–99 (2018). https://doi.org/10.4236/jis.2018.91008 4. M. Nabeel,The many faces of end-to-end encryption and their security analysis, in 2017 IEEE international conference on edge computing (EDGE), pp. 252–259 (2017).https://doi.org/10. 1109/IEEE.EDGE.2017.47 5. E. Ekwonwune, V. Enyinnaya, Design and implementation of end to end encrypted short message service (SMS) using hybrid cipher algorithm. J. Softw. Eng. Appl. 13, 25–40 (2020). https://doi.org/10.4236/jsea.2020.133003 6. R. Adrian, A. Ahmed, B. Karima, W. Marco, Usability of end-to-end encryption in e-mail communication. Front. Big Data https://doi.org/10.3389/fdata.2021.568284. ISSN 2624-909X 7. V. Krapa, S. Prayla Shyry, M. Rahul Sai Krishna, WhatsApp encryption—a research. Int. J. Recent Technol. Eng. (IJRTE) 8(2S3) (2019). ISSN: 2277–3878 8. O.O. Blaise, O. Awodele, O. Yewande, An Understanding and Perspectives of End-To-End Encryption.Int. Res. J. Eng. Technol. (IRJET) 08(04), 1086 (2021). www.irjet.net. e-ISSN: 2395-0056, p-ISSN: 2395-0072. © 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal 9. A.M. Espinoza, W.J. Tolley, J.R. Crandall, Alice and Bob, who the FOCI are they?: Analysis of end-to-end encryption in the LINE messaging application 10. J.C.S. Reis, P. Melo, K. Garimella, Detecting misinformation on whatsapp without breaking encryption

Study of Spike Glycoprotein Motifs in Coronavirus Infecting Animals and Variants of SARS-CoV-2 Observed in Humans Across Countries Akhbar Sha and Manjusha Nair

Abstract The greatest threat the world currently faces is due to the COVID-19 pandemic and its adverse effects. This in turn has obtained greater support in research and study on this field with the aim of a better tomorrow. Due to the large-scale spread of COVID-19 which in turn caused high possibility of mutations in this virus prompted us to conduct a study on the spike glycoprotein sequence of this highly debated organism. This study is conducted on two aspects: first on the spike glycoprotein sequences of coronaviruses infecting animals based on association with humans and the second on variants of SARS-CoV-2 based on geographic location of the sequences collected. Coronavirus is considered to be originated in bats and reached humans through unknown sources. We extend this possibility by conducting studies on the spike glycoprotein of coronaviruses that infect animals having some association with humans directly or indirectly as well as to provide better insights into the different mutations that had occurred to the SARS-CoV-2 as it spread through countries. The most similar organisms sharing a significant motif “KRSFIEDLLFNKV” of spike glycoprotein in our study are coronaviruses found in bats and cat. From the current study of mutations in the surface glycoprotein domain of SARS-CoV-2 observed in samples collected from 15 different countries, the amino acid present at 613th position was found to have the most stable mutation. The computational study detailed here provides better insights to the possible origins and transmission of SARS-CoV-2 viruses. Keywords SARS-COV-2 · Spike glycoprotein · COVID-19 · Coronavirus · Motif discovery

A. Sha · M. Nair (B) Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_56

643

644

A. Sha and M. Nair

1 Introduction The world around us have been severely affected by the effects of the COVID-19 pandemic. Government and organizations are trying their best to improve the performance of vaccines and therapeutics that could be best effective to fight this virus. Nations around the globe have reported about 5.2 million deaths (from beginning of the pandemic till November 29, 2021) classified as “COVID-19 Deaths”, and the numbers are increasing day by day. Many diagnosing techniques using the concept of deep learning, image processing, etc., [1, 2] are used nowadays for the early detection of abnormalities caused by this virus. Studies on approaches to track the virus across different regions are also being conducted [3]. RNA family of viruses has greater rates of mutation and better adaptability to the environment through evolution, which further increases the difficulty in predicting the history (i.e., the origins and transmission to human) of COVID-19. The coronavirus gets its name due to the presence of crown like protrusions from its protein membrane. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the organism responsible to cause the disease 2019 novel coronavirus (COVID19). This virus belongs to the family Coronaviridae, to the order Nidovirales. The viral genome of SARS-CoV-2 has 5 and 3 terminal and the 5 end is a major part of the genome containing open reading frames which helps in encoding proteins responsible for viral reproduction. Coronaviruses can belong to 4 genera of alphacoronavirus, beta-coronavirus, gamma-coronavirus, or delta-coronavirus [4]. SARSCoV-2 genome is a Beta-coronavirus and it closely resembles the SARS-CoV [5]. There are mainly six coronaviruses which are thought to affect humans. alpha-CoVs HCoV-229E and HCoV-NL63 has low ability to cause an infection, whereas betaCoVs HCoV-HKU1 and HCoV-OC43 causes mild respiratory illness which is very similar to the common cold. SARS-CoV and MERS-CoV, which are beta forms, can lead to severe or even fatal respiratory tract infections [6]. The SARS-CoV-2 virus emerged into the human population in December 2019. These viruses are positive stranded RNA viruses and have the largest genome among all RNA viruses which is usually 27–32 kB in length [7]. Glycoproteins are protein sequences that have carbohydrate groups attached to protein counterparts. All coronavirus synthesize spike(S) glycoproteins which protrude out from their surface as spikes. The S-glycoprotein present in coronaviruses is significant sequences because it regulates the cross-species transmission of the virus to other organisms. This protein mediates the fusion of the virus and other organism’s cell membrane. It also helps in recognizing the host’s cellular receptors [8]. The glycoprotein of SARS-CoV-2 spike is a 1273 amino acid polyprotein precursor on the rough endoplasmic reticulum (RER) [9]. Mature S-glycoprotein on the viral surface is a heavily glycosylated trimer. Each of the protomer of which is composed of 1260 amino acids. There are 672 amino acids on the surface subunit S1 and organized into four domains: an N-terminal domain (NTD), a C-terminal domain (CTD), and two subdomains (SD1 and SD2). There are 588 amino acids in the transmembrane S2 subunit and contain an N-terminal hydrophobic fusion peptide (FP), two heptad

Study of Spike Glycoprotein Motifs in Coronavirus Infecting …

645

repeats (HR1 and HR2), a transmembrane domain (TM), and a cytoplasmic tail (CT), arranged as FP-HR1-HR2-TM-CT (34) [10]. Variants of concern or VOCs are strains of virus that had undergone proper changes with respect to the reference genome and is commonly found in large clusters, having high transmission rates or exhibiting immunity towards vaccines and therapeutics. Researches have been done in relation to understanding the probability distribution of many factors associated with the disease [11]. Currently, WHO has identified four VOCs (alpha, beta, gamma, and delta) in the case of SARS-CoV-2. Most of the mutations were observed in the S protein sequence, and this is a major concern as S protein mutations can adversely affect the action of vaccines. We have very limited knowledge of the evolutionary backgrounds of this virus. Possible routes of transmission, its evolutionary chain, and various diversities of this virus and questions on its natural reservoir are still under debate. Recombination, mutation, and reassortment are the main aids in helping the viruses adjust very well to its new host environment. The closest relative to SARS-CoV-2 has been identified in China’s Wuhan province in a horseshoe bat called Rhinolophus affinis. The sequence similarities between the SARS-CoV-2 with this virus were estimated to be 96.2% identical, whereas it shares 79.5% identity to SARS-CoV, and thus the emergence of this virus was confirmed to be from the bat named R. affinis [12]. Coronaviruses are very common among mammals and birds. Bats constitute the main reservoir of coronaviruses, as they provide good support to drive evolutionary and ecological well-being of coronavirus species [13]. Animals such as pigs, dogs, cats, chicken, cows, etc., are also largely affected by coronavirus diseases. Cases which were reported in Wuhan, China were thought to have originated from the seafood wholesale market in Huanan, which can be considered as a zoonotic source since they sold poultry, snakes, bats, and other animals. Currently, there is no evidence pointing to the fact that SARS-CoV-2 originated from the seafood market, but they could have come from bats, which are thought to be a major host to coronaviruses. Also, protein sequences alignment and phylogenetic analysis [9] showed that similar residues of receptor were identified in many organisms like turtles, pangolin, and snakes which in turn increased the possibility of alternative intermediate hosts. Our main objective in this study is to identify patterns in spike glycoprotein of coronaviruses affecting animals which are proximal to humans. This could help understand the various mutations responsible for the change in amino acids, similarities of sequences, and help solve the issues related with the intermediate transmission of SARS-CoV-2 to humans. In a later part of our current study, we are analysing the sequences of potentially important coronaviruses that resembles SARS-CoV-2 in terms similarity of protein motifs. The organisms that have close association with humans like domesticated, ones that humans come frequently in contact with directly or indirectly are given more importance in this work. Protein motifs are small windows of amino acid sequences which are similar or slightly mutated and can be observed in different other proteins. Motif discovery can be described as “the problem of discovering of motifs without any prior knowledge of how the motifs look.” [14]. In the past few years, the scope of using improved meth-

646

A. Sha and M. Nair

ods to identify motif sequences has gained great research interests. There are many ways available across the literature comparing the working of different algorithms like genetic algorithm, using regular expressions [15], various enumerative, and probabilistic and nature-inspired models [16]. Algorithms, such as greedy method which we used here in this study, use probability to capture the motifs.

2 Methodology In order to understand the mutations and similarities occurred in the surface glycoprotein sequence of SARS-CoV-2 across different countries, and structurally understand the comparison between them, we collected the surface glycoprotein sequences of SARS-CoV-2 published in National Center of Biotechnology Information (NCBI) database. The main selective approach was to collect the samples from different countries to study about their changes in spike glycoprotein sequences. The mutations observed in these set of collected sequences with respect to the Wuhan Reference sequence is documented in Table 1. We used basic text processing algorithms in Python to obtain the results. Another dataset used for this study is obtained from UniProt, which is a popular database for obtaining protein sequences. Several animals with whom coronavirus could live as a potential host were identified, and the virus’s spike glycoprotein were collected from UniProt for motif analysis. The following coronaviruses were considered in this study (UniProt IDs in their respective parenthesis): • • • • • • • • • • • •

Rabbit coronavirus (ID: H9AA34) Avian coronavirus (ID: A0A7T7IGI4) Bovine coronavirus (ID: Q91A26) Rhinolophus affinis coronavirus (ID:A0A023PTS3) Murine coronavirus (ID: S5YA08) Feline coronavirus (ID: A0A0C5CJM0) Canine coronavirus (ID: P36300) Coronavirus neoromicia (ID: U5NJG5) Rat coronavirus (ID: I1TMG2) Equine coronavirus (ID: Q6W355) Dromedary camel coronavirus (ID: A0A5J6NRR1) Porcine transmissible gastroenteritis coronavirus (ID: P07946).

Herein we also propose a greedy method to find the motif sequences in the glycoproteins of SARS-CoV-2 related species. The method involves using a greedy approach to maximize the score of similar looking patterns in the protein sequences of different species of coronaviruses. We designed a greedy algorithm by taking inspiration from the greedy algorithm for DNA motif finding [17] (Fig. 1) and improved it to be used in protein sequences as well. In this process, we had to change the 4 letters of the nucleotide occurring in genomic sequences to the 20 amino acids seen

Study of Spike Glycoprotein Motifs in Coronavirus Infecting …

647

Fig. 1 Greedy algorithm on the context of DNA motif discovery

in protein sequences. We then ran the algorithm on the collected data and noted the potential motifs returned by the algorithm. The inputs to the algorithm are a set of protein sequences from the above mentioned coronaviruses and the motif length (k). The algorithm is implemented as follows: Step 1: Create a protein matrix by incorporating spike glycoproteins from selected sequences. Step 2: Select the first k amino acids from each row of protein matrix. Step 3: Create count matrices, which is basically the count of each amino acid at each position of a word having length “k”, once the sequences are arranged in rows. The row-wise sum of counts is to be taken to fill up the count matrix. Step 4: Create profile matrix from those greedily selected sequences. Profile matrix could be seen as a normalized account of the count matrix in which each entry is divided by its total frequency of itself and its counterparts. Step 5:Use this profile matrix to pick the best k-mer from row-1 of protein matrix and label it as Motif1. Step 6: Using this Motif1 evaluate which among all the possible k-mers from the second row of the protein matrix fits as the best candidate to resemble the Motif1. Thus, we get two motifs: Motif1 and Motif2. We repeat these steps, each time with a newly added motif to the motif matrix, and at last returning the set of motifs, called the motif matrix which contains [Motif1, Motif2, ...]. The significance of a profile matrix is that, it says the highest probability for each amino acid to occur at a particular position. Since profile matrices are created by considering the significance to the count of amino acids at each position of a sequence, it will have a bias towards the implanted similar sequences, as a result of which we can use it to further evaluate the sequences.

648

A. Sha and M. Nair

Table 1 Mutations in the variants of SARS-CoV-2 occurring in humans across different countries Accession Id Release date Geo location Mutation location with amino acid YP_009724390 QZY59917

2020-01-13 2021-10-01

Reference India

QNJ99480 UAN19001

2021-09-30 2021-09-14

Myanmar Egypt

QPK67538 QLJ57671 QVQ63041 UAG57585 QJQ84771 QWC77717 QZI00767 QIU78875 QQH16254 BCX25745 UDA74583 QUD15925

2021-03-12 2020-09-25 2021-05-22 2021-10-27 2020-05-08 2021-06-03 2021-08-25 2021-06-30 2021-03-03 2021-05-28 2021-10-18 2021-04-22

Turkey France Hong Kong Germany Thailand New Zealand Saudi Arabia: Jeddah Georgia Pakistan Japan Mexico South Korea

Nil 18 R, 451 R, 477 K, 613 G, 680 R, 949 N 76 M 7 F 18 R 451 R 477 K 548 S 613 G 680 R 845 S 949 N 613 G 613 G Nil 94 I, 244 R, 613 G, 685 G 828 T 613 G 48 Y, 156 S, 500 T, 613 G Nil Nil 613 G 12 I, 151 C, 451 R, 613 G 613 G

3 Results and Conclusion From the current study of mutations in the surface glycoprotein domain of SARSCoV-2 observed in samples collected from 15 different countries, The amino acid present at 613th position was found to be more significant. The mutation at this site which caused the amino acid to turn into Glycine has been found in the following considered countries: South Korea, Mexico, Japan, Saudi Arabia, New Zealand, Germany, France, Turkey, Egypt, and India. In sequences obtained from Mexico, Saudi, Germany, Egypt, and India, there were several other mutations occurred along with the 613th mutation. Another related observation is that, all other mutations that were found to be observed in these sequences coincided with this positional mutation. This clearly spotlights the importance of the glycine mutation observed in spike glycoprotein sequence of SARS-CoV-2. We ran our designed greedy motif search algorithm on the spike glycoprotein sequence of twelve human-related animal species. Using the greedy approach, we were able to find the most related motifs from all animals with respect to the standard glycoprotein motif of Wuhan-Hu-1 Isolate of SARS-CoV-2. Table 2 displays the results obtained. From the classes of animals which we studied, the popular glycoprotein motif “KRSFIEDLLFNKV” discussed in the works of [18] found in SARS-CoV-2, the best match out of the 12 studied organisms was R. affinis coron-

Study of Spike Glycoprotein Motifs in Coronavirus Infecting …

649

Table 2 Similar motif to “KRSFIEDLLFNKV” from coronaviruses infecting animals UniPort ID Motif Mutated Motif Organism position positions H9AA34 A0A7T7IGI4 Q91A26 A0A023PTS3

911 692 912 800

4 3 3 0

SRSAIEDLLFDKV GRSFIEDLLFTSV SRSAIEDLLFSKV KRSFIEDLLFNKV

S5YA08 A0A0C5CJM0 P36300

868 977 959

3 1 3

GRSAIEDLLFDKV KRSAIEDLLFNKV YRSAIEDLLFDKV

U5NJG5

877

6

ARSALEELLFDSV

I1TMG2 Q6W355

904 912

4 2

GRSAIEDVLFDKV SRSAIEDLLFNKV

A0A5J6NRR1

915

3

TRSAIEDLLFDKV

P07946

955

3

YRSAIEDLLFDKV

Rabbit coronavirus HKU14 Avian coronavirus Bovine coronavirus (cattle) Rhinolophus affinis coronavirus (horseshoe bat) Murine coronavirus (rodent) Feline coronavirus Canine coronavirus (strain Insavc-1) (CCoV) (Canine enteric coronavirus) (Dog virus) Coronavirus Neoromicia/PMLPHE1/RSA/2011 (vesper bat) Rat coronavirus Equine coronavirus (isolate NC99) (ECoV) (horse) Dromedary camel coronavirus HKU23 Porcine transmissible gastroenteritis coronavirus (strain Purdue) (TGEV) (pig)

avirus found in horseshoe bat. But bats as a species cannot be completely considered as the source of transmission of SARS-CoV-2 to humans. One of the other bats we studied, vesper bat which carries the coronavirus Neoromicia/PML-PHE1/RSA/2011 had twelve positional dissimilarities with the “KRSFIEDLLFNKV” reference motif. Apart from this, from the remaining 10 animals, Feline coronavirus found in cats which stand at second, with just one mutation. This reveals some connection of SARS-CoV-2 with one of the most popular domesticated animals in the world. During the COVID-19 pandemic, several experiments have shown that cats, ferrets, and hamsters are more prone to get SARS-CoV-2 infection [19]. The high conservation of the motif “KRSFIEDLLFNKV” could potentially have some role to this. Our results basically aims at providing better insights to the possible origins and transmission of SARS-CoV-2, by analysing and documenting 12 animals having human connections using their glycoprotein sequences.

650

A. Sha and M. Nair

References 1. R. Anand, V. Sowmya, E. Gopalakrishnan, K. Soman et al., Modified Vgg deep learning architecture for Covid-19 classification using bio-medical images, in IOP Conference Series: Materials Science and Engineering (IOP Publishing, 2021), p. 012001 2. R.C. Jariwala, M.R. Nalluri , Orthonormal Bayesian convolutional neural network for detection of the novel coronavirus-19, in Innovations in Electrical and Electronic Engineering (Springer, Berlin, 2021), pp. 819–836 3. S.R. Kiran, P. Kumar, Real-time statistics and visualization of the impact of COVID-19 in India with future prediction using deep learning, in Soft Computing for Problem Solving (Springer, Berlin, 2021), pp. 717–731 4. J. Cui, F. Li, Z.-L. Shi, Origin and evolution of pathogenic coronaviruses. Nat. Rev. Microbiol. 17, 181–192 (2019) 5. T.G. Ksiazek, D. Erdman, C.S. Goldsmith, S.R. Zaki, T. Peret, S. Emery, S. Tong, C. Urbani, J.A. Comer, W. Lim et al., A novel coronavirus associated with severe acute respiratory syndrome. N. Engl. J. Med. 348, 1953–1966 (2003) 6. M.E.R.S. Yywr, SARS and other coronaviruses as causes of pneumonia. Respirology 23, 130 (2018) 7. Fehr AR, Perlman S (2015) Coronaviruses: an overview of their replication and pathogenesis. Coronaviruses 1-23 8. S. Zhang, S. Qiao, J. Yu, J. Zeng, S. Shan, L. Tian, J. Lan, L. Zhang, X. Wang, Bat and pangolin coronavirus spike glycoprotein structures provide insights into SARS-CoV-2 evolution. Nat. Commun. 12, 1–12 (2021) 9. Z. Liu, X. Xiao, X. Wei, J. Li, J. Yang, H. Tan, J. Zhu, Q. Zhang, J. Wu, L. Liu, Composition and divergence of coronavirus spike proteins and host ACE2 receptors predict potential intermediate hosts of SARS-CoV-2. J. Med. Virol. 92, 595–601 (2020) 10. D. Wrapp, N. Wang, K.S. Corbett, J.A. Goldsmith, C.-L. Hsieh, O. Abiona, B.S. Graham, J.S. McLellan, Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation. Science 367, 1260–1263 (2020) 11. A. Ashok, P. Gopika, G. Charishma, V. Balakrishnan, O. Deepa, Application of geometric poisson distribution for COVID-19 in selected states of India, in Advances in Interdisciplinary Engineering (Springer, Berlin, 2021), pp. 435–446 12. P. Zhou, X.-L. Yang, X.-G. Wang, B. Hu, L. Zhang , Zhang W, Si H-R, Zhu Y, Li B, Huang C-L, others (2020) A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature 579, 270–273 (2020) 13. W. Li, Z. Shi, M. Yu, W. Ren, C. Smith, J.H. Epstein, H. Wang, G. Crameri, Z. Hu, H. Zhang et al., Bats are natural reservoirs of SARS-like coronaviruses. Science 310, 676–679 (2005) 14. N.C. Jones , P.A. Pevzner , P. Pevzner, An Introduction to Bioinformatics Algorithms (MIT Press, 2004) 15. M. Nair, P. Anusree, A. Babu, Recognizing significant Motifs of corona virus spike Proteins using computational approaches, in 2021 2nd Global Conference for Advancement in Technology (GCAT) (IEEE, 2021), pp. 1-6 16. M.K. Das, H.-K. Dai, A survey of DNA motif finding algorithms. BMC Bioinform. 8, 1–13 (2007) 17. P. Compeau, P. Pevzner, Bioinformatics Algorithms: An Active Learning Approach (Active Learning Publishers, La Jolla, California, 2015) 18. B. Robson, COVID-19 Coronavirus spike protein analysis for synthetic vaccines, a peptidomimetic antagonist, and therapeutic drugs, and analysis of a proposed achilles’ heel conserved region to minimize probability of escape mutations and drug resistance. Comput. Biol. Med. 121, 103749 (2020) 19. J. Shi, Z. Wen, G. Zhong, H. Yang, C. Wang, B. Huang, R. Liu, X. He, L. Shuai, Z. Sun et al., Susceptibility of ferrets, cats, dogs, and other domesticated animals to SARS-coronavirus 2. Science 368, 1016–1020 (2020)

A Systematic Review on Approaches to Detect Fake News Shashikant Mahadu Bankar and Sanjeev Kumar Gupta

Abstract The widespread use of social media and digital content, it is extremely important for people and society to be able to assess the trustworthy sources transmitting information through social media platforms. Fake news is not a new notion, but widespread and frequent circulation of fake news is an issue nowadays. Fake news might lead to annoyance, influencing and deceiving society or even nations. There are a number of ways of identifying fake news. By performing a systemic literary evaluation, we identify the major existing techniques to identify fake news and how such approaches may be implemented in various scenarios. A comprehensive description of factors that promote and circulate fake news, challenges to identify such sources, and techniques used to determine the fake news is prepared and discussed to minimize the impact of fake news. Keywords Fake news · Machine learning · Deep learning · SVM · Naïve Bayes · CNN · LSTM

1 Introduction and Motivation A false statement misled the people by altering the facts and information about events is defined as misinformation, also known as trickery, uncertainty, deception, and misrepresentation [1]. In this growing online world, the spread of deliberate misleading information becomes a trend which not only creates confusion among people but also violates expectations. Misinformation can either be represented as rumor, fake news, spam, and disinformation, where all terminologies share the variant prospects [2]. Rumors are stated as the circulation stories of doubtful information about the event, without any true evidence among people [3]. Intentional misleading S. M. Bankar (B) · S. K. Gupta Dr Babasaheb Ambedkar, Marathwada University, Aurangabad, Maharashtra, India e-mail: [email protected] S. K. Gupta e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_57

651

652

S. M. Bankar and S. K. Gupta

of the readers through news articles carrying verifiably false information define fake news [3]. Spontaneous messages propagating malware or misinformation on the multiple Internet sources are referred as spams [4]. Disinformation differs from misinformation based on intention criteria as the former is always followed on intentional basis, rather both defines the state of spread of false information among people [5]. Passage of this misinformation through varying channels questions over the credibility of the news by disrupting the review outlook of the marketing products, creating miscommunication in society regarding political or commercial issues, and poses one of the greatest threats to journalism. People generally trust their friends, colleagues, and relatives and share the information from those people without any second thought. The sharing of fake news sometimes also affects the confidence and trust level [6, 7]. The reviews of users received on a product could also be generated from the fake profile and accounts, to improvise the sale of the product or any commercial benefits. Multiple scenarios were identified in the past where fake profiles have been created to develop a good image of different products or schemes or people for the commercial or political benefits while hiding the real opinion [8]. Fake news belongs to the category of yellow journalism, consisting of deliberate conduction of misinformation or hoaxes or deceptions, through traditional print or broadcast news sources, online social media, and other social networking sites. Existence of fake news was firstly reported in 1835 with the publication of “Great moon hoax”. Later, it progresses with growing advancements of online social networks, where fake news for multiple commercial and political grounds is circulating in large numbers. Several instances have been observed in the recent time, where the fake news is being broadcasted deliberately through social media networks, in order to get some commercial, industrial, or political benefits at the large scale [9, 10]. Latest and new information is being shared by a large part of the population available on social networking applications without looking for the authenticity of the information. The users get attracted with the catchy headlines and pictures, and spread misleading information. The post electoral statistical report of USA stated that during 2016 and 2021 presidential election, the results are tremendously affected by the fake news broadcasted through various social networking platforms [10, 11]. The spreading of fake news is being handled by various sources accounts for 41.8% of news shared through various electronic and print media [12]. The availability of multiple platforms for sharing news or any content makes the process easier for the people willing to spread the fake news. At the same time, it is a challenging task to identify whether the information is generated from the authentic source or not. While surfing the social networking sites, users do not spend time to verify the facts and source of information [13] which further increases the complexity of identification of source as the same information is being shared from a large number of users. Considering the large number of platforms and extensive user base, manual detection of fake news is a tedious task, therefore generation of large data through various platforms requires an automated detection technique [13]. Several researchers are working on the development of techniques to quantify the source of information in real time and identify unauthentic information on social media applications [5, 8]. In order to identify or classify the information based on authenticity, the extraction

A Systematic Review on Approaches to Detect Fake News

653

of relevant features and characteristics is a prime requirement which makes them different from the pool of information [1, 14]. The manual extraction or selection of features is also a good prospect adopted by few researchers mainly consisting of the suggestions, viewpoint, and feedback of individuals [6, 15].

2 Methodology A methodology has been prepared to select the suitable research papers for each category of this study. The papers are selected from peer reviewed journals indexed in IEEE, Scopus, or SCI/SCIE. Along with the journals, papers were also selected from international conferences and annual reports published on concerned areas by organization of repute. Approximately, 300 papers were selected from all the sources and first screening of papers was performed based on reviewing the abstract of papers and executive summary of reports. After screening, 52 papers were selected for referring and preparation of review papers. The methodology presented in Fig. 1.

Fig. 1 Methodology to perform paper selection for fake news detection

654

S. M. Bankar and S. K. Gupta

2.1 Type of Dataset Nowadays, it is easy to obtain information about any topic, incidence or status of any political movement, projects, or major event happening around the world. Print media was the choice till a few decades ago to gather all the information under a single roof. However, search engines and social media and live broadcasting of news or any major event worldwide allows the users to information as per their requirement [16]. At the same time, the challenging task is to determine the reliability of those news, information, or reports. The reliability can be determined either through fact checking with domain experts or confirming with the authentic sources in the respective area [16–23]. Some of the websites are also available to cross-check the news veracity to verify the facts or information [22]. Few public agencies collect the data pertain to single or multiple events and can be used for data analysis as listed below: BuzzFeedNews: This dataset contains news about a single event just before the US presidential election during September 2016 [23–25]. Dataset collects the news published from nine agencies. The BuzzFeed team, individually verified each and every article comprises 1627 articles including articles published from the end of both the candidates. However, this dataset consists of headlines of news articles and keywords from the article text [26]. The sources covered in the dataset are also very limited is the major limitation of this dataset. LIAR: PolitiFact verifies the news published through various platforms including broadcasted news, print news, magazines, political speeches, radio news, and social media feeds [26]. The verification is performed through individuals on the basis of their comments classified as false, partly true, true, partly false, completely false, and mostly true. The dataset consists of statements recorded from 12,836 individuals. Such a dataset is defined as the fine grained multi-class dataset with a label that can be used with the supervised classification techniques for further analysis. This dataset also covers the key statement out of the complete article [25]. All the key statements were identified based on the reviews of individuals which may or may not be the fake news. BS Detector: This dataset is based on the collection of data from the links provided on the websites after verification and determining the reliability of links [27]. The labels obtained after verifying the various links are generated to minimize the human error, as the data were authenticated in LIAR using human statements. The statements collected in this dataset are based on the data checker tool and become more reliable in comparison with other datasets. However, as no human verification is involved in this dataset, there may be the possibility of development of wrongly trained models. CREDBANK: This is one of the largest datasets developed from the tweets of individuals collected for 96 days from October 2015 [28]. The dataset consists of more than 60 million tweets. The tweets are initially classified into more than 1000 classes on the basis of news category, and all the classes are verified by the Amazon Mechanical Turk to ensure the reliability of the dataset [15]. However, the large dataset also holds some boundaries that make it applicable for limited areas while detecting the fake news. However, all the tweets considered in the dataset are based

A Systematic Review on Approaches to Detect Fake News

655

Table 1 Comparison of various features of fake news dataset Dataset

Dataset category

Dataset type

Text

Post

Images

Comments

Limitations Articles

Webpages

BuzzFeedNews

Yes

Yes

Only headlines

LIAR

Yes

Yes

Short statement

BS detector

Yes

CREDBANK

Yes

Yes

Yes

Not credible for fake news

FacebookHoax

Yes

Yes

Yes

Few samples

Yes

NELA-GT 2018 Yes

Yes

Image manipulation

Yes

The PS battles

Yes

Yes

Yes

Yes

Fakeddit

Yes

Yes

Yes

No human expert

Only text data Very few images Manipulated content

Yes

Not validated

on the personal observation of individuals and do not cover the social engagements on news articles. FacebookHoax: This dataset consists of 15,500 posts collected from 32 pages. From these 32 pages, 14 are identified as non-hoax and the rest is identified as hoax pages which receive more than 0.3 million comments and 2.3 million likes over the period of 6 months [3, 15]. The information in the dataset is considered true or false on the basis of user comments, and most of the posts were collected from 14 conspiracy pages and 18 non-conspiracy pages. A comparison of all the dataset discussed is prepared, as shown in Table 1 based on the dataset type, category, sources covered, and number of sample data points. Fakeddit: This dataset contains both the text and image data from subreddit. The database is one of a kind containing more than 1 million samples of images and text and contains the multi-class dataset that can be classified into two, three or six classes [29]. The database consists of a variety of information covering multiple sectors from Reddit. Therefore, a complete dataset is required to fulfill all the limitations of the dataset stated above. A complete and precise database should contain sufficient databases in terms of number of samples, should cover every aspect of information, social context as well as personal suggestions, and should be validated with a reliable source to ensure the authenticity.

656

S. M. Bankar and S. K. Gupta

2.2 Category of Datasets All the news articles, reports, magazines, social media posts, tweets contain text, images, and videos. Out of these three forms, text and images are dominantly shared by individuals.

2.2.1

Text Datasets

Any machine learning or deep learning technique for determining the flux in samples and patterns of variation requires a significant amount of data whether it could be images or text data. The accuracy of the model is dependent on the sample set significantly, as the smaller number of samples would not allow the sufficient training of the model and could generate vague results. For the assessment of fake news, only few data are available with about 500,000 samples are considered as adequate [8]. CREDBANK [30] and Fake News Corpus [31] contain over a million samples collected from varied sources and largest compared to the size of another dataset. Few datasets contain information from two classes only and data divided into binary classification, however, some datasets such as NELA-GT-2018, LIAR [26], and Fake News Corpus contain multi-class data where data can be divided into more than two classes. Information about the areas covered in the dataset also improves the dataset quality. Few datasets only cover more than one area (politics, entertainment, sports, and business) to collect the information. The data about the single sector and few categories reduce the applicability of dataset to wider areas [32].

2.2.2

Image Datasets

The news, social media posts, tweets, and incidences are significantly shared by using images, whereas most of the dataset are based on the text data that reduces the scope of the dataset. The development of image-based dataset is a wider area of research for the identification of classes, features to capture the data [33, 34]. Image dataset is significantly larger than the text data, due to the greater pixel size of images and information hold by single image compared to one sample of text data. The image manipulation holds a smaller number of images, and images were selected on the basis of self-selection without any validation [29, 35]. The PS Battles dataset contains large number of images compared to Image Manipulation dataset collected from subreddit. Fakeddit extent the image database developed by PS Battles by including data using subreddit and included both image and text data in database [29, 35]. The Fakeddit contains a large number of images compared to other dataset and includes both the text and image data collected from a variety of sources.

A Systematic Review on Approaches to Detect Fake News

657

3 Feature Extraction There are tremendous features that affect the quality of the dataset for the identification of fake news. Different features are identified depending upon the source of data, type of data, and data samples [32, 36]. The fake news detection from the conventional digital media is carried out based on the news broadcasting time and content, whereas source of news is the prime factor for news from social media [10, 37]. The conventional digital media of news broadcasting and social media are the two essential modes of news transmission, and details of features of fake news detection are illustrated in this section.

3.1 Conventional Digital Media The news content is the essential element of fake news detection. A representative sample of data can be extracted from the source of news, headline, story line, and images or video linked with the news story [38, 39]. Source of news content could be the direct reporting or referred article. Headlines are generally designed to attract the viewers like short and appealing. The method of presentation of the story line is the main link that helps to identify the authenticity of the news and time when the news is broadcasting also define the importance of news [39]. The images or videos within a story line of news provide relevant and vast information for extracting the relevant features [40]. There is significant variation in the news broadcasted with image or video or simply textual information. The total number of inflammatory words, dangling words in a sentence, vague sentences, and misrepresentation of words are the significant linguistic features that can be used to detect the authenticity of the news [8]. The linguistic features are also categorized as common and subject specific depending upon the area of application. The common linguistic words are dominantly used in several applications of natural language processing. The features are also identified on sentence level such as clarity and engagement of words in a sentence, sentence delivery method, speech formation, phrases in story, irregular punctuation, or hyphenation in sentence referred as syntactic features [8]. A single image or video can spread significantly large information in comparison with a complete news article [36]. It is easy to spread unrest, provoke people on the basis of religious, political, sentimental, and caste using fake or manipulated images. Such features that can affect the originality of the image or video are required to be identified to avoid the spreading of wrong or fake messages to the viewers. The most relevant features are extracted from images or videos and classified to hold the originality of the content. The classification features are evaluated using the clarity score, coherence score, similarity distribution histogram, diversity score, and clustering score [1, 37, 38]. The statistical features of the images are also accountable

658

S. M. Bankar and S. K. Gupta

to keep the authenticity of content. The major statistical features are pixel size, pixel ratio, hot image ratio, diagonal size of image, colors in image, etc.

4 Social Media Features The news content developed and circulated at the conventional digital media or through social media is significantly divided by the user type based on their age group [41, 42]. In comparison with the features extracted from the news articles, social media provide access to a large number of features. The time devoted by the user on news links, content of the visual news seen by viewers, number of times the news links are being shared and circulated between various platforms, provide useful information about authenticity of news content and user interest [43]. As mentioned above, the large number of users spread fake news for monetary benefits and use catchy headlines and taglines to attract the users. Social media platforms also spread fake news through rumors of hot topics, ongoing political discussions, business deals, and flux in the stock market are used tremendously to produce fake news affecting the market by spreading rumors [15, 44]. The spreading of fake news on social media is categorically dependent upon the type of account, post, and network used to circulate the information. The information about the account type, individual, and type of organization provides significant information about the purpose of sharing fake news [45]. The historical background of the user, pattern of previously shared information can be used as key features to determine the reliability and originality of the news for userbased accounts [21, 46]. The characteristics of a group, page, or company sharing the news are the key features of commercial accounts [47]. The number of followers of a group, domain area of group, link between the news content and domain of group, age of the group, rating of group are the key features that can be used to detect the probability of authenticity of news [21, 46]. The commercial account found to be sharing the fake news for various benefits, and user-based accounts share the fake news in most cases unknowingly or after finding the content attracting. The news obtained from global posts, tweets, and local groups circulated among the groups are commonly observed. By observing the frequency and pattern of sharing information, a matrix can be designed to extract the relevant features and determine the authenticity of data shared. The data classification techniques and clustering techniques can be applied on such data to identify the source, credibility, and reliability score [47]. Lot of people express their opinion, suggestion, expectation, reviews on various news content, government policies, strategies, actions without knowing the source of information, and authenticity of information [48]. The user’s suggestion or reviews or comments can be used as the prominent features to determine the authenticity of news shared as a post [41]. The keywords are used depending upon the news content; it could be common linguistic words, or domain specific words [42]. The use of pretrained word embeddings is an advanced method of determining the credibility score of the news content [43]. The neural network uses the word embeddings through

A Systematic Review on Approaches to Detect Fake News

659

pre-trained multi-dimensional embeddings. The GloVe is an unsupervised learning algorithm consisting of more than one billion tokens generated from the linguistics of 4 million [43].

5 Techniques for Fake News Detection Rapid dispersal of fake news through catalytic outrage and influence of social media has become one of the challenges nowadays. The spread of such news not only influences individuals but also causes societal harm at large scale along with instability in the news ecosystem [44]. Multiple studies have focused on scrutinizing the pattern of fake news dispersal through variants of social media sources, where one of the methods utilized was manual fact checking. Manual fact checking can be performed using two different methods as expert-based fact checking and crowd sourced manual fact checking. The process involves three sequential steps such as fake news detection, identification of source of generation, and authentication of news; authentication can be executed using different websites like International Fast Checking Network (ICFN), Washington post, Snopes, Fast Checker, FastCheck, and TruthOrFiction. Once verification of fake data is completed; the genuine true information is circulated on the social media platform to avoid any further harm. The major benefit of this task is accuracy of the results obtained after verification through portals but still offer some limitations like limited scalability of tasks with large size datasets. Requirement of intensive labor along with involvement of huge cost with time restricts the usage of optimized accuracy of expert-based methods. Similarly, less credibility limits the application of crowd sourced fact checking [26, 49]. Automated fact checkers have been exploited as the new technique by many print media organizations which overcome the data handling issues related with manual fact checking. Various customized web extensions like Decodex show great ability in segregating huge volumes of news data in authentic and fake information across social media platforms with high accuracy and credibility. Automated fake news detection now serves as one of the surging curatives in impeding the dissemination of fake news among social media platforms and drive major attention by various researchers. Use of a specific type of algorithm depends on the type of dataset and objectives of study like one technique may work well for one type of dataset but fail to reproduce the same for another dataset. The work protocol of machine learning and deep learning approaches also varies like machine learning algorithms perform both feature extraction and classification separately while deep learning approaches combine both together, leading to differences in their accuracies. Wang [16] designed a new dataset named Liar and used automated detection algorithms like SVM, logistic regressions (LR), long short-term memory networks (LSTM), and CNN for comparing their efficiency in validation of text data only [16]. Based on the result, it was found that the dataset length is not sufficient for neural network-based advanced models, defining efficacy of dataset for specific model. Multiple machine learning and deep learning-based studies performed by

660

S. M. Bankar and S. K. Gupta

researchers focusing on validation of text data like Facebook posts, twitter data, and other social media content.

5.1 Machine Learning Algorithms Different machine learning algorithms are used in multiple studies focusing on text data validation using different datasets. Machine learning classifiers are mostly employed for fake news detection and accuracy of classifiers yield the best outcomes in controlling the dissemination of fake news in near future [50]. Several types of machine learning algorithms are experimented for detection using different supervised machine learning algorithms support vector machines (SVM), NB, decision trees (DT), extra trees (ET), stochastic gradient descent (SGD), and random forests (RF). Gravanis et al. [50] applied different machine learning algorithms like SVM, random forest (RF), KNN, and Gaussian Naïve Bayes (NB) to validate multilingual text data using five different datasets as TwitterBR, FakeBrCorpus, FakeNewsData1, Fake or Real News, and life style. Three different feature sets were selected for evaluation of the dataset as Word2Vec, DCDistance, and bag-of-words. The study concluded about the out performance of SVM and RF over other classification algorithms with high accuracy of 76%. Simplification of text data and extraction of classified text messages defines feature extraction which not only allow elimination of extraneous features from data but also lessen dimensionality of the text. The process enhances the accuracy of the classifier with reduction in training time. Ahmed et al. [51] presented fake detection model using n-gram analysis with different machine learning techniques (SGD, SVM, LSVM, KNN, and DT), along with two different feature extraction techniques as term frequency (TF) and term frequency-inverse document frequency (TF-IDF). The study concluded about the highest accuracy with the model using unigram feature and LSVM as 92% where use of feature extraction method (term frequency-inverse document frequency) resulted in improved accuracy of the classifier. Hakak et al. [34] experimented on characterization of fake news using feature analysis (believable and unbelievable messages) on twitter news for specific duration. The study utilized three different machine learning classifiers as SVM, neural networks, and NB for detection, and their performance were evaluated using accuracy, precision, F1-score, and recall. The results stated high accuracy by classifiers due to the feature extraction performed during the study as twenty-two attribute selection was the leading step of the study. Neural network and SVM outperformed with accuracy of 99.9% and NB with accuracy of 96.08%. Another approach used for the improvisation of performance is usage of ensemble classification models which integrate different machine learning algorithms together for effective detection of fake news. Hakak et al. [34] used an ensemble model of three algorithms DT, RF, and extra tree classifier to detect the fake news using data collected from Liar and ISOT dataset. The study involved preprocessing steps along with specific extracted features feeding to the classifier which leads to the

A Systematic Review on Approaches to Detect Fake News

661

100% training and testing accuracy for Liar dataset as compared to 99.8% training accuracy for ISOT dataset. Khan et al. [13] investigated the accuracy of different machine learning algorithms (SVM, LR, NB, DT, and KNN), trained with specific features (lexical and sentiment features for SVM, LR and DT; n-gram for NB; and Empath generated features for KNN), for detection of fake news for data collected from Liar dataset, fake or real news dataset, and combined corpus. Among all, NB with n-gram features outperforms all with accuracy of 93% on combined corpus dataset while no such improved effect of feature extraction was observed on rest algorithms for large size dataset.

6 Conclusion Tremendous work has been done for creation of different datasets with variant scopes and application but still their limitations present a need to create one validated type of dataset, consisting of text and multimedia data, over which several classification algorithms can effectively work for detection of fake news. As discussed, several machine learning algorithms effectively work on text data categorization as authentic and fake news but the dataset biasness limits their effective working. This necessitates the need of creating unbiased dataset with large size, and utilization of specific feature extraction techniques to improve the performance of algorithms. In today’s scenario, fake news is widely dispersed in multimedia formats for which detection using supervised and unsupervised algorithms or their hybrid models need to be improvised and can be followed as the upcoming objective. The available dataset mostly consists of text data whereas image-based dataset consists of a smaller number of images. Aggregation of a large number of images to form a dataset that can cover all the shortcomings of previously available dataset would be an exciting work. The model accuracy can be increased with larger dataset, and several new features can be extracted from the database. The machine learning algorithms perform well in detection studies but with increase in dataset size, working of classifiers do not yield effective outcomes, which is found to be improvised using deep learning algorithms in multiple studies.

References 1. Z. Jin, J. Cao, Y. Zhang, J. Zhou, Q. Tian, Novel visual and statistical image features for microblogs news verification. IEEE Trans. Multimed. 19(3), 598–608 (2017). https://doi.org/ 10.1109/TMM.2016.2617078 2. S. Banaji, R. Bhat, WhatsApp vigilantes: an exploration of citizen reception and circulation of WhatsApp misinformation linked to mob violence in India. LSE Media Commun. 2, 1–14 (2020) 3. K. Shu, S. Wang, H. Liu, Understanding user profiles on social media for fake news detection, in Proceedings—IEEE 1st Conference on Multimedia Information Processing and Retrieval

662

S. M. Bankar and S. K. Gupta

MIPR 2018 (2018), pp. 430–435. https://doi.org/10.1109/MIPR.2018.00092 4. G.R. Milner, Violence and warfare among Hunter-Gatherers, ed. by M.W. Allen, T.L., 391 p. (Left Coast Press, Walnut Creek, California, 2014) $39.95 (cloth). ISBN 978-1-61132-939-1 (Am. Antiq. 80(4), 787–788 (2015)). https://doi.org/10.1017/s0002731600003863 5. S. Kumar, R. Asthana, S. Upadhyay, N. Upreti, M. Akbar, Fake news detection using deep learning models: A novel approach. Trans. Emerg. Telecommun. Technol. 31(2), 2020. https:// doi.org/10.1002/ett.3767 6. J. Ma, et al., Detecting rumors from microblogs with recurrent neural networks, p. 9 7. J. Wu, Z. Su, S. Wang, J. Li, Crowd sensing-enabling security service recommendation for social fog computing systems. Sensors 17(8), 1744 (2017). https://doi.org/10.3390/s17081744 8. Y. Liu, Y.-F.B. Wu, FNED: a deep network for fake news early detection on social media. ACM Trans. Inf. Syst. 38(3), 1–33 (2020). https://doi.org/10.1145/3386253 9. E.C. Tandoc, Z.W. Lim, R. Ling, Defining ‘Fake News’: A typology of scholarly definitions. Digit. J. 6(2), 137–153 (2018). https://doi.org/10.1080/21670811.2017.1360143 10. Á. Figueira, L. Oliveira, The current state of fake news: challenges and opportunities. Procedia Comput. Sci. 121, 817–825 (2017). https://doi.org/10.1016/j.procs.2017.11.106 11. L. Lu, Joe Biden’s political communication on Twitter during the 2020 United States presidential elections (2020), p. 66 12. D. Metaxa-Kakavouli, N. Torres-Echeverry, Google’s role in spreading fake news and misinformation. SSRN Electron. J. (2017). https://doi.org/10.2139/ssrn.3062984 13. J.Y. Khan, Md.T.I. Khondaker, S. Afroz, G. Uddin, A. Iqbal, A benchmark study of machine learning models for online fake news detection. Mach. Learn. Appl. 4, 100032 (2021). https:// doi.org/10.1016/j.mlwa.2021.100032 14. V. Gupta, K. Beckh, S. Giesselbach, D. Wegener, T. Wirtz, Supporting verification of news articles with automated search for semantically similar articles (2021). Accessed: Jun 11, 2021 [Online]. Available: http://arxiv.org/abs/2103.15581 15. K. Shu, D. Mahudeswaran, S. Wang, D. Lee, H. Liu, FakeNewsNet: a data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big Data 8(3), 171–188 (2020). https://doi.org/10.1089/big.2020.0062 16. E. Weltevrede, A. Helmond, C. Gerlitz, The politics of real-time: a device perspective on social media platforms and search engines. Theory Cult. Soc. 31(6), 125–150 (2014). https://doi.org/ 10.1177/0263276414537318 17. L. Singh, Fake news detection: a comparison between available deep learning techniques in vector space, in 2020 IEEE 4th Conference on Information & Communication Technology (CICT), Chennai, India (2020), pp. 1–4. https://doi.org/10.1109/CICT51604.2020.9312099 18. G. Haciyakupoglu, J.Y. Hui, V.S. Suguna, D. Leong, Countering fake news, p. 24 19. C.M. Pulido, B. Villarejo-Carballido, G. Redondo-Sama, A. Gómez, COVID-19 infodemic: More retweets for science-based information on coronavirus than for false information. Int. Sociol. 35(4), 377–392 (2020). https://doi.org/10.1177/0268580920914755 20. S.C. Lewis, L. Molyneux, A decade of research on social media and journalism: assumptions, blind spots, and a way forward. Media Commun. 6(4), 11–23 (2018). https://doi.org/10.17645/ mac.v6i4.1562 21. Y. Wang, M. McKee, A. Torbica, D. Stuckler, Systematic literature review on the spread of health-related misinformation on social media. Soc. Sci. Med. 240, 112552 (2019). https://doi. org/10.1016/j.socscimed.2019.112552 22. A. Bondielli, F. Marcelloni, A survey on fake news and rumour detection techniques. Inf. Sci. 497, 38–55 (2019). https://doi.org/10.1016/j.ins.2019.05.035 23. M.D. Vicario, W. Quattrociocchi, A. Scala, F. Zollo, Polarization and fake news: early warning of potential misinformation targets. ACM Trans. Web 13(2), 1–22 (2019). https://doi.org/10. 1145/3316809 24. A. Guess, B. Nyhan, Selective exposure to misinformation: evidence from the consumption of fake news during the 2016 U.S. presidential campaign, p. 49 25. S. Yang, K. Shu, S. Wang, R. Gu, F. Wu, H. Liu, Unsupervised fake news detection on social media: a generative approach. Proc. AAAI Conf. Artif. Intell. 33, 5644–5651 (2019). https:// doi.org/10.1609/aaai.v33i01.33015644

A Systematic Review on Approaches to Detect Fake News

663

26. W.Y. Wang, ‘Liar, Liar Pants on Fire’: a new benchmark dataset for fake news detection, in Proceedings of the 55th annual meeting of the association for computational linguistics (Volume 2: Short Papers) (Vancouver, Canada, 2017), pp. 422–426. https://doi.org/10.18653/ v1/P17-2067 27. X.-Q. Pham, E.-N. Huh, Towards task scheduling in a cloud-fog computing system, in 2016 18th Asia-Pacific network operations and management symposium (APNOMS), Kanazawa, Japan (2016), pp. 1–4. https://doi.org/10.1109/APNOMS.2016.7737240 28. N.X. Nyow, H.N. Chua, Detecting fake news with tweets’ properties, in 2019 IEEE conference on application, information and network security (AINS), Pulau Pinang, Malaysia (2019), pp. 24–29. https://doi.org/10.1109/AINS47559.2019.8968706 29. K. Nakamura, S. Levy, W. Y. Wang, r/Fakeddit: a new multimodal benchmark dataset for finegrained fake news detection (2020). Accessed: Dec. 06, 2021 [Online]. Available: http://arxiv. org/abs/1911.03854 30. T. Mitra, E. Gilbert, CREDBANK: a large-scale social media corpus with associated credibility annotations, p. 10 31. A. Pathak, R. Srihari, BREAKING! presenting fake news corpus for automated fact checking, in Proceedings of the 57th annual meeting of the association for computational linguistics: student research workshop, Florence, Italy (2019), pp. 357–362. https://doi.org/10.18653/v1/ P19-2050 32. C. Zhang, A. Gupta, C. Kauten, A.V. Deokar, X. Qin, Detecting fake news for reducing misinformation risks using analytics approaches. Eur. J. Oper. Res. 279(3), 1036–1052 (2019). https:// doi.org/10.1016/j.ejor.2019.06.022 33. K. Anoop, M.P. Gangan, V.L. Lajish, Leveraging heterogeneous data for fake news detection, in Linking and mining heterogeneous and multi-view data, ed. by A. Jurek-Loughrey (Springer International Publishing, Cham, 2019), pp. 229–264. https://doi.org/10.1007/978-3030-01872-6_10 34. S.R. Sahoo, B.B. Gupta, Multiple features based approach for automatic fake news detection on social networks using deep learning. Appl. Soft Comput. 100, 106983 (2021). https://doi. org/10.1016/j.asoc.2020.106983 35. A. Kirchknopf, D. Slijepcevic, M. Zeppelzauer, Multimodal detection of information disorder from social media (2021). Accessed: Dec. 06, 2021 [Online]. Available: http://arxiv.org/abs/ 2105.15165 36. H. Ahmed, I. Traore, S. Saad, Detection of online fake news using N-Gram analysis and machine learning techniques, in Intelligent, secure, and dependable systems in distributed and cloud environments, ed. by I. Traore, I. Woungang, A. Awad, vol. 10618 (Springer International Publishing, Cham, 2017), pp. 127–138. https://doi.org/10.1007/978-3-319-69155-8_9 37. A. Bonet-Jover, A. Piad-Morffis, E. Saquete, P. Martínez-Barco, M. Ángel García-Cumbreras, Exploiting discourse structure of traditional digital media to enhance automatic fake news detection. Expert Syst. Appl. 169, 114340 (2021). https://doi.org/10.1016/j.eswa.2020.114340 38. V. Bakir, A. McStay, Fake news and the economy of emotions: problems, causes, solutions. Digit. J. 6(2), 154–175 (2018). https://doi.org/10.1080/21670811.2017.1345645 39. S. Hakak, M. Alazab, S. Khan, T.R. Gadekallu, P.K.R. Maddikunta, W.Z. Khan, An ensemble machine learning approach through effective feature extraction to classify fake news. Future Gener. Comput. Syst. 117, 47–58 (2021). https://doi.org/10.1016/j.future.2020.11.022 40. S.B. Parikh, P.K. Atrey, Media-rich fake news detection: a survey, in 2018 IEEE conference on multimedia information processing and retrieval (MIPR), Miami FL (2018), pp. 436–441. https://doi.org/10.1109/MIPR.2018.00093 41. K. Shu, A. Sliva, S. Wang, J. Tang, H. Liu, Fake news detection on social media: a data mining perspective, vol. 19, no. 1, p. 15 42. P.M. Waszak, W. Kasprzycka-Waszak, A. Kubanek, The spread of medical fake news in social media—the pilot quantitative study. Health Policy Technol. 7(2), 115–118 (2018). https://doi. org/10.1016/j.hlpt.2018.03.002 43. K. Shu, S. Wang, H. Liu, Understanding user profiles on social media for fake news detection, in 2018 IEEE conference on multimedia information processing and retrieval (MIPR), Miami, FL (2018), pp. 430–435. https://doi.org/10.1109/MIPR.2018.00092

664

S. M. Bankar and S. K. Gupta

44. Z.I. Mahid, S. Manickam, S. Karuppayah, Fake news on social media: brief review on detection techniques, in 2018 Fourth international conference on advances in computing, communication & automation (ICACCA), Subang Jaya, Malaysia (2018), pp. 1–5. https://doi.org/10.1109/ ICACCAF.2018.8776689 45. H. Rashkin, E. Choi, J.Y. Jang, S. Volkova, Y. Choi, Truth of varying shades: analyzing language in fake news and political fact-checking, in Proceedings of the 2017 conference on empirical methods in natural language processing, Copenhagen, Denmark (2017), pp. 2931–2937. https:// doi.org/10.18653/v1/D17-1317 46. A. Giachanou, E.A. Ríssola, B. Ghanem, F. Crestani, P. Rosso, The role of personality and linguistic patterns in discriminating between fake news spreaders and fact checkers, in Natural language processing and information systems, ed. by E. Métais, F. Meziane, H. Horacek, P. Cimiano, vol. 12089 (Springer International Publishing, Cham, 2020), pp. 181–192. https:// doi.org/10.1007/978-3-030-51310-8_17 47. Y. Keneshloo, S. Wang, E.-H. (Sam) Han, N. Ramakrishnan, Predicting the popularity of news articles, in Proceedings of the 2016 SIAM international conference on data mining (2016), pp. 441–449. https://doi.org/10.1137/1.9781611974348.50 48. J. Cao, P. Qi, Q. Sheng, T. Yang, J. Guo, J. Li, Exploring the role of visual content in fake news detection (2020). https://doi.org/10.1007/978-3-030-42699-6 49. Y. Liu, Y.-F.B. Wu, Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks, p. 8 50. P. Meel, D.K. Vishwakarma, Fake news, rumor, information pollution in social media and web: A contemporary survey of state-of-the-arts, challenges and opportunities. Expert Syst. Appl. 153, 112986 (2020). https://doi.org/10.1016/j.eswa.2019.112986 51. F. Monti, F. Frasca, D. Eynard, D. Mannion, M.M. Bronstein, Fake news detection on social media using geometric deep learning (2019). Accessed: Jun 02, 2021 [Online]. Available: http://arxiv.org/abs/1902.06673

An Analysis of Semantic Similarity Measures for Information Retrieval Preeti Rathee and Sanjay Kumar Malik

Abstract One of the most significant difficulties in information retrieval is measuring semantic similarity. Semantically similar words, phrases, and concepts are measured in this field of study. It is the degree to which two concepts resemble one another depending on their meaning. Various measures of semantic similarity have been suggested several times throughout the years. This paper divides these methodologies into two groups: knowledge-based and corpus-based, to demonstrate how they evolved. On specified parameters, a study of similarity approaches is performed. This assessment, which analyzes each method for semantic similarity, provides an analysis, review of existing methods on semantic similarity to explore, and creates new ideas for researchers. Keywords Semantic similarity · Semantic relatedness · Knowledge-based methods · Corpus-based methods · Information retrieval

1 Introduction The main aim of information retrieval is to retrieve relevant information from the available data. In order to retrieve all relevant information, the non-relevant information must be removed from the search results. Relevant information we get from the search mainly depends on the semantic similarity [1]. Semantic similarity (Fig. 1) is a measure of how similar two concepts are, which can be sentences, words, or paragraphs [2]. It is essential in data processing, artificial intelligence, and data mining, as well as question answering, sentiment analysis, plagiarism, natural language processing (NLP), and other fields. There are two ways to associate concepts: lexically and semantically. Concepts are lexically similar if their letter sequences and P. Rathee (B) MSIT, GGSIPU, Janakpuri, Delhi, India e-mail: [email protected] S. K. Malik GGSIPU, Dwarka, Delhi, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_58

665

666

P. Rathee and S. K. Malik

Fig. 1 Semantic similarity [26]

other words are similar, and concepts, if they have the same meaning, they are semantically comparable despite having different vocabulary patterns [2]. Semantic similarity is basically a numerical value or a metric to compare concepts based on their meaning [3]. Very vast research has been done in this field. We have conducted an analysis that represents how these methods evolved by dividing them into two categories: knowledge-based and corpus-based. In the corpus-based method, information is derived from large corpora, which is used for semantic similarity among words. The “distribution hypothesis” [4] is based on the concept that “similar phrases frequently occur together.” However, the true meaning of the word is not considered. Pointwise mutual information and the latent semantic approach are not the same things. Semantic similarity is determined in the knowledge-based method using information acquired from knowledge sources such as an ontology/lexical database, a thesaurus, and so on. On specified parameters, a full study of similarity approaches is performed. In this study, each method delivers a thorough perspective of currently in use systems with which new ideas can be created on semantic similarity.

2 Semantic Similarity Approaches Semantic similarity approaches have evolved over the past decades. Similarity measures are distinguished on the basis of methods used in them. In this paper, similarity approaches are divided into knowledge-based, as shown in Fig. 2.

2.1 Knowledge-Based Similarity Approaches Data from one or more underlying sources of knowledge, such as ontologies, thesauri, dictionaries, and lexical databases, and so on, are used to compute semantic similarity between two terms. They are provided by the basic knowledge foundation. It is

An Analysis of Semantic Similarity Measures …

667

Fig. 2 Semantic similarity methods

defined as a logical arrangement of terms or concepts linked by semantic relationships [5]. In Sect. 3.1, we will look at different similarity measures in more detail. Depending on how the semantic similarity of words is measured, semantic similarity based on knowledge approaches is further categorized as methods for counting edges, feature-based methods, and information content-based methods [6]. Edge/Path Counting Method Author Lin et al. [7] introduced edge-counting techniques. It was a very basic strategy for determining similarity, using “is-a” linkages. It computes the shortest path distance between the end-to-end connections in their connected ontological model [8]. The similarity between the terms or concepts tends to decrease as path length increases. Rada et al. [8] developed a similarity metric based on the shortest path approach. Even if the phrases or concepts at the bottom of the hierarchy have a more particular meaning, they may be more comparable to one another despite being separated by the same amount of distance. Information Content-Based The information derived from a concept is defined as IC [9]. IC values over 100 indicate the concepts are more specific and less specific with high IC value; the concept is less specific and less ambiguous with low IC value [9]. Resnik [10] proposed res as a measure of semantic similarity which is based on the notion that if two concepts have a shared subsumer, they have more information in common because the IC is shared by both ideas. Furthermore, Lin [11] suggested

668

P. Rathee and S. K. Malik

a modification of the res a metric that takes both into account terms IC values as well as their LCS, indicating the terms’ shared similarity. Jiang and Conrath [12] derive a distance measure from the aggregate of the IC values of the words and their LCS. If ontologies are meaningfully constructed, information count can be quantified, making use of underlying corpora or the ontology’s internal structure [13]. Because some terms may not be covered by a single ontology, several ontologies [14] can be used to determine their relationship to one another. Feature-Based Method Feature-based techniques are used to determine similarity as a function of word attributes such as gloss, neighboring words, and so on. A gloss is a dictionary definition of a term, whereas a glossary is a collection of glosses. Semantic measures come in a variety of forms. Word gloss has been offered as a foundation for similarity algorithms. Semantic similarity measures are based on vocabulary terms with more popular words in their gloss for words with similar meanings. The degree of overlap between words in a sentence analysis determines semantic similarity. The measure [11] is used to determine how closely two words are linked to each other. Hybrid Method Various similarity methods were proposed as a result of integrating various knowledge-based methodologies into one. In the method provided by Goa et al. [15] on the basis of the WordNet ontology, there are three possible strategies for adding weights to the edges. Zhu and Iglesias [16] propose a new weighted path metric that weights the shortest path length with the IC value of the least common subsumer. Lastra-Daz et al. [17] created the Half-Edge Semantic Measures Library program for implementing numerous proposed ontology-based semantic similarity measures (Fig. 3), demonstrating improved model performance time and scalability.

Fig. 3 Ontology relationships among similarity measures [8]

An Analysis of Semantic Similarity Measures …

669

Semantic similarity approaches based on knowledge are being developed and give a strong foundation for models, and the most basic ambiguity problem is easily addressed as synonyms, idioms, and phrases [4, 18]. The knowledge-based semantic similarity measurements are given in Table 1. Table 1 Knowledge-based semantic similarity measures Approach

Author name Ontology type

Formula used for similarity

Features

Edge/Path counting method

Rada [7]

Sim = 2 ∗ max −length(C1, C2)

Edges between the concepts are taken for similarity value

Edge/Path counting method

Leacock and Single Chodorow WordNet [16] ontology

Information Resnik [15] content method

Single WordNet ontology

WordNet single or cross ontology

Sim = The distance  among terms   max − log PLength(C1,C2) and depth is 2∗Depth used for similarity value Sim = IC(LCS(C1, C2)) = IC(C) =

log(depth(c)) log log(deep max)

Information content of common sub-summer is used

Feature-based

Tversky’s [8] WordNet, Sim = mesh, single C1∩C2    or cross C1 C1∩C2+i C2 +(i−1) C2 C1 ontology

Common and uncommon features are used for similarity

Hybrid method

Mostafa, Ehsan [15]

Single or cross ontology

–

Swarm optimization and weighted edge distance is used for similarity

Information Jiang [7] content method

Single WordNet ontology

Sim = IC(C1) + IC(C2) − 2 ∗ Simres(C1, C2)

LCS of concepts is used for similarity

Information Lin [17] content method

WordNet, Sim = mesh, single or cross ontology

Edge/Path counting method

Single WordNet Ontology

Wu and Palmer [7]

Sim =

2∗Simres(C1,C2) IC(C1)+IC(C2)

Ration info and Resnik concepts are used for similarity

2∗D d1+d2+2∗D

LCS and depth of concepts are used for similarity

670

P. Rathee and S. K. Malik

2.2 Corpus-Based Similarity Approaches In this approach, the similarity is measured between terms with large corporate information. “Similar words are often used in conjunction,” and the basic principle called the “distributive hypothesis” takes advantage of the idea. Words are not taken into account. During the construction of the different techniques of text data vector representation, several simultaneous distributional measurements to estimate the similarity between the vectors were proposed in a hypothesis. Mohammad and Hurst have conducted an extensive survey of various distributional semantics measures [19, 20]. However, the cosine similarity has taken on the importance and was widely used among all these measures. Latent Semantic Analysis LSA is a well-known and commonly used corpus methodology for determining semantic similarity. The columns form a word matrix, with the words representing paragraphs and the rows representing columns. Word counts are put into the cells. This matrix has a big corpus beneath it, and a mathematical approach known as singular value minimizes and dismisses dimensionality (SVD). SVD is made up of three matrices: two for rows and two for columns in their own values, and a third column matrix with values that reproduce when multiple. LSA models are generalized in that they are used to determine the degree of resemblance between sentences, paragraphs, and papers by swapping words with different text and columns. Hyperspace Analog to Language Co-occurrences of the word are used to shape a semantic space. The author in Ref. [21] creates a matrix, where both row and column are used to represent the word. The strength of the association of the corresponding column and row words is represented in every matrix element. When the text is analyzed, the focal concept is chosen and in comparison with neighboring concepts that are referred to as a co-occurring condition. The frequency of co-occurrence is related to the remoteness between the focal words and the matrix values. Dimensional decrease is possible selected by removing any columns possessing a low entropy. In addition, HAL concentrates on the written word patterns to deal differently with checking for co-occurrences whether occurred either before or after the term. Explicit Semantic Analysis This approach computes the similarity of two arbitrary texts [22]. Word meaning is represented by Wikipedia in weighted vector form. Each vector represents a different type of TF-IDF weighted relationship among a Wikipedia page and a term, and each of the vectors is mapped as large-dimensional vectors. To compute semantic similarity between vectors, the cosine similarity measure is utilized (Table 2).

An Analysis of Semantic Similarity Measures …

671

Table 2 Corpus-based semantic similarity measures Approach

Author name

Latent semantic analysis (LSA)

Scott Deerwester et al. Works with singular [14] value decomposition, reduces redundancy

Features

Dataset used

Explicit semantic analysis (ESA)

Evgeniy Gabrilovich et al. [23]

Works with machine Collection of 50 learning techniques, documents from news uses Wikipedia, which mail service contains a huge amount of human encoded knowledge

Generalized latent semantic analysis (GLSA)

Irina Matveeva et al. [5]

Efficiently captures semantic relations between terms

Corpus-based method

Ellen Riloff and Jessica shepherd [29]

Text corpus and a small Text corpus (MUC-4) set of seed words do not need additional semantic knowledge

Semantic corpus

Lubomir Stanchev [21]

WordNet can be used to WordSimilarity353 create a phrase graph, dataset high-quality structured knowledge

MED and CISI dataset

TOEFL, TS1, and TS2

3 Conclusion Measuring semantic similarity has proven to be a challenging task. Throughout the years, numerous approaches for evaluating semantic similarity have been created, and this analysis, study, and review cover various aspects, merits, and demerits of each. Methods based on the knowledge take the current situation into account but, they are rigid across disciplines as well as languages. An analysis of different type of ontologies over different measures is shown in Fig. 4. However, corpus-based approaches have a statistical analysis foundation and may be applied by making no distinction between languages and underlying meaning of concepts. According to the analysis, each strategy has benefits and drawbacks, making it difficult to select the best model; yet, the most recent hybrid techniques have demonstrated encouraging outcomes when compared to other standalone models. There are additional research gaps in areas such as design, which addresses the need for an optimal corpus through domain-specific word embedding.

672

P. Rathee and S. K. Malik

Fig. 4 Ontology structure with measures [8]

References 1. K. Saruladha, G. Aghila, Sajina Raj, A survey of semantic similarity methods for ontology based information retrieval, in 2010 International conference on IEEE Computer Society 2. A. Gupta, A. Kumar, A survey on semantic similarity measures. IJIRST –Int. J. Innov. Res. Sci. Technol. 3(12) (2017) 3. S. Anitha Elavarasi, J. Akilandeswari, K. Menaga, A survey on semantic similarity measure. Int. J. Res. Advent Technol. 2(3) (2014) 4. J. Gorman, J.R. Curran, Scaling distributional similarity to large corpora, in Proceedings of the 21st international conference on computational linguistics and 44th annual meeting of the association for computational linguistics (2006), pp. 361–368. 5. D. Sánchez, M. Batet, D. Isern, A. Valls, Ontology-based semantic similarity: a new feature based approach. Expert Syst. Appl. 39(9), 7718–7728 (2012). https://doi.org/10.1016/j.eswa. 2012.01.082 6. D. Chandrasekaran, V. Mago, Evolution of semantic similarity—a survey. 1(1), 35 (2020) 7. D. Lin, An information-theoretic definition of similarity, in Proceedings of ICML (1998), pp. 296–304 8. S. Jain, K.R. Seeja, R. Jindal, Identification of new parameters for ontology based semantic similarity measures. EAI Endorsed Trans. Scalable Inf. Syst. 6(20) (12 2018–03 2019) 9. D. Sánchez, M. Batet, A semantic similarity method based on information content exploiting multiple ontologies. Expert Syst. Appl. 40(4), 1393–1399 (2013). https://doi.org/10.1016/j. eswa.2012 10. P. Resnik, Using information content to evaluate semantic similarity in a taxonomy, in Proceedings of the 14th international joint conference on artificial intelligence, vol. 1 (1995), pp. 448–453 11. D. Lin et al., An information-theoretic definition of similarity. In Icml 98, 296–304 (1998) 12. J.J. Jiang, D.W. Conrath, Semantic similarity based on corpus statistics and lexical taxonomy, in Proceedings of the 10th research on computational linguistics international conference (1997), pp 19–33 13. D. Sánchez, M. Batet, D. Isern, Ontology-based information content computation. Knowl. Based Syst. 24(2), 297–303 (2011) 14. M. Andrea Rodríguez, M.J. Egenhofer, Determining semantic similarity among entity classes from different ontologies. IEEE Trans. Knowl. Data Eng. 15(2), 442–456 (2003)

An Analysis of Semantic Similarity Measures …

673

15. J.-B. Gao, B.-W. Zhang, X.-H. Chen, A WordNet-based semantic similarity measurement combining edge-counting and information content theory. Eng. Appl. Artif. Intell. 39(2015), 80–88 (2015). https://doi.org/10.1016/j.engappai.2014.11.009 16. G. Zhu, C.A. Iglesias, Computing semantic similarity of concepts in knowledge graphs. IEEE Trans. Knowl. Data Eng. 29(1), 72–85 (2017). https://doi.org/10.1109/TKDE.2016.2610428 17. J.J. Lastra-Díaz, A. García-Serrano, M. Batet, M. Fernández, F. Chirigati, HESML: a scalable ontology-based semantic similarity measures library with a set of reproducible experiments and a replication dataset. Inf. Syst. 66(2017), 97–118 (2017) 18. M.C. Lee, A novel sentence similarity measure for semantic-based expert systems. Expert Syst. Appl. 38(5), 6392–6399 (2011) 19. S.M. Mohammad, G. Hirst. Distributional measures of semantic distance: a survey (2012). arXiv preprint arXiv:1203.1858 20. J. Nivre, Inductive dependency parsing (Springer, 2006) 21. K. Lund, C. Burgess, R.A. Atchley, Semantic and associative priming in a high-dimensional semantic space. Cogn. Sci. Proc. (LEA) (1995), pp 660–665 22. E. Gabrilovich, S. Markovitch, Computing semantic relatedness using wikipedia-based explicit semantic analysis, in Proceedings of the 20th International Joint Conference on Artificial Intelligence (2007), pp. 6–12 23. Y. Jiang, X. Zhang, Y. Tang, R. Nie, Feature-based approaches to semantic similarity assessment of concepts using Wikipedia. Inf. Process. Manag. 51(3), 215–234 (2015)

Recommendation Engine: Challenges and Scope Shikha Gupta and Atul Mishra

Abstract On the internet, with the increased number of users, its use for recommending products and services is also increasing. But there is a need to filter that data and provides only relevant recommendations. Recommendation systems help in solving this problem by providing personalized recommendations from a large pool of data. This paper provides an overview of the recommendation system along with its various filtering techniques. The paper also discusses various challenges faced by the current recommendation systems and the possible research areas in this field that can improve its efficiency. Keywords Recommendation engine · Collaborative filtering · Content-based filtering · Hybrid filtering

1 Introduction In the recent years, with the increased use of the Internet, the mode of advertising is also changing. The focus is now changing towards the online medium. As the amount of data available on the Internet is too large leading to information overload, some tool is required which will help in finding the items of interest to the user—recommendation engine or recommendation system (RS) is one such tool for that. In the last few years, recommendation engine has gained a lot of interest among various industries like social networks, data mining, information retrieval, data engineering, machine learning, and many more [1]. RS has also attracted a lot of researchers as well mainly because of the challenges faced by it. Recommendation systems aim to provide personalized support/recommendations to the users. Most of the E-commerce sites use RS for suggesting relevant goods and services to the user which are most suitable to them [2]. RS uses product knowledge and the customer’s information to find the list of most suitable products from a big ocean of products available online. The Tapestry was the first RS, used for filtering a large number of S. Gupta (B) · A. Mishra JC Bose University of Science and Technology, YMCA, Faridabad, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_59

675

676

S. Gupta and A. Mishra

incoming documents [3]. But providing recommendations that best matches the user requirement is a challenging task. In the immortal words of Steve Jobs: “A lot of times, people don’t know what they want until you show it to them”. There are many techniques used by the RS. Some of them are discussed below in this paper [4]. (a) Content-based filtering: Content-based filtering is based on the recommendation of items based on the user history. Those items were recommended that were similar to the items liked by the user in the past. This technique is domaindependent and focuses on the item attributes for making predictions. The user profile is used for making recommendations as it uses the features extracted from the items that were evaluated by the user in the past. This approach can use various methods like probabilistic method, vector space model, decision trees, Bayes classifier, or neural networks for modelling the relationship between various documents [5]. (b) Collaborative filtering: Collaborative filtering works by recommending those items to the target user that other users with similar choices have liked in the past. This technique is domain-independent. In this approach, the recommendations are made by searching the users with similar profiles for finding common interests or choices. A group of similar users is made called a neighbourhood. Now, recommendations are based on the ratings of the users in the same neighbourhood. This approach can use various methods like Jaccard distance, cosine distance, normalizing ratings, or clustering. Collaborative filtering techniques can be broadly divided into two categories: model-based and memory-based [6]. (i) Memory-based: Memory-based algorithms predict the ratings by using the complete collection of previously rated items done by the user. It can be done in two ways: item-based and user-based techniques [5]. (ii) Model-based: In contrast, the model-based approach learns a model by using the collection of ratings, and then, the model is used to make predictions. (c) Hybrid filtering: Hybrid filtering uses the features of various recommendation techniques. It combines different techniques to take their advantage, and the limitations of one technique can be eliminated by the other. It will provide more effective and accurate recommendations. The combinations of various techniques can be done in many ways. Some of them are discussed below: (i)

(ii)

Weighted Hybridization: In weighted hybridization, the results of various recommenders are combined by integrating the scores generated from various techniques by a linear formula for making the predictions or recommendations. Switching Hybridization: In switching hybridization, switching between various approaches is done depending on the heuristic to make the best prediction. In this, problems specific to one method can be avoided.

Recommendation Engine: Challenges and Scope

677

(iii) Cascade Hybridization: In cascade hybridization, iterative refinement is done, i.e. the recommendations generated from one recommendation technique are refined by another technique. (iv) Mixed Hybridization: In mixed hybridization, recommendation results of various techniques are combined at the same time. Multiple recommendations associated with each item are combined. (v) Feature combination Hybridization: In feature combination hybridization, the features generated from one recommendation technique are fed into another technique. (vi) Feature augmentation Hybridization: In feature augmentation hybridization, a recommender system uses the ratings, additional functionality, and other information produced by the previous recommender. They are superior to feature combination hybridization. (vii) Meta-level Hybridization: In meta-level hybridization, the internal model produced by a recommender is fed as input into another recommendation technique. Table 1 shows a brief comparison of various recommendation techniques. This paper presents a brief overview of the recommendation engine with the various issues faced by it. There are 5 sections in this paper. In Sect. 2, a literature review is given. Section 3 contains a brief introduction of the latest research areas of the recommendation engine. In Sect. 4, this paper talks about the current challenges faced by the recommendation engine. Section 5 includes the future scope for improving these approaches and a brief conclusion of this paper. Table 1 Comparison of various recommendation techniques Approach

Content-based filtering

Collaborative filtering

Hybrid filtering

Logic

Based on user history

Based on liking of similar users

Based on combination of various approaches

Commonly used techniques

• • • • • • •

• Jaccard distance • Cosine distance • Normalizing ratings • Clustering • Nearest neighbour • Graph theory • Linear regression

• Weighted hybridization • Switching hybridization • Cascade hybridization • Mixed hybridization • Feature combination hybridization • Feature augmentation hybridization • Meta-level hybridization

Limitations

• Limited content analysis • Overspecialization • Cold start problem

TF-IDF Clustering Probabilistic method Vector space model Decision trees Bayes classifier Neural networks

• • • •

New user problem • Overspecialization New item problem • Scalability Sparsity Scalability

678

S. Gupta and A. Mishra

2 Literature Review The Tapestry was the first recommendation system for mailing system, used to refine a large number of incoming documents based on using the concept of usercollaboration filtering. There are a number of approaches for all three types of recommendation techniques. Firstly, talking about content-based filtering uses various approaches to generate the result. Term frequency/inverse document frequency (TF-IDF) measure used cosine similarity measure to calculate the utility function. Then, a Bayesian classifier was used which was based on a model learned from the underlying data instead of a heuristic formula. It uses machine learning and statistical learning techniques. But an assumption was made that the keywords are independent. Then, the adaptive filtering approach was introduced. It observes the document one by one to provide more relevant documents incrementally. After that, a threshold setting was introduced which helps in deciding that to what extent the documents should match a given query to be relevant [3]. The next approach is collaborative-based filtering, which is further divided into memory-based and model-based. In the memory-based approach, earlier, an aggregation function was used which aggregated the ratings of other users. It can be a simple average or weighted sum. But the issue with simple average is that it does not consider the fact that rating scale may be used differently by different users. To overcome this problem, the weighted sum approach uses the deviation from the average rating of the respective user. The similarity between the users can be calculated using a correlation or cosine-based approach. In the model-based approach, a probabilistic approach was used. It can use cluster models, where similar users are grouped together but with the limitation that a user can be clustered into a single cluster only. Another approach is the Bayesian network which represents an item in the domain as a node and the states of each node represent its possible rating values.

3 Latest Research Areas in Recommendation Engine There are a number of research areas for the recommendation engine. This section discusses some of them in brief. 1) Understanding the user better for increasing the sales of the service provider: Recommender systems are mostly used for commercial purposes. They help in increasing the sale by recommending the products or services which are of interest to the user. But gathering and understanding the user data to predict the user choices and behaviour is of concern [7]. 2) Exploitation vs. Exploration: During the recommendation, it should be considered that only relevant products are not recommended but diversity should be maintained. The balance between relevant and diverse top-recommended items is of major concern as it helps in improving the efficiency of the system.

Recommendation Engine: Challenges and Scope

679

4 Challenges It is evident from the various research that recommendation systems can help users in making better decisions with less effort. But still, there are various issues and challenges faced by the recommender system and some of them are discussed below in this section. 1) Cold start problem: Cold start problem generally arises when the user or the product is new, i.e. when there is no information about users’ preferences or the product is not rated by anyone. Collaborative filtering fails due to a lack of information. 2) Item churn: Sometimes the environment is dynamic, where the recommender system is implemented so, there is constantly adding and removing of items that are difficult to manage. 3) Short- and long-term preferences: The short- and long-term preferences of the users are different and they should be taken into account while recommending items. 4) Recommending the same items repeatedly: The frequency of recommending the same product is important. They found out that recommending the same product multiple times makes no sense and at the same time by recommending only once can miss interesting recommendations. So, the count of how many times a product should be recommended is an area of concern. 5) Negative feedback: Negative feedbacks are very rarely given by the users so, they are very important and should be considered. In the case of implicit feedback, not clicking an ad even after displaying it ‘n’ times can be considered as negative feedback. 6) Overspecialization: The recommender system should not recommend too similar products; some amount of diversity is required.

5 Future Scope and Conclusion The recommender system proves to be an important tool in providing personalized data to the user on the Internet. With its use, users’ access to online products and services have increased remarkably. This paper talked about the various types of approaches used by the recommendation engine. Although a lot of progress has been done by it, still some improvement is required to increase their efficiency. This paper discusses the various issues and challenges faced by the current approaches. It also discusses the various research areas in this field that need to be worked on in future.

680

S. Gupta and A. Mishra

References 1. L. Cao, Non-IID recommender systems: a review and framework of recommendation paradigm shifting. Engineering 2(2), 212–224 (2016). ISSN 2095-8099. https://doi.org/10.1016/J.ENG. 2016.02.013. (https://www.sciencedirect.com/science/article/pii/S2095809916309481) 2. S. Gupta, A. Mishra, Evolution of online marketing tools, approaches, and strategies with associated challenges: a survey. Int. J. Technol. Diffus. (IJTD) 12(3), 61–82 (2021). https://doi.org/ 10.4018/IJTD.2021070104 3. J. Myrberg, A recommender system for an online auction (2016). https://sal.aalto.fi/publicati ons/pdf-files/tmyr16_public.pdf 4. A. Rajaraman, J. Leskovec, J. Ullman, Mining of massive datasets (2014).https://doi.org/10. 1017/CBO9781139058452 5. F. Isinkaye, Y. Folajimi, B. Ojokoh, Recommendation systems: Principles, methods and evaluation. Egypt. Inform. J. 16https://doi.org/10.1016/j.eij.2015.06.005 6. G. Adomavicius, A. Tuzhilin, Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005). https://doi.org/10.1109/TKDE.2005.99 7. S. Gupta, A. Mishra, Big data analytics: approaches and applications in online marketing (July 12, 2021). Available at 2SSRN: https://ssrn.com/abstract=3884656 or https://doi.org/10.2139/ ssrn.3884656

Learning Impact of Non-personalized Approaches for Point of Interest Recommendation Rachna Behl and Indu Kashyap

Abstract Location-based social networks (LBSNs) have impacted our lives recently to a great deal. Typical location-based social networking sites have provision of declaring a check-in at a venue for users. Users can communicate this information with their friends, thus, generating huge dataset. This large geographical information has paved the way to build location-based recommender systems. Location recommendation service is an integral feature of location-based social networks. More and more users are using this service to explore new places and take timely and effective decisions. These systems provide a rich knowledge about a new place that a user has never visited before and also recommend interesting locations to the user after mining socio-spatial check-in data. In this paper, the authors present non-personalized techniques to utilize the check-in information for recommending popular and interesting locations to users. Background of location-based social networks and various techniques to develop location recommender system is discussed initially, followed by existing work and research issues of location-based recommender system. Authors have presented illustrative examples to mine the available spatial information of real-world location-based social network to suggest best interesting locations. Keywords POI · Generic recommendation · Information filtering · LBSN

1 Introduction Recommender systems are information filtering techniques that identify and present desirable items (products or services) to users (individuals or businesses) by predicting a user’s interest in an item based on relation between user and item information content [1]. Due to vast information overload, it is becoming a challenging task R. Behl (B) · I. Kashyap Manav Rachna International Institute of Research and Studies, Faridabad, India e-mail: [email protected]; [email protected] I. Kashyap e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9_60

681

682

R. Behl and I. Kashyap

for users to select appropriate items and take best decision. Recommender systems are helping people to make choices out of too many available alternatives. Thus, the main goal of developing recommender systems is to reduce information overload by filtering the most relevant information and services from vast available data, thereby providing personalized services. Recommender systems accomplish this by ‘guessing’ a user’s interests and liking after examining their profile [2, 3]. A general recommender system has following steps to identify top list of items: • • • •

Acquire the user-item preference data. Build user and item profile. Analyze the profiles to identify similarities between user and item. Present top-n list of items to user.

Figure 1 presents the steps of generating recommendations. With enhancements in location-based social network, the concept of location recommendation systems has been emerged. The goal of location recommender systems is to learn users’ implicit preferences according to users’ check-in history and provide users with new locations that user may be interested in. POI recommendation is a fascinating task of recommending new, captivating places to users. These systems have just boosted and researchers have developed keen interest in designing and developing better algorithms to generate accurate and effective recommendations. Recommendation could be personalized or generic/non-personalized based on whether user’s history is considered or not. Fig. 1 Steps of generating recommendations

Acquire the Data

Build User/ItemProfiles

Analyze the Profiles

Generate Top-n Recommendations

Learning Impact of Non-personalized Approaches …

683

1.1 Basic Recommendation Approaches Recommendations can be personalized or non-personalized depending upon the approach that is utilized in recommender systems.

1.1.1

Personalized Approach

Personalized recommender system generates recommendations taking into consideration the user’s personal preferences and his previous patterns. They are based on user’s personal choices and what items are liked/visited/purchased by a user in the past [4]. For example, in Google News, personalization system users are recommended news based on their clicks. When a user clicks on a news article that is considered as a positive rating for news article. This is an example of implicit ratings, as it is not specified explicitly by the user. On the other hand, an example of explicit rating is ratings in Amazon on a 5-point scale, to quantify the likeliness of users toward items. Least liked products are rated as 1-star, and the highest rating being 5-star.

1.1.2

Non-Personalized Approach

Personalized recommender systems are the most popular and effective, being able to predict products as per the taste of a person [3–5]. However, they demand lot of data well in advance. Content-based recommenders are the choice in situations when item attributes do not change. Moreover, these systems perform extremely well when user contextual information is available. However, sometimes the user data may not be possible to gather because of security constraints or other reasons. This is where non-personalized recommendations are preferred. Being the simplest one, non-personalized recommender systems do not rely on individual preferences and do not consider their historical behavioral patterns. Rather, these provide with general recommendations to the user determined from overlying consumer groups and product categories. In e-commerce site, Amazon, for example, users are shown list of most popular items when they visit the site. Popularity of items is computed on different parameters like geography, age, sex, etc. Moreover, non-personalized recommendations are sorted, such that newest items are being shown first. Figure 2 demonstrates the key differences between personalized and non-personalized recommender systems. The main contributions of the present work are summarized as 1. Commonly utilized traditional non-personalized approaches have been scrutinized to identify their limitations for point of interest recommendation. 2. Preprocessing has been applied on the dataset, and the approaches have been implemented to study their advantages and shortcomings in various scenarios.

684 Fig. 2 Personalized versus non-personalized

R. Behl and I. Kashyap Personalized RecommendationApproach

Non Personalized Recommendation Approach

Personal Preference

Generic Recommendation

Demand Historical Data

Rely on Overlying Consumer Groups

Implicit/Explicit Ratiings

Popularity Based

2 Existing Work Existing work in traditional recommender systems focused on generic and personalized approaches. In generic approaches, POIs are commonly ranked by scores computer using their popularity and top best items are recommended [6–8]. Reference [8] proposed a link analysis-based recommendation system for recommending experienced users and popular locations. But the system is unable to provide personalized recommendations and is preferred only in cases, where generic recommendations are anticipated. Reference [9] developed loosely coupled non-personalized and personalized module to attract tourists to the popular and interesting locations. Reference [10] has extensively studied recommendation systems and applied deep learning algorithms, later developed collaborative deep learning (CDL) model. In the proposed system, the authors integrated stacked denoising autoencoder and simple latent factor-based collaborative filtering model for providing movie recommendation. However, in this proposed model, only rare users and implicit interactions between users and items have been accentuated, and a very basic CF model is considered. In CDL, top-n items have been emphasized which are not applicable for the explicit rating prediction. Auxiliary information is defined as the set of attributes of users and items. For user attributes, for more concise prediction, raw ratings are included with the trust network. References [11, 12] worked on user social tag data and design a diffusion-based recommendation algorithm which may be utilized for social tagging recommendation. Reference [13] selected users’ demographic data and utilized aggregated weighted ratings made by similar users to generate ratings for new users. Reference [14] amalgamated attribute selection and local learning into the recommendation model for cold start users. References [13, 14] used the ToU approach in models to recommend products to users with very few profile. To learn and build user profiles, [15] initiated an additional interview process. For item attributes, collaborative topic modeling (CTR) [16] applies topic model and latent Dirichlet allocation (LDA) to learn item content feature. However, this model only

Learning Impact of Non-personalized Approaches …

685

focused on implicit rating and fails to learn latent representation with highly sparse content information.

3 Methodology Non-personalized recommendations goal is to help users decide if they would visit any place or not. Recommendation scores are computed whenever a user searches for a product or any location. Figure 3 shows the request flow for a query made by a user and interaction with recommendation engine. These recommendations could be generated by applying various techniques. In this section, we describe couple of algorithms that we studied for generic recommendation problem. Visiting Popular Venues: This method arranges each user’s unvisited venues by calculating their popularity using following techniques. (a) Mean rating: Mean rating = (total ratings of item)/(Number of Ratings of item)

(1)

Average rating works fine if you always have a ton of ratings, however, averages can be misleading if very few persons have rated any item. For example, Fig. 3 Recommendation engine request flow

User Queries/searches for an Item (Venue, Book, Music)

Recommendation engine runs Learning Algorithm

Generate Scores and Find Most Popular Similar Items and their Scores

Return Scores and Top-n Products to User

686

R. Behl and I. Kashyap

it could be that an item has got average rating of 5-stars because only one person really likes that item. We may be less confident of whether a good rating provided to an item is actually good or not by using mean rating. (b) Damped Mean: Damped mean is solution to the problem that arises when items have very few ratings. The damped mean solution assumes that, every item is average without any evidence/ratings. That is, this will not allow any item to be rated positive on the basis of only few good ratings until large number of good ratings is not provided. To achieve this, modification in the formula of mean is done as shown in Eq. 2: ∑ s(i ) =

r ui + αμ |U i| + α

u∈U i

(2)

where α controls the strength of the evidence. (c) Rating Count (popularity): This method calculates popularity by counting how many people have rated any item. (d) Popularity by percent liking: This method calculates the percentage of ratings for each item that are 4 or higher. This is shown in Eq. 3. Popularity by percent liking = (count of ratings ≥ 4 of an item)/total number of ratings (3) Visiting associated locations: Location association recommender is one of the techniques of ‘non-personalized recommender systems’. It is classified as nonpersonalized because it does not consider user’s choice or their past patterns. Also recommendations do not consider item’s characteristics or attribute information; rather, recommendations rely on user’s context. This is short-term in nature as it depends on user’s current pattern or behavior, e.g., the recommendation for next item will completely rely on what a customer just added into the cart. Using item association, the score can be computed using the Affinity score or lift value as show in Eqs. 4 and 5. P(Y |X ) P(Y )

(4)

P(Y X ) P(Y ).P(X )

(5)

Affinity Score = Lift Value =

Learning Impact of Non-personalized Approaches …

687

Fig. 4 User-venue frequency count

4 Experimental Setup and Results In this section, we analyze the check-in ratings data to generate non-personalized recommendations. All experiments were performed in Python and on real-world foursquare dataset as mentioned in [17].

4.1 Dataset Foursquare dataset: Dataset used for our study is accessed from Refs. [17–19]. This dataset has check-ins of New York City from April 12, 2012 to February 16, 2013. This dataset contains 227,428 check-ins in New York City made by 1083 users to 38,333 venues. Each check-in has timestamp, geo-coordinates, and category associated with it. We preprocessed it and fetched the frequency count of check-ins to a venue by a user. The resultant dataset is shown in Fig. 4. The next step was to apply the techniques and generate top 10 venues according to mean ratings, count, location association, etc., we also calculated correlations of one venue with all other venues to check the strength of each venue with chosen one. A snippet of top 10 venues as per mean ratings and corresponding graph is shown in Figs. 5 and 6. Recommendations generated using damped mean and its corresponding graph are shown in Figs. 7 and 8. Figure 9 shows top 5 venues according to popularity count, and Fig. 10 displays venue % ratings ≥4. Following Fig. 11 shows venues that are visited whenever venue with id ‘v2291’ is visited by any user.

5 Conclusions Recommender systems are the backbone of every online platform. Be it an online retail platform like AMAZON or movie service provider like NETFLIX, or Point of Interest services like Foursquare, all are investing lot on recommender system to assist the customers to explore more products and subsequently earn more profits.

688

R. Behl and I. Kashyap

Fig. 5 Top 10 venues (mean ratings)

Fig. 6 Top 10 venues (mean ratings) graph

This paper focuses on building non-personalized point of interest recommendation for location-based service like Foursquare. These are generic recommendations and are preferred over personalized, when few ratings are available. We implemented various techniques and bring out the differences between each technique. Through this paper, we generated non-personalized recommendation and suggested top the 10 locations to users based on average mean ratings, popularity count, and percent

Learning Impact of Non-personalized Approaches …

689

Fig. 7 Top 10 venues (damped mean)

Fig. 8 Top 10 venues (damped mean) graph

linking with ratings greater than four and many more. Mean rating is easy to implement and effective. However, it does not always assure actual taste of an individual. Generating recommendations using popularity count may recommend a point of interest even if it receives negative rating. Other method ignores the rating below a threshold and might result in losing a pretty useful rating. A variety of aggregation techniques of various types has been developed so far. However, they only recognize one aspect of overall assessment (for example, frequency counts, high averages, percent likings, etc.), which imposes some limitations in capturing a user’s interest.

690

R. Behl and I. Kashyap

Fig. 9 Top 10 venues (popularity count) graph

Fig. 10 Top 100 venues (score with % ratings ≥4)

In the future, hybridized aggregation technique may be developed to strengthen the existing non-personalized approaches.

Learning Impact of Non-personalized Approaches …

691

Fig. 11 Top 100 venues (location association) graph

References 1. J. Bobadilla, F. Ortega, A. Hernando, A. Gutiérrez, Recommender systems survey. Knowl. Based Syst. 46, 109–132 (2013) 2. P. Resnick, H.R. Varian, Recommender systems. Commun. ACM 40(3), 56–58 (1997) 3. J. Bao, Y. Zheng, D. Wilkie, M. Mokbel, Recommendations in location-based social networks: a survey. GeoInformatica 19(3), 525–565 (2015) 4. H. Li, Y. Ge, R. Hong, H. Zhu, Point-of-interest recommendations: Learning potential check-ins from friends, in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (2016), pp. 975–984 5. T. Silveira, M. Zhang, X. Lin, S. Ma, How good your recommender system is? A survey on evaluations in recommendation. Int. J. Mach. Learn. Cyber. 10, 813–831 (2019). https://doi. org/10.1007/s13042-017-0762-9 6. X. Cao, G. Cong, C.S. Jensen, Mining significant semantic locations from gps data, in Proceedings of the VLDB Endowment, vol. 3, no. 1–2 (2010), pp. 1009–1020 7. P. Venetis, H. Gonzalez, C.S. Jensen, A. Halevy, Hyper-local, directions-based ranking of places, in Proceedings of the VLDB Endowment, vol. 4, no. 5 (2011), pp. 290–301 8. Y. Zheng, L. Zhang, X. Xie, W.Y. Ma, Mining interesting locations and travel sequences from GPS trajectories. In WWW (2009), pp. 791–800 9. A. Smirnov, A. Ponomarev, A. Kashevnik, Tourist attraction recommendation service: an approach, architecture and case study, in International conference on enterprise information systems, vol. 3 (SCITEPRESS, 2016), pp. 251–261 10. H. Wang, N. Wang, D.Y. Yeung, Collaborative deep learning for recommender systems, in Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (2015), pp. 1235–1244 11. P. Victor, C. Cornelis, M.D. Cock, Key figure impact in trust-enhanced recommender systems. AI Commun. 21(2), 127–143 (2008) 12. Z.K. Zhang, C. Liu, Y.C. Zhang, T. Zhou, Solving the cold-start problem in recommender systems with social tags. EPL Europhys. Lett. 92(2), 28002 (2010) 13. B. Lika, K. Kolomvatsos, S. Hadjiefthymiades, Facing the cold start problem in recommender systems. Expert Syst. Appl. 41(4), 2065–2073 (2014) 14. U. Ocepek, J. Rugelj, Z. Bosni´c, Improving matrix factorization recommendations for examples in cold start. Expert Syst. Appl. 42(19), 6784–6794 (2015)

692

R. Behl and I. Kashyap

15. K. Zhou, S.H. Yang, H. Zha, Functional matrix factorizations for cold-start recommendation, in Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval (2011), pp. 315–324 16. C. Wang, D.M. Blei, Collaborative topic modeling for recommending scientific articles, in Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (2011), pp. 448–456 17. D. Yang, D. Zhang, V.W. Zheng, Z. Yu, Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man Cybern. Syst. 45(1), 129–142 (2015). https://doi.org/10.1109/TSMC.2014.2327053 18. B.C. Chen, D. Agarwal, Recommender problems for web applications, in Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (2010), pp. 1–1 19. M. Pazzani, D. Billsus, Content-based recommendation systems. Adapt. Web 4321, 325–341 (2007)

Author Index

A Abeyrathne, Avishka Heshan, 287 Adinarayna, S., 137, 611 Agrawal, Alka, 3 Agarwal, Nidhi, 181 Ahmed, Syed Irfan, 89 Aljohani, Abeer A., 313 Amarasinghe, D. A., 115 Anbarasan, M., 243, 577 Ansari, Md Tarique Jamal, 3 Ashoda, Chinthaka, 487

B Babu, P. Haran, 125 Babu, Siva Surya, 89 Bai, Pinky, 215 Banerjee, Shobhan, 393 Bankar, Shashikant Mahadu, 651 Bansal, Abhay, 537 Batra, Usha, 151 Behl, Rachna, 681 Beski Prabaharan, S., 427 Bhanusri, B., 477 Bhatia, Kamaljit Singh, 457

C Chachoo, Manzoor Ahmad, 529 Chadha, Harshita, 303 Chatterjee, Siddhartha, 255 Chaurasia, Ankur, 537 Chawla, Pronika, 599 Chhabra, Kashish, 467 Chinmay, Behera, 27

Chirag, 467 Chowdhury, Atiqul Islam, 277 Chowdhury, Bushra Rafia, 383

D Dash, Satya Ranjan, 367 Das, Soumen, 255 Debnath, Narayan C., 345, 403 Deep, Prakhar, 181 Deshpande, Sujit, 47 Deshpande, Vivek, 623 Dey, Sudeepa Roy, 89 Dham, Hridya, 599 Divya, S., 89 Dohare, Upasana, 215 Dubey, Tushar, 599 Dusane, Suprit, 635 Dutta, Soumi, 255

F Ferdosi, Bilkis Jamal, 383

G Gamage, N. D. U., 115 Gamage, T. C. T., 115 Garg, Neha, 13 Gayathri, S., 243, 577 Geetha Ramani, R., 519 Ghosal, Sayani, 589 Gore, Deipali Vikram, 623 Gunathilake, Prasadini Ruwani, 287 Gupta, Archana, 269

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 P. Dutta et al. (eds.), Emerging Technologies in Data Mining and Information Security, Advances in Intelligent Systems and Computing 1348, https://doi.org/10.1007/978-981-19-4676-9

693

694 Gupta, Ayush, 269 Gupta, Gourav, 13 Gupta, Prashant K., 537 Gupta, Sanjeev Kumar, 651 Gupta, Shikha, 675 Gupta, Sunanda, 413 Gusain, Raj, 103

H Haque, Md. Mokammel, 169 Herath, Anuradha Sadesh, 287 Hrushikesava Raju, S., 611

I Iliyaz, Md, 635 Immaculate Rexi Jenifer, P., 335 Islam, Md Khairul, 553 Islam, Md Nurul, 193 Islam, Shahidul, 231 Islam, Towhidul, 383

J Jadhav, B. T., 503 Jain, Amita, 589 Jamwal, Sanjay, 231 Janardhan Rao, S., 611 Jayasinghe, D., 115 Jemulesha, Shaik, 611 Jeslin Shanthamalar, J., 519 Jeyapiriya, K., 243, 577 Jogdand, Rashmi, 47 Jos, Dereck, 511

K Kakkar, Mohit Kumar, 13 Kalai Selvi, T., 335 Kanade, Vijay A., 207 Kashyap, Indu, 681 Kaur, Amandeep, 553 Kaur, Simarpreet, 457 Kaushal, Chetna, 553 Khadke, Kunal S., 355 Khandelwal, Kunal, 599 Khan, Qamar Rayees, 231 Khan, Raees Ahmad, 3 Kieu, Manh-Kha, 345, 403 Krishan, Abhilash, 487 Krishna, Gopika, 323 Kumar, Sushil, 215, 445, 565 Kundu, Kumud, 181

Author Index L Le, Ngoc-Bich, 345, 403 Le, Ngoc-Huan, 345, 403

M Madan, Deeksha, 303 Madhubabu, Kotakonda, 125 Madhumali, Dilini, 487 Mahadik, Omkar, 511 Makandar, Aziz, 67 Malik, Sanjay Kumar, 665 Mamun, Khondaker A., 277 Manhas, Pratima, 413 Manikandan, S., 335 Mani, Prashant, 77 Mariam Bee, M. K., 477 Minz, Sonajharia, 445 Mir, Mahmood Hussain, 231 Mishra, Atul, 675 Mishra, S. K., 27 More, Sharmila S., 503 Murthy, Akash, 89

N Nachappa, M. N., 427 Nair, Manjusha, 643 Naqvi, S. K., 193 Narad, Priyanka, 537 Narain, Bhawna, 503 Natarajan, Harish, 511 Nguyen, Duc-Canh, 345, 403 Nguyen, Vu-Anh-Tram, 345, 403 Nguyen, Xuan-Hung, 345, 403 Ninh, Tran-Thuy-Duong, 345, 403

P Parida, Shantipriya, 367 Pathak, Nitish, 59 Pattabiraman, Lalitha, 59 Phan, Minh-Dang-Khoa, 345 Pradeep, Gayan, 487 Prakash, Rishi, 103 Prasad, Sanjeev Kumar, 181 Prasanna, M. Merrin, 137 Pratama, Mahardhika, 537

Q Quadri, S. M. K., 193

Author Index R Rajesh, S., 243, 577 Raju, S. Hrushikesava, 125, 137 Rana, Deepika, 303 Rankothge, W. H., 115 Rao, P. Venkateswara, 137 Rathee, Preeti, 665 Rather, Ishfaq Hussain, 445 Rath, Manas Kumar, 393 Rathnasekara, Prasanna Vikasitha, 287 Refat, Md Abu Rumman, 553 Reno, Saha, 169

S Sagar, Aman Kumar, 467 Saibaba, CH. M. H., 137 Samantaray, Annapurna, 367 Samanta, Tapaswini, 393 Sarkar, Debasree, 255 Sarker, Soumen, 553 Sarmah, Kshirod, 437 Sha, Akhbar, 643 Shafia, 529 Shahare, Yogesh, 511 Sharma, Aditi, 367 Sharma, Neelam, 303 Sharma, Tripti, 181 Sheoran, Kavita, 467 Shukla, A. K., 103 Shweta, Kombe, 27 Singla, Jajji, 13 Sinha, Ashish Kumar, 623 Soni, Kritika, 599 Srivastava, Mohit, 457

695 Suresh, A., 635 Suresh, Sweety, 323 Swain, Prasanta Kumar, 393 Swain, Tanmaya, 393

T Thakral, Shaveta, 413 Thelijjagoda, Samantha, 287 Thushara, M. G., 323 Tiwari, Anupam, 151

U Uwanpriya, S. D. L. S., 115

V Velvizhi, V. A., 243, 577 Verma, Jay Prakash Narayan, 77 Verma, Jyoti, 413 Vidyarthi, Anurag, 103 Vidyullatha, P., 125 Vignesh, N. Arun, 125 Vivekanandhan, V., 335

W Wangi, Kanchan, 67

Y Yaduwanshi, Ritesh, 565 Yogeswara Rao, K., 611 Youki, Ravina Akter, 383