Recent Trends in Signal and Image Processing: ISSIP 2020 (Advances in Intelligent Systems and Computing) 9813369655, 9789813369658

Citation preview

Advances in Intelligent Systems and Computing 1333

Siddhartha Bhattacharyya · Leo Mršić · Maja Brkljačić · Joseph Varghese Kureethara · Mario Koeppen   Editors

Recent Trends in Signal and Image Processing ISSIP 2020

Advances in Intelligent Systems and Computing Volume 1333

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Nikhil R. Pal, Indian Statistical Institute, Kolkata, India Rafael Bello Perez, Faculty of Mathematics, Physics and Computing, Universidad Central de Las Villas, Santa Clara, Cuba Emilio S. Corchado, University of Salamanca, Salamanca, Spain Hani Hagras, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK László T. Kóczy, Department of Automation, Széchenyi István University, Gyor, Hungary Vladik Kreinovich, Department of Computer Science, University of Texas at El Paso, El Paso, TX, USA Chin-Teng Lin, Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan Jie Lu, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia Patricia Melin, Graduate Program of Computer Science, Tijuana Institute of Technology, Tijuana, Mexico Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro, Rio de Janeiro, Brazil Ngoc Thanh Nguyen , Faculty of Computer Science and Management, Wrocław University of Technology, Wrocław, Poland Jun Wang, Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing such as: computational intelligence, soft computing including neural networks, fuzzy systems, evolutionary computing and the fusion of these paradigms, social intelligence, ambient intelligence, computational neuroscience, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, Web intelligence and multimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results. Indexed by DBLP, EI Compendex, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and Technology Agency (JST), SCImago. All books published in the series are submitted for consideration in Web of Science.

More information about this series at http://www.springer.com/series/11156

Siddhartha Bhattacharyya · Leo Mrši´c · Maja Brkljaˇci´c · Joseph Varghese Kureethara · Mario Koeppen Editors

Recent Trends in Signal and Image Processing ISSIP 2020

Editors Siddhartha Bhattacharyya Rajnagar Mahavidyalaya Birbhum, West Bengal, India Maja Brkljaˇci´c Algebra University College Zagreb, Croatia

Leo Mrši´c Research and Development Algebra University College Zagreb, Croatia Joseph Varghese Kureethara CHRIST (Deemed to be University) Bengaluru, Karnataka, India

Mario Koeppen Graduate School of Creative Informatics Kyushu Institute of Technology Fukuoka, Japan

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-981-33-6965-8 ISBN 978-981-33-6966-5 (eBook) https://doi.org/10.1007/978-981-33-6966-5 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

The editors would like to dedicate this book to those deceased and infected due to the ongoing COVID-19 pandemic.

Preface

Of late, computational intelligence has been the driving force behind almost every technological innovation in the present times. The ignoble presence of computational intelligence methods in different spheres of human civilization can be attributed to the rapid exploration of research in this direction. The versatility of computational intelligence-based techniques has enabled applications of intelligence to several interdisciplinary areas of science, technology, medicine and business in the likes of signal processing, smart manufacturing, predictive control, robot navigation, smart cities, sensor design, to name a few. Analysis and understanding of information in the form of signals and images have assumed utmost importance in recent times given the vast amount of data and signals generated across all platforms and applications. These signal data emanate from several sources which include speech, audio, images, video, sensor data, telemetry, electrocardiograms or seismic data among others. The related application areas include transmission, display, storage, interpretation, classification, segmentation or diagnosis. In an effort to effectively process and analyze different types of signals encountered in day-to-day life, scientists and researchers are continuously investing their efforts to evolve efficient algorithms which, besides being robust and failsafe, are also intelligent enough to stand out in varied conditions and parameter settings. The classical techniques in vogue are limited by some stringent requirements as regards knowledge of signal strengths and degrees of channel noises affecting the signalto-noise ratio. In order to overcome these limitations of the classical techniques, computational intelligence-based methods have proved to be promising enough to cater to the evolving amount of signal and image data in various incarnations. The 2020 Third International Symposium on Signal and Image Processing (ISSIP) is one such attempt in this direction. This volume, which constitutes the proceedings of ISSIP 2020, comprises 16 well-versed papers/chapters which have been accepted and presented in the symposium. COVID-19 is a pandemic that breaks out through the world and has a high mobility to transfer between humans. Developing intelligent bioinformatics tools is mandatory to aid in the analysis of the disease. One of these tools is local aligner which aims to find the longest common subsequence between two biological sequences. Fragmented Local Aligner Technique (FLAT) is developed based on meta-heuristic vii

viii

Preface

algorithms to accelerating the alignment process, especially for sequences with huge length. In Chapter “Chaotic Ions Motion Optimization (CIMO) for Biological Sequences Local Alignment: COVID-19 as a Case Study,” the performance of Ions Motion Optimization (IMO) algorithm for implementing FLAT has been measured. Further, a chaotic parameter was added in the exploration equations of IMO to enhance its performance for FLAT when applied on a dataset of a set of real proteins having a product length ranging from 250,000 to 9,000,000. With the increase of multimedia devices on the Internet, an enormous number of videos are being added as part of digital content. There is a huge challenge in the retrieval of these videos as mostly the videos are kept in unstructured form. Intended users try to retrieve video content as per the relevancy and need. Shot boundary detection is a significant and critical approach in the domain of digital video processing. It is the foremost critical job of content-based video retrieval and indexing. In Chapter “Differential Evolution-Based Shot Boundary Detection Algorithm for Content-Based Video Retrieval,” a novel approach for shot boundary detection algorithm based on differential evolution algorithm (DE) with SVM classifier is proposed. In this method, the authors first calculate the curve difference of U-component histograms as the feature of difference between video frames. In the next step, slide-window mean filter has been used to filter difference curves and SVM classifier has been applied with DE to detect and classify the shot transitions. The well-known TRECVID 2005, 2006 and 2007 datasets have been used to test the performance of our proposed approach. The result shows the superior performance of our approach, and this method can achieve high recall, precision rate, accuracy and better computation time. Thresholding of hyperspectral images is a tedious task. The interactive information value between three bands is used to reduce the redundant bands in the pre-processing stage. Chapter “Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding” presents a qutrit-inspired genetic algorithm for thresholding the minimized hyperspectral images with improved quantum genetic operators. In this chapter, a quantum disaster operation is implemented to rescue the qutrit-inspired genetic algorithm from getting stuck into local optima. The proposed algorithm produces better results than classical genetic algorithm and qutrit-inspired genetic algorithm in most of the cases. Among the solid components of the human blood, the type that has the largest number of them is well known as red blood cells (RBCs). These cells have flatrounded shapes, where their centers are depressed like a donut missing its hole. When the cell shape is changed from circular to sickle, then this case is a blood disease named sickle cell anemia (SCA). Based on its number, the dangerous level is obtained. Chapter “Employing Parallel Hardware Architectures to Diagnose Sickle Cell Anemia in Real-Time Basis” employs parallel hardware architectures to detect the sickle cells and its dangerous level in a real-time basis. These parallel architectures include the field-programmable gate array and the graphical processing unit (GPU). In addition, the central processing unit (CPU) as the common serial architecture is also employed for comparison basis in terms of time-consuming and power consumption. The circular Hough transform (CHT) method is employed for detecting the sickle

Preface

ix

cells. To determine the dangerous level, the number of sickle cells and the number of normal ones are counted. The detection, counting and classification algorithms are all coded in the Verilog language (for the FPGA) and in the MATLAB software (for the GPU and CPU). The findings have achieved well-behaved performances, and acceptable results are obtained. For recognizing vegetable and fruit maturing, various techniques have been released in the last two decades. These techniques have different accuracies and are generally time-consuming. To speed up the recognition performance to be suitable for real-time basis, a hardware accelerator is needed to implement. Chapter “Implementing a Hardware Accelerator to Enhance the Recognition Performance of the Fruit Mature” introduces a field-programmable gate array (FPGA), as a parallel hardware architecture, to solve the problem of time-consuming. Moreover, color threshold and k-means clustering are two techniques utilized for recognition purposes and for comparison principles. The findings showed that the color threshold technique required 16% of the total logic elements and performed the recognition task in 10.25 ms. In contrast, the k-means clustering technique required 62% of the logic elements and performed the recognition task in 64.88 ms. Thus, color threshold technique is more efficient and much faster than the k-means technique. Managing overcrowding with fluctuating patient arrivals in emergency department (ED) of hospitals requires a quantitative approach to make decisions related to resource planning and deployment by hospital administrators. In this context, analyzing patient flow and predicting demand will enable better decision making. In Chapter “Time Series Modelling and Forecasting of Patient Arrivals at an Emergency Department of a Select Hospital,” 7748 ED arrivals were recorded from a multi-specialty hospital in Bengaluru. The patient flow in each of the working shifts of the ED was analyzed separately. Time series modeling techniques have shown to be useful in generating short-term forecasts. Shift-wise modeling approach has been used since hospital resources were planned according to the shifts. Exponential smoothing techniques proposed by Hyndman was used in this study. Model validation was further carried out along with residual analysis. The prediction intervals shift-wise have been obtained with an average confidence level of 90% which will help hospital management to redeploy resources and handle demand with increased operational efficiency. The morbidity rate of breast cancer is among the highest exhibited by all forms of known cancer. It accounts for a high mortality rate in women. Detection in the early stages and corresponding attempts to treatment can help avert the fatality. The most common method used for the screening of breast cancer is mammography. However, the isolation of the breast lesion using this method is quite difficult and requires highly skilled radiologists. The most cardinal form of breast cancer is invasive ductal carcinoma (IDC) where the malignant growth spreads over to the breast fatty tissue after originating from the ducts. The difficulty in the analysis of images for invasive ductile carcinoma and the time spent on the diagnosis give rise to the urgent need for an accurate computer-aided diagnosis system. In Chapter “Deep Convolutional Neural Network-Based Diagnosis of Invasive Ductal Carcinoma,” the authors use a deep convolutional neural network (DCNN) for the automating the detection of

x

Preface

invasive ductal carcinoma in the early stages. This would serve as an efficient tool for assisting radiologists in the decision-making process and further save more lives in the long term. Chapter “Speaker Identification in Spoken Language Mismatch Condition: An Experimental Study” describes the impact of spoken language variation in a multilingual speaker identification (SID) system. The development of speech technology applications in low-resource languages (LRL) is challenging due to the unavailability of proper speech corpus. This chapter illustrates an experimental study of SID on Eastern and NorthEastern (E&NE) Indian languages in language mismatch conditions. For this purpose, several experiments are carried out using the LRL data to build speaker identification models. Here, spectral features are explored for investigating the presence of speaker-specific information. Mel-frequency cepstral coefficients (MFCCs) and shifted delta cepstral (SDC) are used for representing the spectral information. Gaussian mixture model (GMM) and support vector machine (SVM)based models are developed to represent the speaker-specific information captured through the spectral features. Apart from that, to build the modern SID i-vectors, time-delay neural networks (TDNNs) and recurrent neural network with long shortterm memory (LSTM-RNN) have been considered. For the evaluation, equal error rate (EER) has been used as a performance matrix of the SID system. Performances of the developed systems are analyzed with native and non-native corpus in terms of speaker identification (SID) accuracy. The best SID performances are observed to be EER 10.52% after the corpus fusion mechanism. B-mode ultrasound imaging is a popular medical imaging technique. Like other image processing tasks, deep learning has been used for analysis of B-mode ultrasound images in the last few years. However, training deep learning models requires large labeled datasets, which is often unavailable for ultrasound images. The lack of large labeled data is a bottleneck for the use of deep learning in ultrasound image analysis. To overcome this challenge, the authors exploit auxiliary classifier generative adversarial network (ACGAN) in Chapter “Ultrasound Image Classification Using ACGAN with Small Training Dataset” that combines the benefits of data augmentation and transfer learning in the same framework. We conduct experiment on a dataset of breast ultrasound images that shows the effectiveness of the proposed approach. The present age is the age of technology. People are increasingly using technology in their daily activities. That is why when we go to consider people’s attention to something, our head detection comes first. Detecting the head is not the only thing that ends here; the thing that is directly related to it is the eyeball movement. For example, when a student is studying, the level of attention means his concentration deeply. And the depth of this attention depends not only on the head movement but also on how many times his eyes are moving from left to right or from right to left. Because it is seen that looking at the same object at a glance does not mean that he is not paying attention, maybe he is thinking of something else or is immersed in another thought. And by using this motivation, eyeball and head movements play a vital role in Chapter “Assessment of Eyeball Movement and Head Movement Detection Based on Reading” which presents a system to read the video frames and

Preface

xi

determine the number of eyeball and head movements in real time. Eyeball and head movement from left to right and right to left is counted per minute. After one minute, the previous data will be refreshed, and new data will be recorded for the next minute. Thus, the system will give us the result of each minute movement numbers, and very nicely, our system can detect eyeball and head movement in case of reading. In Chapter “Using Hadoop Ecosystem and Python to Explore Climate Change,” the data storing and processing capabilities of Apache Hadoop ecosystem and its components—Hive, Impala and Sqoop—are demonstrated. Also, it demonstrates Python programming language’s capabilities in data analysis, plotting and statistical computing. It does so by exploring climate change problem, one of today’s most relevant and detrimental problems. Apache Sqoop was employed to migrate data from RDBMS system and store it into Hive database, where Hive and Impala were used for data processing and ELT. Finally, data were analyzed using Python, showing strong evidence for global warming presence, as well as exploring the relationship between carbon dioxide (CO2 ) emissions and climate change. Rule extraction is a process of extracting rules which helps in building domain knowledge. Rules plays an important role in reconciling financial transactions. Chapter “A Brief Review of Intelligent Rule Extraction Techniques” presents a brief study of intelligent methods for rule extraction. The chapter touches upon heuristic, regression, fuzzy-based, evolutionary and dynamic adaptive techniques for rule extraction. This chapter also presents the state-of-the-art techniques used in dealing with numerical and linguistic data for rule extraction. The objective of the chapter is to provide directional guidance to researchers working on rule extraction. Features of skin cancer have a certain impact on computer-aided diagnosis (CAD) systems. Researchers had used different techniques to experience with patterns. The melanoma lesion could also be identified with a different texture, shape and clinical features. The proposed study has used 22 features of texture and 12 features of shape. Chapter “The Effect of Different Feature Selection Methods for Classification of Melanoma” exposes three feature selection (FS) techniques like gradient boosting (GB), particle swarm optimization (PSO) and statistical approach. The features are evaluated with these methods and highlighted the effectiveness of each feature for the classification of melanoma. Selected key features have less than the cost of computation. The reduced feature set can make classification better than per the selection of the model. The random forest has the highest performance based on accuracy as it got the highest accuracy of 97.1 (%) on GB feature sets. Decision tree and k-nearest neighbors have shown a decent accuracy of 96.8 (%) and 93.3(%) on GB feature sets. The study rewards upcoming explorations to select an effective subset of features for machine learning and deep learning techniques. E0 algorithm is the most popular when it comes to data transmission in Bluetooth communication among devices. E0 is having a symmetric stream cipher key of 128-bit length. Many types of attacks at Bluetooth protocol and cryptanalysis of E0 have proved that it would be broken by using 264 operations. In Chapter “Intelligent Hybrid Technique to Secure Bluetooth Communications,” the authors have proposed hybrid encryption based on Blowfish and MD5 algorithms to improve the security of transferring data between two computers connected using Bluetooth technique.

xii

Preface

Because of the advantages of key management of the MD5 algorithm, the authors have used it to encrypt the secret key of Blowfish algorithm which is used for encryption of plaintext. Therefore, the proposed hybrid encryption (Blowfish and MD5) will positively improve the data security during communication in Bluetooth media. Networks are very important in the world. In signal processing, the towers are modeled as nodes (vertices), and if two towers communicate, then they have an arc (edge) between them, or precisely, they are adjacent. The least number of nodes in a network that can uniquely locate every node in the network is known in the network theory as the resolving set of a network. One of the properties that is used in determining the resolving set is the distance between the nodes. Two nodes are at a distance one if there is a single arc that can link them whereas the distance between any two random nodes in the network is the least number of distinct arcs that can link them. In Chapter “Parallel Algorithm to find integer k where a given Well-Distributed Graph is k-Metric Dimensional,” the authors propose two algorithms with the proofs of correctness. The first one is in line with the BFS that finds the distance between a designated node to every other node in the network. This algorithm runs in O(log n). The second algorithm is to identify the integer k, such that the given graph is k-metric dimensional. This can be implemented in O(log n) time with O(n2) processors in a CRCW PRAM. Fog computing is an emerging technology that offers high-quality cloud services by providing high bandwidth, low latency and efficient computational power and storage capacity. Although cloud computing is an efficient solution so far to store and retrieve the huge data of IoT devices, it is expected to limit its performance due to low latency and storage capacity. Fog computing addresses these limitations by extending its services to the cloud at the edge of the network. In Chapter “A Fog-Based Retrieval of Real-Time Data for Health Applications,” the authors use a fog computing network approach for efficiently retrieving the real-time patient data. The performance of our proposed approach has been compared with the cloud computing approach in terms of retrieval time of real-time data. The editors strongly believe that this volume benefit researchers and practitioners in the interdisciplinary fields of intelligent signal and image processing to evolve failsafe and efficient algorithms in the future. The editors would also like to take this opportunity to render their heartfelt gratitude to Mr. Aninda Bose, Senior Editor, Springer, for the overall guidance during the entire project. Birbhum, India Zagreb, Croatia Zagreb, Croatia Bengaluru, India Fukuoka, Japan November 2020

Siddhartha Bhattacharyya Leo Mrši´c Maja Brkljaˇci´c Joseph Varghese Kureethara Mario Koeppen

Contents

Chaotic Ions Motion Optimization (CIMO) for Biological Sequences Local Alignment: COVID-19 as a Case Study . . . . . . . . . . . . . . Mohamed Issa, A. M. Helmi, Mohamed Abd Elaziz, and Siddhartha Bhattacharyya Differential Evolution-Based Shot Boundary Detection Algorithm for Content-Based Video Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Abhishek Dixit, Ashish Mani, and Rohit Bansal Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tulika Dutta, Sandip Dey, Siddhartha Bhattacharyya, and Somnath Mukhopadhyay

1

9

19

Employing Parallel Hardware Architectures to Diagnose Sickle Cell Anemia in Real-Time Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohammed A. Fadhel and Omran Al-Shamma

29

Implementing a Hardware Accelerator to Enhance the Recognition Performance of the Fruit Mature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohammed A. Fadhel and Omran Al-Shamma

41

Time Series Modelling and Forecasting of Patient Arrivals at an Emergency Department of a Select Hospital . . . . . . . . . . . . . . . . . . . . V. Rema and K. Sikdar

53

Deep Convolutional Neural Network-Based Diagnosis of Invasive Ductal Carcinoma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Smaranjit Ghose, Suhrid Datta, C. Malathy, and M. Gayathri

67

Speaker Identification in Spoken Language Mismatch Condition: An Experimental Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joyanta Basu and Swanirbhar Majumder

77

xiii

xiv

Contents

Ultrasound Image Classification Using ACGAN with Small Training Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sudipan Saha and Nasrullah Sheikh Assessment of Eyeball Movement and Head Movement Detection Based on Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Saadman Sayeed, Farjana Sultana, Partha Chakraborty, and Mohammad Abu Yousuf

85

95

Using Hadoop Ecosystem and Python to Explore Climate Change . . . . . . 105 Ivan Ksaver Šušnjara and Tomislav Hlupi´c A Brief Review of Intelligent Rule Extraction Techniques . . . . . . . . . . . . . 115 Abhishek Gunjan and Siddhartha Bhattacharyya The Effect of Different Feature Selection Methods for Classification of Melanoma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Ananjan Maiti and Biswajoy Chatterjee Intelligent Hybrid Technique to Secure Bluetooth Communications . . . . 135 Alaa Ahmed Abbood, Qahtan Makki Shallal, and Haider Khalaf Jabbar Parallel Algorithm to find Integer k where a given Well-Distributed Graph is k-Metric Dimensional . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Xavier Chelladurai and Joseph Varghese Kureethara A Fog-Based Retrieval of Real-Time Data for Health Applications . . . . . 155 I. Diana Jeba Jingle and P. Mano Paul

About the Editors

Siddhartha Bhattacharyya, FIET (UK), is currently serving as Principal in Rajnagar Mahavidyalaya, Birbhum, India. He was a Professor in CHRIST (Deemed to be University), Bangalore, India from December 2019 to March 2021. He served as Senior Research Scientist at the Faculty of Electrical Engineering and Computer Science of VSB Technical University of Ostrava, Czech Republic, from October 2018 to April 2019. Prior to this, he was Professor of Information Technology at RCC Institute of Information Technology, Kolkata, India. He is a co-author of 6 books and a co-editor of 75 books and has more than 300 research publications in international journals and conference proceedings to his credit. His research interests include soft computing, pattern recognition, multimedia data processing, hybrid intelligence and quantum computing. Leo Mrši´c is Vice Dean and Assistant Professor in Algebra University College, Croatia. He has strong organizational skill set during 20 years of management experience. He is active in the EDU community and a member of ESCO Maintenance Committee (ThirdMAI). With a wide range of different experiences, he is capable of providing a deep and usable approach in several areas related to law, digital technology, math, structured decision making and educational methodologies. He is an independent EU Project Funds consultant and a registered consultant in GOPA Consulting Group database. Maja Brkljaˇci´c received her doctorate from the Central European University, Budapest. She worked as a researcher at the Institute for Human Sciences, Vienna, and as a project manager for the Westfalische Wilhelms-Universitat, Munster. She is a winner of Erich Maria Remarque’s scholarship at New York University and the Alexander-von-Humboldt Foundation for Freie Universitat, Berlin. She worked as a project manager at CEU and as a guest lecturer at Northwestern University, USA. She also worked as Head of Educational Programmes for Students in Algebra, and at the moment, she leads Algebra LAB’s start-up incubator.

xv

xvi

About the Editors

Joseph Varghese Kureethara completed his Ph.D. from MS University, Tirunelveli, and master’s degree in Economics and Mathematics from Madras University. He has over 16 years of experience in teaching and research at Christ University, Bengaluru, and has published over 100 articles in the fields of graph theory, number theory, history, religious studies and sports both in English and Malayalam. Dr. Joseph co-edited three books and authored one book. He has delivered invited talks in over thirty conferences and workshops. He is the Mathematics issue editor of Mapana Journal of Sciences and a member of the Editorial Board and a reviewer of several journals. Mario Koeppen studied physics at the Humboldt University, Berlin, and received his master’s degree in solid-state physics in 1991. In 2006, he became JSPS Fellow at the Kyushu Institute of Technology, Japan, in 2008 Professor at the Network Design and Research Center (NDRC) and in 2013 Professor at the Graduate School of Creative Informatics of the Kyushu Institute of Technology, Japan. His research interests include multi-objective and relational optimization, digital convergence and human-centered computing. He has published more than 150 peer-reviewed papers in conference proceedings, journals and books. He was actively involved in the organization of the WSC on-line conference series on Soft Computing in Industrial Applications, and the HIS conference series on Hybrid Intelligent Systems. He is a founding member of the World Federation of Soft Computing, and since 2016 an editor-in-chief of its Elsevier Applied Soft Computing Journal.

Chaotic Ions Motion Optimization (CIMO) for Biological Sequences Local Alignment: COVID-19 as a Case Study Mohamed Issa, A. M. Helmi, Mohamed Abd Elaziz, and Siddhartha Bhattacharyya

Abstract COVID-19 is a pandemic that broke out throughout the world and has a high mobility to transfer between humans. Developing intelligent bioinformatics tools is a mandatory to aid in the analysis of the disease. One of these tools is local aligner which aims to find the longest common subsequence between two biological sequences. Fragmented local aligner technique (FLAT) was developed based on meta-heuristic algorithms to accelerating the alignment process, especially for sequences with huge length. In this paper, the performance of ions motion optimization (IMO) algorithm for implementing FLAT was measured. The performance was poor, and a chaotic parameter was added in the exploration equations of IMO to enhance its performance for FLAT. A set of real proteins having a product length which ranges from 250,000 to 9,000,000 were used as a dataset to test the performance of IMO and its developed version. Besides, COVID-19 virus was aligned using FLAT according to IMO and chaotic IMO to verify the enhancement of IMO. All results were compared to the results founded by Smith–Waterman approach. The tests prove the superiority of chaotic IMO over IMO for implementing FLAT on all datasets.

1 Introduction Sequence alignment is a basic operation in bioinformatics for aligning DNA, protein and RNA sequences to measure its evolutionary relationships [1] and protein M. Issa (B) · A. M. Helmi Computer and Systems Department, Faculty of Engineering, Zagazig University, Zagazig, Egypt e-mail: [email protected] M. A. Elaziz Department of Mathematics, Faculty of Science, Zagazig University, Zagazig, Egypt e-mail: [email protected] S. Bhattacharyya Rajnagar Mahavidyalaya, Birbhum, India e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_1

1

2

M. Issa et al.

secondary structure prediction [2]. Pairwise local sequence alignment aims to align portion of the sequences to determine the longest common subsequence between two subsequences [3], and Smith–Waterman (SW) alignment algorithm [3] is the standard algorithm for producing the accurate longest common consecutive subsequences between two sequences. It is based on dividing the alignment into sub-problems and solving it on accumulative manner using dynamic programming (DP). However, it is time-consuming due to the long of biological sequences. The main objective of local sequence alignment operation is finding the binding sites in the protein sequences for molecular docking process of drug design and finding the common features between biological sequences which aim to construct the phylogenetic tree of sequences. SW alignment algorithm works as following to find LCCS between pair of sequences. First, a matrix with size (k + 1) X(z + 1) is constructed where the two sequences have lengths of k and z. The cells of matrix are filled by the various alignment scores that are computed using Eq. 1. ⎫  ⎧ Score(i − 1, j − 1) + Sim_Fun Seq1 (i), Seq2 ( j) ⎪ ⎪ ⎪ ⎪ ⎬ ⎨ maxm=1:i−1 (Score(i, m) + g0 + kge ) Score(i, j) = max ⎪ ⎪ maxm=1:i−1 (Score(m, j) + g0 + kge ) ⎪ ⎪ ⎭ ⎩ 0

(1)

In Eq. (1), Seq1 and Seq2 denote the sequences to be aligned and each of them has lengths m and n, respectively. i and j represent the indices of row and column in order, and i lies in the range from 1 to k, and j lies in the range from 1 to z. Sim_Fun() is a function that evaluates the similarity of residues of protein. In some cases of partial alignment, it may be need to shift until matching occurs. For shifting a gap, ‘–’ can be inserted which is evaluated as a g0 score for first gap and ge for other k inserted consecutive gaps. Note that g0 and ge has a negative score that reduces the alignment score. (g0 + kge ) is a linear gap function which is used for maximizing the alignment score instead of using individual scoring scheme due to g0 has negative score lower than that of ge [4]. The second step is the trace back stage which is started to construct the alignment of the two sequences (aligning portion of the two sequences) after computing the scoring matrix. It starts from the cell that has maximum score and traces back until reach the first cell has score zero. After constructing the alignment, Eq. 2 is used to evaluate the overall alignment score. AlignmentScore

L  if Seq_Ai == Seq_Bi penalize(+1) score = otherwise penalize zero

(2)

i=1

where Seq_A andSeq_B refer to aligned sequences and L denotes the length of these sequences. represents the sum of scores of corresponding residues over L positions.

Chaotic Ions Motion Optimization (CIMO) for Biological Sequences …

3

However, these methods provide the accurate local alignment between two sequences, but it consumes huge time since the computational complexity of this    algorithm is O n 3 and the space complexity is O n 2 , where n denotes the length of the two sequences to be aligned. This alignment method was optimized to speed up the execution time of alignment process, especially for sequences for very long lengths [5]. FLAT [5] is an improved of local sequence alignment depends on using metaheuristic algorithms. FLAT divides the long sequences into fragments with small lengths, and these fragments are aligned using Smith–Waterman algorithm. Hence, the execution time is decreased since small length of fragments instead of the overall length of sequences. The aim of using meta-heuristic algorithm is determining the positions of cutting fragments to be aligned. Therefore, the solution is the position at which two fragments to be cut and align using SW algorithm, and these solutions are updated to move toward the positions that have fragments produced by the longest common subsequences. The basic idea of FLAT is fragmenting the longer sequences into short ones and aligning these fragments using SW algorithm and the search be concentrated around the position that have the longest common subsequences founded. As shown in Fig. 1, three positions are determined in each sequences and the LCCS between each pair of sequences is computed. All other positions update their locations toward the (PA1 in Seq1 and PB1 in Seq2 ) since they produce the maximum LCCS. An objective function is used that evaluates the corresponding residues for the optimization of finding the positions of cutting fragments. Equation 2 is used as an objective function that is applied to control the agent’s movement and evaluate the finding alignments. Therefore, the search process based on the where is the positions at which the fragments are cut to be aligned and the search be concentrated around the position that have LCCS found and trying to enhance it. The length of sequences is huge to be explored so meta-heuristic algorithm (MA) algorithms are applied to accelerate the search process based on stochastic mechanism.

Fig. 1 Cutting the fragments at different positions of two sequences

4

M. Issa et al.

MA is a search-based algorithm that accelerates exploring the search space of the problem based on random movement to find the best solutions [6]. MA algorithm mimics the search methods from nature, physics, or human [6] such as particle swarm optimization (PSO) [7], ions motion optimization (IMO) [8], lightning attachment procedure optimization [9]. Gravitational search algorithm (GSA) [10], electromagnetic field optimization (EFO) [11], moth-flame optimization (MFO) [12], and other hundreds of algorithms are developed. The general idea of MA is updating the solutions toward the best solution based on certain updating operators depend on the optimization technique of the used MA. The general procedure for implementing FLAT is as described in Algorithm 1. In general, the time complexity of FLAT is  T K L 2F C , where K, T, and L F denote the length of fragments, the number of generations, and number of solutions. C denotes the time–cost of updating equation of MA in order. IMO algorithm [8] was presented that mimics the attraction of ions with different signs (anions and cations). In this paper, a modified IMO was evaluated for implementing FLAT. Since the performance of IMO was poor for these problems because of the large length of sequences, it fails in local optima. Chaos theory was embedded in IMO to enhance its exploration capability, and its performance was evaluated on finding the LCCS between COVID-19 and other viruses. Algorithm (1) FLAT Procedure 1. A K search agents (Pi , i in the range (1, K)) are initialized randomly with a one position in each sequences and are chosen in the range (1, length ((Seq1 or Seq1 ) − LF)) where LF represents the length of fragment. 2. Two fragments (one in each sequences) having a length LF is cut at the position (solution) of each search agents 3. Each two fragments that are cut by each solution are aligned using SW alignment aligns, and Eq. 2 is used to estimate the alignment score as a fitness function. 4. The positions of solutions are updated according to the update mechanism of the MA used toward the position that has maximum LCCS founded. 5. Repeat the steps (from 2 to 5) for a number of iterations.

The rest of the paper is organized as follows: Sect. 2 describes IMO algorithm, and the developed IMO (chaotic IMO) is presented in Sect. 3. Section 4 proposes the experimental results, and Sect. 5 presents the conclusion and future work.

2 IMO Algorithm The IMO algorithm [8] emulates the behavior of attracting of ions in physics. The individuals of IMO are separated into two groups of cations (represents the ions with negative signs) and anions (represents the ions with positive signs). The basic idea of updating the solutions is attracting the anions to the best cation and cations to the best anion founded. The algorithm execution is divided into two phases:

Chaotic Ions Motion Optimization (CIMO) for Biological Sequences …

5

• exploration phase: In this stage, each solution updates its movement using Eqs. 3 and 4.   Ai, j = Ai, j + AFi, j ∗ Cbest j − Ai, j

(3)

  Ci, j = Ci, j + C Fi, j ∗ Abest j − Ci, j

(4)

where j is the dimension, i is the index of ions, C i,j is the cation candidate solution, Ai,j refers to anion. C bestj denotes the best cation found, Abestj is the best anion. AF i,j and CF i,j refer the mathematical modeling that represents the distance between cations and ions. 1

AFi, j =  where ADi,j is  Ai, j

0.1 − AD

, C Fi, j =

1

1 + e i, j 1+e    − Cbest j  and CDi,j is Ci, j − Abest j .

− C 0.1 D

(5)

i, j

• Solid Phase (exploitation): In solid phase, ions have difficult move become in stable phase therefore to give these ions an energy an external force is used and this leads to avoids local minima. This task is achieved using Eqs. 6 and 7 to improve the exploration of the solutions. Ai = Ai + α1 ∗ (Cbest )

(6)

Ci = Ci + α2 ∗ (Abest )

(7)

IMO has advantages such as low computational complexity few number of parameters that to be tuned, and fast convergence. But the main limitation is poor exploitation capabilities and trapping in local minima.

3 Chaotic IMO (CIMO) Chaos is the simulation process of dynamic behavior for the nonlinear systems where the small deviation or change of the start condition of chaos will lead the future behavior to update with nonlinear behavior that is due to its same properties of randomness but with better statistical properties. Recently are used to enhance the performance of the stochastic and meta-heuristic algorithms [13]. Chaos are used in many applications in different sciences such as optimization researches [14] and chaos control [15]. In this study, chaotic variables are embedded into the IMO algorithm in trial to improve its performance in problem fragmentation of local aligner. In the update

6

M. Issa et al.

equations of the liquid phase (Eqs. 1 and 2), the only control variable is the distance between the best ion location and the ions with no random parameters that enhance the diversity. Hence, there is a random variable (r) which is embedded to enhance the diversity and the new update equations as follows:   Ai,j = Ai,j + AFi,j ∗ r ∗ Cbestj − Ai,j

(8)

  Ci,j = Ci,j + CFi,j ∗ r ∗ Abestj − Ci,j

(9)

Chaotic maps replaced the random variable (r) with the chaotic variables since the variable r will balance between the exploration and exploitation of the search space as assumed. In this study, the chaotic maps that are used are shown in Eq. 10 (Chebyshev) and Eq. 11 (Iterative) where s is the index of chaotic sequence and the start of chaotic sequence is 0.7 for all functions.   Os+1 = cos s cos−1 (Os )  cπ , c = 0.7 = sin Os

(10)



O s+1

(11)

4 Experimental Results and Discussions Chaotic IMO for FLAT was tested on real biological data gathered from National Center for Biotechnology Information (NCBI) (https://www.ncbi.nlm.nih.gov/) with various lengths having product ranges from 250,000 to 9,000,000. Besides, COVID19 virus was analyzed using FLAT based on chaotic IMO versus other viruses to find the LCCS between them for guarantee the enhancement of IMO. Figure 2 shows the percentage of LCCS obtained by the IMO, chaotic versions using chaotic function Chebyshev function (CIMO_1), and Iterative function (CIMO_2). As shown in Fig. 2, the performance of chaotic IMO is better than that of IMO; however, the performance of CIMO_1 is little better than that of CIMO_2. The performance of IMO and its developed versions was assessed to determine the LCCS of COVID-19 versus other virus such as Human immunodeficiency virus, Hepatitis C virus, Influenza A, Malaria, and alveolar proteinases. As shown in Table 1 SW finds the exact LCCS between COVID-19 and each disease. FLAT depends on CIMO-1 which finds better results than that of IMO and CIMO-2 in most of diseases but not the exact LCCS or part of it except the first and third diseases. That is due to the huge length of COVID-19 that exceeds 7000 bp and also the length of other diseases besides the sequences can share many LCCS with different lengths so the Chaotic IMO can be enhanced for longer sequences to overcome this obstacle.

80 70 60 50 40 30 20 10 0

7

IMO CIMO_1 CIMO_2 250000 350000 550000 750000 1000000 1400000 1800000 2200000 2600000 3000000 4000000 5000000 6000000 7000000 8000000 9000000

LCCS prcentage relave to Exact ones

Chaotic Ions Motion Optimization (CIMO) for Biological Sequences …

The product of lengths of sequences Fig. 2 Percentage of FLAT based on chaotic IMO over various lengths Table 1 Results obtained by CIMO and other methods to find LCCS of COVID-19 protein Virus protein name (length)

Method

Score

LCCS obtained

Human immunodeficiency

SW

24

TYPSLETIQITSSFKWDLTAFGLV

IMO

3

GAC

Hepatitis C

Influenza A

Malaria

Alveolar proteinases

CIMO_1

4

AFGLV

CIMO_2

3

LLS

SW

10

TSSGDATTAY

IMO

3

SLL

CIMO_1

4

NGSI

CIMO_2

3

SLL

SW

16

TGSSKCVCSVIDLLLD

IMO

4

SLVP

CIMO_1

3

LLD

CIMO_2

3

VLV

SW

6

EEEQEE

IMO

2

DF

CIMO_1

4

GLFK

CIMO_2

3

DFL

SW

22

IDAMMFTSDLATNNLVVMAYIT

IMO

4

GAVC

CIMO_1

5

VGGSC

CIMO_2

4

GGSC

8

M. Issa et al.

5 Conclusion This word presented a developed IMO algorithm based on chaos theory for implementing FLAT algorithm to find the LCCS with high percentage of the exact ones as possible. FLAT based on chaotic IMO beat the quality of FLAT depends on IMO when testing on real protein sequences have various lengths. Besides, the two technique was applied to determine LCCS between COVID-19 and other viruses as a verification of the enhancement of IMO based on chaotic operators. Chaotic IMO has little enhancement of the quality of FLAT according to IMO, but it still needs more enhancement in the future.

References 1. Xiong, J.: Essential Bioinformatics. Cambridge University Press (2006) 2. Di Francesco, V., Garnier, J., Munson, P.: Improving protein secondary structure prediction with aligned homologous sequences. Protein Sci. 5(1), 106–113 (1996) 3. Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147(1), 195–197 (1981) 4. Gotoh, O.: An improved algorithm for matching biological sequences. J. Mol. Biol. 162(3), 705–708 (1982) 5. Issa, M., et al.: ASCA-PSO: adaptive sine cosine optimization algorithm integrated with particle swarm for pairwise local sequence alignment. Expert Syst. Appl. 99, 56–70 (2018) 6. Talbi, E.-G.: Metaheuristics: From Design to Implementation, vol. 74. Wiley, New York (2009) 7. Kennedy: Particle swarm optimization. Neural Networks (1995) 8. Javidy, B., Hatamlou, A., Mirjalili, S.: Ions motion algorithm for solving optimization problems. Appl. Soft Comput. 32, 72–79 (2015) 9. Shareef, H., Ibrahim, A.A., Mutlag, A.H.: Lightning search algorithm. Appl. Soft Comput. 36, 315–333 (2015) 10. Rashedi, E., Nezamabadi-Pour, H., Saryazdi, S.: GSA: a gravitational search algorithm. Inf. Sci. 179(13), 2232–2248 (2009) 11. Abedinpourshotorban, H., et al.: Electromagnetic field optimization: a physics-inspired metaheuristic optimization algorithm. Swarm Evolut. Comput. 26, 8–22 (2016) 12. Mirjalili, S.: Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowl.-Based Syst. 89, 228–249 (2015) 13. dos Santos Coelho, L., Mariani, V.C.: Use of chaotic sequences in a biologically inspired algorithm for engineering design optimization. Exp. Syst. Appl. 34(3), 1905–1913 14. Tavazoei, M.S., Haeri, M.: Comparison of different one-dimensional maps as chaotic search pattern in chaos optimization algorithms. Appl. Math. Comput. 187(2), 1076–1085 (2007) 15. Zhang, L., Zhang, C.: Hopf bifurcation analysis of some hyperchaotic systems with time-delay controllers. Kybernetika 44(1), 35–42 (2008) 16. Feng, D.-F., Doolittle, R.F.: Progressive alignment and phylogenetic tree construction of protein sequences. Methods Enzymol. 183, 375–387 (1990) 17. Li, L., Khuri, S.: A comparison of DNA fragment assembly algorithms. In: METMBS (2004)

Differential Evolution-Based Shot Boundary Detection Algorithm for Content-Based Video Retrieval Abhishek Dixit, Ashish Mani, and Rohit Bansal

Abstract With the increase of multimedia devices on internet, enormous number of videos are being added as part of digital content. There is a huge challenge in retrieval of these videos as mostly the videos are kept in unstructured form. Intended users try to retrieve video content as per the relevancy and need. Shot boundary detection is a significant and critical approach in the domain of digital video processing. It is the foremost critical job of content-based video retrieval and indexing. In this paper, a novel approach for shot boundary detection algorithm based on Differential evolution algorithm (DE) with SVM classifier is proposed. In this method we first calculate the curves difference of U-component histograms as the feature of difference between video frames. In the next step, Slide-Window Mean Filter to filter difference curves and SVM Classifier applying DE to detect and classify the shot transitions. The wellknown TRECVID 2005, 2006, and 2007 datasets are used to test the performance of our proposed approach. The result shows the superior performance of our proposed approach, and this method can achieve high recall, precision rate, accuracy, and better computation time. Keywords Shot boundary detection · Differential evolution · SVM

A. Dixit (B) Department of Computer Science, Amity School of Engineering and Technology, Amity University, Noida, U.P, India e-mail: [email protected] A. Mani Department of EEE, Amity School of Engineering and Technology, Amity University, Noida, U.P, India e-mail: [email protected] R. Bansal Department of Management Studies, Rajiv Gandhi Institute of Petroleum Technology, Raebareli, U.P., India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_2

9

10

A. Dixit et al.

1 Introduction Shot boundary detection is an approach in which video sequence is divided into shots which is the initial important step for analyzing video content and retrieval and browsing of content-based video [1]. The unique shots captured can be presented visually by associated key frames mined from camera images in continuous time period. These shots refer as continuous action. For the content-based video retrieval process, segmentation of video is the first step. Therefore, video sequence is required to be divided into shots to extract the vital important frames from video and classify video sequence. In the field of video analysis, shot boundary detection is important and relevant area for researchers, and various research is done in this area. In literature, various different algorithms were proposed to showcase the efficiency and performance of a shot boundary detection system mainly on the basis of recall and precision parameters [2, 3]. The formulae for both the parameters are: correct correct + missed correct Precision = correct + false alarm Recall =

(1)

The feature extraction and similar scales in a shot-based boundary detection algorithm can be divided into three groups: pixel-based approach [4]; block-based approach [5]; histogram-based approach [6]. The pixel-based approach is the simplest way to identify the discontinuous video content, but it is too costly approach and results impacted global and local motions. There is another approach based on histogram which is based on the fact that in one shot the sequential frames have common global visual properties. This is because histogram difference is less as compared to frame difference lying on one shot boundary [7]. The histogram-based approach mainly focuses on the global distribution, and performance of this approach is not impacted by local motions of objects. During the global motions, the histogram is also changed, and this results in missing the transitions of shots. Due to this, different images may have similar histograms. To resolve these problems, block-based approach is used. In block-based approach, video frames are divided into blocks and a transformation function is used to extract the features. The selected features are then kept in memory location and processed sequentially. From these, features are not processed in sequential way in memory. A well-known problem of the overall computer performance is the difference in memory speed and CPU speed [8]. To process the image (video frames) sequentially is a disadvantage, and to overcome this Abdulhussain et al. [9] proposed a feature extraction approach based on discrete transform. These selected features are then processed to SVM classifier for classification. With this approach the overall performance of SBD is improved. However, the overall performance of this model can further be improved by adding a feature selection approach so that only relevant features are fed to the

Differential Evolution Based Shot Boundary Detection Algorithm …

11

classifier. Motivated by this idea, we introduce a new approach of feature selection with shot-based boundary detection algorithm based on differential evolution algorithm and SVM classifier. The paper is organized as follows; Sect. 2 discusses on the differential evolution algorithm; Sect. 3 showcases the proposed model. In Sect. 3, experiment and results are presented to highlight the effectiveness of our proposed model. Finally, Sect. 4 concludes the overall work and future research direction.

2 Proposed Method 2.1 Shot Boundary Detection Algorithm with DE and SVM The first step in our proposed model is pre-processing in which number of shots for a target video is determined. For the initial analysis, a sample from the original video is taken as video set may contain numerous frames based on the length of the video. Chiu et al. [10] suggested a way to sample frames. For pre-processing step, this is most relevant approach in different domains as it assumes longer shot length with 15 frames or 0.5 s. In an image (video frame), pixels p1 × p2 are divided into block of size b1 × b2. Total number of blocks n1 × n2 and n1 = p1/b1 and n2 = p2/b2. The blocks can be represented in matrix form as: ⎤ bv1,1 · · · bv1,n2 ⎢ .. ⎥ I = ⎣ ... . . . . ⎦ bvn 1 ,1 · · · bvn1,n2 ⎡

(2)

where bvx,y is, referred as image block in x- and y-directions and can be represented as: bvx,y = I ( j, k)

(3)

j = (x − 1)b2, (x − 1)b2 − 1, . . . , xb2 − 1 k = (y − 1)b1, (y − 1)b1 − 1, . . . , yb1 − 1 For any video, if we identify and classify filtered curves, we should be able to determine the cuts and gradual transitions. In this paper, we have identified the gradual transitions by dividing them into multiple classes and then created the training set of cuts and training set of gradual transitions as T Sc T Sg, respectively, where T Sc = {c1 , c2 , . . . cm }, T Sg = {g1 , g2 , . . . .gn }.

12

A. Dixit et al.

Therefore, we are able to identify that which classification a testing sample gcin fits in to by looking for the training sets to search the training sample matches gcin . Typically, to ensure the fitness of any classifier, training set is chosen as a large, and therefore, it receives high precision by finding the training samples all over the training sets. This impacts the computational cost of the classification algorithm and thus impacts the overall efficiency of the detection algorithm, To solve this, we organize the training data of as per the likeness and apply DE to search in between 2 sets and find the matching training and testing sample gcin and calculate the boundary types as per the gcin . In this way, we can achieve high accuracy and precision. For the given population, in our proposed model, we have initialized the positions of particles as the positions of the training samples in T Sc and T Sg, . The particle’s fitness value is initialized as the matching function value among graphic gcin and a training sample. Fitness function: Typically, the width of sample ck (1 ≤ k ≤ m) or gl (1 ≤ l ≤ n) in training set and gcin width is not equal. Therefore, gcin width is tuned as per the wc   or wg as the ck and gl width to get gcin . Lastly, gcin . value is calculated to equalize the value among training and testing sample. Fitness of the particle is calculated as per the below equations. Fitness(ck ) =

wc 



wc

 1    ck [i] − gcin ck [i] − gcin [i] − [i] wc i=1

i=1

Fitness(gl ) =

wg 

gl [i] −

i=1

 gcin [i]

2

2 wg  1 

 gl [i] − gcin [i] − wg i=1

(4)

(5)

where height values are denoted as ck [i] and gl [i] in the coordinates I of training  sample graphics. Testing samples height values are denoted as ck and gl , gcin [i] for testing sample graphics gcin in coordinates i. Mutation and crossover: After initializing the population, setting up the fitness value of the population and rank is calculated. Further, the individual population is set and is randomly selected to generate next-generation population based on the equation below. 

vi (k) = xrk1 + F ∗ xrk2 − xrk3 , i = 1, 2, 3 . . . N

(6)

where F is the scaling factor, r1 , r2 , r3 are the mutually exclusive random values in the range [0, 1], k the generation count. The chances of selection of an individual is based on the ranking value. More the value, greater is the chance to be selected. Crossover is applied on the selected individual as per the equation below.

u ikj

=

vikj if rand ≤ C R xikj otherwise

(7)

Differential Evolution Based Shot Boundary Detection Algorithm …

13

C R is the crossover rate, and rand is the random value in range [0, 1]. Based on the generation gap value, the number of individuals is selected. Crossover is applied based on the segmented boundary values. There is an equal probability of selection for each boundary value. As an example, below are the selected 10 frames chromosome having value as 2. 0001000010 0100100000 In the initial shot from the parent 1 and at the pointer, 4 is chosen by random selection as the point for crossover. The selected parents are divided as below. 000 | 1000010 010 | 0100000 As shown below, from the recombination of the parents, child individuals are generated and combining the outstanding segments. 0000100000 0101000010 This generates 1 child having 1 boundary segment shot and generates one more child having 3 boundaries shot. Mutation happens for the first child, and this will translate to 1 from 2 of the 0’s. Then, for the next child, mutation is applied to change one of the 1’s into a 0. Hence, for k-2 shot boundary, the outcome is 2 children. 0100100000 0100000010 Supplementary mutation through reversing a bit randomly in the string of chromosome is not advised as it has a tendency to produce unstable shot sections. Thus, we have applied the mutation which happens by adjusting back to k shots. Lastly, if the set value of generational gap indicates that the population with odd number value should be replaced, then from the generated children set, any one is removed based on random selection with equal probability. Selection: Finally, selection operation is performed based on Eq. 4. The criteria of termination is chosen based on the objective function value compared with previous generation value, and the same value comes continually. Optimal solution is achieved if the given value of objective function is constant for 2 or more generation. There is a possibility where optimal solution is not achieved, in that case the termination criteria is the maximum number of iterations.



  k vi if f u ik ≤ f xik (8) xik+1 = xik otherwise

14

A. Dixit et al.

f (u) is the fitness value.

3 Experiment and Results The proposed shot boundary detection method based on DE and SVM as classifier is evaluated on the well-known datasets TRECVID2005, TRECVID2006, and TRECVID2007 [11]. We have compared our proposed approach with recently proposed feature selection approach [9]. This experiment is repeated over 100 iterations, and average vales are calculated. The selection of iteration number also plays an important role in determining the efficiency of an algorithm and usually done on the basis of past experiences. If the iteration value is large, efficiency will reduce without having similar testing and training samples. And if small value is chosen, then search can be stopped without getting proper training sample. Therefore, in our experiment, iteration number is set as 100 to ensure the efficiency of algorithm. The various parameters used in the experiments are shown in Table 1. The experiment is performed in MATLAB 2016 on laptop with configuration as Intel ® Core™ i7-8665U CPU at 1.90 GHZ 2.11 GHZ and 16 GB RAM. Table 2 shows the datasets details. For each dataset, we have used 4 * 4, 8 * 8 and 16 * 16 blocks with 10 and 20% of selected moments. Table 2 also showcases the accuracy comparison results which include precision, recall and F1-score. From Table 2, we can see that with the increase of block numbers percentage of accuracy is also increased. As seen from the results, for TRECVID2005 dataset with 10% moment, the value of F1-score increases from 98.00 to 98.57 with the increase in number of blocks from 4 * 4 to 16 * 16. The similar results can be observed for the other datasets. This shows that as block numbers increases, the impact of camera motions like local and global motions are reduced. From the table, we can see that our proposed approach is giving better results on all the three datasets and for different combination of bocks and selected moments. For TRECVID2007, the highest accuracy value of 99.20% is obtained on 16 * 16 block with 10% selected moments. From the results, we can see that the proposed model for SBD with DE and SVM improves the overall efficiency of algorithm and feature selection approach plays a major role in achieving this. DE optimizes the feature set and fed only relevant features to the Table 1 Parameters

S. No.

Parameter name

Value

1

Iteration

100

2

Default population

50

3

w

0.9

4

c1 and c2

2.05, 2.05

5

Scaling factor—F min , F max

0.5, 1.0

6

Crossover rate—Cr

0.8

Differential Evolution Based Shot Boundary Detection Algorithm …

15

Table 2 Accuracy comparison results of SBD Dataset Block 2005

2006

2007

Selected moments % Paper [9]

Proposed

P%

R%

F1%

P%

R%

F1%

4*4

10

95.18

96.99 96.08 97.08

98.93 98.00

4*4

20

94.93

96.69 95.80 96.83

98.62 97.72

8*8

10

95.47

97.29 96.37 97.38

99.24 98.30

8*8

20

95.59

97.37 96.47 97.50

99.32 98.40

16 * 16 10

95.51

97.33 96.41 97.42

99.28 98.34

16 * 16 20

95.76

97.54 96.64 97.68

99.49 98.57

4*4

10

93.18

96.28 94.70 95.04

98.21 96.59

4*4

20

92.91

95.71 94.28 94.77

97.62 96.17

8*8

10

93.14

96.54 94.81 95.00

98.47 96.71

8*8

20

93.09

95.90 94.47 94.95

97.82 96.36

16*16

10

93.66

96.60 95.11 95.53

98.53 97.01

16 * 16 20

93.26

96.73 94.97 95.13

98.66 96.87

4*4

10

97.15

96.63 96.89 99.09

98.56 98.83

4*4

20

96.81

96.38 96.59 98.75

98.31 98.52

8*8

10

97.39

96.87 97.13 99.34

98.81 99.07

8*8

20

96.62

96.38 96.50 98.55

98.31 98.43

16 * 16 10

97.54

96.97 97.25 99.49

98.91 99.20

16 * 16 20

96.91

96.58 96.74 98.85

98.51 98.67

SVM classifier thus improving the efficiency of SVM classifier. This shows the efficiency of our proposed approach in comparison to results obtained for approach [9]. Table 3 showcase the computation time for our proposed approach and approach [9] in testing phase. Table shows the computational cost per dataset and per frame. From the results, we can see that our proposed method reduces the computational cost of shot-based algorithm (SBD) in comparison to other method. From the results, we can also observe that as the block size increases there is no significant increase in computation time for both the forms. This shows that computational cost of our proposed approach is lower as compared to other approaches.

4 Conclusion In this paper, a shot boundary detection algorithm based on differential evolution algorithm (DE) with SVM classifier is proposed. In the algorithm, DE is added as feature selection approach with SVM classifier to improve the efficiency of shotbased algorithm. In this algorithm, video frames are first extracted and then the SVM classifier is applied with DE as a feature selection step to classify, and finally,

16

A. Dixit et al.

Table 3 Computation time measure Dataset

2005

2006

2007

Block

Selected moments %

Paper [9]

Proposed

Computation cost

Computation cost

/dataset (s)

/frame (ms)

/dataset (s)

/frame (ms)

4*4

10

723

1.229

713

1.167

4*4

20

773

1.314

752

1.301

8*8

10

712

1.210

679

1.205

8*8

20

789

1.341

765

1.321

16 * 16

10

722

1.227

705

1.212

16 * 16

20

793

1.348

765

1.314

4*4

10

533

1.186

521

1.089

4*4

20

580

1.290

564

1.245

8*8

10

540

1.201

523

1.189

8*8

20

583

1.297

565

1.265

16 * 16

10

558

1.241

547

1.221

16 * 16

20

590

1.312

576

1.302

4*4

10

781

1.355

765

1.332

4*4

20

852

1.478

834

1.234

8*8

10

778

1.350

743

1.323

8*8

20

857

1.487

854

1.467

16 * 16

10

798

1.384

756

1.321

16 * 16

20

861

1.494

848

1.434

positions of shot transitions are detected, and types of them are classified. To evaluate the performance of the proposed model, two experiments are performed. In the first experiment, we have calculated the recall, precision, and accuracy values of our proposed model on different datasets with various parameters. In the second experiment, computational cost is also calculated based on per frame and per dataset. Experimental results show that the performance of our proposed algorithm shows better results in comparison to the other approach. We wanted to further evaluate the performance of this model by applying other evolutionary approaches and classification algorithms.

References 1. Hanjalic, A.: Shot-boundary detection: unraveled and resolved. IEEE Trans. Circuits Syst. Video Technol. 12(2) (2002) 2. Xi, Z., Gang, L.X.: Survey on video temporal. Chin. J. Comput. 27(8):1027–1035 (2004) 3. Boreczky, J., Rowe, L.: Comparison of video shot boundary detection techniques. J. Electron. Imaging 5(2) (1996)

Differential Evolution Based Shot Boundary Detection Algorithm …

17

4. Abdulhussain, S.H., Ramli, A.R., Saripan, M.I., Mahmmod, B.M., AlHaddad, S., Jassim, W.A.: Methods and challenges in shot boundary. Entropy 20(4), 214 (2018) 5. Tippaya, S., Sitjongsataporn, S., Tan, T., Khan, M.M., Chamnongthai, K.: Multi-modal visual features-based video shot boundary. IEEE Access 5, 12563–12575 (2017) 6. Bhaumik, H., Bhattacharyya, S., Nath, M.D., Chakraborty, S.: Hybrid soft computing approaches to content based video retrieval: a brief review. Appl. Soft Comput. 46, 1008–1029 (2016) 7. Bhaumik, H., Bhattacharyya, S., Chakraborty, S.: A vague set approach for identifying shot transition in videos using multiple feature amalgamation. Appl. Soft Comput. 75, 633–651 (2019) 8. Alted, F.: Why modern CPUs are starving and what can be done about it. Comput. Sci. Eng. 12(2), 68–71 (2010) 9. Abdulhussain, S.H., Ramli, A.R., Mahmmod, B.M., Saripan, M.I., Al-Haddad, S., Baker, T., Flayyih, W.N., Jassim, W.A.: A fast feature extraction algorithm for ımage and video processing. In: %1 içinde International Joint Conference on Neural Networks, Budapest, Hungary, 14–19 July 2019 10. Storn, K., Price, R.: Differential evolution—a simple and efficient heuristic for global optimization. J. Global Optim. 11(4), 341–359 (1997) 11. TRECVID. https://trecvid.nist.gov (2016)

Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding Tulika Dutta, Sandip Dey, Siddhartha Bhattacharyya, and Somnath Mukhopadhyay

Abstract Thresholding of hyperspectral images is a tedious task. The interactive information value between three bands is used to reduce the redundant bands in the pre-processing stage. A qutrit-inspired genetic algorithm is proposed for thresholding the minimized hyperspectral images with improved quantum genetic operators. In this paper, a quantum disaster operation is implemented to rescue the qutrit-inspired genetic algorithm from getting stuck into local optima. The proposed algorithm produces better results than classical genetic algorithm and qubit-inspired genetic algorithm in most of the cases. Keywords Hyperspectral image thresholding · Qutrit-inspired genetic algorithm · Quantum mutation operator

1 Introduction A hyperspectral image (HSI) consists of high-dimensional and highly correlated spectral channels, called bands. Redundant bands and increased computational complexity of HSI create difficulty in processing them accurately. To overcome these

T. Dutta · S. Mukhopadhyay Department of Computer Science & Engineering, Assam University, Silchar, Assam, India e-mail: [email protected] S. Mukhopadhyay e-mail: [email protected] S. Dey Department of Computer Science, Sukanta Mahavidyalaya, Dhupguri, Jalpaiguri, West Bengal, India e-mail: [email protected] S. Bhattacharyya (B) Rajnagar Mahavidyalaya, Birbhum, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_3

19

20

T. Dutta et al.

limitations, a number of dimensionality reduction techniques are used. These can be broadly classified as feature selection [5] and feature extraction techniques. Mutual information [11], information gain-based methods and interactive information methods are few widely used feature selection techniques [5]. Segmentation is an important step in image processing, as it reduces the computational complexity. Thresholding is a widely implemented tool for image segmentation. Clustering-based technique like Otsu’s method [12] is extensively used thresholding-based method. In [9], a modified Otsu’s criterion is implemented which speeds up the process by 2.45 times of the original Otsu method. Metaheuristics are widely used for solving exhaustive search problems in an effective manner. They can be classified into nature and non-nature algorithms based on their origins. Nature-inspired metaheuristics have gained more popularity due to their capability of finding effective results in lesser time. Based on Darwin’s concepts of evolution, Holland proposed genetic algorithm (GA) [7] in 1975. Differential evolution [13] is another popular meta-heuristic inspired by Darwin’s evolutionary concepts. Sir Richard Feynman first proposed the idea of developing quantum computer in 1982. Deutsch [2] further elaborated the concepts of quantum computers in 1992. Applying the various properties of quantum mechanics, like superposition and entanglement in computing paradigm, has accelerated the rate of information processing. With the development of quantum computers, quantum metaheuristics or quantum-inspired metaheuristics have also gained popularity. They produce faster and more robust results compared to their classical counterparts. In [10], a framework of quantum-inspired GA is proposed. In [3], two quantum-inspired meta-heuristics using GA, particle swarm optimization and chaotic map model are introduced for thresholding of gray-level images. A qutrit GA is implemented by introducing two quantum phenomenon, viz. disaster operations and quantum rotation gates in [14]. The main objective of this paper is to propose a qutrit-inspired genetic algorithm (QutritGA) for thresholding of hyperspectral images. An interactive informationbased band selection technique is applied in the pre-processing stage for choosing three bands with maximum information content [5]. The QutritGA is designed with new quantum-based mutation operator. Quantum disaster operation [14], is implemented to improve the fitness values. To speed up the process, a modified Otsu criterion (MOC) [9] is used as the fitness function to optimize the threshold values. The main contributions of the proposed algorithm are as follows: • Use of interactive information-based band selection method, for choosing optimal bands. • Qutrit-based population is used for extensive search and faster results. • Qutrit-inspired genetic algorithm is proposed for optimal thresholding of HSI. • Qutrit-based modified mutation method is introduced for maintaining diversity in the population. The layout of the paper is in the following manner. Sections 2 and 3 contain important background concepts. The proposed methodology is discussed in details in Sect. 4.

Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding

21

In Sect. 5, experimental results and their analysis are presented. A brief conclusion of the paper is drawn in Sect. 6.

2 Fundamental Concepts of Quantum Computing The smallest and basic unit of quantum computers is called qubit [8]. Similar to classical bits, qubits also exist in two basis states, viz. |0 and |1. Benjamin Schumacher coined the term qubit. ‘Ket’ in Dirac’s notation (“|”) is used to represent the basis states. Qubits are represented using the following column vectors     1 0 |0 = and |1 = (1) 0 1 The main advantage of a qubit is that apart from existing in the basis states, it can also exist in a linear combination of the states. This is called quantum superposition and can be explained using a wave function as follows. |ψ = α0 |0 + α1 |1

(2)

In Eq. (2), α0 and α1 are complex numbers. The superposition collapses into one of the basis states when a quantum system is measured. |α0 |2 and |α1 |2 are the probabilities of finding the qubit in |0 and |1 states, respectively. This implies the certainty of an event’s occurrence. This is mathematically expressed in the form of the following normalization equation. (3) α02 + α12 = 1 Quantum information can have more than two states. An n state quantum unit is called qudit. The ternary unit of quantum computing is called qutrit. It consists of three states, viz. |0, |1 and |2 and can be represented in the following manner using column vectors. ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 1 0 0 |0 = ⎝ 0 ⎠ , |1 = ⎝ 1 ⎠ and |2 = ⎝ 0 ⎠ (4) 0 0 1 The superposition state of a qutrit is determined by |ψ = α0 |0 + α1 |1 + α2 |2

(5)

The normalization constraint for a qutrit state is given by α02 + α12 + α22 = 1

(6)

22

T. Dutta et al.

3 HSI Band Selection Based on Interactive Information Mutual Information (MI) is one of the easiest and effective approaches for band reduction in hyperspectral images. The first step in this method is to divide the bands into groups based on the M I values of individual bands (bi ) and the ground truth image (GT) [5]. MI(bi , GT) = H (bi ) + H (GT) − H (bi , GT) =  P(bi ,GT) bi ,GT P(bi , GT) log2 P(bi )P(GT)

(7)

where H (bi ) =



P(bi ) log2 P(bi )

(8)

bi

In Eqs. (7) and (8), MI is the mutual information between the ith band designated by (bi ) and the ground truth image (GT). H represents the Shannon entropy [11], which is used to evaluate the information present in the ith band. A M I value of 1 indicates total dependence and that of 0 indicates total independence. Dependent bands are grouped together, and the information contained is evaluated using the Interaction Information (IN) [5], as follows. (IN) = 1/S ∗ MI(GT, GTe , bi )

(9)

where, MI(GT, GTe , bi ) = MI(GT, GTe , bi ) − MI(GT, bi ) − MI(GTe , bi )

(10)

In Eq. (9), S is the total number of bands in a group. Three bands that are taken to predict the interactive information are ground truth image, estimated ground truth image (GTe ) and individual bands.

4 Proposed Methodology In this paper, a qutrit-inspired genetic algorithm is proposed for HSI image thresholding. The process is divided into two parts, viz. (1) HSI pre-processing Stage and (2) qutrit-inspired genetic algorithm. The entire process is explained with the help of the flow diagram shown in Fig. 1.

Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding

23

Fig. 1 Flowchart of QutritGA for HSI thresholding

4.1 HSI Pre-processing Stage Initially, the MI is calculated for each band using Eq. (7). The bands are divided into groups on the basis of similar information content. From each group, the band with the highest information is designated as the GTe . The interactive information [5] is calculated for each band considering the respective estimated ground truth of that group using Eq. (9). If any band’s interactive information value is found to be higher than the GTe , then that band is replaced as the GTe . After obtaining the IN values for all the bands, three bands are selected, taking one band from each group. If the number of groups is more than three, one band from each group is selected having the highest IN value. Then three bands having the highest I N values are chosen to obtain the fused image for applying thresholding.

24

T. Dutta et al.

4.2 QutritGA Initially, a random population (P) of N number of chromosomes is generated using the following equation [14]. 1 1 1 |c = √ |0 + √ |1 + √ |2 3 3 3

(11)

As the initial population does not contain any prior information, hence, they are assigned with equal state amplitudes using Eq. (11). The corresponding classical states (C) are obtained by comparing the values of α0 , α1 and α2 with a randomly generated number, r n (between [0, 1]) [14]. If α02 is less than r n, then C is considered to be 0 or if the value of α02 + α12 is less than r n, then C is considered to be 1. If both the above-mentioned cases do not satisfy the necessary conditions, then C is assigned to 2. Once the quantum population and classical population are obtained, the proposed algorithm is run for Max. Gen number of generations. Otsu’s thresholding is one of the most widely used thresholding algorithms [12]. For segmenting the image into K parts, K − 1 thresholds (th1 , th2 , . . . , th K −1 ) are selected. The between class variance is obtained using the following equation.

 th∗1 , . . . , th∗K −1 =

arg max

(th1 ,th2 ,...,th K −1 )∈T

2

σ B (th1 , th2 , . . . , th K −1 )

(12)

The constant term used to depict whole image mean intensity is eliminated from the original Otsu’s thresholding equation [12] to obtain the following equation.

σ B2 =

K −1 k=0

wk ∗ μ2k =

K −1 k=0

 wk

1 N



i∈Ck i ∗ li

wk

2

K −1

1 ∗ = N k=0

 thk+1 −1 i=thk

i ∗ li

tk+1 −1 i=tk

2

li (13)

In Eq. (13), li is the frequency of gray level i in the image. The cumulative probability and mean gray level are depicted as (wk ) and (μk ), respectively. [9]. Here, the value of σ B is substituted from Eq. (13) in Eq. (12) to get the fitness value of each chromosome (MOC) [9]. Tournament selection [3] procedure is used for choosing the fittest individuals for each successive generation. Then, a quantum crossover operation is performed by choosing a random gene position and interchanging the chromosomes. A predefined crossover probability value is considered for selecting few chromosomes that undergo crossover operations. A new quantum mutation operation is introduced in the proposed algorithm. A random variable r1 is initialized (between [0, 1]). If the value of r1 is less than α02 , then the corresponding gene values for α0 and α1 are interchanged. If the above condition does not hold true, then the value of r1 is compared to (α02 + α12 ), and if r1 is found to be less, then α0 and α2 values are interchanged. If both the above-mentioned conditions are not satisfied, then α1 and α2 values are

Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding

25

interchanged. The number of mutated genes depends on a predefined number called the mutation probability. After mutation is completed, the non-participating quantum coefficients are checked and altered to maintain the quantum superposition principle, as mentioned in Eq. (6). To obtain superior results, the fitness values are checked and if they do not improve in consecutive steps, quantum disaster operation is performed [14]. Few chromosomes are re-initialized to their initial states using Eq. (11) for this purpose. This prevents the algorithm from getting stuck into local minima. The highest value of σ B , obtained from Eq. (12), after executing the algorithm for Max. Gen times and corresponding threshold values is considered as the optimum result.

5 Experimental Results A brief description of the dataset is given in Sect. 5.1. The qutrit-inspired GA is compared with the classical GA [7] and qubit-inspired GA (QubitGA) [3]. The analysis of the results are briefly discussed in Sect. 5.2.

5.1 Dataset The Salinas Dataset was collected by the AVIRIS Sensor [1]. It was captured over the Salinas Valley, California. The image contains 224 bands, with a spatial resolution of 512 × 217. It was captured over a 3.7-m area. The ground truth image contains 16 classes which comprise of vegetables, bare soils, and vineyard fields.

5.2 Analysis The proposed algorithm is compared to classical GA [7] and QubitGA [3]. The modified Otsu’s criterion [9] is used as fitness function. The fused image, ground truth image and the thresholded images are presented in Fig. 2. All the algorithms are run for 1000 generations with a population size of 20. The crossover probability and mutation probability are taken as 0.9 and 0.1, respectively, for all the algorithms. The experiments are conducted for different threshold values (K = 4 and 7). In Table 1, the optimum threshold values for K = 4, 7 are presented. The best fitness values (BFV) recorded for all the three methods are also given in Table 1. The mean, standard deviation (STD) and the best execution times are recorded in Table 2. To compare the quality of segmentation, Sørensen-Dice similarity index [4] (SDI) is applied. The segmented images from all the three processes are compared with the ground truth image. SDI value ranges from 0 to 1, where a perfect segmen-

26

T. Dutta et al.

(a) Fused Image

(b) Ground Truth Image

(c) GA [7](K=4)

(d) QubitGA [3](K=4)

(e) QutritGA(K=4)

(f) GA [7](K=7)

(g) QubitGA [3](K=7)

(h) QutritGA(K=7)

Fig. 2 a Fused Image using Interactive Information, b ground truth image for Salinas Dataset, c–f thresholded Images (K = 4,7) for GA [7], QubitGA [3] and QutritGA Table 1 Threshold values for GA [7], QubitGA [3] and QutritGA on Salinas Dataset [1] and BFV K GA [7] BFV QubitGA [3] BFV QutritGA BFV 4 7

53, 99, 160 5814.1878 26, 42, 62, 5916.7607 79, 115, 165

51, 100, 158 5814.1855 25, 46, 67, 5917.0402 111, 153, 199

51, 97, 158 18, 47, 66, 105, 147, 186

5814.2022 5917.4128

Table 2 Mean, STD, Time and SDI [4] values for GA [7], QubitGA [3] and QutritGA on Salinas Dataset [1] K Process Mean STD Time SDI 4 4 7 7

GA [7] QubitGA [3] QutritGA GA [7] QubitGA [3] QutritGA

5813.8667 5814.1860 5814.1958 5916.2622 5916.8810 5917.4515

0.18210 0.00300 0.00810 0.14520 0.13260 0.29760

1.5791 1.1411 1.2026 1.5791 1.1497 0.6025

0.00004 0.00008 0.18870 0.03753 0.10272 0.26450

tation is designated by 1. QutritGA produces better results compared to GA [7] and QubitGA [3] in terms of SDI. As each gene is considered to be a qutrit in superposed state, a wide search space is traversed for the same population size compared to classical GA [7] and

Qutrit-Based Genetic Algorithm for Hyperspectral Image Thresholding

27

QubitGA [3]. Hence, QutritGA provides better results. Due to the implementation of quantum superposition, QutritGA has a faster execution time as shown in Table 2. A statistical analysis test, called the two-tailed t-test [6], is conducted. A 5% confidence level is considered for the test. The p values are found to be 0.02745 for K = 4 and 0.00001 for K = 7. As the p values are less than 0.05, this indicates that the null hypothesis is discarded and the results are significant. The algorithms are executed in MATLAB R2019a. An Intel(R) Core(TM) i7 8700 Processor with Windows 10 environment is used for executing the process.

6 Conclusion In the proposed algorithm, a qutrit-inspired genetic algorithm is developed for thresholding of hyperspectral images, by reducing the number of bands. The application of a qutrit-based population helps in traversing a huge search space by employing lesser number of particles. The superiority of QutritGA over its classical counterpart and qubit-inspired GA is established by means of the two-tailed t-test and the Sørensen-Dice similarity index. Evaluating the performance of the qudit version of genetic algorithm on hyperspectral image is a future scope of research. Acknowledgements This work was supported by the AICTE sponsored RPS project on Automatic Clustering of Satellite Imagery using Quantum-Inspired Metaheuristics vide F.No 8-42/RIFD/RPS/ Policy-1/2017-18.

References 1. Graña, M., Veganzons, M.A., Ayerdi, B.: Hyperspectral remote sensing scenes— grupo de inteligencia computacional (gic) (2019). http://www.ehu.eus/ccwintco/index.php? title=Hyperspectral_Remote_Sensing_Scenes. Accessed 7 Oct 2019 2. Deutsch, D., Jozsa, R.: Rapid solution of problems by quantum computation. Proc. R. Soc. Lond. Ser. A: Math. Phys. Sci. 439(1907), 553–558 (1992) 3. Dey, S., Bhattacharyya, S., Maulik, U.: Quantum inspired genetic algorithm and particle swarm optimization using chaotic map model based interference for gray level image thresholding. Swarm Evol. Comput. 15, 38–57 (2014) 4. Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945) 5. Elmaizi, A., Nhaila, H., Sarhrouni, E., Hammouch, A., Nacir, C.: A novel information gain based approach for classification and dimensionality reduction of hyperspectral images. Procedia Comput. Sci. 148, 126–134 (2019) 6. Flury, B.: A First Course in Multivariate Statistics. Springer, New York (1997) 7. Holland, J.H.: Adaptation in Natural and Artificial Systems : An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. University of Michigan Press, Ann Arbor (1975) 8. McMahon, D.: Quantum Computing Explained. Wiley, Hoboken, New Jersey (2008) 9. Merzban, M.H., Elbayoumi, M.: Efficient solution of Otsu multilevel image thresholding: a comparative study. Expert Syst. Appl. 116 (2019)

28

T. Dutta et al.

10. Narayanan, A., Moore, M.: Quantum-inspired genetic algorithms. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 61–66 (1996) 11. Nhaila, H., Elmaizi, A., Sarhrouni, E., Hammouch, A.: New wrapper method based on normalized mutual information for dimension reduction and classification of hyperspectral images. In: 2018 4th International Conference on Optimization and Applications (ICOA), pp. 1–7 (2018) 12. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979) 13. Storn, R., Price, K.: Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997) 14. Tkachuk, V.: Quantum genetic algorithm based on qutrits and its application. Math. Prob. Eng. 2018(8614073) (2018)

Employing Parallel Hardware Architectures to Diagnose Sickle Cell Anemia in Real-Time Basis Mohammed A. Fadhel and Omran Al-Shamma

Abstract Among the solid components of the human blood, the type that has the largest number of them is well known as red blood cells (RBCs). These cells have flat-round shapes, where their centers are depressed like a doughnut missing its hole. When the cell shape is changed from circular to sickle, then this case is a blood disease named sickle cell anemia (SCA). Based on its number, the dangerous level is obtained. This paper employs parallel hardware architectures to detect the sickle cells and its dangerous level in a real-time basis. These parallel architectures include the field programmable gated array (FPGA) and the graphical processing unit (GPU). In addition, the central processing unit (CPU) as the common serial architecture is also employed for comparison basis in terms of time consuming and power consumption. The circular Hough Transform (CHT) method is employed for detecting the sickle cells. To determine the dangerous level, the number of sickle cells and the number of normal ones are counted. The detection, counting, and classification algorithms are all coded in the Verilog language (for the FPGA) and in the MATLAB software (for the GPU and CPU). The findings have achieved well-behaved performances and acceptable results are obtained. Keywords Red blood cells · Sickle cell anemia · Parallel architecture · FPGA · GPU · Circular Hough transform

1 Introduction Among the solid components of the human blood, the type that has the largest number of them is well known as red blood cells (RBCs). The primary task of these cells is in carrying oxygen to the all-human body [1]. One of the factors that has significant M. A. Fadhel (B) College of Computer Science and Information Technology, University of Sumer, ThiQar, Iraq e-mail: [email protected] O. Al-Shamma University of Information Technology and Communications, Baghdad, Iraq e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_4

29

30

M. A. Fadhel and O. Al-Shamma

impact on the human health is the quantity of the hemoglobin, which is a scale of the red blood. When this quantity becomes low, this leads to a shortage in the supplied oxygen to the tissues, and in turn, a shortage in breathing and could cause a fatigue [2]. In contrast, if the hemoglobin molecule has abnormal structure due to genetic defect, then the case is so-called hemoglobinopathy, or simply, sickle cell anemia [3]. The easiest tool to diagnose the sickle cell, in addition, to normal ones, is the simple blood test aided a microscope. It is very important in determining the dangerous level of the SCA to calculate the number of normal/abnormal RBCs. However, the manual counting of the RBCs is extremely a boring task and often leads to errors in the counting process. Thus, successful SCA diagnosis requires automatic, effective, and accurate tools. Image processing approach is in front of these tools [4]. FPGA and GPU have been utilized to solve very heavy computational tasks [5–9]. This paper introduces parallel hardware architectures to diagnose SCA in realtime basis. The FPGA and the GPU platforms are employed. In addition, the usual serial architecture represented by the CPU is also employed for comparison purposes in terms of time consuming and power consumption. The CHT technique, which includes five basic steps, is utilized to diagnose the SCA. These steps are entering image data, filtering, detecting the edges, normal/abnormal cell counting, and classifying. Successful performances have achieved and reasonable accuracies are obtained.

2 Literature Review Developing automatic diagnosis of erythrocytes has been started in the last decade. It is based on image processing with the aid of MATLAB [10]. One earlier research studied the development of the gene that creates hemoglobin with sickle shape [11]. At a transferable disease description, the natural selection ability of the human gene in providing better adaptive fitness was investigated, as well [12]. However, in most preceding researches, the correlated features of the area, perimeter, diameter, and shape are compared with specific thresholds to diagnose the blood diseases [13]. In contrast, traditional, artificial, and convolutional neural networks were also employed as different diagnostic techniques in several studies. Feature extraction, segmentation, and deep learning methods have applied in these neural networks, respectively [14]. For example, the sickle cells have identified by Barpanda [15], whereas the Plasmodium parasites that existing on the microscope slides have recognized by Memeu [16]. Clustering-based segmentation method has applied in both works. For cell recognition, the feature extraction of the image that is based on texture, color, and geometry has applied. On the other side, an algorithm to obtain the extended and normal cells within a group of cells was introduced by Gonzalez-Hedalgo et al. [17]. This algorithm employed ellipse adjustment in an expert approach to detect the cells. In addition, it incorporated an innovative technique to effectively recognize the interested concave and convex points. Note that no need for pre-processing is required in this method.

Employing Parallel Hardware Architectures …

31

Due to the capability of the method in achieving legal ellipses using the presented criteria, the realized findings showed an excellent performance. Furthermore, the technique of interested point detection was extremely efficient. Conversely, a novel technique called fractal dimension performs the shape analysis of fractal objects. It is usually used to anywhere detect all man-made objects in the image is necessary. In images with a black–white scale, the local fractal method is extremely efficient to recognize any object [18]. The box-counting algorithm is commonly applied algorithm to compute the local fractal dimension in real-time basis [19, 20]. Based on shape and range, this algorithm demonstrates the uncertainties in the mass scale technique. It illustrates the basic concept of using fractal dimension in the role of an important algorithm to recognize objects. In general, such algorithm has many varied fields of applications [21]. For example, it is used to extract the significant features of the face recognition using the fractal code [22]. Exclusive of the need for segmentation, this algorithm can furthermore expanded to identify the segments of the face rather than the complete face. However, for studying RBC aggregation, it is actually advantageous tool [23]. All these techniques used CPU (serial architecture) to perform classification and diagnosis. The problem is these techniques are time consuming and are not applicable for real-time purposes. It is necessary to include parallel processing to achieve real-time mode. This paper presented parallel hardware architectures to diagnose SCA in real-time basis. The FPGA and the GPU platforms are employed.

3 Parallel Hardware Implementation Generally, processing architectures are categorized into two types, either serial or parallel processing. In serial processing, the task is executed step after step, hence, time consuming. In contrast, parallel processing executes the task steps in parallel, hence, time saving. The processing time is inversely proportional to cost. Thus, parallel processing is much faster and much costly. On the other hand, parallel processing can be implemented either software or hardware. Software parallel processing is slower, cheaper, and lesser complexity or system size. Most real-time applications use hardware parallel architectures to achieve a successful performance. In addition, long-life applications pay cost once at system development and last long time with free of charge. Thus, parallel hardware architectures are not costly if they are long-life applications [24]. However, this study applied two types of parallel hardware architectures, which are FPGA and GPU, and one common serial architecture, i.e., the CPU, for time and power comparison purposes. The FPGA and the GPU architectures are briefly described in sequence, as in the following:

32

M. A. Fadhel and O. Al-Shamma

3.1 FPGA Over the last two decades, the FPGA has turned out to be at the forefront of the main digital circuit employment instruments. Its architecture, which dominates the nature of its programmable interconnection and its programmable logic functionality, is the significant part of its creation. In addition, the FPGA architecture has a powerful impact on the value of the finishing device’s power consumption, area efficiency, and speed performance [25]. It is important to consider the data flow inside the FPGA more specifically to understand the FPGA operation. Initially, the input image, which is picked up from the microscope, is in analogue form. This image is converted to digital via the TV decoder unit, which represents the FPGA input unit. The ITU 656 unit receives the digital format and converts it to the standard YUV 4:2:2 format. Then, it stores in the SDRAM for buffering purposes. Then, the stored format is passed through a multiplexer to detect and synchronize the stream format. The format is then converted to YUV 4:4:2 format to be able to convert it to 10-RGB format in the YCbCr-RBG unit. The YCbCr format is the standard format of the YUV system. The letter Y in the YCbCr represents Luma (luminous), whereas the blue and the red difference Chroma are represented by Cb and Cr, respectively [26, 27]. Then, the 10-RGB format is decoded to RGB format via the RGB decoder and is passed to the CHT block. The CHT block has three modules to filter, detect edges, and perform circle Hough transform (CHT) algorithm. The output of the CHT block is passed to the video DAC unit to convert the digital format to an analogue signal for displaying on VGA unit. Figure 1 shows the FPGA dataflow diagram.

Fig. 1 FPGA dataflow diagram

Employing Parallel Hardware Architectures …

33

3.2 GPU In general, the GPUs have different architectural specifications based on their manufacturers. Thus, the following describes the general GPU architecture. Initially, all GPUs start when a task invokes specific application programming interfaces (APIs) to create a vertices’ stream that identify the sight geometry. The streamer unit inside the GPU receives this stream. It has a cash memory joined together for capturing localities. Then, these vertices are passed to a group of shaders. These shaders perform shading features to these vertices using a few elements joined together with them. Next, the primitive assembly stage takes place to convert the received shaded vertices to triangles. The following stage is the clipper stage, which performs a trivial triangle rejection test. Then, the triangle setup stage takes place to calculate the triangle depth and edge interpolation equations. The next stage is the fragment generator, which converts the triangles into pixels (fragment). The non-visible fragments are then removed in the Z and stencil test stage based on the scene depth-complexity to reduce the calculated load within the fragment shaders. It should be noted that a cash memory named “ZST cache” is utilized I each Z and stencil test stage. Then, the interpolator stage receives the remaining visible fragments and generates triangle attributes from the fragment attributes using perspective corrected linear interpolation. These interpolated fragments are supplied to the shader group. However, the programmable stage in the GPU is the shader, which allow generalpurpose algorithms to perform at the shader top. It comprised of several texture units and a pipeline. These texture units have a cash memory joints together with each unit and identified “TU cache.” In addition, they provide mip-mapping (a method utilized to decrease aliasing artifacts and improve rendering speed), cube-map and n-dimensional textures, anisotropic, trilinear, and bilinear filtering. The following stage is the color write stage, which blends the fragments. The frame buffer receives these bent fragments for displaying [28, 29]. A cash memory is jointed together with each color write unit. The modern GPU architecture is shown in Fig. 2.

Fig. 2 GPU architecture [28]

34

M. A. Fadhel and O. Al-Shamma

4 Diagnostic Algorithm of SCA The main diagnostic algorithm is based on the CHT technique, which includes five key stages, as in the following: 1.

2.

3.

4.

5.

Entering image data: In common cases, the input image is picked up from a microscope. It is also possible to input stored images with the aid of glass slides. These images are colored and have JPEG formats. Pre-processing: It consists of two steps. In the first step, the colored input image is converted to grayscale image, while in the second step, noise elimination is takes place. Detecting edges: One of the sophisticated techniques in image processing is the edge detection. It is utilized as a reference for different segmentation techniques. The edges and the surrounded areas are much correlated due to the specific intensity changes in these areas. For the shape of the sickle cell, three features are extracted, which are the highest, lowest, and mean cell radius. As compared with standard cell sizes, these features are recognized [30]. This technique has the ability to detect the overlapped blood cells and the imperfect boundary cells. Applying CHT technique: It is utilized to recognize the distinct blood cells. This technique calculates a form factor to accurately solve the problem of overlapping cells [1]. This form factor has a value range from 0.5 to 1 for healthy (normal) blood cells. Unfortunately, the blood half-cells cannot recognize in this technique. In general, it has an accuracy of 91%. Calculating sickle/normal cells and evaluating: In the last stage, the normal/abnormal cells are counted to classify the case under consideration as a healthy or sickle cell anemia. Figure 3 illustrates the flow diagram of the sickle cell diagnostic technique:

5 Results and Discussion The FPGA hardware connection to diagnose sickle cell anemia is shown in Fig. 4. At the beginning, the medical analyst has set uniformly a blood droplet in a specific method on a single slide. A digital microscope captures a blood image and send it to the FPGA input port. The image is in JPEG format, as illustrated in Fig. 5. The received image is converted to grayscale image (see Fig. 6). Gaussian filter is utilized for noise reduction purposes. For feature extraction, the canny edge-detection technique is then executed, as obtained in Fig. 7. Next, the CHT technique is applied to diagnose the healthy (normal) blood cells (see Fig. 8) and the sickle cells (see Fig. 9). The last stage after CHT technique is the calculation of the normal and the sickle cells. Table 1 lists the findings, which shows that about 43% of the total cells are sickle cells. Thus, it is a sickle cell anemia.

Employing Parallel Hardware Architectures …

35

Fig. 3 Diagnosing sickle cell anemia flowchart

Pre-processing

Entering image data Converting to Grayscale conversion Filtering the noises Detecting the edges CHT Counting the red blood cells

Digital

Video in Cable

VGA Out

Microscope with camera

V G A M FPGA

oni

Fig. 4 FPGA hardware connection for diagnosing sickle cell anemia

Fig. 5 Blood droplet on a slide (in JPEG format)

However, the diagnostic procedure was applied on different architectures, two of them are parallel and one is serial. First, the FPGA architecture is utilized. It performed the procedure in 5.474 ms. The used FPGA type was Altera DE2

36 Fig. 6 Grayscale blood image

Fig. 7 Blood image after applying the canny edge-detection technique

Fig. 8 Output of CHT technique (healthy cells)

Fig. 9 Output of CHT technique (sickle cells)

M. A. Fadhel and O. Al-Shamma

Employing Parallel Hardware Architectures … Table 1 Counting results after applying CHT technique

Table 2 Execution time and the power consumption of different hardware platforms

37

Normal

Abnormal

Total cell count

140

105

245

FPGA Execution time (ms) Power consumption (mW)

5.474 149.48

GPU

CPU

4.919

8214

176,000

31,000

Cyclone II, which contains 33,216 logic blocks, 4 PLLs, and two clock oscillators of 50 and 27-MHz. Next, the GPU architecture was utilized. It performed the procedure in 4.919 ms. The used GPU type was GeForce gtx 1080 NVIDIA, which has 2560 CUDA cores with two clocks (1607-MHz for graphic and 1733-MHz for the processor). In contrast, the serial architecture CPU performed the procedure in 8213 ms. Its type is Core i7 eighth generation, which has 4 cores, 16 GB RAM, and 2.3-GHz base clock and up to 4 GHz with Intel turbo boost technology. From the point view of the power consumption, the lowest consuming power architecture was the FPGA with 149.48 mW, the CPU is the second with 31,000 mW, and the GPU has the highest power consumption with 176,000 mW (see Table 2). The main drawback of the GPU architecture is that it consumes very high power because the complete entire logic elements are operated during the execution of the procedure. In contrast, the FPGA architecture consumes the lowest power because it uses only the required logic elements needed to carry out the procedure. It should be noted that the biggest benefit of using the GPU architecture is its simplicity of use (i.e., plug and play) and it does not need a specific language to program the GPU card. The key drawback of the FPGA is that it uses a special language (Verilog language) to program the FPGA board.

6 Conclusion This paper introduced the technique of circle Hough transform to diagnose sickle cell anemia using three different architectures. This technique was simple to implement, fast in execution, and has a reasonable accuracy of 91%. The FPGA and the GPU are two parallel architectures utilized to perform the diagnosis algorithm. These parallel architectures have well-behaved performances for real-time basis, as compared to the common serial architecture. The GPU consumes considerable power but it is very easy to implement and it has not need for a special language to program it. In contrast, the FPGA has a special language (Verilog) but it consumes lowest power.

38

M. A. Fadhel and O. Al-Shamma

References 1. Fadhel, M.A., Humaidi, A.J., Oleiwi, S.R.: Image processing-based diagnosis of sickle cell anemia in erythrocytes. In: 2017 Annual Conference on New Trends in Information & Communications Technology Applications (NTICT). IEEE (2017) 2. Alzubaidi, L., Al-Shamma, O., Fadhel, M.A., Farhan, L., & Zhang, J.: Classification of red blood cells in sickle cell anemia using deep convolutional neural network. In International Conference on Intelligent Systems Design and Applications, pp. 550–559. Springer, Cham (2018) 3. Alzubaidi, L., Fadhel, M.A., Al-Shamma, O., Zhang, J., Duan, Y.: Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell anemia diagnosis. Electronics 9(3), 427 (2020) 4. Alzubaidi, L., Fadhel, M.A., Al-Shamma, O., Zhang, J.: Robust and efficient approach to diagnose sickle cell anemia in blood. In International Conference on Intelligent Systems Design and Applications, pp. 560–570. Springer, Cham (2018) 5. Alzubaidi, L., Al-Shamma, O., Fadhel, M.A., Farhan, L., Zhang, J., Duan, Y.: Optimizing the performance of breast cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics 9(3), 445 (2020) 6. Alzubaidi, L., Fadhel, M.A., Al-Shamma, O., Zhang, J., Santamaría, J., Duan, Y., Oleiwi, S.R.: Towards a better understanding of transfer learning for medical imaging: a case study. Appl. Sci. 10(13), 4523 (2020) 7. Hasan, R.I., Yusuf, S.M., Alzubaidi, L.: Review of the state of the art of deep learning for plant diseases: a broad analysis and discussion. Plants 9(10), 1302 (2020) 8. Alzubaidi, L., Al-Shamma, O., Fadhel, M.A., Arkah, Z.M., & Awad, F.H.: A deep convolutional neural network model for multi-class fruits classification. In: International Conference on Intelligent Systems Design and Applications, pp. 90–99. Springer, Cham (2019) 9. Fadhel, M.A., Al-Shamma, O., Alzubaidi, L., Oleiwi, S.R. Real-time sickle cell anemia diagnosis based hardware accelerator. In: International Conference on New Trends in Information and Communications Technology applications, pp. 189–199. Springer, Cham (2020) 10. Gonzalez, R.G., Woods, R.G., Eddins, S.L.: Digital image processing. Pearson Education, Inc., NJ. (2007) 11. Paunipagar, P.V., Pati, S.K.B., Singh, C.M., Arya, R.C.: Sickle cell gene in tribal area of Rajnandgaon district of Chhattisgarh. Indian J. Prev. Soc. Med. 37(3 & 4) (2006) 12. Buford, J.A.: Sickle cell hemoglobin and malaria: an adaptive study of natural selection on an infectious disease (2004) 13. Taherisadr, M., Nasirzonouzi, M., Baradaran, B., Mehdizade, A.: New approach to red blood cell classification using morphological image processing. Shiraz E-Med J 14(1) (2013) 14. Veluchamy, M., Perumal, K., Ponuchamy, T.: Feature extraction and classification of blood cells using artificial neural network. Am. J. Appl. Sci. 9(5):615–619 (2012). ISSN 1546-9239 15. Barpanda, S.S.: Use of Image Processing Techniques to Automatically Diagnose Sickle-Cell Anaemia Present in RBC Smear. National Institute of Technology (ODISHA), May 2013 16. Memeu, D.M.: A Rapid Malaria Diagnostic Method Based on Automatic Detection and Classification of Plasmodium Parasites in Stained Thin Blood Smear Images. University of Nairobi, Mar 2014 17. Gonzalez-Hidalgo, M., Guerero-pena, F.A., Herold-Garcia, S.: Red blood cell cluster separation from digital images for use in sickle cell disease. IEEE J. Biomed. Health Informat. JBHI.2356402 (2014) 18. Beaver, P., Quirk, S.M. (U.S Military Academy), Sattler, J.P. (Army Research Lab): Object Characterization in Grey Scale Imagery Using Fractal Dimension. U.S. Army Research Laboratory, Dec 2015 19. Alzubaidi, L., Fadhel, M.A., Oleiwi, S.R., Al-Shamma, O., Zhang, J.: DFU_QUTNet: diabetic foot ulcer classification using novel deep convolutional neural network. Multimedia Tools Appl. 79(21), 15655–15677 (2020)

Employing Parallel Hardware Architectures …

39

20. Ebrahimpour-Komleh, H., Chandran, V., Sridharan, S.: Face recognition using fractal codes. In: Proceedings of International Conference on Image Processing. IEEE, Thessaloniki (2001) 21. Rapa, A., Oancea, S., Creanga, D.: Fractal dimensions in RBC. Turk. J. Vet. Anim. Sci. 29, 1247–1253 (2005) 22. Les, T., Kruk, M., Osowski, S.: Objects Classification Using Fractal Dimension and Shape Based on Leaves Classification. Warsaw University of Technology & Life sciences (2013) 23. de Araujo Mariath, J.E., dos Santos, R.P., dos Santos, R.P.: Fractal dimension of the leaf vascular system of three Relbunium species (Rubiaceae). Brazilian J. Biosci (2010). ISSN 1980-4849 (on-line)/1679-2343 (print) 24. Esquembri, S., Nieto, J., Ruiz, M., de Gracia, A., de Arcas, G.: Methodology for the implementation of real-time image processing systems using FPGAs and GPUs and their integration in EPICS using Nominal Device Support. Fusion Eng. Des. 130, 26–31 (2018) 25. Parab, J.S., Gad, R.S., Naik, G.M.: Hands-on experience with Altera FPGA Development Boards. Springer, Berlin (2018) 26. Al-Shamma, O., Fadhel, M.A., Hameed, R.A., Alzubaidi, L., Zhang, J.: Boosting convolutional neural networks performance based on FPGA accelerator. In: International Conference on Intelligent Systems Design and Applications, pp. 509–517. Springer, Cham (2018) 27. Fadhel, M.A., Al-Shamma, O., Oleiwi, S.R., Taher, B.H., Alzubaidi, L.: Real-time PCG diagnosis using FPGA. In: International Conference on Intelligent Systems Design and Applications, pp. 518–529. Springer, Cham (2018) 28. Al Maashri, A., Sun, G., Dong, X., Narayanan, V., Xie, Y.: 3D GPU architecture using cache stacking: Performance, cost, power and thermal analysis. In: 2009 IEEE International Conference on Computer Design, pp. 254–259. IEEE (2009) 29. Hong, S., Kim, H.: An analytical model for a GPU architecture with memory-level and threadlevel parallelism awareness. In: Proceedings of the 36th Annual International Symposium on Computer Architecture, pp. 152–163 (2009) 30. Humaidi, A.J., Fadhel, M.A., Ajel, A.R.: Lane detection system for day vision using altera DE2. TELKOMNIKA 17(1), 349–361 (2019)

Implementing a Hardware Accelerator to Enhance the Recognition Performance of the Fruit Mature Mohammed A. Fadhel and Omran Al-Shamma

Abstract For recognizing vegetable and fruit maturing, various techniques have been released in the last two decades. These techniques have different accuracies and are generally time consuming. To speed up the recognition performance to be suitable for real-time basis, a hardware accelerator is needed to implement. This paper introduced a field programmable gate array (FPGA), as a parallel hardware architecture, to solve the problem of time-consuming. Moreover, color-threshold and k-means clustering are two techniques utilized for recognition purposes and for comparison principles. The findings showed that the color-threshold technique required 16% of the total logic elements and performed the recognition task in 10.25 ms. In contrast, the k-means clustering technique required 62% of the logic elements and performed the recognition task in 64.88 ms. Thus, color-threshold technique is more efficient and much faster than the k-means technique. Keywords Parallel hardware architecture · FPGA · Fruit mature recognition · Color-threshold · K-means clustering

1 Introduction Images play a key role in the recognition of foodstuff and food industry. For instance, the “best before” date is set owing to the inspection of the fruits and vegetables, as well as, in determining their market prices. The manual inspection method is considerably inconsistent, time-consuming, and error decisions among human inspectors. Automatic systems as machine vision systems are the optimal solution for quality assurance, conventional analysis, and continuous tasks. The main source of data and

M. A. Fadhel (B) College of Computer Science and Information Technology, University of Sumer, Thi-Qar, Iraq e-mail: [email protected] O. Al-Shamma University of Information Technology and Communications, Baghdad, Iraq e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_5

41

42

M. A. Fadhel and O. Al-Shamma

information for these systems creates from photographic images. Thus, digital image processing enhances the image processing and analyzing. In the agriculture field, image processing has several applications. These applications involve detecting plant disease, automated classification, and recognizing the pest infected regions based on color, texture, and shape features [1, 2]. As computer vision and information sciences have been rapidly growing in the last decade, computerized inspection techniques of fruits and vegetables have mutually grown, as well. In food industries, the recognition of fruit maturity based on manual human perspective has several faults [3]. Automation enhances the work performance, saves task-time, and reduces the system failures. This encourages researchers to develop a number of techniques to recognize the maturity of fruit and vegetable [4–6]. Conversely, the color feature is a key factor in recognizing the maturity of the fruit. It represents an indirect measure of quality features such as variety, desirability, and freshness [7]. It is commonly used in image processing due to its; powerful in distinguishing an image among others, robust to background complexity, efficient in characterizing visual image contents, independent in orientation and size, as well as, simplicity in the extracting color information from images. In addition, various color spaces like CIELab, HIS, and RBG are widely used in inspecting the maturity of fruits and vegetables. Among these spaces, the RBG is used in this paper since images are commonly picked up using RBG models [8]. However, several researchers introduced various methods to recognize and classify the fruit maturity. In 2013, Prabha and Kumar [9] have used calibration images to extricate the mean color intensity from the perimeter, area, and histogram. They have achieved accuracies of 85 and 99.1% for the mean color algorithm and area algorithm, respectively. Visible optical-fiber sensors, which involve RGB LEDs of 470, 525, and 635 nm wavelengths, are proposed to evaluate the fruit quality [10]. At various ripeness indices, the data set obtained a coefficient of determination (R2 = 0.879). Recently, the color features were used in algorithm for presenting the citrus estimate [11]. The maker-controlled watershed algorithm and distance transform were employed to perform automated watershed segmentation. This approach obtained well-behaved findings with 0.93 correlation coefficient. In addition, random forests and digital images were used to estimate the papaya ripeness [12]. Every peel that has low computational cost was used to calculate the color features. The results showed an accuracy of 94.3% in classification. Recently, deep learning plays an important role in agricultural and other fields, but need huge dataset to train its networks [13–16]. However, among different approaches and technique, the k-means and colorthreshold techniques are the most familiar, easy to implement, have a considerable classification accuracies. Thus, they are employed in this paper for recognizing the fruit mature. For real-time application, a parallel hardware architecture called FPGA is used, as well.

Implementing a Hardware Accelerator to Enhance …

43

2 Fruit Recognition Process In the fruit recognition process, the first step after image data entering is the image segmentation, given that the image is partitioned into meaningful areas relating to the color. Implementing image segmentation is based on monochromatic images, where the main information source is the intensity. In contrast, the human eye has the ability to detect thousands of color intensities and shades. Regarding to grayscale images, the human eye is able to detect around double dozens of shades. Thus, it is popular to process color images as they have extra capacity, extra information, and speed in processing the information [3]. The color detecting cells in the human eye system are the basis of storing the realtime images in RGB format. Processing color images need to convert its color format from RGB to YCbCr to make use of the lower resolution capability of the human eye system for color, regarding to the brightness [3]. It is commonly used in image processing. Note that the letter Y in YCbCr stands for luminance, while Cb and Cr are the chrominance-blue and chrominance-red components, respectively. The main benefit of color conversion is to separate the luminance component for transmitting at high bandwidth or storing with high resolution. The other two components (Cb and Cr) can be treated separately, compressed, subsampled, or bandwidth-reduced for enhanced system efficiency. From the medical point view, there are around 120 million rods and 6–7 million cones in the human eye. The rods are much more sensitive than cones, but are not sensitive to color. Therefore, most image variation is in intensity. In addition, the largest part of the signal energy is focused into the luminance component. The intensity decimal values of R, G, and B of various red shades are listed in Table 1 [17]. Table 1 Red shade Red shades

Light

Hex

RGB

90%

#FFCCCC

RGB(255, 204, 204)

85%

#FFB3B3

RGB(255, 179, 179)

80%

#FF9999

RGB(255, 153, 153)

75%

#FF8080

RGB(255, 128, 128)

70%

#FF6666

RGB(255, 102, 102)

65%

#FF4D4D

RGB(255, 77, 77)

60%

#FF3333

RGB(255, 51, 51)

55%

#FF1A1A

RGB(255, 26, 26)

50%

#FF0000

RGB(255, 0, 0)

44

M. A. Fadhel and O. Al-Shamma

2.1 Color-Threshold In this method, a label, mostly similar to scalar threshold, is assigned to each pixel in the image. According to a color set of interest, the core objective of this technique is to identify each pixel of which color of the color set it belongs. The technique results an image of labels, where every pixel belongs to a color class. To simplify this technique, each color class is represented by a rectangular box with two thresholds, known as boundaries of the box [5]. As mentioned earlier, it is not possible to work with RGB format unless fixing the Illumination, due to its highly correlated with the other three components; red, green, and blue. More specifically, any change in the intensity allows all points to move diagonally inside the RGB space, which leads to enlarge the box. Therefore, color discrimination becomes more difficult and just some distinct colors can be identified [18]. However, a few enhancements can be achieved by converting to YCbCr format, since the rectangular boxes, which are made parallel with the axis of YCbCr space, become diagonal in RGB space. In contrast, the chrominance components can be resized along with the luminance to achieve well-behaved distinction, or instead resizing the chrominance components with maximum the three basic components (red, green, and blue) [19].

2.2 K-Means Clustering It is one of the highly effective techniques in fruit image processing. It partitions a dataset into subsets (clusters) or classifies objects into different number of groups. In each cluster, all data have some shared characteristics or features. Partitioning data is a common way to analyze statistical data and is commonly used in many applications. Machine learning, pattern recognition, and image analysis are some of these applications [20]. Unsupervised learning is a computational task that segments the dataset into k number of subsets (clusters). Various clustering approaches were developed for various applications. The k-means is a typical clustering algorithm. In general, k-means technique is very fast and it is straightforward; thus, it is practically attractive. It is usually employed in determining the presence of pixel natural groupings inside the image. The cluster center (so-called adaptive changing center), which represents each cluster, starts from certain initial values known as seed points. This technique calculates the distance between the centers and the input data points (simply known as inputs), which are allocated to the closest center. In addition, it categorizes the input data objects into a number of classes according to their inherent distance between each other. It is also known as unsupervised clustering technique. However, the vector space in a clustering algorithm is designed from features of the data. A clustering algorithm attempts to classify natural clustering inside these features. Note that clustering the objects should be about the centroids µi∀i = 1 [21].

Implementing a Hardware Accelerator to Enhance …

45

Fig. 1 Hardware setup for the ripe apple recognition system

An iterative version of K-means algorithm is applied as part of this design. The input should be a color image as required by the algorithm.

2.3 Hardware Implementation The FPGA technologies have become widely used in video and image processing applications due to their architecture. The core objective of this work is to design and implement a ripe apple recognition using FPGA. In image processing, if the image size and the bit depth increase, the software become less useful in real-time applications. These real-time systems need powerful processors to enhance their performances. The problem is how to deal with huge data. Since the FPGA architecture is parallel inherent, it can perform the logic required by the application via constructing independent hardware for each function. These aspects make a benefit of the FPGA speed in enhancing the performance with relatively less cost. Thus, FPGA architecture is very suitable for real-time experiments [22–25]. Figure 1 shows the hardware implementation for the ripe apple recognition system. Initially, the input image is picked up using a real-time camera on the conveyor belt that carries the apples. Next, these images are sent to the FPGA (here using Altera DE2 Cyclone II) for classifying apples as ripe or unripe fruits, depending on the color segmentation algorithms. It is worth to mention that these steps are programmed inside the FPGA hardware using the Verilog language.

3 Results and Discussion The recognition sequence starts with inputting the image data from the camera into the ITU656 unit of the FPGA model as an RGB format. This unit transforms the

46

M. A. Fadhel and O. Al-Shamma

input format to YUV 4:2:2 format, or so-called YCbCr. Next, the system shrinks the samples of the input signal from 720 to 640 horizontal pixels and buffers the output frame into a frame buffer (SDRAM FIFO). The FIFO output is transformed from YUV 4:2:2 to 4:4:4 format. Finally, a 10-bit RGB format is generated based on the new YUV format. The RGB data is delivered to the VGA controller for display on the VGA monitor, either directly or through one or more modules, such as noise filtering, morphology technique, or color segmentation algorithms. Figure 2 illustrates the data flow diagram of the video decoder hardware for ripe apple recognition. Figure 3 shows the input image in RGB format. In Fig. 4, the color image is transformed to a black and white format, where the “1” represents all detected pixels and “0” represents the other pixels. The binary thresholding explained that the fruit is shown as black and the background is shown as white. The first step is segmenting the ripe apple by the color thresholding technique, which, in turn, depends on the color shading.

Camera/Video input device

VGA Controller

ITU656 to YUV 4:2:2

Down sample from 720 to 640 pixels

Noise filtering, Morphology technique, and color segmentation algorithms

Fig. 2 Mature apple recognition steps

Fig. 3 Original image

Fig. 4 Raw segmentation step

YUV 4:4:4 to RGB

SDRAM frame buffer

YUV 4:2:2 to YUV 4:4:4

Implementing a Hardware Accelerator to Enhance …

47

The following code written in the Verilog language illustrates the range of red shades: if (((aRed-aGreen) > 10’d0) && ((aRed-aGreen) < 10’d74)) begin raw_R < = 10’h3FF; raw_G < = 10’h3FF; raw_B < = 10’h3FF; data_reg1 [VGA_A1] < = 1’b1; end else begin raw_R < = 10’h0; raw_G < = 10’h0; raw_B < = 10’h0; data_reg1 [VGA_A1] < = 1’b0; end The range of color shading (10’d0 to 10’d74) is tested randomly by the trial-anderror method to select the required color. To enhance the binary image, a morphology technique is applied, such as the erosion process to remove the separated pixels. Next, the dilation process is used for filling the holes in the black region, as shown in Figs. 5, 6 and 7. Figure 8 illustrates the processing sequence of the K-means clustering method. Converting the image format from RGB to L * a * b color space is the first Fig. 5 Erosion result

Fig. 6 Filled region

48

M. A. Fadhel and O. Al-Shamma

Fig. 7 Final output

Input image

Mature an apple are segmented

Image format conversion (RGB to L*a*b) color Divide the original image by color

Color assorting (K-means clustering method) Labelling all pixels in the image

Fig. 8 Overall procedure of K-means clustering algorithm

step in the sequence. Next, the color is classified using the K-means method. Labeling all pixels of the image is the next step. Finally, separating the original image based on color. Figures 9, 10, 11 and 12 illustrate these findings. Figures 13 and 14 illustrate the flow summary obtained from Quartus II 11.1 Fig. 9 Original image

Fig. 10 Image filtered by Gaussian

Implementing a Hardware Accelerator to Enhance …

49

Fig. 11 Noise-removed binary image

Fig. 12 Cropped apple area

Fig. 13 Flow summary of color thresholding

web edition (32 bit) of the FPGA model (Altera DE2 Cyclone II (EP2C35F672C8) family). In considering these two figures, we found that the total memory, total registers, total logic elements (total combinational functions and dedicated logic register), and embedded multiplier bits for the color-threshold technique are greater

50

M. A. Fadhel and O. Al-Shamma

Fig. 14 Flow summary of K-means clustering

Table 2 Execution time on FPGA Altera DE2 model

Technique

Execution time (ms)

Color thresholding

10.2478

K-mean clustering

64.8741

than K-means clustering technique in terms of hardware components, due to the complex operation of the latter technique. Table 2 lists the execution time for both techniques. The time exhausted in the color-threshold technique is smaller than K-means clustering due to its process simplicity that have a considerable effect on the required hardware design.

4 Conclusions This paper employed two techniques for performing apple mature recognition, which are color-threshold and K-means clustering. The findings obtained the following points: • The color-threshold technique is much simpler than the K-means algorithm, since it needs only the intensity information for the detection process, while the K-means requires training and learning algorithms for finding the clustering center. • By changing the luminance, the K-means method has the ability to determine the chosen red shade, while the color-threshold method can obtain only a binary color at another contrast band. This procedure needs to be repeated for the execution of the searching algorithm at any change in the environment.

Implementing a Hardware Accelerator to Enhance …

51

• The color-threshold method is faster than the K-means method, since it needs less significant hardware design.

References 1. Patil, J.K., Kumar, R.: Advances in image processing for detection of plant diseases. J. Adv. Bioinform. Appl. Res. 2(2), 135–141 (2011) 2. Krishna, M., Jabert, G.: Pest control in agriculture plantation using image processing. IOSR J. Electron. Commun. Eng. (IOSR-JECE) 6(4), 68–74 (2013) 3. Fadhel, M.A., Hatem, A.S., Alkhalisy, M.A.E., Awad, F.H., Alzubaidi, L.: Recognition of the unripe strawberry by using color segmentation techniques. Int. J. Eng. Technol. 7(4), 3383–3387 (2018) 4. Naik, S., Patel, B.: A machine vision based fruit classification and grading: a review. Int. J. Comput. Appl. 170(9), 22–34 (2017) 5. Dubey, S.R., Jalal, A.S.: Application of image processing in fruits and vegetables analysis: a review. J. Intell. Syst. 24(4), 405–424 (2015) 6. Zhang, B., Huang, Z., Li, J., Zhao, C., Fan, S., Wu, J., Liu, C.: Principle, developments and applications of computer vision for external quality inspection of fruits and vegetables: a review. Food Res. Int. 62, 326–343 (2014) 7. Pathare, P.B., Opara, U.L., Al-Said, F.A.: Colour measurement and analysis in fresh and processed foods: a review. Food Bioprocess Technol. 6(1), 36–60 (2013) 8. Mustafa, N.B.A., Arumugam, K., Ahmed, S.K., Sharrif, Z.A.M.: Classification of fruits using probabilistic neural networks-improvement using color features. In: IEEE International Conference TENCON, pp. 264–269 (2011) 9. Prabha, D.S., Kumar, J.S.: Assessment of banana fruit maturity by image processing technique. J. Food Sci. Technol. 52, 1316–1327 (2013) 10. Kalsom, O., Yahaya, M., MatJafri, M.Z., Aziz, A.A., Omar, A.F. Non-destructive quality evaluation of fruit by color based on RGB LEDs system. In: International Conference in Electronics Design, pp. 230–233 (2014) 11. Dorj, U.O., Lee, M., Yum, S.: An yield estimation in citrus orchards via fruit detection and counting using image processing. Comput. Electron. Agric. 140, 103–112 (2017) 12. Pereira, L.F.S, Jr, S., B., Valous, N.A., Barbin, D.F.: Predicting the ripening of papaya fruit with digital imaging and random forests. Comput. Electron. Agric. 145, 76–82 (2018) 13. Alzubaidi, L., Fadhel, M.A., Al-Shamma, O., Zhang, J., Santamaría, J., Duan, Y., Oleiwi, S.R.: Towards a better understanding of transfer learning for medical imaging: a case study. Appl. Sci. 10(13), 4523 (2020) 14. Alzubaidi, L., Al-Shamma, O., Fadhel, M.A., Farhan, L., Zhang, J., Duan, Y.: Optimizing the performance of breast cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics 9(3), 445 (2020) 15. Alzubaidi, L., Al-Shamma, O., Fadhel, M.A., Arkah, Z.M., Awad, F.H.: A deep convolutional neural network model for multi-class fruits classification. In: International Conference on Intelligent Systems Design and Applications, pp. 90–99. Springer, Cham (2019, December) 16. Hasan, R.I., Yusuf, S.M., Alzubaidi, L.: Review of the state of the art of deep learning for plant diseases: a broad analysis and discussion. Plants 9(10), 1302 (2020) 17. Shades of Red: http://www.w3schools.com/colors/colors_shades.asp. Last accessed 15 Jun 2020 18. Humaidi, A.J., Fadhel, M.A., Ajel, A.R.: Lane detection system for day vision using altera DE2. Telkomnika 17(1), 349–361 (2019) 19. Xu, H., Ye, Z., Ying, Y.: Identification of citrus fruit in a tree canopy using color information. Trans. Chin. Soc. Agric. Eng. 21(5), 98–101 (2005)

52

M. A. Fadhel and O. Al-Shamma

20. Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 333–342 (2010) 21. Hassan, M.R., Ema, R.R., Islam, T.: Color image segmentation using automated K-means clustering with RGB and HSV color spaces. Glob. J. Comput. Sci. Technol. 17(2), 25–33 (2017) 22. Fadhel, M.A., Al-Shamma, O., Alzubaidi, L., Oleiwi, S. R.: Real-time sickle cell anemia diagnosis based hardware accelerator. In: International Conference on New Trends in Information and Communications Technology Applications, pp. 189–199. Springer, Cham (2020, June) 23. Fadhel, M.A., Al-Shamma, O., Oleiwi, S.R., Taher, B.H., Alzubaidi, L.: Real-time PCG diagnosis using FPGA. In: Abraham, A., Cherukuri, A., Melin, P., Gandhi, N. (eds.) ISDA 2018. AISC, vol. 940, pp. 518–529. Springer, Cham (2018) 24. Al-Shamma, O., Fadhel, M.A., Hameed, R.A., Alzubaidi, L., Zhang, J.: Boosting convolutional neural networks performance based on FPGA accelerator. In: International Conference on Intelligent Systems Design and Applications, pp. 509–517. Springer, Cham, (2018, December) 25. Fadhel, M.A., Al-Shamma, O., Alzubaidi, L.: Hardware accelerator for real-time holographic projector. In: International Conference on Intelligent Systems Design and Applications, pp. 132–139. Springer, Cham (2019, December)

Time Series Modelling and Forecasting of Patient Arrivals at an Emergency Department of a Select Hospital V. Rema and K. Sikdar

Abstract Managing overcrowding with fluctuating patient arrivals in emergency department (ED) of hospitals requires a quantitative approach to make decisions related to resource planning and deployment by hospital administrators. In this context, analysing patient flow and predicting demand will enable better decision making. In this study, 7748 ED arrivals were recorded from a multi-specialty hospital in Bengaluru. The patient flow in each of the working shifts of the ED was analysed separately. Time series modelling techniques have shown to be useful in generating short-term forecasts. Shift-wise modelling approach has been used since hospital resources were planned according to the shifts. Exponential smoothing techniques proposed by Hyndman were used in this study. Model validation was further carried out along with residual analysis. The prediction intervals shift-wise have been obtained with an average confidence level of 90% which will help hospital management to redeploy resources and handle demand with increased operational efficiency. Keywords Emergency department · Patient arrivals · Time series · ETS model

1 Introduction An emergency department (ED) of a hospital is a key department that gives immediate medical attention sought by patients. This makes the ED a section of a hospital, which is likely to face intense overcrowding. Patient arrivals into an ED is clearly not deterministic in nature, it is stochastic [1]. Resources involving doctors, nurses, staff, beds, necessary equipment, etc. are limited. This makes it important for healthcare administrators to know the nature of patient arrivals into the ED to manage resources V. Rema (B) Ramaiah Institute of Management, Bengaluru, Karnataka, India e-mail: [email protected] K. Sikdar Department of Mathematics, BMS Institute of Technology and Management, Bengaluru, Karnataka, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_6

53

54

V. Rema and K. Sikdar

effectively. The capacity to forecast patient demand into the ED will enable hospitals to design strategies, facilitating optimal utilisation of the limited available resources. Kadri et al. [2] have demonstrated that time series is an effective mathematical model that can aid effective short-term forecasting of patient arrivals, enabling decision makers to improve healthcare operational efficiency. Descriptive statistics of patient arrivals in each of the three working shifts are computed. Further, this is followed by model building, forecasting and validation for each of the operational shifts of the ED. For each of the operational shift, the time series is plotted and stationarity is established using the Augmented Dickey–Fuller (ADF) test prior to applying the forecasting model. Data is divided into training and test set. The exponential smoothing models are fit. Error measures are computed, and thereafter, forecasts are generated with about 90% prediction intervals for each of the operational shifts of patient arrivals.

2 Review of Literature Technology adoption in healthcare is growing in India [3]. The ecosystem of free data flow and exchange is still building. Adoption of IoT and AI will enable achieve operational excellence in healthcare [4]. Review of literature applying mathematical models and techniques in healthcare applications shows that there are several models such as discrete-event simulation, queueing, time series among many others that study how efficiency in healthcare operations may be achieved. Literature discusses choice of models on the basis of patient flow characteristics. Analytical queueing theoretic models may be used to model simple patient flows; Markov chains and compartmental models for capacity planning and resource allocation decisions; discrete-event simulation for complex patient flows; statistical or empirical models being more relevant for modelling clinical and operational patient flows as discussed by Bhattacharjee and Ray [5]. Queueing models have been applied by Almehdawe et al. [6], Singer and Donoso [7] to analyse various performance measures with respect to offload delays for ambulance patients and key performance indicators from a hospital administrator’s and patient’s perspective. Lakshmi and Iyer [8] and Fomundam and Herrmann [9] provide a robust review of queueing applications in healthcare systems design, system operations and system analysis. Konrad et al. [10], Paul and Lin [11] used discrete-event simulation modelling to support process improvements in a hospital to demonstrate reduction in waiting time measures and overall patient length of stay. Zeng et al. [12] carried out computer simulation study to improve the quality of care at the emergency department of a community hospital through evaluation of performance metrics such as length of stay, waiting times and patient elopement. Bernstein et al. [13], Jones et al. [14] devised quantitative measures and scales to model ED operations such as the Emergency Department Work Index (EDWIN) and simulation using tools like Forecast ED have been used by Hoot et al. [15].

Time Series Modelling and Forecasting of Patient …

55

Carvalho-Silva et al. [16] in their study pointed out that time series has been used as an accurate and powerful tool to provide short-range forecasts of future patient arrivals in the emergency department. Gershon et al. [17] carried out time series modelling on healthcare data in Ontario at Canada to analyse trends in acute care hospitalisation and emergency department visit rates for a certain disease. Autoregressive integrated moving average models were used to generate forecasts. Villani et al. [18] carried out a time series based study to facilitate planning and resource allocation of emergency management services for diabetic emergencies. A time series analysis on monthly diabetic cases was conducted using data from the ambulance victoria (AV) electronic database for a duration of seven years. Forecasts were generated using seasonal autoregressive integrated moving average (SARIMA). Juang et al. [19] recorded monthly ED visits from 2009 to 2016 and applied autoregressive integrated moving average (ARIMA) analysis. The model that gave the minimum Akaike Information Criterion (AIC) and Schwartz Bayesian criterion and followed the assumptions of residual independence was selected as the best model. Capan et al. [20] used daily NICU census data between 2008 and 2012 for model development and data from January to December 2013 for validation. ARIMA and seasonal linear regression models were used to carry out forecasts. The study revealed that time series models provided higher prediction accuracy under different census conditions compared with the fixed average census approach. Luo et al. [21] developed seasonal ARIMA model and single exponential smoothing (SES) models on daily time series and day of the week time series data respectively of outpatient arrivals of a large hospital in Chengdu. A combination of the above models were used to forecast the daily outpatient visits. The combinatorial model seems to have achieved better prediction performance with lower residual variances and smaller residual errors. Kim et al. [22] used patient volume data from the Northwestern Medicine Enterprise Data Warehouse from January 2009 to June 2012 and analysed univariate methods including exponential smoothing, autoregressive integrated moving average (ARIMA), seasonal ARIMA and generalised autoregressive conditional heteroskedasticity (GARCH) methods with results being compared with benchmark historical means. Hyndman et al. [23, 24] provided the method to automatic forecasting on the basis of a range of exponential smoothing methods. The models enable evaluation of the likelihood, the AIC, other model selection criteria, prediction intervals and random simulations from the state space model. There are not many studies in the Indian context that use real-time data to analyse healthcare operational efficiency. Data availability, accessibility and privacy are among the key concerns. This study focuses on real-time data recorded from a particular hospital’s ED log book to analyse patient arrivals for different shifts of the hospital.

56

V. Rema and K. Sikdar

3 Context of the Study In this study, data is collected from a 50 bedded multi-speciality level II referral hospital located in a popular residential area in Bengaluru city. The study is restricted to analysing patient dynamics in the ED. The hospital’s ED operates in three working shifts classified as Shift 1 from 12 AM to 8 AM, Shift 2 from 8 AM to 8 PM and Shift 3 from 8 PM to 12 AM. Availability and accessibility of data in many hospitals is a major concern. The hospital records in its logbook details of patients such as name, age, gender, time of arrival, patient MRN, doctor’s name, chief complaints and treatment given. Patient arrivals in ED for a period of three months from September to November was shared by the hospital consisting of a total of 7748 arrivals across all the three shifts. Data was manually recorded in a spreadsheet. The number of arrivals shift-wise was recorded for the above three-month period. The objective of the research is to understand the patient flow through analysis of various descriptive metrics and further apply time series model to generate accurate short-term forecasts of patient volumes at the ED, which can enable the hospital management for better planning. The key limitation of the study is that patient arrivals only during three months from historical data was collected due to data confidentiality and the time period of the data which the hospital was willing to share. Further, data was recorded by the authors from the hospital’s log book by recording on a spreadsheet. For an empirical investigation, the sample evidence suffices.

4 Data Analysis 4.1 Descriptive Analysis Descriptive analysis of patient arrivals in the ED (Fig. 1) and descriptive statistics of patient arrivals in all the three shifts (Table 1) is summarised. Fig. 1 Shift-wise total ED patient arrivals

2500 2000

2042

2131 1694

1500 1000 500 0

164

505

167

479

Sep

Oct

Shi1

Shi2

158

408

Nov Shi3

Time Series Modelling and Forecasting of Patient … Table 1 Descriptive statistics of patient arrivals

57

Summary measure

Shift 1

Shift 2

Shift 3

Mean Median

5

64

15

5

65

15

Mode

4

66

15

Std. deviation

2.54

12.07

4.51

Range

18

65

28

Minimum

1

34

0

Maximum

19

99

28

Sum

489

5867

1392

4.2 Predictive Analytics: Modelling Patient Arrivals for Operational Shift One from 12 AM to 8 AM Description and Initial Analysis In this section, predictive analytics is applied to model patient arrivals and also forecast the same for each of the working shifts. A general trend of large volume of patient arrivals in the second shift which spans between 8 AM and 8 PM is seen since this is the longest shift occurring during the day. As every shift has specific resources and arrivals are likely to be shift or time-dependent, it makes sense to model patient arrivals for each shift category. R-Studio has been used for analysis. A time series is analysed by understanding its components-level, trend, seasonality and random fluctuations or the error component. From the time series plot of arrivals in Shift 1 (Fig. 2), it is evident that there is no steady increase or decrease pattern, showing that the series does not have a trend. Forecasting methods in time series require that the series be stationary [25, 26] to make future predictions since the condition of stationarity means that the properties of the series do not change over time. The ADF test is run to confirm the stationarity condition of time series data. The ADF tests the null hypothesis that the series is nonstationary [25]. The ADF test run using R for arrivals in Shift 1 gives the test statistic value as −4.891 and p value is 0.01 ( 0, without reusing previously selected features penalty. It has been learned from an ensemble of regression trees. This method deals with non-leave our interaction between features. It is a random forest algorithm that is used as a feature selection. It unites merged both processes into an optimized FS method. The method consists of- O(nd) complexity; here, d- depicts the number of- features in dimensionality, and n depicts the number of feature points. It has pre-specific feature cost structures as reference information. The selected feature focuses on a specific region of interest and it works similarly to lasso liner methods. It has measured inter-feature dependencies. The incorporated feature competed differed features with the accuracy of a trained bag of features. It is a non-learn feature selection. It has allowed bias in the probability of sampling by diving two pants of the necessary and unnecessary part. Although this method depends on the number of elements of the sample with A if dimension is too large, then there will be a problem of cost computation. Random forest builds a tree structure  √ and requiring the time complexity of O dn log n − for each tree. Particle Swarm Optimization (PSO) The performance of the classifier has been improved by selecting the proper feature subset. The test relevant features are suitable for the classification task. The binary PSO works as in the position of the particles which are expressed into two terms [25]. Generally, this binary approach deals with 0 and 1. Here, x is a particle on d-dimensional data, Eq. 1 shows mathematical derivation of binary PSO. D = [x1 , x2 , . . . , xd ] where, Yi→d = [y1 , y2 , y3 , . . . , yd ] here, yi ∈ 0, 1   nf , f (Y ) = α[(1 − β) + (1 + α)] 1 − nt Here, α = Hyperparameter

(1)

The method worked on each feature in the dimension of each particle. The classifier drops features of irrelevant contain by hyperparameter tuning. The parameter could be hyperparameter trade-off between the performances of classifier. The model was set as several neighbors are equal to the number of particles; the hyperparameter was locally set arbitrarily. The bitwise parameter metrics are evaluated and optimal metrics of bits are selected. In this way, binary PSO utilizes a bit string of O and 1’s bits and check its neighbors.

130

A. Maiti and B. Chatterjee

Mutual Information Mutual information has been applied to observe the behavior of random variables. The random variables can interact with each other and it has worked as a dimensionless manner. So, if I, J (mutual information of x and y) is high, then reduction of uncertainty is high, y is low then there can be a small reduction [26]. If mixy is zero, then both the variables are independent. The formula of mutual information has been derived in Eq. 2.

M Ii, j



pi, j (i, j) pi (i) p( j) pi, j (i, j) pi, j did j = βpi, j log = ∫i ∫ j pi, j log pi (i) p( j) pi p j

M Ii, j =

pi, j (i, j) log

(2)

There has been the distribution of each variable and replace sum of variables with integral between them. It has divided two sets of random variables based on the reduction of entropy. It checks information gain and tried to minimize the entropy by splitting the data set. When it comes to choosing the best feature, which should be primarily based on the distribution of characteristics of probability. It has although it has more error variance than other models.

4 Results and Discussion The performance analysis for all features of texture and shapes has been done through different FS methods and classifiers. The CAD system has included features that are subjective to the skin lesion classification. The features set contained irrelevant features that are removed through different feature selection methods. Table 1 shows the different features selected by the feature section’s different methods, and Table 2 shows the results obtained by the feature section method by each sub-set feature. The features are segregated according to the feature selection method applied to them. The following table shows the selected set of features as per its highest importance in the dataset. The MI method has been arranged for the top 20% of important features. The investigation of binary PSO has also selected the best possible features according to interaction on each mutation. The GB method has identified 0.9 co-linear as per threshold. It has removed the features which have less than this Table 1 Shape and texture features selected by different feature selection methods Method

Shape features

GB

1, 2, 5, 7, 11, 12

MI PSO

Texture features 1, 2, 4, 8, 11, 13

Shape + texture features S-5, 10, 12 T-1, 2, 8, 10

4, 8, 10

3, 14, 15, 17, 18

S-1, 3, 4, 5, 8, 10, 12

2, 4, 11, 12

1, 2, 6, 7, 10, 15, 16, 17, 20, 21

S-11, 12 T-2, 3, 4, 6, 7, 10, 21

The Effect of Different Feature Selection Methods … Table 2 Accuracy of the classifiers for each subset of features selected by different feature selection methods

131 PSO

MI

GB

Random forest

74.1

87.2

97.1

Decision tree

72.3

85.5

96.8

K-neighbors

73.1

82.0

93.3

Fig. 3 Results obtained from classifiers for different subset of features by FS methods

threshold. The final features sets have been balanced by merging both texture and shape features. The classification on feature sets of each classifier of KNN, DT, and RF appears relevant. The RF has the highest performance based on accuracy as it got the highest accuracy of 97.1 (%) on GB feature sets. The hybrid feature sets of texture and shape features are also evaluated through these classifiers which have been displayed in the table. The model selection justifies the endurance of the different feature selection-based CAD system when a balanced feature set of texture and shape features are practiced. Figure 3 demonstrates results achieved by FS methods from classifiers for a specific subset of features.

5 Conclusion This research presented a thorough review of the 22 texture and 12 shape features of melanoma and Naevus classes. The features have been processed under relevant image pre-processing techniques and segmentation like active conture model. The typical approach of hair removal through black hat hair removal has reduced the chances of getting noisy features. The performance analysis for all features of texture and shapes has been done through different FS methods and classifiers. The CAD

132

A. Maiti and B. Chatterjee

system has included features that are subjective to the skin lesion classification. The features set contained irrelevant features that are removed through different feature selection methods. The RF has the highest performance based on accuracy as it got the highest accuracy of 97.1% on GB feature sets. DT and KNN have shown decent accuracy of 96.8 and 93.3% on GB feature sets. Researchers hope that this research will be a guide for the selection of texture and shape features for CAD systems in advance studies. Finally, the investigation has reviewed different FS techniques on skin cancer detection and tested accordingly with different popular machine learning models. The CAD system has more focused on features and selection of features.

References 1. Benco, M., Kamencay, P., Radilova, M., Hudec, R., Sinko, M.: The comparison of color texture features extraction based on 1D GLCM with deep learning methods. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP). pp. 285–289 (2020) 2. Tiwari, P., Prasanna, P., Rogers, L., Wolansky, L., Badve, C., Sloan, A., Cohen, M., Madabhushi, A.: Texture descriptors to distinguish radiation necrosis from recurrent brain tumors on multiparametric MRI. In: Medical Imaging 2014: Computer-Aided Diagnosis, vol. 9035, p. 90352 (2014) 3. Filali, Y., Sabri, M.A., Aarab, A.: Improving Skin Cancer Classification Based on Features Fusion and Selection (2020) 4. Afza, F., Khan, M.A., Sharif, M., Rehman, A.: Microscopic skin laceration segmentation and classification: a framework of statistical normal distribution and optimal feature selection. Microsc. Res. Tech. 82(9), 1471–1488 (2019) 5. Tapia, D.I., Corchado, J.M.: An ambient intelligence based multi-agent system for alzheimer health care. Int. J. Ambient. Comput. Intell. (IJACI) 1(1), 15–26 (2009) 6. Gaonkar, R., Singh, K., Prashanth, G.R., Kuppili, V.: Lesion analysis towards melanoma detection using soft computing techniques. Clin. Epidemiol. Glob. Health 8(2), 501–508 (2020) 7. Rehman, A., Khan, M.A., Mehmood, Z., Saba, T., Sardaraz, M., Rashid, M.: Microscopic melanoma detection and classification: a framework of pixel-based fusion and multilevel features reduction. Microsc. Res. Tech. 83(4), 410–423 (2020) 8. Matallah, H., Belalem, G., Bouamrane, K.: Towards a new model of storage and access to data in big data and cloud computing. Int. J. Ambient. Comput. Intell. (IJACI) 8(4), 31–44 (2017) 9. Dey, N., Ashour, A.S.: Ambient Intelligence in healthcare: a state-of-the-art. Glob. J. Comput. Sci. Technol. (2017) 10. Damian, F.A., Moldovanu, S., Dey, N., Ashour, A.S., Moraru, L.: Feature selection of nondermoscopic skin lesion images for nevus and melanoma classification. Computation 8(2), 41 (2020) 11. Reshma, M., Shan, B.P.: A clinical decision support system for micro panoramic melanoma detection and grading using soft computing technique. Measurement 163, 108024 (2020) 12. Ain, Q.U., Al-Sahaf, H., Xue, B., Zhang, M.: Generating knowledge-guided discriminative features using genetic programming for melanoma detection. In: IEEE Transactions on Emerging Topics in Computational Intelligence (2020) 13. Akram, T., Lodhi, H.M.J., Naqvi, S.R., Naeem, S., Alhaisoni, M., Ali, M., Haider, S.A., Qadri, N.N.: A multilevel features selection framework for skin lesion classification. Human-centric Comput. Inform. Sci. 10, 1–26 (2020) 14. Adjed, F., Gardezi, S.J.S., Ababsa, F., Faye, I., Dass, S.C.: Fusion of structural and textural features for melanoma recognition. IET Comput. Vis. 12(2), 185–195 (2018). https://dx.doi. org/10.1049/iet-cvi.2017.0193

The Effect of Different Feature Selection Methods …

133

15. Hagerty, J.R., Stanley, R.J., Almubarak, H.A., Lama, N., Kasmi, R., Guo, P., Drugge, R.J., Rabinovitz, H.S., Oliviero, M., Stoecker, W.V.: Deep learning and handcrafted method fusion: higher diagnostic accuracy for melanoma dermoscopy images. IEEE J. Biomed. Health Inform. 23(4), 1385–1391 (2019) 16. Li, X., Wu, J., Jiang, H., Chen, E.Z., Dong, X., Rong, R.: Skin lesion classification via combining deep learning features and clinical criteria representations. bioRxiv, 382010 (2018) 17. Abbas, Q., Celebi, M.E.: DermoDeep—a classification of melanoma-nevus skin lesions using multi-feature fusion of visual features and deep neural network. Multimed. Tools Appl. 78(16), 23559–23580 (2019) 18. Tan, T.Y., Zhang, L., Lim, C.P.: Adaptive melanoma diagnosis using evolving clustering, ensemble and deep neural networks. Knowl. Based Syst. 187, 104807 (2020) 19. Filali, Y., Khoukhi, H.E., Sabri, M.A., Aarab, A.: Efficient fusion of handcrafted and pretrained CNNs features to classify melanoma skin cancer. Multimed. Tools Appl. 1–20 (2020) 20. Kassem, M.A., Hosny, K.M., Fouad, M.M.: Skin lesions classification into eight classes for ISIC 2019 using deep convolutional neural network and transfer learning. IEEE Access 8, 114822–114832 (2020) 21. Belarbi, M.A., Mahmoudi, S.A., Mahmoudi, S., Belalem, G.: A new parallel and distributed approach for large scale images retrieval. In: International Conference of Cloud Computing Technologies and Applications. pp. 185–201. Springer, Cham (2017) 22. Khamparia, A., Singh, P.K., Rani, P., Samanta, D., Khanna, A., Bhushan, B.: An internet of health things-driven deep learning framework for detection and classification of skin cancer using transfer learning. Trans. Emerg. Telecommun. Technol. e3963 (2020) 23. Satheesha, T.Y.: Classification of Skin Lesion Using (Segmentation) Shape Feature Detection (2020) 24. Sheridan, R.P., Wang, M., Liaw, A., Ma, J., Gifford, E.: Correction to extreme gradient boosting as a method for quantitative structure—activity relationships. J. Chem. Inf. Model. 60(3), 1910–1910 (2020) 25. Rostami, M., Forouzandeh, S., Berahmand, K., Soltani, M.: Integration of multi-objective PSO based feature selection and node centrality for medical datasets. Genomics (2020) 26. Tao, G., Liu, Z., Cao, J., Liang, S.: Local difference ternary sequences descriptor based on unsupervised min redundancy mutual information feature selection. Multidimens. Syst. Signal Process. 31(3), 771–791 (2020)

Intelligent Hybrid Technique to Secure Bluetooth Communications Alaa Ahmed Abbood, Qahtan Makki Shallal, and Haider Khalaf Jabbar

Abstract E0 algorithm is the most popular which used for data transmission Bluetooth communication among devices. E0 is having a 128-bit of symmetric stream cipher key length. Many types of attacks at Bluetooth protocol and cryptanalysis of E0 has proved that it would be broken by using 264 operations. In this work, we have proposed hybrid encryption based on blowfish and md5 algorithms to im-prove the security of transferring data between two computers connected using Bluetooth technique. Because of the advantages of key management of the MD5 algorithm, we used it to encrypt the secret key of Blowfish algorithm which used for encryption of plaintext. Therefore, the proposed hybrid encryption (Blowfish and MD5) will positively improve the data security during communication in Bluetooth media. Keywords Bluetooth · Encryption algorithms · E0 algorithm · MD5 algorithm · Blowfish

1 Introduction Bluetooth is a technology of wireless link that provides high-speed data transferability. Bluetooth technology depends on the on-chip that delivers a wireless link in order to connect devices together [1]. The Bluetooth technology is used for swapping the data over short distances between electronic devices, such as mobile, laptops, and so on. The Bluetooth technology could be work in the range of radio frequency

A. A. Abbood Faculty Of Business Informatics, University Of Information Technology and Communications, Baghdad, Iraq e-mail: [email protected] Q. M. Shallal (B) Management Technical College of Basra, Southern Technical University, Basra, Iraq e-mail: [email protected] H. K. Jabbar Imam Jafar Al-Sadiq University Mysan, Information Technology College, Mysan, Iraq e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 S. Bhattacharyya et al. (eds.), Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing 1333, https://doi.org/10.1007/978-981-33-6966-5_14

135

136

A. A. Abbood et al.

between 2.40 GHz to 2.480 GHz. The actual range of Bluetooth devices is approximately 10 m. The data transfer rate in Bluetooth technology is 1 Mbps [2]. The technology of Bluetooth is allowed two digital devices to be connected wirelessly while they are not far away from each other. Further, to connect cell phones, Bluetooth technology can also be suitable for personal computers, smartphones, digital cameras, laptops, GPS receivers, printers and many more. Due to wireless technology, the Bluetooth is exposed to attack, remote access and spying. It presents a range of major security weaknesses. Data-leaking often comes up along with Bluetooth technology which could result in compromising the networks and devices. Thus, the issues of security are the most important in the technology of Bluetooth [3–5].

2 Popular Security Issues of Bluetooth In this part, a discussion about the Bluetooth hacking types will be conducted. Therefore, this will explain how real the security issue in Bluetooth devices. Bluetooth hacks are mainly categorized to [6]: • Blue Jacking: In this method, the hacker will send business cards or contact numbers to nearby mobile. The contact’s ‘name’ field of the contact can be easily replaced with a suggestive text, so the destination will read it in the context of intimation query which been shown on its screen. • Bluesnarfing: In this method, the hacker is able to steal or accesses data like phone-book, calendar, messages …etc., from the destination device in an illegal way bypassing the requirement of usual paring. At this point, the problem issue is greater since there will always be tools reports that use different approaches which can include brute force and device address guessing as a way to a break-in, no matter if the device configuration is ‘invisible’. • Blue bugging: In which the attacker controls the device of the victim, the attacker sends commands to execute the actions which typically get the physical access to Bluetooth device this is an analogous functionality to Trojans. • Bluetoothing: it indicates social networking in terms of short-range, and there may be a chance of harassment due to the security perspective. There will be programmers to obtain cracking of Bluetooth PIN code. E0 stream cipher algorithm is used for encrypting transferred data in Bluetooth technology. But unfortunately, this algorithm had many drawbacks, because of E0 stream cipher is having 128-bit, it could possibly be hacked using some methods with the complexity of time O(264 ). Therefore, the security of data in the traditional algorithm of E0 is insufficient for most applications that required a high level of priority to confidentiality.

Intelligent Hybrid Technique to Secure Bluetooth Communications

137

3 Authentication and Encryption Used in Bluetooth Media 3.1 Encryption Process The system of Bluetooth encryption is encrypting the data. This process will be done using E0 stream cipher, that is certainly resynchronized for just about any type of data. The E0 divides into two parts: generator of key stream and the encryption/decryption. The data key generator integrates the input bits within desirable order and then shifts them to the four Linear Feedback Shift Registers (LFSR) of the key stream generator [7, 8]. The data key generator integrates the bits which are used as input in an effective order and will shift them according to whether the device is using the master key or semi-permanent link key; there are many encryption modes are exist. In case of using a combination key or unit key, broadcast traffic will not be encrypted. The data traffic Individually might be encrypted or not. In the case of using the master key, there will be three encryption faces. In the first encryption face, no encryption action will be accomplished. In the second encryption face, broadcast traffic would not be encrypted, but the individual traffic is encrypted using the master key. In the third encryption face, all of the traffic is encrypted using the master key [7, 9]. while the size of the encryption key is quite different from 8 up to 128 bits, then the encryption key’s size utilized among two devices need to be negotiated. In every single device, there will be a parameter explaining the maximum length of the key allowed. In the negotiation of key size, the master will send its idea of key length to the slave device. The slave device may either receive and admit it, or ignore it and send another idea [7, 10]. This will be continuously running until both sender and slave are agreed or one of them terminates the negotiation. The negotiation terminate is done by using the application. In every single application, there exists a minimum key length specified, and in case if the significant requirement does not really meet one of the participant’s requirements, the application will ignore the negotiation, hence the encryption will never be accomplished. This is essential to prevent the specific circumstances in which the malicious device is forcing the encryption process to become low to do harm on the device [11]. Figure 1 shows the stream cipher e0 algorithm.

3.2 Authentication Process The scheme of Bluetooth authentication is using a strategy of challenge-response, in which two protocols are utilized to verify if the closed node aware of the secret key. In fact, the symmetric keys are used in the protocol, so the positive sign for the authentication process will be accomplished if both nodes share exactly the same key. Calculation of Authenticated Ciphering Offset (ACO) will be accomplished, and that will be stored in all nodes, ACO is employed to generating the cipher key afterward. Moreover, master node sends a random number to the slave node in order

138

A. A. Abbood et al.

RAN

Addre

Cloc

K

Payload Key Generator Payload Key

Key Stream Generator Z Plaintext/ Ciphertext

Cipherte xt/Plaintext Fig. 1 The stream cipher e0 algorithm

Verifier

Claimant AU_RANDA

AU_RANDA BD_ADDRB

BD_ADDRB

E1

Link Key

SRES

AU_RANDA

SRES

ACO

E1

SRES

Link Key

ACO

Fig. 2 Bluetooth Challenge response [7]

to get authenticated [12]. Thereafter, both nodes use E1 which is an authentication function along with that random number, the slave nodes Bluetooth Device Address as well as the present link key to acquire the reply. The slave node delivers the reply to the master node, who will make sure that the response is matched. The application used is referred to as who will be authenticated. A few numbers of applications have the need for one-way authentication only, so there is a need for authenticating

Intelligent Hybrid Technique to Secure Bluetooth Communications Fig. 3 The Encryption Process in Bluetooth [15]

PIN

E2

139

E2 Authentication Process LINK KEY

LINK KEY

E3

E3 Encryption

Encryption Key

Encryption Key

one node only. This case is not always the same, as there is frequently a mutual authentication, in which the two nodes are authenticated consequently. In case the process of authentication is unable, there will be another time to retry the second attempt of authentication. The time frame doubles for any subsequent not succeed attempt coming from the identical node address, up to the time when it reached to highest possible waiting time [7, 11, 13]. The value of waiting time will be decreased exponentially to the lowest value when no any failed in authentication attempts [7] (Fig. 2). The mechanism of Bluetooth security is providing encryption, authentication, and functions of key management in the Link layer [4]. It uses the algorithms of E0, E1, E2, and E3. 4-bit PIN inserted by the user creates Link key by using the E2 algorithm that’s is then will be used by the E3 algorithm to create the key of encryption. Then the stream’s key generated by the E0 algorithm and the encryption key will be used to produce ciphertext by encrypting the plaintext. The process of encryption in Bluetooth is explained by Fig. 3 [7, 14].

4 The Framework Details of Proposed Hybrid Technique In this work, we combine both blowfish and MD5 algorithms, which will upgrade the security in Bluetooth communication.

4.1 Blowfish Algorithm It is 64-bit block, it has a different size of key starting from 32 to 448 bits. There are two procedures are considered in the Blowfish algorithm which can be consisted

140

A. A. Abbood et al.

of key initialization and the phase used to data decryption [16]. The variable key of the user is consumed in the first phase to the arrays of sub key which is 4168/8336byte, that’s presented with 4-byte or 8-byte of arrays size. The arrays of sub-key (P entry-18 and four S arrays entry-256) the generation actions is depending on the key of user. There is an improvement in security strength considering the difficulty of sub-keys as well as user key relation [16, 17]. Algorithm 1 Blowfish Algorithm

Encryption Process: Step no.1: Input 64-bits of plaintext, let say X. Step no.2: divide the text into two parts, each is 32-bit: XLe and XRi. Step no.3: For I =1 to 16 XLe =XLe

PI

XRi =F (XLe) XRi Swap XLe and XRi I= I+1 end for Swap XLe and XRi one more time (in which, final swap undo) XRi = XRi

P17

XLe = XLe P18 Combine both XLe and XRi Step no.4: Calculate F Function: chopped the XLe into 4 groups, each having eight-bit quarters: W, X, Y, and Z F(XLe) = ((S1, W+ S2, X mod 232)

S3, Y) + S4, Z mod 232

Decryption Process: The decryption process is similar encryption except using inverse arrangement of P1, P2……….P18. Generate the Key: Step no.1: initialize both arrays of S boxes and P. Step no.2:

P arrays with key bits (i.e., P1

the first 32 bits of key, and P2

the second 32 bits of key…..etc.). Step no.3: using above method, encrypt all zero strings. Step no.4: both (P1, P2) are having fresh output. Step no.5: using sub-keys (modified), encrypt the new produced P1 and P2. Step no.6: a new output is generated (P3 and P4). Step no.7: repeating the same process 521 times to calculate the new P-array as well as 4 S-boxes.

Intelligent Hybrid Technique to Secure Bluetooth Communications

141

4.2 MD5 Hashing Algorithm It is Message-Digest algorithm, the data will split up to numerous blocks, each block will contain 512-bits, where each one will have a sixteen 32-bit of sub-blocks. Once the procedure completed, MD5 will generate128-bits information message by four linked 32-bit hinders for the document integrity [18]. Algorithm 2 MD5 Algorithm [19]

Step no.1: Check the number of Input bits. Step no.2: Append some bits to input message (IM), as the data bits is equivalent to a multiple of 512, the addition bits are 0 1 0….0) Step no.3: Append 64 bits to the output of previous step (IM), and then the obtained output is considered as ms. Step no.4: ms to bs(blocks), each divided to 512 bits. Step no.5: bs(blocks) to 16 blocks (x), where each x has 32bits. Step no.6: There are 4 rounds in the algorithm, where each round have 16 steps, so the total steps are 64. Step no.7: Four group of 32 bits that have shift register (hex.) value are described as: register a1= [7 6 5 4 3 2 1 0] which is 32- bits[a1]=[a4]. register a2= [f e d c 8 a 9 7] which is 32- bits[a2]=[a3]. register a3= [8 9 a b c d e f ] which is 32- bits [a3]=[a4]. register a4= [0 1 2 3 4 5 6 7] which is 32-bits [a4]=[a1]. Step no.8: Store the value of a1, a2, a3, and a4temporarily in aa, ab, ac, and ad respectively. Step no.9: The processing of algorithm is including the four rounds with several functions, which are r1, r2, r3, and r4. The operation of Single step function is shown below: a1 = a2 + (( a1 + r1(a2, a3, a4) + xi[w] + t[r4]