Challenges and Applications for Implementing Machine Learning in Computer Vision 2019018175, 9781799801825, 9781799801832, 9781799801849, 9781522590231, 9781522595786, 9781522585312, 9781522582953

Citation preview

Challenges and Applications for Implementing Machine Learning in Computer Vision Ramgopal Kashyap Amity University, Raipur, India A.V. Senthil Kumar Hindusthan College of Arts and Science, India

A volume in the Advances in Computer and Electrical Engineering (ACEE) Book Series

Published in the United States of America by IGI Global Engineering Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA, USA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2020 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Names: Kashyap, Ramgopal, 1984- editor. | Kumar, A. V. Senthil, 1966editor. Title: Challenges and applications for implementing machine learning in computer vision / Ramgopal Kashyap and A.V. Senthil Kumar, editors. Description: Hershey, PA : Engineering Science Reference, an imprint of IGI Global, [2020] | Includes bibliographical references. | Summary: “This book examines the latest advances and trends in computer vision and machine learning algorithms for various applications”-- Provided by publisher. Identifiers: LCCN 2019018175 | ISBN 9781799801825 (hardcover) | ISBN 9781799801832 (paperback) | ISBN 9781799801849 (ebook) Subjects: LCSH: Computer vision. | Machine learning. Classification: LCC TA1634 .C44 2020 | DDC 006.3/7--dc23 LC record available at https://lccn.loc.gov/2019018175 This book is published in the IGI Global book series Advances in Computer and Electrical Engineering (ACEE) (ISSN: 2327-039X; eISSN: 2327-0403) British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. For electronic access to this publication, please contact: [email protected].

Advances in Computer and Electrical Engineering (ACEE) Book Series Srikanta Patnaik SOA University, India

ISSN:2327-039X EISSN:2327-0403 Mission

The fields of computer engineering and electrical engineering encompass a broad range of interdisciplinary topics allowing for expansive research developments across multiple fields. Research in these areas continues to develop and become increasingly important as computer and electrical systems have become an integral part of everyday life. The Advances in Computer and Electrical Engineering (ACEE) Book Series aims to publish research on diverse topics pertaining to computer engineering and electrical engineering. ACEE encourages scholarly discourse on the latest applications, tools, and methodologies being implemented in the field for the design and development of computer and electrical systems. Coverage • Programming • Circuit Analysis • Computer Architecture • Applied Electromagnetics • Digital Electronics • Computer Hardware • VLSI Fabrication • Algorithms • Power Electronics • Analog Electronics

IGI Global is currently accepting manuscripts for publication within this series. To submit a proposal for a volume in this series, please contact our Acquisition Editors at [email protected] or visit: http://www.igi-global.com/publish/.

The Advances in Computer and Electrical Engineering (ACEE) Book Series (ISSN 2327-039X) is published by IGI Global, 701 E. Chocolate Avenue, Hershey, PA 17033-1240, USA, www.igi-global.com. This series is composed of titles available for purchase individually; each title is edited to be contextually exclusive from any other title within the series. For pricing and ordering information please visit http://www.igi-global.com/book-series/advances-computer-electricalengineering/73675. Postmaster: Send all address changes to above address. Copyright © 2020 IGI Global. All rights, including translation in other languages reserved by the publisher. No part of this series may be reproduced or used in any form or by any means – graphics, electronic, or mechanical, including photocopying, recording, taping, or information and retrieval systems – without written permission from the publisher, except for non commercial, educational use, including classroom teaching purposes. The views expressed in this series are those of the authors, but not necessarily of IGI Global.

Titles in this Series

For a list of additional titles in this series, please visit: https://www.igi-global.com/book-series/advances-computer-electrical-engineering/73675

Novel Practices and Trends in Grid and Cloud Computing Pethuru Raj (Reliance Jio Infocomm Ltd. (RJIL), India) and S. Koteeswaran (Vel Tech, India) Engineering Science Reference • © 2019 • 374pp • H/C (ISBN: 9781522590231) • US $255.00 Blockchain Technology for Global Social Change Jane Thomason (University College London, UK) Sonja Bernhardt (ThoughtWare, Australia) Tia Kansara (Replenish Earth Ltd, UK) and Nichola Cooper (Blockchain Quantum Impact, Australia) Engineering Science Reference • © 2019 • 243pp • H/C (ISBN: 9781522595786) • US $195.00 Contemporary Developments in High-Frequency Photonic Devices Siddhartha Bhattacharyya (RCC Institute of Information Technology, India) Pampa Debnath (RCC Institute of Information Technology, India) Arpan Deyasi (RCC Institute of Information Technology, India) and Nilanjan Dey (Techno India College of Technology, India) Engineering Science Reference • © 2019 • 369pp • H/C (ISBN: 9781522585312) • US $225.00 Applying Integration Techniques and Methods in Distributed Systems and Technologies Gabor Kecskemeti (Liverpool John Moores University, UK) Engineering Science Reference • © 2019 • 351pp • H/C (ISBN: 9781522582953) • US $245.00 Handbook of Research on Cloud Computing and Big Data Applications in IoT B. B. Gupta (National Institute of Technology Kurukshetra, India) and Dharma P. Agrawal (University of Cincinnati, USA) Engineering Science Reference • © 2019 • 609pp • H/C (ISBN: 9781522584070) • US $295.00

701 East Chocolate Avenue, Hershey, PA 17033, USA Tel: 717-533-8845 x100 • Fax: 717-533-8661 E-Mail: [email protected] • www.igi-global.com

xv

Preface

Participation is prime significant for both the educator and understudy of an instructive association. So it is imperative to keep a record of the involvement. The issue emerges when we consider the conventional procedure of gauging participation in study hall. Calling name or move number of the understudy isn’t just an issue of time utilization yet; also, it needs vitality. So a programmed participation framework can tackle every single above problem. There are some programmed attendances making framework which are as of now utilised by much organisation. One such structure is a biometric procedure. Even though it is processed and a stage in front of customary strategy, it neglects to meet the time requirement. The understudy needs to sit tight in line for giving participation, which is time taking. This task presents an automatic participation stamping framework, without any obstruction with the ordinary instructing strategy. The structure can be additionally actualised during test sessions or in other instructing exercises where participation is profoundly essential. This framework disposes of old-style understudy recognisable proof, for example, calling the name of the understudy, or checking individual ID cards of the understudy, which can meddle with the progressing showing process, yet additionally can be distressing for understudies during assessment sessions. The objective of computational neuroscience is to discover unthinking clarifications of how the sensory system forms data to offer ascent to intellectual capacity and conduct. At the core of the field are its models, for example, numerical and computational depictions of the framework examined, which guide tangible improvements to neural reactions as well as neural to social responses. These models run from easy to complex. As of late, deep neural systems (DNNs) have come to rule a few spaces of Artificial Intelligence (AI). Current DNNs disregard numerous subtleties of organic neural systems. These improvements add to their computational proficiency, empowering them to perform complex accomplishments of knowledge, extending from perceptual (for example visual article and sound-related discourse acknowledgement) to individual assignments (for example machine interpretation), and on to engine control (for instance, playing computer games or controlling a robot arm). Notwithstanding their capacity to demonstrate sophisticated shrewd practices,

Preface

DNNs exceed expectations at anticipating neural reactions to novel tactile boosts with exactnesses well past some other as of now accessible model sort. DNNs can have a considerable number of parameters, which are required to catch the space information required for fruitful undertaking execution. In spite of the instinct that this renders them into impervious secret elements, the computational properties of the system units are the aftereffect of four straightforwardly manipulable components: input measurements, arrange structure, useful target, and learning calculation. With full access to the action and availability everything being equal, propelled perception systems, and logical instruments for organizing portrayals to neural information, DNNs speak to a fantastic structure for structure task-performing models and will drive significant bits of knowledge in computational neuroscience. Restorative Imaging Techniques are non-intrusive strategies for peering inside the body without opening up the shape precisely. It used to help determination or treatment of various ailments. There are numerous therapeutic imaging methods; each system has multiple dangers and advantages. This paper introduces an audit of these methods; ideas, favourable circumstances, disservices, and applications. The concerning systems are; X-ray radiography, X-ray Computed Tomography (CT), Magnetic Resonance Imaging (MRI), ultrasonography, Elastography, optical imaging, Radionuclide imaging incorporates (Scintigraphy, Positron Emission Tomography (PET) and Single Photon Emission Computed Tomography (SPECT)), thermography, and Terahertz imaging. The ideas, advantages, dangers and uses of these systems will give subtleties. An examination between these strategies from perspective, picture quality (spatial goals and complexity), security (impact of ionising radiation, and warming effect of pollution on the body), and framework accessibility (continuous data and cost) will exhibit. The ascent of human-made brainpower as of late is grounded in the achievement of profound learning. Three noteworthy drivers caused the performance of (intelligent) neural systems: the accessibility of immense measures of preparing information, incredible computational foundation, and advances in the scholarly community. In this way, profound learning frameworks begin to beat old-style techniques, yet also personal benchmarks in different assignments like picture arrangement or face acknowledgement. It makes the potential for some, problematic new organisations utilising profound figuring out how to take care of genuine issues. Computer vision innovation is exceptionally flexible and can adjust to numerous enterprises in altogether different ways. Some utilisation cases off-camera, while others are increasingly obvious. No doubt, you have officially utilised items or administrations improved by computer vision. Tesla has finished the absolute most popular uses of computer vision with their Autopilot work. The automaker propelled its driver-help framework in 2014 with just a couple of highlights, for example, path focusing and self-cleaving. However it’s set to achieve completely self-driving autos xvi

Preface

at some point in 2018. Highlights like Tesla’s Autopilot are conceivable gratitude to new businesses, for example, Mighty AI, which offers a stage to produce exact and different comments on the datasets to prepare, approve, and test calculations identified with independent vehicles. Computer vision has made a sprinkle in the retail business also. Amazon Go store opened up its ways to clients on January 22 this year. It’s a mostly mechanised store that has no checkout stations or clerks. By using computer vision, profound learning, and sensor combination clients can necessarily leave their preferred store with results and get charged for their buys through their Amazon account. The innovation isn’t 100% impeccable yet, as a few authority trials of the store’s change demonstrated that a few things were let alone for the last bill. Notwithstanding, it’s a fantastic positive development. While computers won’t supplant medicinal services workforce, there is a decent probability of supplementing routine diagnostics that require a great deal of time and skill of human doctors yet don’t contribute altogether to the last conclusion. Like these computers fill in as a helping apparatus for the social insurance workforce. For instance, Gauss Surgical is creating a continuous blood screen that takes care of the issue of wrong blood misfortune estimation during wounds and medical procedures. The screen accompanies a straightforward application that uses a calculation that investigations images of careful wipes to precisely foresee how much blood lost during a therapeutic process. This innovation can spare around $10 billion in unnecessary blood transfusions consistently. One of the primary difficulties the medicinal services framework is encountering is the measure of information that is being delivered by patients. Today, we as patients depend on the information bank of the restorative workforce to break down every one of that information and produce the right analysis. It can be troublesome on occasion. Microsoft’s venture InnerEye is chipping away at tackling portions of that issue by building up an instrument that utilizations AI to investigate three-dimensional radiological images. The innovation possibly can make the procedure multiple times snappier and recommend the best medications. As shown above, computer vision has made some fantastic progress as far as what it can accomplish for various ventures. Be that as it may, this field is still moderately youthful and inclined to difficulties. One unique angle that is by all accounts the foundation for the vast majority of the problems is the way that computer vision is as yet not similar to the human visual framework, which is the thing that it attempts to emulate. Computer vision calculations can be very weak. A computer can perform undertakings it was prepared to execute and misses the mark when acquainted with new errands that require an alternate arrangement of information — for instance, encouraging a computer what an idea is hard yet it essential with the end goal for it to learn independently from anyone else. A genuine model is the idea of a book. As children, we realise what a book is and inevitably can recognise a book, a magazine or a comic while understanding that they have a place with a xvii

Preface

similar generally speaking classification of things. For a computer, that learning is significantly more troublesome. The issue is raised further when we add digital books and book recordings to the condition. As people, we comprehend that each one of those things falls under a similar idea of a book, while for a computer the parameters of writing and a book recording are too unique even to consider being put into related gatherings of things. To beat such deterrents and capacity ideally, computer vision calculations today require social inclusion. Information researchers need to pick the right design for the information type with the goal that the system can naturally learn highlights. Engineering that isn’t ideal may deliver results that have no incentive for the venture. At times, a yield of a computer vision calculation can upgrade with different sorts of information, for example, sound and content, to deliver exceedingly exact outcomes. As such, computer vision still does not have the abnormal state of precision that is required to work in a whole, a different world ideally. As the advancement of this innovation is still in progress, much resilience for mix-ups needed from the information science groups taking a shot at it. Neural systems utilised for computer vision applications are more straightforward to prepare than any time in recent memory yet that requires a great deal of excellent information. It implies the calculations need a ton of information that explicitly identified with the undertaking to create unique outcomes. Regardless of the way that images are accessible online in more significant amounts than any time in recent memory, the answer for some real issues calls for brilliant marked preparing information. That can get rather costly because an individual must finish the naming. How about we take the case of Microsoft’s undertaking InnerEye, a device uses computer vision to dissect radiological images. The calculation behind this undoubtedly requires well-commented on models where several physical oddities of the human body are named. Given that around 4-5 images can dissect every hour, and a satisfactory informational collection could contain a considerable number of them, appropriate marking of images can get over the top expensive. Information researchers now and again use pre-prepared neural systems that initially prepared on a massive number of images as a base model. Without great information, it’s a sufficient method to show signs of improvement results. Be that as it may, the calculations can find out about new articles just by “looking” at this present reality information. Pharmaceutical and therapeutic gadget producers are attempting to get genuine, and controllers are empowering them. That is because randomised clinical preliminaries, the highest quality level for testing treatment viability and security, may give an inappropriate thought regarding how well medications work in regular day to day existence. That, alongside expense and effectiveness concerns, is driving energy for more prominent utilisation of information assembled outside of clinical preliminaries. Genuine information originates from numerous sources, for example, electronic xviii

Preface

medicinal records, libraries of patients with a specific condition and social insurance claims data. It can emerge out of patients, as well, including through their versatile applications or wearable wellbeing trackers. A geographic information system (GIS), or geospatial data framework is any framework that catches, stores, examines, oversees, and exhibits information that connects to the location(s). GIS is the converging of cartography, measurable examination, and database innovation, ordinary asset the board, exactness horticulture, photogrammetric, urban arranging, a crisis the board, route, ethereal video, and limited hunt engines. Today the idea of GIS is generally utilised in various territories of research and humanity. The catastrophe the board frameworks are most founded on GIS today, and the centre thought of those frameworks is the past seismic tremor encounters. Remote detecting picture preparing can enable the government to evaluate tremor harm rapidly. Taking this innovation coordinated with the quondam structure can improve the appraisal’s accuracy of this sort of framework. It talks about a technique which quickly assesses the tremor harm. Right off the bat, we talk about various cases and snappy assessment of seismic tremor harm by methods for GIS-based framework. Besides, breaks down the outcome by brisk evaluation. Thirdly, coordinating the picture handling module with the old structure to get the new framework, in which the evaluation’s accuracy can improve. Computer-based intelligence innovations will keep upsetting in 2019 and will turn out to be much more generally accessible because of reasonably distributed computing and enormous information blast. I don’t review some other tech area right now that pulls in such vast numbers of brilliant individuals and huge assets from both the open-source/producer network and the most significant ventures simultaneously. What is the distinction between Artificial Intelligence (AI), Machine Learning (ML) and Deep Learning (DL)? While individuals frequently utilise these terms reciprocally, I think underneath is a decent applied delineation to separate these three terms. Simulated intelligence is remarkably a broad term, and to some degree this likewise makes each organisation guarantee their item has AI nowadays then ML is a subset of AI and comprises of the further developed systems and models that empower computers to make sense of things from the information and convey AI applications. ML is the study of getting computers to act without being unequivocally Content-Based Image Retrieval (CBIR) is one of the extraordinary zones in Computer Vision and Image Processing. ContentBased Image Retrieval frameworks recover images from that database, which are like the question picture. It is finished by really coordinating the substance of the question picture with the images in the database. Picture database can be an embrace, containing many thousands or a considerable number of images. In like manner case, those just listed by catchphrases entered in database frameworks by human administrators. Images Content of a picture can be portrayed regarding shading, shape and surface of a picture. CBIR in radiology has been a subject of research xix

Preface

enthusiasm for about ten years. Substance Based Image Retrieval in restorative is one of the conspicuous zones in Computer Vision and Image Processing. CBIR can be utilised to find radiology images in massive radiology picture databases. The primary objective of CBIR in restorative is to recover images that are outwardly like a question proficiently. Examination of enormous information by AI offers excellent focal points for digestion and assessment of a lot of complex social insurance information. Be that as it may, to successfully utilise AI instruments in human services, a few impediments must tend to and key issues considered, for example, its clinical usage and morals in social insurance conveyance. Preferences of AI incorporate adaptability and versatility contrasted and conventional biostatistical techniques, which makes it deployable for some undertakings. The objective of the proposed book is to introduce the reader to the Challenges and Applications for Implementing Machine Learning in Computer Vision. In particular, the book looks into the use of Machine Learning techniques and Artificial Intelligence to model various technical problems of the real world. This book organized in ten chapters; it includes essential chapters written by researchers from prestigious laboratories/ educational institution. A brief description of each of the chapters in this section given below: Chapter 1 presents an approach of student attendance monitoring system in Universiti Tun Hussein Onn Malaysia is slow and disruptive. As a solution, biometric verification based on face recognition for student attendance monitoring presented. The face recognition system consisted of five main stages. Firstly, face images under various conditions were acquired. Next, face detection was performed using the Viola-Jones algorithm to detect the face in the original image. The original image was minimised, and transformed into grayscale for faster computation. Histogram techniques of oriented gradients were applied to extract the features from the grayscale images, followed by the principal component analysis (PCA) in dimension reduction stage. Face recognition, the last stage of the entire system, using a support vector machine (SVM) as a classifier. The development of a graphical user interface for student attendance monitoring was also involved. The highest face recognition accuracy of 62% achieved. The obtained results are less promising, which warrants further analysis and improvement. Chapter 2 addresses the Computational neuroscience is inspired by the mechanism of the human brain. Neural networks have reformed machine learning and artificial intelligence. Deep learning, it is a type of machine learning method that teaches computers to do that comes indeed to individuals: acquire by example. It inspired by biological brains and became the essential class of models in the field of machine learning. Deep Learning involves several layers of computation. In the current scenario, researchers and scientists around the world are focusing on the implementation of

xx

Preface

different deep models and architectures. This chapter consists the information about significant structures of the deep network. That will give the information about Convolutional neural network, recurrent neural network, multilayer perceptron and many more. Further, it discusses CNN (Convolutional Neural Network) and its different pre-trained models due to its significant requirements in visual imaginary. This chapter also deliberates about the similarity of deep model and architectures with the human brain. Chapter 3 evaluates the Boltzmann distribution, which is the distribution that holds the statistical treatment of the method. The Boltzmann equation explained next to the main difficulty of this equation, the integral of the collision operator solved by the BGK-Approximation where a long term substitutes this essential. The discretisation of the Boltzmann comparison with the BGK-Approximation introduced, all along with the lattice and the different lattice configurations, to define the lattice framework where the method is applied. Also, in this framework, the algorithm of the process was described. The boundary conditions summarized in this chapter, where one can see that they represent macroscopic conditions acting locally in every node. Chapter 4 reviews big data is required to analysis process or analytics tools or techniques. For that, in the past several decades, a lot of research has been using data mining, machine learning and deep learning techniques. Here, machine learning is a subset of artificial intelligence, and deep learning is a subset of machine learning. Deep learning is much efficient than machine learning technique because, in this, it use perceptron and neuron or back propagation method, i.e., in these techniques, solve a problem by learning by itself. Todays in several applications like healthcare, retails, etc., deep learning used everywhere. But, using deep learning techniques in such applications creates several problems or raised several critical issue and challenges which are highly needed to overcome to determine accurate results. Chapter 5 aims to the current growth in the market insist on adopting machine learning the smart technology utilised by our nation widely. The proposed chapter focuses on deep introductory of Machine learning. Why is it Important? Machine learning generally used to find out knowledge from unknown data. There are many approaches and algorithms are available for performing machine learning. The different kind of algorithms is possible to find out a different type of patterns from the data. This chapter focuses on different approaches and usage. Chapter 6 analyses the medical image resolution, automatic segmentation is a challenging task, and it’s still an unsolved problem for most medical applications due to the wide variety connected with image modalities, encoding parameters and organic variability. In this chapter, medical image segmentation using clustering, compression, histogram, edge detection, parametric, variational model and level

xxi

Preface

set-based methods. Here also discuss modes of segmentation like manual, semiautomatic, interactive and automatic. Aim and motivation for doing fast, interactive and correct segmentation, the medical image modalities X-ray, CT, MRI and PET discussed in this chapter. Chapter 7 reviews a genuine seismic tremor; it sets aside more extended effort to handle harm, regardless of whether substantial damages concentrated in such a circumstance, remote detecting innovation can assume significant jobs. Since the time has come devouring to recognize the harm by human eyes, it is successful in applying picture Handling. This paper presents the relevance of picture handling to distinguish different sorts of Harm. For areal-type harm, 1) edge extraction, 2) unsupervised arrangement, 3) texture examination, and 4) edge improvement is Suitable to distinguish harmed zone. For liner-type harm, it is hard to improve the permeability of harm partition by picture preparing the impact of overlaying office information to help staff to discover damage at extraction by human described. Chapter 8 considers the recent development in technologies and integration of millions of internet of thing’s devices; a lot of data generated every day. It required to analysis for improving the growth of several organizations or in applications like e-healthcare, etc. Also, we are entering into an era of smart world, where Robotics is going to take place in most of the applications (to solve the world’s problems). Implementing Robotics in applications like medical, automobile, etc., is an aim/ goal of computer vision. Computer Vision (CV) objective fulfilled by several components like Artificial Intelligence (AI), Machine Learning (ML) and Deep Learning (DL). Here, Machine Learning and Deep learning techniques/ algorithms used to analysis Big Data. Today’s various organizations, like Google, Facebook, etc., are using ML techniques to search particular data or recommend any post. Hence, the requirement of a computer vision is fulfilled these three terms AI, ML, and DL. Chapter 9 considers the Content-Based Image Retrieval is a promising technique to access visual data. With the massive development of computer storage, networking, and the transmission technology now it becomes possible to retrieve the image data besides the text. Traditionally we use to find the content of the image by the tagged model with some indexed text. In the development of a machine learning technique in the domain of Artificial Intelligence, the feature extraction techniques become easier for CBIR. The medical images are continuously increasing day by day, where each image holds some specific and unique information about some particular disease. The objectives of using CBIR in medical diagnosis are to provide correct and useful information to the specialist for the quality and efficient diagnosis of the disease. In the medical image, content required different types of CBIR technique for different medical image acquisition technique such as MRI, CT, PET scan, USG, MRS etc. So in this concern the each CBIR technique have its unique feature extraction algorithm for each acquisition technique. xxii

Preface

Chapter 10 presents machine learning that is most ever-growing subfield in health data analytics. At present, public health and population health are a primary concern and the study highlights the significant challenges through a few case studies of the application of machine learning for health data analytics to focus on regression four categories of machine learning methods meaningful supervised, unsupervised, semisupervised and reinforcement learning. Machine learning provides new opportunities for automatically learning inexpressive models. Regression Models, including Multiple and Multivariate Regression techniques, are suitable for modelling Air Pollution and heart disease prediction. The applicability of STATA and R packages for multiple linear regression and predictive modelling for Crude Birth Rate and Crude Mortality Rate is well established in this study. Decision Tree is a class of Machine Learning model and applied for Brain Tumor. In predictive modelling, the measurement of the uncertain estimated model is the confidence interval. The confidence interval evaluated as a case study. The prospective audience of this book will be on one side undergraduate students, postgraduate students, and researchers who are interested in the emerging area of machine learning, computer vision and on the other side industry people who are working in the mentioned areas. The potential use of this book could be course supplement for upper-level courses; library reference for students’ projects; resource for researchers in machine learning, Challenges and Applications for Implementing Machine Learning in Computer Vision area. As the book will give an overview of both major domains together with their current research progress, issues and ideas; reference for the industry field, as essential research issues could address in the real-world environment. The Challenges and Applications for Implementing Machine Learning in Computer Vision book is published by IGI Global (formerly Idea Group Inc.), publisher of the “Information Science Reference” (formerly Idea Group Reference), “Medical Information Science Reference,” “Business Science Reference,” and “Engineering Science Reference” imprints. For additional information regarding the publisher, please visit www.igi-global.com. The editors wish you a pleasant reading.

xxiii

Table of Contents

Foreword............................................................................................................. xiv Preface.................................................................................................................. xv Acknowledgment.............................................................................................. xxiv Chapter 1 Development of Class Attendance System Using Face Recognition for Faculty of Mechanical and Manufacturing Engineering, Universiti Tun Hussein Onn Malaysia............................................................................................1 Pauline Ong, Universiti Tun Hussein Onn Malaysia, Malaysia Tze Wei Chong, Universiti Tun Hussein Onn Malaysia, Malaysia Woon Kiow Lee, Universiti Tun Hussein Onn Malaysia, Malaysia Chapter 2 Deep Learning in Computational Neuroscience...................................................43 Sanjay Saxena, Department of Computer Science and Engineering, IIIT Bhubaneswar, Bhubaneswar, India Sudip Paul, North-Eastern Hill University, India Adhesh Garg, Department of Computer Science and Engineering, IIIT Bhubaneswar, Bhubaneswar, India Angana Saikia, North-Eastern Hill University, India Amitava Datta, The University of Western Australia, Australia Chapter 3 Advanced Diagnosis Techniques in Medical Imaging..........................................64 Ramgopal Kashyap, Amity School of Engineering and Technology, Amity University, Raipur, India Chapter 4 Challenges of Applying Deep Learning in Real-World Applications...................92



Amit Kumar Tyagi, School of Computing Science and Engineering, Vellore Institute of Technology, Chennai, India G. Rekha, Department of Computer Science and Engineering, Koneru Lakshmaiah Educational Foundation, Hyderabad, India Chapter 5 Challenges and Applications for Implementing Machine Learning in Computer Vision: Machine Learning Applications and Approaches.................119 Hiral R. Patel, Ganpat University, India Ajay M Patel, AMPICS, India Satyen M. Parikh, FCA, India Chapter 6 Medical Imaging Importance in the Real World................................................136 Ramgopal Kashyap, Amity School of Engineering and Technology, Amity University, Raipur, India Chapter 7 Image Processing Approaches and Disaster Management..................................163 Surendra Rahamatkar, Amity University, Raipur, India Chapter 8 Artificial Intelligence and Machine Learning Algorithms..................................188 Amit Kumar Tyagi, School of Computing Science and Engineering, Vellore Institute of Technology, Chennai, India Poonam Chahal, MRIIRS, Faridabad, India Chapter 9 Application of Content-Based Image Retrieval in Medical Image Acquisition.220 Vinayak Majhi, North-Eastern Hill University, India Sudip Paul, North-Eastern Hill University, India Chapter 10 Machine Learning for Health Data Analytics: A Few Case Studies of Application of Regression...................................................................................241 Muralikrishna Iyyanki, Independent Researcher, India Prisilla Jayanthi, Administrative Staff College of India, India Valli Manickam, Administrative Staff College of India, India Compilation of References............................................................................... 271



About the Contributors.................................................................................... 288 Index................................................................................................................... 292

Detailed Table of Contents

Foreword............................................................................................................. xiv Preface.................................................................................................................. xv Acknowledgment.............................................................................................. xxiv Chapter 1 Development of Class Attendance System Using Face Recognition for Faculty of Mechanical and Manufacturing Engineering, Universiti Tun Hussein Onn Malaysia............................................................................................1 Pauline Ong, Universiti Tun Hussein Onn Malaysia, Malaysia Tze Wei Chong, Universiti Tun Hussein Onn Malaysia, Malaysia Woon Kiow Lee, Universiti Tun Hussein Onn Malaysia, Malaysia The traditional approach of student attendance monitoring system in Universiti Tun Hussein Onn Malaysia is slow and disruptive. As a solution, biometric verification based on face recognition for student attendance monitoring was presented. The face recognition system consisted of five main stages. Firstly, face images under various conditions were acquired. Next, face detection was performed using the Viola Jones algorithm to detect the face in the original image. The original image was minimized and transformed into grayscale for faster computation. Histogram techniques of oriented gradients was applied to extract the features from the grayscale images, followed by the principal component analysis (PCA) in dimension reduction stage. Face recognition, the last stage of the entire system, using support vector machine (SVM) as classifier. The development of a graphical user interface for student attendance monitoring was also involved. The highest face recognition accuracy of 62% was achieved. The obtained results are less promising which warrants further analysis and improvement. Chapter 2 Deep Learning in Computational Neuroscience...................................................43 Sanjay Saxena, Department of Computer Science and Engineering, IIIT



Bhubaneswar, Bhubaneswar, India Sudip Paul, North-Eastern Hill University, India Adhesh Garg, Department of Computer Science and Engineering, IIIT Bhubaneswar, Bhubaneswar, India Angana Saikia, North-Eastern Hill University, India Amitava Datta, The University of Western Australia, Australia Computational neuroscience is inspired by the mechanism of the human brain. Neural networks have reformed machine learning and artificial intelligence. Deep learning is a type of machine learning that teaches computers to do what comes naturally to individuals: acquire by example. It is inspired by biological brains and became the essential class of models in the field of machine learning. Deep learning involves several layers of computation. In the current scenario, researchers and scientists around the world are focusing on the implementation of different deep models and architectures. This chapter consists the information about major architectures of deep network. That will give the information about convolutional neural network, recurrent neural network, multilayer perceptron, and many more. Further, it discusses CNN (convolutional neural network) and its different pretrained models due to its major requirements in visual imaginary. This chapter also deliberates about the similarity of deep model and architectures with the human brain. Chapter 3 Advanced Diagnosis Techniques in Medical Imaging..........................................64 Ramgopal Kashyap, Amity School of Engineering and Technology, Amity University, Raipur, India The Boltzmann distribution was derived in this chapter. The Boltzmann equation was explained next to the main difficulty of this equation, the integral of the collision operator, which was solved by the BGK-approximation where a longterm substitute is essential. The discretization of the Boltzmann comparison with the BGK-approximation was introduced along with the lattice and the different lattice configurations to define the lattice framework where the method is applied. Also, in this framework, the algorithm of the process was described. The boundary conditions were summarised, where one can see that they represent macroscopic conditions acting locally in every node. Chapter 4 Challenges of Applying Deep Learning in Real-World Applications...................92 Amit Kumar Tyagi, School of Computing Science and Engineering, Vellore Institute of Technology, Chennai, India G. Rekha, Department of Computer Science and Engineering, Koneru Lakshmaiah Educational Foundation, Hyderabad, India



Due to development in technology, millions of devices (internet of things: IoTs) are generating a large amount of data (which is called as big data). This data is required for analysis processes or analytics tools or techniques. In the past several decades, a lot of research has been using data mining, machine learning, and deep learning techniques. Here, machine learning is a subset of artificial intelligence and deep learning is a subset of machine leaning. Deep learning is more efficient than machine learning technique (in terms of providing result accurate) because in this, it uses perceptron and neuron or back propagation method (i.e., in these techniques, solve a problem by learning by itself [with being programmed by a human being]). In several applications like healthcare, retails, etc. (or any real-world problems), deep learning is used. But, using deep learning techniques in such applications creates several problems and raises several critical issues and challenges, which are need to be overcome to determine accurate results. Chapter 5 Challenges and Applications for Implementing Machine Learning in Computer Vision: Machine Learning Applications and Approaches.................119 Hiral R. Patel, Ganpat University, India Ajay M Patel, AMPICS, India Satyen M. Parikh, FCA, India The chapter introduces machine learning and why it is important. Machine learning is generally used to find knowledge from unknown data. There are many approaches and algorithms available for performing machine learning. Different kinds of algorithms are available to find different patterns from the data. This chapter focuses on different approaches with different usage. Chapter 6 Medical Imaging Importance in the Real World................................................136 Ramgopal Kashyap, Amity School of Engineering and Technology, Amity University, Raipur, India In the medical image resolution, automatic segmentation is a challenging task, and it’s still an unsolved problem for most medical applications due to the wide variety connected with image modalities, encoding parameters, and organic variability. In this chapter, a review and critique of medical image segmentation using clustering, compression, histogram, edge detection, parametric, variational model. and level set-based methods is presented. Modes of segmentation like manual, semi-automatic, interactive, and automatic are also discussed. To present current challenges, aim and motivation for doing fast, interactive and correct segmentation, the medical image modalities X-ray, CT, MRI, and PET are discussed in this chapter. Chapter 7



Image Processing Approaches and Disaster Management..................................163 Surendra Rahamatkar, Amity University, Raipur, India This chapter presents the relevance of picture handling to distinguish different sorts of harm. For areal-type harm, 1) edge extraction, 2) unsupervised arrangement, 3) texture examination, and 4) edge improvement are suitable to distinguish harmed zone. For liner-type harm, it is hard to improve the permeability of harm partition by picture preparing. Likewise, the impact of overlaying office information to help staff to discover harm at an extraction is described. Chapter 8 Artificial Intelligence and Machine Learning Algorithms..................................188 Amit Kumar Tyagi, School of Computing Science and Engineering, Vellore Institute of Technology, Chennai, India Poonam Chahal, MRIIRS, Faridabad, India With the recent development in technologies and integration of millions of internet of things devices, a lot of data is being generated every day (known as Big Data). This is required to improve the growth of several organizations or in applications like e-healthcare, etc. Also, we are entering into an era of smart world, where robotics is going to take place in most of the applications (to solve the world’s problems). Implementing robotics in applications like medical, automobile, etc. is an aim/goal of computer vision. Computer vision (CV) is fulfilled by several components like artificial intelligence (AI), machine learning (ML), and deep learning (DL). Here, machine learning and deep learning techniques/algorithms are used to analyze Big Data. Today’s various organizations like Google, Facebook, etc. are using ML techniques to search particular data or recommend any post. Hence, the requirement of a computer vision is fulfilled through these three terms: AI, ML, and DL. Chapter 9 Application of Content-Based Image Retrieval in Medical Image Acquisition.220 Vinayak Majhi, North-Eastern Hill University, India Sudip Paul, North-Eastern Hill University, India Content-based image retrieval is a promising technique to access visual data. With the huge development of computer storage, networking, and the transmission technology now it becomes possible to retrieve the image data beside the text. In the traditional way, we find the content of image by the tagged image with some indexed text. With the development of machine learning technique in the domain of artificial intelligence, the feature extraction techniques become easier for CBIR. The medical images are continuously increasing day by day where each image holds some specific and unique information about some specific disease. The objectives of using CBIR in medical diagnosis are to provide correct and effective information



to the specialist for the quality and efficient diagnosis of the disease. Medical image content requires different types of CBIR technique for different medical image acquisition techniques such as MRI, CT, PET Scan, USG, MRS, etc. So, in this concern, each CBIR technique has its unique feature extraction algorithm for each acquisition technique. Chapter 10 Machine Learning for Health Data Analytics: A Few Case Studies of Application of Regression...................................................................................241 Muralikrishna Iyyanki, Independent Researcher, India Prisilla Jayanthi, Administrative Staff College of India, India Valli Manickam, Administrative Staff College of India, India At present, public health and population health are the key areas of major concern, and the current study highlights the significant challenges through a few case studies of application of machine learning for health data with focus on regression. Four types of machine learning methods found to be significant are supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. In light of the case studies reported as part of the literature survey and specific exercises carried out for this chapter, it is possible to say that machine learning provides new opportunities for automatic learning in expressive models. Regression models including multiple and multivariate regression are suitable for modeling air pollution and heart disease prediction. The applicability of STATA and R packages for multiple linear regression and predictive modelling for crude birth rate and crude mortality rate is well established in the study as carried out using the data from data.gov.in. Decision tree as a class of very powerful machine learning models is applied for brain tumors. In simple terms, machine learning and data mining techniques go hand-inhand for prediction, data modelling, and decision making. The health analytics and unpredictable growth of health databases require integration of the conventional data analysis to be paired with methods for efficient computer-assisted analysis. In the second case study, confidence interval is evaluated. Here, the statistical parameter CI is used to indicate the true range of the mean of the crude birth rate and crude mortality rate computed from the observed data. Compilation of References............................................................................... 271 About the Contributors.................................................................................... 288 Index................................................................................................................... 292

1

Chapter 1

Development of Class Attendance System Using Face Recognition for Faculty of Mechanical and Manufacturing Engineering, Universiti Tun Hussein Onn Malaysia Pauline Ong Universiti Tun Hussein Onn Malaysia, Malaysia Tze Wei Chong Universiti Tun Hussein Onn Malaysia, Malaysia Woon Kiow Lee Universiti Tun Hussein Onn Malaysia, Malaysia

ABSTRACT The traditional approach of student attendance monitoring system in Universiti Tun Hussein Onn Malaysia is slow and disruptive. As a solution, biometric verification based on face recognition for student attendance monitoring was presented. The face recognition system consisted of five main stages. Firstly, face images under various conditions were acquired. Next, face detection was performed using the Viola Jones algorithm to detect the face in the original image. The original image was minimized and transformed into grayscale for faster computation. Histogram techniques of oriented gradients was applied to extract the features from the grayscale images,

DOI: 10.4018/978-1-7998-0182-5.ch001 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Development of Class Attendance System Using Face Recognition

followed by the principal component analysis (PCA) in dimension reduction stage. Face recognition, the last stage of the entire system, using support vector machine (SVM) as classifier. The development of a graphical user interface for student attendance monitoring was also involved. The highest face recognition accuracy of 62% was achieved. The obtained results are less promising which warrants further analysis and improvement.

INTRODUCTION Nowadays, biometric applications are widely applied across industries, institutions and government establishments. Biometric verification means that a person can be uniquely identified by evaluating one or more distinguishing biological traits (Zhang, 2013). Unique identifiers include fingerprints, face and hand geometry, retina and iris patterns, voice waves, DNA, and signatures. Every human has unique fingerprint, which is constructed by numerous ridges and valley on the surface of finger (Jain, Ross, & Prabhakar, 2004). Voice recognition is a technology that transforms the sound and words of human into electrical signals. These signals will then be converted into code design (Baumann, 1993). Iris recognition is one of the most accurate biometric verification techniques. The digital camera is used to take the impression of an iris, and then, the iris will be evaluated in the stored version (Saini & Rana, 2014). For hand geometry recognition, the shape, size of palm, and the lengths and width of the fingers are taken from the human hand based on a number of measurements (Jain, Flynn, & Ross, 2007). Face recognition involves an analysis of facial features. Typically, it involves a computer system which determines or verifies an individual automatically from a digital image or video framework. The selected facial features from the image are then compared against the available features in facial database to identify the individual (Saini & Rana, 2014). Face recognition, in general, consists of three principal phases, specifically, face detection – detection of a human face from within the image phase for further scrutinization, feature extraction – extracting the distinguishable features from faces using the techniques of segmentation, image rendering and scaling, and face identification - applying the mathematical models or artificial intelligence methods to identify the face based on the features extracted from the facial area within an image. Face recognition has been widely used in numerous applications, for instance, surveillance system (Zafar et al., 2019), photo album organization (Oh, Benenson, Fritz, & Schiele, 2018), identity verification in financial services (Szczuko, Czyżewski, Hoffmann, Bratoszewski, & Lech, 2019) and airport (Carlos-Roca, Torres, & Tena, 2018), and owner verification of mobile phone (Mahbub, Sarkar, & Chellappa, 2

Development of Class Attendance System Using Face Recognition

2019). Using face recognition to monitor student attendance in universities is another potential application area to pursue. This is due to universities are using class attendance as a marker of student engagement for reasons including admission to the final examination. Most standard attendance monitoring methods, in fact, require trade-off between the effort spent to record the attendance and the aspect of accuracy (Joseph & Zacharia, 2013). For instance, lecturers take the student attendance at the expense of the available teaching time. This approach can be disruptive, imagine the case where the attendance list is passed around from attendee to attendee during an important part of the lecture (Alia, Tamimi, & Al-Allaf, 2013). Hence, utilization of biometric verification based on face recognition for student attendance seems to be a viable solution. Moreover, attendance monitoring using face recognition does not require any action from students. Student just simply let the camera or webcam to scan his/her face, and the attendance would be recorded automatically. Face recognition performs massive identification which usually other biometric systems could not do (Klokova, 2010). Taking the attendance based on face recognition helps to avoid attendance fraud on one hand, and reduces human mistake on another hand. Moreover, the paper-less attendance monitoring system contributes to a cleaner environment (Saini & Rana, 2014). The most important thing is that facial recognition technology can be easily programmed into real-time attendance monitoring system. As such, an automated student attendance system based on biometric verification, particularly, face recognition used in this study, is proposed as an alternative solution to the traditional attendance sheet. Using face recognition as a system to monitor the student attendance does not only prevent fraud attendance, it reduces human mistake as all students have unique biometric. The objectives of this study are: 1. To develop an automated student attendance system using face recognition technique 2. To develop a graphical user interface (GUI) in order to improve the efficiency of managing students’ attendance.

THEORETICAL FRAMEWORKS Face recognition has become more popular due to the advancement in computer vision and machine learning. This section reviews some publicly available databases of face images, feature extraction methods, classifiers used in face recognition, and some prior work in attendance monitoring.

3

Development of Class Attendance System Using Face Recognition

PUBLICLY AVAILABLE DATABASES OF FACE IMAGES AR Database AR database was collected at the Computer Vision Center in Barcelona, Spain in 1998 (Martinez, 1998). This database has 116 images (63 men and 53 women). The recording conditions (camera parameters, lighting setting, and camera distance) are controlled carefully and always checked to make sure that settings are identical across subjects. The images are in 768 x 576 pixels in size. Various facial expressions, illumination and occlusion were captured during each session. Table 1 summarizes the recording condition of AR database while Figure 1 shows an example for each condition.

BANCA Database The BANCA database was collected as part of the European BANCA project. This project develops and implements a secure system with improved authentication, identification, and access control schemes for applications over the internet (BaillyTable 1. Record condition of AR database No of subject

116

Conditions

Image Resolution

Facial expressions

4

Illumination

4

Occlusion

2

Time

2

768 x 576

Figure 1. AR database with different condition (Gross, 2005)

4

No of Images

3288

Development of Class Attendance System Using Face Recognition

Bailliére et al., 2003). Data were collected for 52 individuals (26 men and 26 women). In a period of 3 months, 12 different sessions are recorded for each subject. Table 2 summarizes the recording conditions of BANCA database. Images from three recording conditions are shown in Figure 2. Controlled and adverse conditions was recorded by a high quality digital camera. The image of the degraded was taken by a low quality digital camera.

CAS-PEAL Database The CAS-PEAL database was collected at the Chinese Academy of Sciences (CAS) from August 2002 to April 2003. Data were collected for 1040 subjects (595 men and 445 women). The collected data fall in 7 categories, which are pose, expression, accessory, lighting, background, distance and time (Gao et al., 2004). Nine cameras were set in a semicircle around the individual for the pose subset. Images were recorded within a short time period (2 seconds). In addition, subjects were required to look up and look down each time by roughly 30° for additional recordings. Table

Table 2. Record condition of BANCA database No of Subject 208

Conditions

Image Resolution

Image quality

3

Time

12

720 x 576

Figure 2. BANCA database (Gross, 2005)

5

Development of Class Attendance System Using Face Recognition

3 summarizes the recording conditions of CAS-PEAL database while Figure 3 shows the example of the images. To record the face under varying lighting conditions, the light were set at (-90°, -45°, 0°, 45°, 90°) azimuth and (-45°, 0°, 45°) elevation. Figure 4 shows the image for all illumination conditions. For the expression conditions, the subjects were required to smile, to look surprise, to close their eyes, to frown, and to open their mouth. Some of the subjects were recorded wearing 3 type of glasses and hats. Subjects were also recorded with five plain colored background, which were blue, white, black, red, and yellow. Furthermore, images were obtained at 1.2 and 1.4 meters distances from camera. Lastly, small number of subjects were required to come back after 6 months for additional recordings. To simplify database distribution, the released images were stored as cropped gray-scale images of size 360 × 480. Figure 5 shows the currently distributed images. Table 3. Record condition of CAS-PEAL database No of Subjects

Conditions

Image Resolution

1040

Pose

21

377

Facial expressions

6

438

Accessory

6

233

Illumination

9– 15

297

Background

2–4

269

Distance

1–2

66

Time

2

360 x 480

Figure 3. Pose variation in the CAS-PEAL database (Gross, 2005)

6

No of Image

30900

Development of Class Attendance System Using Face Recognition

Figure 4. Illumination variation in the CAS-PEAL database (Gross, 2005)

Figure 5. Released image of the pose in grayscale and 360x480 size (Gross, 2005)

FERET Database The Facial Recognition Technology (FERET) database was collected at George Mason University and the US Army Research Laboratory facilities, sponsored by the US Department of Defense Counterdrug Technology Development Program (Phillips, Moon, Rizvi, & Rauss, 2000; Phillips, Wechsler, Huang, & Rauss, 1998). The FERET database has 24 facial image categories. Fifteen recording sessions were conducted from August 1993 until July 1996. The 35 mm camera was used to record the image. The captured images were digitized after that and then transformed to 8-bit grayscale images. The images were 256 × 384 pixels in size. Table 4 summarizes the recording condition of the FERET database while five frontal images are displayed in Figure 6.

7

Development of Class Attendance System Using Face Recognition

Table 4. Record condition of FERET database No of Subjects

1199

Conditions

Image Resolution

Facial expression

2

Illumination

2

Pose

9– 20

Time

2

256 x 384

No of Images

14051

The ia and ib images were obtained in close succession. The subjects were required to show other facial expressions for the ib image. The changes in facial expression were switched between neutral and smiling. The image with a different camera and under different lighting conditions were categorized in the ic image. Some of the subjects were required to return at later date for another shot. For the images in duplicate I, at least 0 to 1031 days gap was in between two consequent recordings sessions. For the images in duplicate II, at least 18 months gap was in between two separate recording sessions. The remaining image categories were set for pose variation. Figure 7 shows categories ba to bi. The subjects were asked to rotate the head and body when recording and the pose angles were ranged from +60° to −60°. Two hundred subjects were available for these pose data. Diverse set of images are presented in Figure 8. These images were collected by using different head aspects: right and left face (labeled pr and pl), right and left half face (hr, hl), and right and left quarter face (qr, ql). There were three categories, specifically, right and left profile, right and left half profile and last is right and left quarter profile. There were 508 to 908 subjects recorded in these categories. Figure 6. Frontal image categories used for FERET evaluations (Gross, 2005)

8

Development of Class Attendance System Using Face Recognition

Figure 7. Pose variation in the FERET database (Gross, 2005)

Figure 8. Additional set of pose images from the FERET database (Gross, 2005)

From the literature review of publicly available face recognition database, it can be seen that the factors of pose, illumination condition, facial expression and recording period are the main considered features, as summarized in Table 5.

Feature Extraction for Face Recognition The aim of feature extraction is to extract a compact set of interpersonal discriminating geometrical or/and photometrical features of the face. Feature extraction starts from Table 5. Summary of the record condition for the database Database

No of Subject

Pose

Facial Expressions

Illumination 4

Time

AR

116

1

4

2

BANCA

208

1

-

1

12

CAS-PEAL

66 – 1040

21

9 – 15

6

2

FERET

1199

9 – 20

2

2

2

9

Development of Class Attendance System Using Face Recognition

a set of measured data and derives values to be used for subsequent learning. The speeded up robust features (SURF) extraction technique extracts the features using the nearest neighbors (Patel & Shah, 2017). The application of SURF features for face recognition in (Du, Su, & Cai, 2009) by utilizing the FERET database achieved the face recognition rate of 96%. Although the recognition rate of SURF is only slightly better than scale invariant feature transform (SIFT) with recognition rate of 95.9%, an improvement on matching speed is observed. SIFT extraction method has been widely used to extract the features in face recognition (Lenc & Král). SIFT forms descriptors of the region of interest which are invariant to scale and rotation, and provides good performance under illumination changes. Purandare et al. utilized SIFT and contour matching for effective heterogeneous face recognition (Purandare & Talele, 2014). It has been shown in their work that the proposed algorithm gave better results in efficiency and accuracy, where this method is robust for illumination changes, facial expression changes and the changes in pose. Gabor wavelet provides analysis of images, considering different scales or resolution and orientations (Liu & Wechsler, 2003). Chung et al. applied Gabor filter as part of the face recognition method (Chung, Kee, & Kim, 1999). The Gabor filter method has been combined with the principal component analysis (PCA) to recognize the face. Due to the images are sensitive to illumination and pose variant, Gabor filter responses as an input of PCA are used to overcome the problems. The experimental result of the proposed method showed the enhancement of recognition rate of 19% and 11% in SAIT dataset and Olivetti dataset, respectively. Eigenfaces decomposes facial images into a small set of characteristic feature images. The lower dimensional space is found by using the eigenfaces (Turk & Pentland, 1991). An automatic face annotation using face features, outfit colors and period has been proposed by Nursuriati et al. (Jamil & Sa’Dan, 2014). The face is first detected by the Viola and Jones Haar-like detector in this approach. The extracted facial features, colors of clothing and date of picture are then stored as feature vectors after the face detection. The facial features are extracted by eigenfaces approach, while the clothing colors are extracted by color histogram and Julian date format is used.

Face Recognition Classifier Classification is the problem of identifying to which of a set of categories belongs, on the basis of a training set of data containing observations whose category membership is known. An example will be assigning the features belong to which person. Regression model is one of the methods that has been utilized in face recognition. Huang et al. (Huang, Lai, Gao, Yang, & Yang, 2016) applied linear 10

Development of Class Attendance System Using Face Recognition

regression classification (LRC) and its improved form, called linear discriminant regression classification (LDRC) to boost the effectiveness of LRC. The adaptive linear discriminant regression classification (ALDRC) algorithm was also proposed in which consideration of different contributions of the training samples was taken into account. Simulation on some face databases showed the advantage of using ALDRC. Machining learning approaches, too, have been widely used for classification of face images. Machine learning is a field of computer science that uses statistical techniques to provide computer systems the ability to “learn” itself without being clearly programmed. Machine learning involves understanding of the complex patterns until it can learn new pattern adeptly or recognize existing one (Christopher, 2016). One of the most outstanding machine learning techniques is artificial neural networks (ANNs), which use some parallel strategies such as simulating neurons and layering in order to learn concepts from experience. Réda and Aoued (Réda & Aoued, 2004) utilized ANNs based face recognition on face images with and without noise. The experimental result showed that the recognition rates of 100% and 94.68% are attained for images without and with Gaussian noise, respectively. The k-Nearest Neighbor (kNN), a variant of machine learning, has been used by Wrosey (Worsey, 2016) in his work. The features are extracted by the SIFT and each descriptor is matched using a Euclidean distance based kNN. The best result achieved in his work was approximately 86% matching rate. Jingyu (Jingyu, 2018) applied three popular algorithms, specifically, K-means clustering, auto-encoder, and convolutional neural network (CNN), to deal with face recognition problem, in terms of sentiment classification from images. It was shown that the classification using CNN produced the highest accuracy. The advent of the deep learning (DL) methodology has successfully overcome the issues with real time face recognition system of ANNs (Vinay et al., 2015). DL operates by modelling high-level abstractions based on how the human brain recognizes patterns. Deep Neural Networks (DNNs) apply a large number of layers for the purpose of mimicking human learning. It can adjust the strength of connections between the simulated neurons within the multiple layers accordingly (Vinay et al., 2015). It has been shown that DNNs are capable of easily being scaled to substantially large datasets which other methods such as SVM and LDA cannot perform well (Vinay et al., 2015). The DNN has been introduced in (Vinay et al., 2015), where it has been shown that the DNN performed perfectly in face recognition even in images with various illuminations and camera angles. The outstanding accuracy of about 97.3% on the Labelled Faces in the Wild (LFW) database has been obtained.

11

Development of Class Attendance System Using Face Recognition

Attendance Monitoring Prior Work Shilwant and Karwankar (Shilwant & Karwankar, 2012) designed a simple web based attendance monitoring system that can be used across campus. The developed system worked in two different ways, first was actual face recognition and second was wider system administrative purposes from post processing of the gathered information. Before passing the captured image with face coordinates to the server for recognition, they used the continuously operating web cam to detect the faces. Alia et al. (Alia et al., 2013) developed a semi-automated system where the camera was used to capture the student’s image when they entered the class. The lecturer can instruct the system to process the images and to confirm the recognition. Before submitting a final attendance report to the administrative system, the lecturer can manually add undetected or unrecognized attendees. Worsey (Worsey, 2016) presented an automated system to monitor the attendance. Students’ images were recorded using the camera. It is then assigned to the identity of the library image that it best matched for each captured face image. The identity with all related data was excluded from further consideration when the identity of the best matching face image was assigned. Kar et al. (Kar, Debbarma, Saha, & Pal, 2012) presented an automated system to monitor the attendance. The attendance of the students in classroom environment will be recorded automatically by this system. The student’s information can be accessed easily because the system provides the facilities to the faculty to maintain a log for clock-in and clock-out time. The system takes attendance of each student at the entry and exit points. Their work showed the improvement of this system as compared to the traditional black and white attendance system. Chintalapati and Raghunadh (Chintalapati & Raghunadh, 2013) developed an automated attendance management system. The system will automatically mark the attendance by recognizing the student when it detects the student enters the classroom, using the camera at the entrance of the classroom. For further processing, face region was extracted and pre-processed. Due to the size of entrance, no more than two persons can enter the classroom at the same time. It has been shown that this system is less time consuming and with high security.

RESEARCH METHODS Nowadays, institutions utilize several methods for student attendance system. In Faculty of Mechanical and Manufacturing Engineering (FKMP), Universiti Tun Hussein Onn Malaysia (UTHM), attendance monitoring using the traditional approach, where students are required to sign on the attendance list. This approach is slow if the size of attendee is large, and it may be disruptive, since the attendance list is 12

Development of Class Attendance System Using Face Recognition

passed around in class during an important part of the lecture. Also, taking attendance using the sign-in attendance sheet, or maybe a short-in-class quiz, may lead to the issue of fraud attendance, where the student who is present may fraudulently sign on the behalf of an absent friend. At times, lecturers call the name one by one to take the attendance in order to prevent fraud attendance at the expense of the available teaching time. All these manual approaches are not only time-consuming, but also less efficient for keeping the student attendance record validly and correctly. UTHM has started to implement Student Attendance System (SASAR) by using the QR code since Semester 2, 2017/2018. Students scan the QR code using their registered student account during the class as proof of attendance. SASAR may be more efficient than the traditional sign-in attendance sheet; however, the issues of time-consuming and validity of attendance are remained unsolved. Lecturers have to display the QR code for the students to scan at the expense of the available teaching time. Also, fraudulent behavior still cannot be prevented. By using two or more smartphones, student can help their absent friends to scan the QR code easily. Hence, the feasibility of applying face recognition to monitor student attendance in FKMP, UTHM is used as the case study in this work. This section discusses the methods used to develop the students’ attendance monitoring system using face recognition in this study. A standard face recognition pipeline, consists of face images acquisition, face detection, pre-processing of face images, feature extraction, dimension reduction and face recognition, are introduced in this section. The technique used in each stage of face recognition is also presented in this section.

Standard Face Recognition Pipeline Basically, the face recognition system consists of five main stages, specifically, collection of face images, face detection, feature extraction, dimension reduction and face recognition, as illustrated in Figure 9. The details of each main stage are explained in the following section. Various conditions of the face images were recorded in the data collection stage. The different conditions, specifically, pose, facial expression, illumination, time, and accessory are considered in this study. Continuing from the data collection, face detection which is to detect the face in the raw image, is performed. Face detection detects the face region and removes the unwanted background for further process. Prior to face detection, the raw image is transformed into grayscale, and is resized to smaller size for faster computation. For the feature extraction stage, HOG technique is applied to extract the features from the face images. Gradients are calculated to generate the histogram in this stage. Subsequently, PCA is used in dimension reduction stage. The purpose of applying the PCA dimensionality reduction approach is to reduce the feature dimension because 13

Development of Class Attendance System Using Face Recognition

Figure 9. Standard pipeline for face recognition

the original data dimension of the extracted features is high. Face recognition is the last stage for the entire system, where SVM classifier has been chosen in this study. The SVM finds the optimal hyperplane to separate different samples into their corresponding categories, such that the owner of the face image can be identified correctly.

Face Image Acquisition For the face recognition system, data acquisition plays an important role. The condition of the face image will affect the accuracy of the entire face recognition system indirectly. After scrutinizing some publicly available database, the recording 14

Development of Class Attendance System Using Face Recognition

condition of face images used in this study are poses, facial expression, illumination, time and accessory. For pose, the face images are taken from left side of the face until right side of the face, which is 90° to −90°, with the difference of 45° in each time of recording. In addition, the images of frontal face looking upside and downside are taken. For the facial expression, the images are taken in smile, neutral, frown, open mouth, close eye, and speech conditions. For the illumination, lights are put in parallel with face, with the sequence set at 45°, 0°, and −45°. Indoor, outdoor, light on and light off conditions are also considered in the illumination of the image. For the time factor, the image is taken 1 to 5 times over a 6 weeks period. Lastly, for the accessory condition, the images of subject with and without glasses are considered. All of these images are taken by Apple IPhone 6s, in the dimension of 3024×4032. The examples of face images in all different recording conditions are shown in Figure 10. The conditions are (A1) glasses, (E1) neutral, (E2) smile, (E3) frown, (E4) closed eye, (E5) open mouth, (E6) speech, (I) outdoor, (I2) on light, (I3) light -45°, (I4) light 0°, (I5) light 45°, (P1) pose -90°, (P2) pose -45°, (P3) pose 0°, (P4) pose 45°, (P5) pose 90°, (P6) look up, (P7) look down. Figure 10. Face image record conditions

15

Development of Class Attendance System Using Face Recognition

Face Detection Using Viola Jones Algorithm Following from the face image acquisition, face detection is generally considered the first step in face recognition. The original image is converted to grayscale prior to face detection as shown in Figure 11. In addition, since the acquired images are in hundreds in this study, longer processing time is required to process the image if large size of image is used. Hence, the original image with dimension 3024×4032 is resized to dimension 1512×2016, which is 50% of the original image. This is to speed up the image processing. Face detection, basically, is to detect the faces from non-faces from the image. The Viola Jones algorithm (Viola & Jones, 2004) is the face detection technique used in this study. It is selected due to the advantages of its robustness and high computational efficiency (Chaudhari, Shanta, & Vanjare, 2015). There are four stages involved in the Viola Jones algorithm to detect the object, specifically: 1. 2. 3. 4.

Haar-like feature Integral image for rapid features detection AdaBoost machine-learning method Cascade classifier

Figure 11. Grayscale image after converting

16

Development of Class Attendance System Using Face Recognition

Haar-Like Feature Human faces have similar characteristics, for instance, the eyes are normally darker than the surrounding areas, and the nose bridge region is usually brighter than the eye region. Hence, the variation in the black and light portion of the image can be detected by Haar-like features. The Haar-like features with two-, three- or fourrectangles, are used as a template to scalable geometric structures, as shown in Figure 12. Haar-like feature computes the differences between distinct rectangular areas within the image space. For example, as shown in Figure 13, a two-rectangles Haar-like features can locate the eye region by searching the areas within the image such that the upper rectangle contrasts the lower rectangle significantly. If a threerectangles Haar-like features is used, the areas of image with the most distinctive difference between the two outer rectangles and the inner rectangle, are potentially the eye region.

Integral Images A main key of the Viola Jones detection algorithm is the use of the integral image. The image space transformed by a single pass can be defined as ii  x, y    i  x, y   |  x  x    y   y 

(1)

where ii is the integral image, the summation of the pixels taken from i from the column and row location by x and y. The integral image is a simple summation. Without using the integral image to compute the sum of pixels in a rectangular area of the image space would require the summation of every pixel. Simple calculation

Figure 12. Haar-like features are identified by the number of rectangles here contain (2, 3, 4) and the position of the rectangles (Worsey, 2016)

17

Development of Class Attendance System Using Face Recognition

Figure 13. Illustrating the strength inherent within the domain knowledge in Haarlike features (Worsey, 2016)

of the four corners of the integral image that sums up the pixel intensity values of the target rectangle area is given as: b

d

  i  x, y   ii  b, d   ii  a, c   ii  a, d   ii  b, c  x  a y c

(2)

where ii is the integral image and i is the source image. This is illustrated in Figure 14.

Adaboost Training Adaboost training can make the Viola Jones algorithm fast and easy computation because the Adaboost algorithm can help to find little features from the face (Viola & Jones, 2001). AdaBoost algorithm provides desired area of the object by discarding pointless background. Neural networks can be used to interpret the working model (Viola & Jones, 2001). The learning process of Adaboost is fast and gives more number of wanted data. This data can be classified by a classifier. A classifier holds minor features of the face. It is frequently utilized for pattern detection. This technique has high accuracy and fast detection but required more time to train (Viola & Jones, 2001). The explanation of Adaboost machine learning is as follows: • • 18

Given image is in the form (x1,y1)…(xn,yn) yi= 0,1 for negative and positive examples

Development of Class Attendance System Using Face Recognition

Figure 14. Integral image to compute the total pixel intensities of an image source rectangle (Worsey, 2016)

1 1 , for yi= 0,1respectively, where m and l 2m 2l

•

Initialize the weights w1,i =

•

are number of positives and negatives respectively. For t = 1,..,: 1. Normalize the weights,

wt ,i 

wt ,i

 j 1 wt ,i n



(3)

wt is the probability distribution 2. For each feature j, train a classifier hj which is restricted to use a single feature. The error is evaluated with respect to

wt , Et   i wi h j  xi  , yi

(4)

3. Choose the classifier ht with the lowest error Et 4. Update the weights

19

Development of Class Attendance System Using Face Recognition

wt 1,i  wt ,i Bt1ei

(5)

where ei= 0 examples xi is classified correctly; ei=1 otherwise

Bt 

et 1  et

•

The final classifier is:

(6)

h(x)=1



T

a h  x 

t 1 t t

where at = log

1 T  at 2 t 1

(7)

1 Bt

Cascading Classifier Using a cascade of stages in the Viola Jones face detection algorithm can quickly eliminates face candidates. The cascade in each stage has stricter requirement to eliminate candidates. After that, it will be more difficult to pass at later stage. A candidate will exit the cascade if they fail in any stage or pass at final stage. If a candidate passes at final stage, then a face will be detected. The process of cascade classifier is shown in Figure 15. A face will be show up when a candidate pass all stages in the cascade. The face will be detected when these four main stages in Viola Jones approach has been done. Figure 16 shows the detected face on the image using the Viola-Jones algorithm. Figure 15. The example cascade of stages (Chaudhari et al., 2015)

20

Development of Class Attendance System Using Face Recognition

Figure 16. The example of detected face on image using the Viola-Jones algorithm

Feature Extraction by Using Histogram of Oriented Gradients Features can be extracted once the face detection has been done. The technique of histogram of oriented gradients (HOG) (Shu, Ding, & Fang, 2011) is used as the feature extraction method in this study, due to its advantages of low computational complexity. Apart from this, HOG are not sensitive to both light and geometric changes. The steps to extract the HOG features are explained in the following section. The first step is to calculate the difference value for x and y directions, using the following equations:

 f x  x, y   f  x  1, y   f  x  1, y   1 1 f x , y f x , y f x , y            y

(8)

where f(x,y) is the brightness value of the image in (x,y). Subsequently, the calculation of magnitude Arg and direction θ in (x,y) is performed, according to Equation (3.8) and Equation (3.9), respectively.

Arg  x, y  

f x  x, y   f y  x, y 

  x, y   arctan

2

f x  x, y 

f y  x, y 

2



(9)

(10)

For example, consider an image of 64×128 dimension as in Figure 17, a 8×8 pixel cell is used to divide the image, forming 8×16 = 128 cell.

21

Development of Class Attendance System Using Face Recognition

Figure 17. Window division process (Xiang-Yu Li et al., 2018)

A 2×2 cells compose a block of 16×16 pixels, then 7x15 blocks are composed. The block is with step size of 8 pixels. (64−16)/8 + 1 = 7 is the number of blocks in the horizontal direction, and (128−16)/8 + 1 = 15 is the number of blocks in the vertical direction, as shown in Figure 18. For example, each cell takes a histogram of 9 gradient directions. Such a block has 4x9 = 36 feature vectors, an image of 36x105 = 3780 HOG features are formed by 105 blocks of feature vectors that are connected in series. The HOG feature extraction display are shown in Figure 19.

Figure 18. The example of block stepping process (Xiang-Yu Li et al., 2018)

22

Development of Class Attendance System Using Face Recognition

Figure 19. The example of the HOG feature extraction display

Feature Extraction by Using Histogram of Oriented Gradients The dimension of features is often higher than the number of samples. Although the classification still can be performed well using the raw extracted features, the computational complexity needs to be addressed. Hence, to achieve faster computation, the dimension reduction using the principal component analysis (PCA) is used in this study. PCA is a widely used technique for dimension reduction (Abdi & Williams, 2010; Jolliffe, 2002; Zhu, Zhu, & Li, 2018). PCA is an unmanaged learning algorithm which makes a linear transformation of the dimensions within the original area to an area that excellent describes the significance of every dimension. In theoretical, it determines the coordinates of the information that minimizes the mean square error over all linear changes of coordinate. It is done by locating the eigenvectors and eigenvalues of a related covariance matrix. A set of vectors is ordered by the variance. Since the quantity of the information contained is proportional to the variance within the component, thus, the components of low variance with less impact can be removed, contributing to dimensionality reduction. An outline of how to carry out the PCA on a set of image is described as follows: 1. Define X as a matrix of stacked images where each image is a column. The images are the xn component of the set of training data. 23

Development of Class Attendance System Using Face Recognition

 x , t  , n  1 N n

n

(11)

2. Averaging the rows of X to compute the mean vector x by using

xi 

1 X i N

(12)

where i is the ith row of X. 3. Subtract the mean and placing the resultant column in a matrix to center the data by using

A  X 1x T

(13)

where 1 is a vector for 1s 4. Compute the sample covariance matrix C by

C

1 AAT N 1

(14)

5. Compute the eigenvectors and the corresponding eigenvalue for the covariance matrix C by solving Cv = λv

(15)

where v are the eigenvectors and λ are the eigenvalues. The eigenvectors are orthonormal. The eigenvalues show the significance of the eigenvectors. A two dimensional examples obtained from Figure 20 in which the biggest component is clearly the direction with the best variance. The second component is shown perpendicular to the first component.

Face Recognition using Support Vector Machine The classification is the last stage for face recognition, where the support vector machine (SVM) is used as the classifier in this study. SVM is selected due to its superiority of strong classification ability for small samples and high dimension data (Li & Lin, 2018). SVM is supervised learning algorithm that maximizes separation between classes by constructing a hyperplane (Cortes & Vapnik, 1995). Figure 21 shows an optimal hyperplane which separates the data points into positive class and negative class. 24

Development of Class Attendance System Using Face Recognition

Figure 20. A simple example of the principal components of a set of two dimensional data is shown (Worsey, 2016)

Figure 21. A two state classification (Cortes et al., 1995)

25

Development of Class Attendance System Using Face Recognition

To classify a sample as being either positive or negative class, a decision rule given by

TRUE  positive  w  u  b  0   FALSE  negative 

(16)

is applied, where u is an unclassified sample, w is perpendicular to the decision boundary and scalar b defines the boundary. To decide which side of the boundary the unclassified sample resides, u is projected onto a vector that is perpendicular to the boundary. Initially, the vector w and the corresponding scalar b are in unknown conditions to describe the maximum boundary. To find the extrema, the Lagrange multiplier is applied. Its partial derivatives are solved by defining a set of constraints. The constraints are derived using: yi(w∙xi + b) – 1>=0

(17)

and yi(w∙xi + b) – 1=0

(18)

where yi is a positive sample xi for +1 and negative sample for −1. Those values of y that equal to zero in the constraint are samples that reside at the edges closest to the boundary. When the boundary edges reside by both positive x+ and negative x_ sample, the width of the boundary is computed by projecting the difference between x+ and x_ unto a unit normal version of w. In fact, this simplifies to 2/∥w∥ and the minima of the magnitude of w which maximizes the width needs to be identified. This can be done using the Lagrange multiplier. The Lagrangian L is described by

L

1 2 w    i  yi  w  xi  b   1 2

(19)

L  w    i yi xi  0 w

(20)

L    i yi  0 w

(21)

26

Development of Class Attendance System Using Face Recognition

where the partial derivatives for both w and b can be solved. The vector w is a linear sum of samples of x where the Lagrange multiplier α is not zero. The extremum is dependent on the dot product of pairs from the sample space by substituting the Equation of 3.21 and 3.22 into the Lagrange, as:

w    i yi xi

(22)

 y

(23)

i

i

0

DATA ANALYSIS AND DISCUSSION In this section, the results obtained from the developed face recognition monitoring attendance system were discussed. The developed system uses histogram of oriented gradients (HOG) as the feature extraction method and support vector machine (SVM) is selected as classifier. The Viola Jones algorithm is used to detect the face in the image. The principal component analysis (PCA) is used to reduce the dimension of the data.

Face Detection Face detection is the first important step to deploy this attendance monitoring system, in which Viola Jones algorithm is utilized in this regard. Prior to face detection, the face images are resized to smaller size (150x150) for fast computation, and are converted from RGB images to grayscale images. Figure 22 presents the original image, and the converted grayscale face images. The original image is in the size of 3024x4032 and is converted into grayscale image in the size of 520x600. The final image that used for feature extraction has been resized to 150x150. Viola Jones algorithm uses cascade object detector to detect the faces, mouth, noses, eyes or upper body. In this study, frontal face has been selected for the cascade object detector in Viola Jones algorithm. Figure 23 shows the detected frontal face using Viola Jones algorithm. As been discussed in the previous section, during image acquisition, the face images are taken from different angles, which is 90° to -90°, including frontal face looking upside and downside. In addition, different facial expressions, and different lighting conditions are considered, as illustrated in Figure 10. As illustrated in Figure 24, among 588 acquired face images, the face detection using Viola Jones algorithm could not work for 82 images – mostly associated with images with side 27

Development of Class Attendance System Using Face Recognition

Figure 22. Original image (left) and the grayscale image (right)

Figure 23. The face was detected successfully using Viola Jones algorithm

faces and some of 45° faces. This is due to frontal face detector is used in Viola Jones algorithm for face detection and thus, the effectiveness to detect side face is less promising in this case. It is pertinent to note that changing the cascade object detector from frontal face to mouth, noses, eyes or upper body detector does not solve the aforesaid limitation. Figure 25 presents some example of face images that the Viola Jones algorithm fails to detect the face.

28

Development of Class Attendance System Using Face Recognition

Figure 24. The success rate of face detection using Viola Jones algorithm

Figure 25. Example of face images that the Viola Jones algorithm fails to detect the face

29

Development of Class Attendance System Using Face Recognition

Since development of a fully automated attendance monitoring system is the main focus of this study, cropping the faces manually for those images which could not be handled using Viola Jones algorithm is not feasible in this case. As such, for these images, morphological processing of edge detection, contrast enhancement, noise removal, skeletonization and image segmentation is performed. Subsequently, the face is identified as the largest object in the image and is extracted. Figure 26 shows the detected side face using this additional approach.

Feature Extraction by Using Histogram of Oriented Gradients After the face detection process has been completed, the following step is to extract the features of each image using HOG. For each image, the extracted feature is in the dimension of 1x10404. Combing all extracted features from all 588 images, the raw features are in the dimension of 588 x 10404, meaning the number of observations is 588 while the number of variables or features is 10404. It can be observed that the dimension of features is higher than the number of observations. Hence, the dimension reduction using the PCA is required in order to address the computational complexity. From the dimension reduction, the explained variance in PCA shows that only the first 14 principal components (PCs) explain more than 1% of the variance. Explained variance in the PCA means the percentage of the variance explained by each principal component. For the PCs with less than 1% of variance, it can be regarded as less contributing to the variability, meaning that not a significant feature to be considered to distinguish among different observations. Figure 26. The successful detected side face using additional morphological operation

30

Development of Class Attendance System Using Face Recognition

Hence, first 14 PCs are selected in this study for the face recognition using SVM subsequently. The input to SVM has been reduced to the dimension of 588×14, comparing to 588 x 10404 originally.

Face Recognition Using Support Vector Machine Following from the dimension reduction using PCA, face recognition using SVM classifier is performed. In this case, the 588 images are separated into training samples and testing samples, in which the training samples are used to build the SVM classification model, while the testing samples are used to evaluate the predictive capability of the developed SVM classifier. The 588 face images of 33 students are acquired in this study. For each individual, two face images are randomly selected as the testing samples, while the remaining are used as the training samples. Hence, the training data in the dimension of 522x14 and testing data in the dimension of 66x14 are obtained. Initially, all 14 PCs are used as the input of SVM. However, it has been found that the recognition accuracy is less satisfying, with 55% of accuracy is attained. Hence, further analysis on the effect of the number of PCs with the classification accuracy is conducted. The number of PCs is varied from 1 to 14, and the obtained recognition accuracy is summarized in Figure 27. As shown in this figure, the highest face recognition accuracy is achieved, when the SVM classifier is trained on 9 PCs. It can be seen that the classification accuracies are increasing from 1 PC to 9 PCs. After the maximum classification accuracy achieved at 9 PCs, the accuracy starts to decline, implying that adding more PCs does not improve the SVM predictive competence. From here, the SVM with 9 PCs as the input is selected as the face recognition model in the following process. Figure 27. The effect of using different number of PCs with the recognition accuracy

31

Development of Class Attendance System Using Face Recognition

Design of the Graphical User Interface for Race Recognition Attendance Monitoring System Following from the training and testing process of SVM using different number of PCs as the input, the best model of SVM with 9 PCs has been selected and constructed as the face recognition model in the attendance monitoring system. Subsequently, the GUI of the attendance monitoring system is developed using MATLAB software. When the user run the system, the first shown window is as illustrated in Figure 28. Figure 29 shows the attendance monitoring system interface developed using MATLAB software, while Figure 28 is the resulted interface during execution. The attendance monitoring system window shown in Figure 28 is the main part of the developed attendance system, as it will classify the captured face of student, and try to match it with the available database using the developed SVM classifier, and display his/her name if there is a match of face. As shown in Figure 29, there are 2 axes on both sides of the window. The axes 1 on the left hand side is to display the live video taken using the built in webcam of computer, while the axes 2 is to display the image captured from webcam. There are two pushbuttons been designed, which are snap pushbutton with caption ‘Take Your Picture’ and namelist pushbutton with caption ‘Generate Table’. When the snap pushbutton is activated, it will capture the current frame of the video in axes 1, and display the captured image in axes 2. Subsequently, all the image process, feature extraction, dimension reduction and face recognition using the predefined SVM classifier are performed. The identified student’s ID is then displayed in the static textbox, located Figure 28. Window for monitoring attendance system

32

Development of Class Attendance System Using Face Recognition

Figure 29. Interface for monitoring attendance system

at the top of the window. The namelist pushbutton is to generate the excel file for attendance recording purposes. Figure 30 shows the window of the monitoring attendance system with the displayed name. The red colour rectangular in axes 2 is the detected face using the cascade object detector. All preprocessing methods are run at background. When the face is matched with the database, the name will be displayed, for example, the “id16” shown in Figure 30.

FUTURE RESEARCH DIRECTIONS From the results and analysis, the obtained face recognition accuracy of 62% using SVM with 9 PCs is less satisfying. As such, several recommendations are proposed in order to increase the predictive accuracy of SVM. During the image acquisition, it has been found that the position and the pose of the student are critical problems that need to be dealt with. The face detection using Viola Jones algorithm works effectively for front face, but not for side face. Hence, it is recommended that in future work, only the images with frontal face are considered, with the assumption that student will stand still in front the system during image acquisition. Furthermore, the lighting condition during image acquisition is a critical problem that may affect the results. The images in this study are captured in different places and under different lighting conditions. Some images are taken under

33

Development of Class Attendance System Using Face Recognition

Figure 30. Window of the monitoring attendance system with matched ID

the natural light, and glare been induced in the images. In future, the images of all students should be captured under stable lighting condition and in the same venue. From the obtained results, it seems that using HOG for feature extraction needs more work since only 43.34% variance been explained by the PCA. In future, it is suggested that feature extraction using HOG can be replaced with Speeded Up Robust Features (SURF). This is due to from literature review, it has been shown that the face recognition rate using the FERET database for SURF is up to 96%.

CONCLUSION This study proposes an attendance monitoring system that automatically records student’s name when the matched face is detected successfully. The automated student attendance system using face recognition technique has been successfully developed. However, due to the frontal face detector is utilized in the Viola Jones algorithm for face detection, only images with frontal face are detected correctly. The detection of the side face and 45° face are less promising. From the results obtained, the first 14 PCs only explain 43.34% of variance, and the face recognition accuracy of only 62% is attained, by using SVM with 9 PCs, which warrants further improvement.

34

Development of Class Attendance System Using Face Recognition

REFERENCES Abdi, H., & Williams, L. J. (2010). Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4), 433–459. doi:10.1002/ wics.101 Alia, M. A., Tamimi, A. A., & Al-Allaf, O. N. (2013). Integrated system for monitoring and recognizing students during class session. The International Journal of Multimedia & Its Applications, 5(6), 45. doi:10.5121/ijma.2013.5604 Bailly-Bailliére, E., Bengio, S., Bimbot, F., Hamouz, M., Kittler, J., Mariéthoz, J., . . . Porée, F. (2003). The BANCA database and evaluation protocol. Paper presented at the International Conference on Audio-and video-based biometric person authentication. doi:10.1007/3-540-44887-X_74 Baumann, J. (1993, Fall). Voice recognition. Human Interface Technology Laboratory. Carlos-Roca, L. R., Torres, I. H., & Tena, C. F. (2018, 8-13 July 2018). Facial recognition application for border control. Paper presented at the 2018 International Joint Conference on Neural Networks (IJCNN). doi:10.1109/IJCNN.2018.8489113 Chaudhari, M., Shanta, S., & Vanjare, G. (2015). A review on Face Detection and study of Viola-Jones method. International Journal of Computer Trends and Technology, 25(1), 54–61. doi:10.14445/22312803/IJCTT-V25P110 Chintalapati, S., & Raghunadh, M. V. (2013). Automated attendance management system based on face recognition algorithms. Paper presented at the 2013 IEEE International Conference on Computational Intelligence and Computing Research. doi:10.1109/ICCIC.2013.6724266 Christopher, M. B. (2016). Pattern recognition and machine learning. Springer-Verlag. Chung, K.-C., Kee, S. C., & Kim, S. R. (1999). Face recognition using principal component analysis of Gabor filter responses. Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV’99 (Cat. No. PR00378). Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. doi:10.1007/BF00994018 Du, G., Su, F., & Cai, A. (2009). Face recognition using SURF features. Paper presented at the MIPPR 2009: Pattern Recognition and Computer Vision.

35

Development of Class Attendance System Using Face Recognition

Gao, W., Cao, B., Shan, S., Zhou, D., Zhang, X., Zhao, D., & Al, S. (2004). The CAS-PEAL large-scale Chinese face database and evaluation protocols. Technique Report No. JDL-TR_04_FR_001, Joint Research & Development Laboratory, CAS. Huang, P., Lai, Z., Gao, G., Yang, G., & Yang, Z. (2016). Adaptive linear discriminant regression classification for face recognition. Digital Signal Processing, 55, 78–84. doi:10.1016/j.dsp.2016.05.001 Jain, A. K., Flynn, P., & Ross, A. A. (2007). Handbook of biometrics. Springer Science & Business Media. Jain, A. K., Ross, A., & Prabhakar, S. (2004). An introduction to biometric recognition. IEEE Transactions on Circuits and Systems for Video Technology, 14(1), 4–20. doi:10.1109/TCSVT.2003.818349 Jamil, N., & Sa’Dan, S. A. (2014). Automated face annotation for personal photo management. Paper presented at the 2014 International Conference on Computational Science and Technology (ICCST). doi:10.1109/ICCST.2014.7045176 Jingyu, Q. (2018). A Survey on Sentiment Classification in Face Recognition. Journal of Physics: Conference Series, 960(1), 012030. Jolliffe, I. T. (2002). Principal component analysis and factor analysis. Principal Component Analysis, 150-166. Joseph, J., & Zacharia, K. (2013). Automatic attendance management system using face recognition. International Journal of Science and Research, 2(11), 328–330. Kar, N., Debbarma, M. K., Saha, A., & Pal, D. R. (2012). Study of implementing automated attendance system using face recognition technique. International Journal of Computer and Communication Engineering, 1(2), 100. Klokova, A. (2010). Comparison of various biometric methods. In Interactive Multimedia Systems. University of Southampton. Li, X.-Y., & Lin, Z.-X. (2018). Face Recognition Based on HOG and Fast PCA Algorithm. Academic Press; doi:10.1007/978-3-319-68527-4_2 Liu, C., & Wechsler, H. (2003). Independent component analysis of Gabor features for face recognition. [PubMed]. IEEE Transactions on Neural Networks, 14(4), 919–928. doi:10.1109/TNN.2006.875987 Mahbub, U., Sarkar, S., & Chellappa, R. (2019). Partial face detection in the mobile domain. Image and Vision Computing, 82, 1–17. doi:10.1016/j.imavis.2018.12.003 Martinez, A. M. (1998). The AR face database. CVC Technical Report24. 36

Development of Class Attendance System Using Face Recognition

Oh, S. J., Benenson, R., Fritz, M., & Schiele, B. (2018). Person Recognition in Personal Photo Collections. [PubMed]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–1. doi:10.1109/TPAMI.2018.2877588 Patel, T., & Shah, B. (2017). A survey on facial feature extraction techniques for automatic face annotation. Paper presented at the 2017 International Conference on Innovative Mechanisms for Industry Applications (ICIMIA). doi:10.1109/ ICIMIA.2017.7975607 Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000). The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1090–1104. doi:10.1109/34.879790 Phillips, P. J., Wechsler, H., Huang, J., & Rauss, P. J. (1998). The FERET database and evaluation procedure for face-recognition algorithms. Image and Vision Computing, 16(5), 295–306. doi:10.1016/S0262-8856(97)00070-X Purandare, V., & Talele, K. (2014). Efficient heterogeneous face recognition using scale-invariant feature transform. Paper presented at the 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA). doi:10.1109/CSCITA.2014.6839277 Réda, A., & Aoued, B. (2004). Artificial neural network-based face recognition. Paper presented at the Control, Communications and Signal Processing, 2004. First International Symposium on. doi:10.1109/ISCCSP.2004.1296323 Saini, R., & Rana, N. (2014). Comparison of various biometric methods. International Journal of Advances in Science and Technology, 2(1), 24–30. Shilwant, D. S., & Karwankar, A. (2012). Student Monitoring By Face Recognition System. International Journal of Electronics Communication and Soft Computing Science & Engineering, 2(2), 24. Shu, C., Ding, X., & Fang, C. (2011). Histogram of the oriented gradient for face recognition. Tsinghua Science and Technology, 16(2), 216–224. doi:10.1016/S10070214(11)70032-3 Szczuko, P., Czyżewski, A., Hoffmann, P., Bratoszewski, P., & Lech, M. (2019). Validating data acquired with experimental multimodal biometric system installed in bank branches. Journal of Intelligent Information Systems, 52(1), 1–31. doi:10.100710844-017-0491-2

37

Development of Class Attendance System Using Face Recognition

Turk, M. A., & Pentland, A. P. (1991). Face recognition using eigenfaces. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. doi:10.1109/CVPR.1991.139758 Vinay, A., Shekhar, V. S., Rituparna, J., Aggrawal, T., Murthy, K. B., & Natarajan, S. (2015). Cloud-based big data analytics framework for face recognition in social networks using machine learning. Procedia Computer Science, 50, 623–630. doi:10.1016/j.procs.2015.04.095 Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. doi:10.1109/ CVPR.2001.990517 Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154. doi:10.1023/B:VISI.0000013087.49260.fb Worsey, J. N. (2016). Face recognition in an unconstrained environment for monitoring student attendance. Academic Press. Zafar, U., Ghafoor, M., Zia, T., Ahmed, G., Latif, A., Malik, K. R., & Sharif, A. M. (2019). Face recognition with Bayesian convolutional networks for robust surveillance systems. Eurasip Journal on Image and Video Processing, 2019(1). doi:10.118613640-019-0406-y Zhang. (2013). Automated biometrics: Technologies and systems (Vol. 7). Springer Science & Business Media. Zhu, Y., Zhu, C., & Li, X. (2018). Improved principal component analysis and linear regression classification for face recognition. Signal Processing, 145, 175–182. doi:10.1016/j.sigpro.2017.11.018

ADDITIONAL READING Damale, R. C., & Pathak, B. V. (2018, June). Face Recognition Based Attendance System Using Machine Learning Algorithms. In 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 414-419). IEEE. doi:10.1109/ICCONS.2018.8662938 Devan, P. A. M., Venkateshan, M., Vignesh, A., & Karthikraj, S. R. M. (2017). Smart attendance system using face recognition. Advances in Natural and Applied Sciences, 11(7), 139–145. 38

Development of Class Attendance System Using Face Recognition

Elias, S. J., Hatim, S. M., Hassan, N. A., Latif, L. M. A., Ahmad, R. B., Darus, M. Y., & Shahuddin, A. Z. (2019). Face recognition attendance system using Local Binary Pattern (LBP). Bulletin of Electrical Engineering and Informatics, 8(1), 239–245. doi:10.11591/eei.v8i1.1439 Guo, G. (2018). Design and Implementation of Smart Campus Automatic Settlement PLC Control System for Internet of Things. IEEE Access : Practical Innovations, Open Solutions, 6, 62601–62611. doi:10.1109/ACCESS.2018.2877023 Hossain, M. I. A., Hossain, I., Banik, M., & Alam, M. A. (2018, June). IOT based Autonomous Class Attendance System using Non-Biometric Identification. In 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR) (pp. 268-271). IEEE. Jain, A. K., Nandakumar, K., & Ross, A. (2016). 50 years of biometric research: Accomplishments, challenges, and opportunities. Pattern Recognition Letters, 79, 80–105. doi:10.1016/j.patrec.2015.12.013 Khaleel, A., & Yussof, S. (2016). An investigation on the viability of using iot for student safety and attendance monitoring In Iraqi primary schools. Journal of Theoretical and Applied Information Technology, 85(3), 394. Kumar, N. A., Swarnalatha, P., Chowdary, P., Naidu, J. K., & Kumar, K. S. (2019). Smart Attendance Marking System using Facial Recognition. Research Journal of Science and Technology, 11(2), 101–108. doi:10.5958/2349-2988.2019.00016.0 Kumar, P. M., Gandhi, U., Varatharajan, R., Manogaran, G., Jidhesh, R., & Vadivel, T. (2017). Intelligent face recognition and navigation system using neural learning for smart security in Internet of Things. Cluster Computing, 1–12. Kutty, N. M., & Mathai, S. (2017). Face Recognition–A Tool for Automated Attendance System. International Journals of Advanced Research in Computer Science and Software Engineering, 7(6), 334–336. doi:10.23956/ijarcsse/V7I6/0268 Labati, R. D., Genovese, A., Muñoz, E., Piuri, V., Scotti, F., & Sforza, G. (2016). Computational intelligence for biometric applications: A survey. International Journal of Computing, 15(1), 40–49. Li, C., Wei, W., Li, J., & Song, W. (2017). A cloud-based monitoring system via face recognition using Gabor and CS-LBP features. The Journal of Supercomputing, 73(4), 1532–1546. doi:10.100711227-016-1840-6

39

Development of Class Attendance System Using Face Recognition

Limkar, S., Jain, S., Kannurkar, S., Kale, S., Garsund, S., & Deshpande, S. (2019). iBeacon-Based Smart Attendance Monitoring and Management System. In First International Conference on Artificial Intelligence and Cognitive Computing (pp. 637-646). Springer, Singapore. doi:10.1007/978-981-13-1580-0_61 Mohammed, K., Tolba, A. S., & Elmogy, M. (2018). Multimodal student attendance management system (MSAMS). Ain Shams Engineering Journal, 9(4), 2917–2929. doi:10.1016/j.asej.2018.08.002 Olagunju, M., Adeniyi, E. A., & Oladele, T. O. (2018). Staff Attendance Monitoring System using Fingerprint Biometrics. International Journal of Computers and Applications, 179(21), 8–15. doi:10.5120/ijca2018916370 Pathak, P., Paratnale, M. M., Khairnar, D. R., Yadhav, P. V., & Wadgaonkar, P. R. (2016). Student Attendance Monitoring System Via Face Detection and Recognition System. International Journal of Science Technology & Engineering, 2(11). Prakash, C., Kumar, R., & Mittal, N. (2018). Recent developments in human gait research: Parameters, approaches, applications, machine learning techniques, datasets, and challenges. Artificial Intelligence Review, 49(1), 1–40. doi:10.100710462-0169514-6 Priya, S., & Mukesh, R. (2018). Multimodal Biometric Authentication using Back Propagation Artificial Neural Network. International Journal of Simulation--Systems, Science & Technology, 19(6). Rewari, S., Shaha, A., & Gunasekharan, S. (2016). Facial Recognition Based Attendance System. Journal of Image Processing & Pattern Recognition Progress, 3(2), 43–49. Rjeib, H. D., Ali, N. S., Al Farawn, A., Al-Sadawi, B., & Alsharqi, H. (2018). Attendance and information system using RFID and web-based application for academic sector. International Journal of Advanced Computer Science and Applications, 9(1). Samy, S. S., & Parthiban, L. (2018). A Novel Feature Extraction Approach Using Principal Component Analysis and Quantum behaved Particle Swarm OptimizationSupport Vector Networks for Enhancing Face Recognition. Journal of Computational and Theoretical Nanoscience, 15(9-10), 3012–3016. doi:10.1166/jctn.2018.7582 Sayeed, S., Hossen, J., Kalaiarasi, S. M. A., Jayakumar, V., Yusof, I., & Samraj, A. (2017). Real-time face recognition for attendance monitoring system. Journal of Theoretical & Applied Information Technology, 95(1).

40

Development of Class Attendance System Using Face Recognition

Surekha, B., Nazare, K. J., Raju, S. V., & Dey, N. (2017). Attendance recording system using partial face recognition algorithm. Intelligent techniques in signal processing for multimedia security (pp. 293–319). Cham: Springer; doi:10.1007/9783-319-44790-2_14 Vidya, M. S., & Arul, K. (2016). Automated Attendance System Through Eigen Faces Using Image Processing. International Journal of Informatics and Communication Technology (IJ-ICT), 111-118. Wagh, P., Thakare, R., Chaudhari, J., & Patil, S. (2015, October). Attendance system based on face recognition using eigenface and PCA algorithms. In 2015 International Conference on Green Computing and Internet of Things (ICGCIoT) (pp. 303-308). IEEE. doi:10.1109/ICGCIoT.2015.7380478 Zaman, F. H. K., Sulaiman, A. A., Yassin, I. M., Tahir, N. M., & Rizman, Z. I. (2017). Development of mobile face verification based on locally normalized Gabor wavelets. International Journal on Advanced Science. Engineering and Information Technology, 7(4), 1198–1205.

KEY TERMS AND DEFINITIONS Biometric Verification: It’s an application which conducts measurement and identification based on physical or behavioral characteristics about a person (such as fingerprint or voice patterns), that can be applied to differentiate personal identities between individuals. Face Detection: Face detection attempts to identify a human face in digital images. Face detection is not the same as face recognition. Face Recognition: Face recognition involves an analysis of facial features, where in general, a computer system is utilized to identify an individual automatically from a digital image or video framework. Image Acquisition: In image processing, image acquisition is an action of retrieving image from an external source for further processing. It’s always the foundation step in the workflow since no process is available before obtaining an image. Principal Component Analysis: Principal component analysis is a statistical approach that transforms a set of observations of possibly correlated variables into a set of linearly uncorrelated principal components using orthogonal transformation.

41

Development of Class Attendance System Using Face Recognition

Support Vector Machine: Support vector machine is a supervised learning model with an associated learning algorithm that primarily used for classification. Viola Jones Algorithm: Viola-Jones algorithm is an object detection framework, with primary concern to solve the problem of face detection. Four stages are involved in the algorithm, namely, Haar feature selection, creating an integral image, Adaboost training, and cascading classifier.

42

43

Chapter 2

Deep Learning in Computational Neuroscience Sanjay Saxena https://orcid.org/0000-0002-8288-1010 Department of Computer Science and Engineering, IIIT Bhubaneswar, Bhubaneswar, India Sudip Paul https://orcid.org/0000-0001-9856-539X North-Eastern Hill University, India Adhesh Garg Department of Computer Science and Engineering, IIIT Bhubaneswar, Bhubaneswar, India Angana Saikia North-Eastern Hill University, India Amitava Datta The University of Western Australia, Australia

ABSTRACT Computational neuroscience is inspired by the mechanism of the human brain. Neural networks have reformed machine learning and artificial intelligence. Deep learning is a type of machine learning that teaches computers to do what comes naturally to individuals: acquire by example. It is inspired by biological brains and became the essential class of models in the field of machine learning. Deep learning involves several layers of computation. In the current scenario, researchers and scientists around the world are focusing on the implementation of different deep models and architectures. This chapter consists the information about major architectures of DOI: 10.4018/978-1-7998-0182-5.ch002 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Deep Learning in Computational Neuroscience

deep network. That will give the information about convolutional neural network, recurrent neural network, multilayer perceptron, and many more. Further, it discusses CNN (convolutional neural network) and its different pretrained models due to its major requirements in visual imaginary. This chapter also deliberates about the similarity of deep model and architectures with the human brain.

INTRODUCTION Computational neuroscience is the area of investigation in which different mathematical tools and theories are implemented to investigate brain function. It achieves human-level execution in specific undertakings, and new analyses demonstrate that they can do. It is designed to empower the shallow models, enhance their performance. This type of networks addresses the fundamental question of how the brain processes the information, how a nervous system processes the information given to it, audio video or in any other form. These models commonly outline stimuli to neural response or potentially neural to functional responses. Feedforward neural network and recurrent neural networks architecture have gain popularity over the recent years. The biological brain inspires these deep learning neural networks. Their reflections add to their computational productivity, empowering to perform complex accomplishments of insight, running from perceptual errands (for example visual item and sound-related discourse acknowledgment) to intellectual errands (for example machine interpretation), and on to motor control errands (for example controlling a robot arm). Over the years several models were designed to process the information with the of revolutionizing artificial intelligence. Neural networks inspired by the brain are at the center of this development. Deep neural networks have given better results than any of the other existing models. Neural networks differ from the biological neural network, but they exhibit structural similarity. For example, convolutional neural networks used in computer vision inspired by the natural view and processes segments of the given data, its neurons exhibit spatial restricted receptive fields that increase the invariance, size, and complexity along with the different levels and similar feature detector exists. These models are very less complicated as compared to the actual biological neural network, but they possess a certain degree of similarity, which is providing an edge in the current computation scenario. One of the major problems faced by the researchers concerning neural networks is, what the system is precisely doing. The interface can predict the data, but it is like a black box scenario where exactly it is too complex to understand the functioning of the neural network (Aggarwal, C. C.,2018), (Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Webster, D. R., 2016). Historically simpler models present 44

Deep Learning in Computational Neuroscience

were easy to understand with a degree of freedom nevertheless since the brain has over billions of parameters and is a very complex structure. We have drawn insights from such complex models by considering a high level of abstraction. It leads us to another argument that states that neural networks are too abstract from the biological reality. A model can be more straightforward and sophisticated at the same time concerning different frames. The main questions researchers are trying to solve to which feature to model and which to omit from the biological brain while deigning network. The merit of these models is not dependent on how exactly they replicate the natural brain but on how accurately they can provide insights into the working of the mind at a given scenario (Waoo, N., Kashyap, R., & Jaiswal, A.,2010). This chapter summarizes the different architecture of deep models inspired by neuroscience. Further, it will draw the attention of the researchers who are working on the field of deep neural network.

SIMILARITY OF DEEP NEURAL NETWORK AND BRAIN Deep neural networks are a set of algorithms that are modeled loosely after the human brain. They have an input layer, an output layer, and a minimum of one hidden layer in between. Each layer aims to extract features from the input to recognize patterns. The general structure of the deep neural network given in figure 1. The hidden layer sums up the inputs, taking their respective weights into account, and makes a non-linear decision to activate a feature by identifying it and sends the activation calculation further to another layer of neurons as input. In terms of the brain, a neuron is the fundamental processing unit of mind as well as a deep neural network. Inside the brain, these are the cells responsible for every action/function in the body with just simple inputs from the external world. The general structure of the brain neuron shown in figure 2, a brain neuron has three main components at a high level: dendrites, cell body (soma), and axon. Dendrites act as the input mechanism as they receive connections from other neurons. The cell body or soma is the calculation mechanism in which receives inputs from all dendrites and calculates a response such as to whether spike a signal or not. The axon carries out the output mechanism as a delivery agent of the message to the other dendrite of the next neuron layer if the soma decides to fire an output. The point at which the axon joins the dendrite of another neuron is called a synapse, and this connection is called a synaptic connection. In brain neurons, the signals transfer from one neuron to another with the help of chemical transmitters released by synapse causing the variation of potential in the soma of another neuron. The neuron does not carry any signal in the resting state. In the given an example below shown in figure 3, a minimum of 15 mV (-55 45

Deep Learning in Computational Neuroscience

Figure 1. Deep neural network

- -70) is required to fire an output. It called the threshold, as shown in figure 3. It means that the total ‘sum’ of all the input signals received must cross the limit for the message to pass through that neuron. On crossing the threshold, the impulse reaches the other dendrite through a synapse, and this process goes on (“Conduction of Nerve Impulse,” 2019), (“CS231n Convolutional Neural Networks for Visual Recognition”, 2019). Figure 2. Structure of the brain neuron

46

Deep Learning in Computational Neuroscience

Figure 3. The response of a single neuron to an incoming signal from its dendrite

This process is very similar to DNN (Deep Neural Networks) in which the inputs added before being passed to further neurons. Other such similarities include synaptic weight, neural backpropagation, and plasticity in the learning method. Synaptic weight in DNNs refers to the strength of the connection between two nodes belonging to consecutive layers of the neural network. Its magnitude can be determined by the intensity of propagation of the signal through the axon, density of neurotransmitter vesicle and receptors, as well as the no. of synapses (i.e., the connections in between axon and dendrites. In biological neurons, as the signal passes through dendrites, the dendrites have some potential charge or intensity values which move from one cell to another. Deep neural networks these connection strengths known as synaptic weights. In DNN, each neuron in one layer connected to a neuron in the next layer through this synaptic weight. It has some value associated with the connection link that informs about the strength of the connection and the intensity of that feature. Synaptic plasticity is the term coined to the occurrence of changes in synaptic weight. Another similarity is the neural backpropagation. It is the phenomenon in which after the action potential of a neuron fires a voltage spike through the axon (feedforward propagation) another impulse is produced from the cell body and spreads towards the tip of dendrites from which the original input signal originated. Though the backpropagating action potential can cause alteration in the synaptic weights, yet there is no such process for an error signal to move through multiple neural layers, as seen in the computer backpropagation algorithms. The brain learns and memorizes using local methods like Hebbian learning or STDP (Spike Timing Dependent Plasticity). These methods strengthen the connections between neurons 47

Deep Learning in Computational Neuroscience

that frequently transfer signals. A fundamental principle in training the deep neural networks is the concept of plasticity. During backpropagation in DNNs, the weights in between two consecutive neural layers in altered in every propagation based on the inputs and their corresponding outputs.

LITERATURE REVIEW Major Architectures of Deep Network Several architectures implemented in deep learning. This section describes different deep learning architectures in the last few years.

Convolutional Neural Network (CNN) A convolutional neural network (CNN) is a combination of multiple convolutional layers, pooling, or fully connected layers. Variations of a multilayer perceptron are used to design the system. Convolutional operations performed at convolution layers to the input data passing the result to the next layer. With much fewer parameters, convolution operation allows the network to be more profound. Image and speech recognition applications show outstanding results in convolutional neural networks. Semantic parsing, paraphrase detection are also the applications of CNN’s.

Multilayer Perceptron (MLP) Multilayer perceptron network (MLP) has three or more layers. The general structure of the MLP shown in figure 5. It classifies data that is not linearly separable using mainly nonlinear activation function that is hyperbolic tangent or logistic function. It is a fully connected network, i.e., every node in a layer connects to each node in Figure 4. Convolutional neural network Ma, Y., Xiang, Z., Du, Q., & Fan, W. (2018), (Peng, M., Wang, C., Chen, T., & Liu, G., 2016)

48

Deep Learning in Computational Neuroscience

Table 1. Work done in last 5 years YEAR

AUTHOR

WORK DONE

Santiago A. et.al

They modeled spiking activity in the primary visual cortex of monkeys using deep convolutional neural networks (CNN), which have been successful in computer vision. CNN’s were directly trained to fit the data, and have used CNN’s trained to solve a high-level task. With these approaches, they were able to outperform previous models and improve the state of the art in predicting the responses of early visual neurons to natural images (Cadena, S.A., Denfield, G.H., Walker, E.Y., Gatys, L.A., Tolias, A.S., Bethge, M. and Ecker, A.S., 2019).

2019

Ullman. S

He discovered that some additional aspects of brain circuitry could supply clues for guiding network models toward broader aspects of cognition and general AI (Artificial Intelligence). He concluded that combining the empirical and computational approaches to the problem is likely to benefit in the long run of both neuroscience and AGI(Artificial General Intelligence), and could eventually be a component of a theory of intelligent processing that will apply to both (Ullman, S., 2019).

2018

Pandarinath et al.

They described an artificial neural network model that captures some key aspects of the activity of populations of neurons in the primary motor cortex (Batista, A.P. and DiCarlo, J.J., 2018).

Gruber et. al

In this study, they created non-dynamic visual competitions for humans by briefly presenting the mixtures of two images. They tested feedforward DNNs with similar combinations and examined their behavior and found that both humans and DNNs tend to perceive only one picture when presented with a mixture of two. These findings can be used to both improve DNNs as models, as well as potentially improve their performance by imitating biological behaviors (Gruber, L.Z., Haruvi, A., Basri, R. and Irani, M., 2018).

Dinggang et.al

They found that computational modeling for medical image analysis had a significant impact on both clinical applications and scientific research. Recent progress in deep learning has shed new light on medical image analysis by enabling the discovery of morphological and textural patterns in images solely from data (Shen, D., Wu, G. and Suk, H.I., 2017).

Jordan et. al

They showed that a deep learning algorithm that utilizes multi-compartment neurons help to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in their model receive sensory information and higher order feedback in electrotonically segregated compartments. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons (Guerguiev, J., Lillicrap, T.P., and Richards, B.A., 2017).

Irina et. al

They implemented an unsupervised approach for learning disentangled representations of the underlying factors of variation. By enforcing redundancy reduction, encouraging statistical independence, and exposure to data with transform continuities analogous to those to which human infants are exposed, they obtained a Variational autoencoder (VAE) framework capable of learning disentangled factors. This approach makes few assumptions and works well across a wide variety of datasets. Furthermore, their solution has useful emergent properties, such as zero-shot inference (Higgins, I., Matthey, L., Glorot, X., Pal, A., Uria, B., Blundell, C., Mohamed, S. and Lerchner, A., 2016).

2019

2018

2017

2017

2016

continues on following page

49

Deep Learning in Computational Neuroscience

Table 1. Continued YEAR

2016

2015

AUTHOR

WORK DONE

Migel et.al

They present a method for synthesizing deep neural networks using Extreme Learning Machines (ELMs) as a stack of supervised autoencoders. They tested the technique using standard benchmark datasets for multi-class image classification and showed that the classification error rate could progressively improve with the inclusion of additional auto encoding ELM modules in a stack. This approach simultaneously offers a significantly faster training algorithm to achieve its best performance relative to a single ELM hence offers lower error rates and rapid implementation (Tissera, M.D. and McDonnell, M.D., 2016).

Peter et. al

They analyzed the effects of converting deep ANNs (Analog Neural Network) into SNNs (Spiking Neural Network) concerning the choice of parameters for spiking neurons such as firing rates and thresholds (Tiwari S., Gupta R.K., & Kashyap R. 2019). A set of optimization techniques was presented to minimize performance loss in the conversion process for ConvNets and fully connected deep networks. The presented analysis and optimization techniques boost the value of spiking deep networks as an attractive framework for neuromorphic computing platforms which aimed at fast and efficient pattern recognition (Diehl, P.U., Neil, D., Binas, J., Cook, M., Liu, S.C. and Pfeiffer, M., 2015).

the next layer making the network. For example, speech recognition and machine translation are applications of multilayer perceptron natural language processing (NLP).

Recursive Neural Network (RNN) A recursive neural network (RNN), shown in figure 6, is designed by applying the same set of weights recursively over a structure is one of the simple deep neural networks. It is intended to predict input structures of variable-size, or by traversing given data in topological order, a scalar prediction on it (Kashyap, R., 2019c). In the most straightforward architecture, nodes are combined into parents using nonlinearity like tanh, and a weight matrix that shared across the entire network.

Recurrent Neural Network (RNN) A recurrent neural network (RNN), as shown in figure 7, is a class of a recursive artificial neural network in which neurons make directed cyclic connections. Unlike a feedforward neural network output depends on the previous step’s neuron state and the present inputs also. It has a memory that lets users solve NLP following data problems like handwriting recognition or speech recognition. It can build a relationship between the data most recently processed to the data processed now. They are mainly used in the case of sequential data and can generate new sentences and document summaries as it has a better understanding of the data fed to the system. 50

Deep Learning in Computational Neuroscience

Figure 5. Multilayer perceptron (Guo, W. W., & Xue, H.,2014)

Figure 6. Recursive neural network (Le, P., & Zuidema, W., 2015)

Figure 7. Recursive neural network

51

Deep Learning in Computational Neuroscience

Long Short-Term Memory (LSTM) Long Short-Term Memory (LSTM) was designed to increase the accuracy in models with temporal sequences and their long-range dependencies. It is a specific type of recurrent neural network (RNN) architecture. It is a type of memory element. Within its repetitive component’s activation functions are not used by LSTM, the values stored are not modified, and during training, the gradient tend does not vanish. LSTM, several units are implemented in “blocks.” These blocks generally have three or four “gates” that is the input gate, forget gate, output gate that controls the logistic function drawing the information flow. LSTM as a fundamental element into their products incorporated by companies such as Apple, Amazon, Google, Microsoft, and others.

Sequence-To-Sequence Models A sequence-to-sequence model has of two recurrent neural networks: one to process the input an encoder and one to produce the output a decoder as presented in figure 8. Same or different sets of parameters can be used by the encoder and decoder or by the networks (Shukla R., Gupta R.K., & Kashyap R., 2019). Question answering systems, chatbots, and machine translation mainly use the sequence-to-sequence models. Sequence-to-Sequence models have multi-layer cell structure, which used for reading in Neural Networks study with Sequence to Sequence Learning. The encoder takes the input data sequence, and train model on it then it passes the last state of its recurrent layer as an initial state to the first recurrent layer of the decoder, the input of the decoder is the output sequence. Figure 8. Sequence to sequence models

52

Deep Learning in Computational Neuroscience

CONVOLUTIONAL NEURAL NETWORK (CNN) AND ITS DIFFERENT ARCHITECTURES CNN or Convnet is a distinct type of multi-layer neural networks, intended to identify graphic patterns unswervingly from pixel images with nominal preprocessing. In a convolutional network (ConvNet), there are three types of layers: Convolution layer, Pooling layer, Fully connected layer. As shown in figure 4. Convolution layer: The Convolution layer is the core building block of a Convolutional Network that does most of the computational heavy lifting. The Convolution layer’s parameters consist of a set of learnable filters. Every filter is small spatially (along width and height), but extends through the full depth of the input volume. For example, a typical filter on the first layer of a ConvNet might have size 5x5x3 (i.e., 5 pixels width and height, and three because images have depth 3, the color channels). During the forward pass, we slide (more precisely, convolve) each filter across the width and height of the input volume and compute dot products between the entries of the screen and the input at any position Cui, Z., Yang, J., & Qiao, Y. (2016), Duggal, R., Gupta, A., Gupta, R., Wadhwa, M., & Ahuja, C. (2016), Georgakopoulos, S. V., Iakovidis, D. K., Vasilakakis, M., Plagianakos, V. P., & Koulaouzidis, A. (2016), (Raina, R., Madhavan, A., & Ng, A. Y.,2009). Pooling layer: This layer is used to diminish the size of the inputs and henceforth speed up the calculation. Let suppose a 4 X 4 matrix, as shown below: Table 2. 1

5

6

2

2

9

1

1

2

3

3

6

5

6

2

1

Applying max pooling on this matrix will result in a 2 X 2 output:

Figure 9. Explanation of pooling layer

53

Deep Learning in Computational Neuroscience

Fully Connected Layer: The convolution and pooling layers would only be able to extract features and reduce the number of parameters from the original images (Deng, J., Dong, W., Socher, R., Li, L., Li, K., & Fei-Fei, L., 2009), (Grósz, T., & T., I. N.,2014). However, to generate the final output need to apply a fully connected layer to create an output equal to the number of classes we need. Till now, there are several trained popular CNN architectures are there. By following the transfer learning of the different existing buildings, we can implement in our dataset. Now we are going to give a brief introduction of different networks architectures of CNN (Krizhevsky, A., Sutskever, I., & Hinton, G. E.,2017), (Das, S., & Das, S.,2017). AlexNet​: Test accuracy of 84.6% Toronto’s best deep CNN that won the 2012 ImageNet competition. It is designed using five convolutional layers, max-pooling layers, ReLUs as activation function, three fully-convolutional layers, and dropout. VGG-16 (Visual Geometry Group-16)​: With 92.7% accuracy this Oxford’s model won the 2013 ImageNet competition. It uses a stack of convolution layers with small receptive fields in the first layers instead of few layers with big receptive fields. VGG-19 (Visual Geometry Group-19): ​VGG-19 is a convolutional neural network that trained on more than a million images from the ImageNet database. The system is 19 layers deep and can classify images into 1000 object categories, such as a keyboard, mouse, pencil, and many animals. As a result, the network has learned rich feature representations for a wide range of images. The system has an image input size of 224-by-224. Google Net​: With the accuracy of 93.3% this Google’s network won the 2014 ImageNet competition. It composed of 22 layers and a newly introduced building block called inception module. The module consists of a Network-in-Network layer, a pooling operation, a large-sized convolution layer, and small-sized convolution layer. ResNet: With 96.4% accuracy this Microsoft’s model won the 2016 ImageNet competition. It is well-known due to the introduction of residual blocks and to its depth (152 layers). The remaining blocks address the problem of training a profound architecture by introducing identity skip connections so that sheets can copy their inputs to the next layer. Shuffle Net​: It is an extremely efficient CNN architecture, designed for mobile devices with the computing power of 10–150 MFLOPs (Mega floating-point operations per second). While maintaining accuracy, the Shuffle Net utilizes pointwise group convolution and channel shuffle to reduce computation cost. It manages to obtain lower top-1 error than the Mobile Net system on Image Net classification (Kashyap, R., 2020). Squeeze Net: ​For the same accuracy of AlexNet, SqueezeNet can be three times faster and 500 times smaller. The squeeze module only contains 1x1 filters, which means it works like a fully-connected layer working on feature points in the same position. As its name says, one of its benefits is to reduce the depth of the feature 54

Deep Learning in Computational Neuroscience

map. Reducing thickness means the following 3x3 filters in the expand layer has computation to do. It boosts the speed as a 3x3 filter need as nine times computation as a 1x1 filter. Dense Net: ​DenseNet connects each layer to every other layer in a feed-forward fashion. It has L layers L(L+1)/ 2 direct connections. For each layer, the featuremaps of all other layers used as inputs, and its feature-maps used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. ENet: ​Efficient Neural Network gives the ability to perform pixel-wise semantic segmentation in real-time. ENet is up to 18x faster, requires 75x fewer FLOPs (Floating point operations per second), has 79x fewer parameters, and provides similar or better accuracy to existing models. Enet is the fastest model in semantic segmentation. Inception: ​The Inception network was an essential milestone in the development of CNN classifiers. Before its inception CNN’s just stacked convolution layers deeper and deeper, hoping to get better performance. It performs convolution on input, with three different sizes of filters (1x1, 3x3, 5x5). Additionally, max pooling is also shown (Kashyap, R., & Piersson, A. D., 2018). The outputs are concatenated and sent to the next inception module. Following table illustrate different CNN models, year of establishment, and its accuracy.

Table 3. Summary table of CNN architecture S.N.

Year

CNN

Developed By

Top Accuracy

1

2012

AlexNet

Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton

55%

2

2014

VGG-16

Simonyan and Zisserman

72%

3

2014

VGG-19

Simonyan and Zisserman

71%

4

2014

Inception

Szegedy

80%

5

2014

GoogleNet

Google

68%

6

2015

ResNet

Kaiming

74%

7

2016

SqueezNet

DeepScale, University of California

58% 69%

8

2016

ENet

Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello

9

2017

ShuffleNet

Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, Jian Sun

62%

DenseNet

Gao Huang, Zhuang Liu, Laurens van der Maaten and Kilian Weinberger

75%

10

2017

55

Deep Learning in Computational Neuroscience

Figure 10.

From the above figure, we can see that the accuracy has been gradually increasing in most of the cases with time (Years). Hence, it can deep learn has taken over most of the learning systems in the field of neuroscience due to its high accuracy and efficiency.

NEUROSCIENCE AND ITS APPLICATIONS THROUGH DEEP LEARNING The word Neuroscience is on the application of computation, studying neural codes, dynamics, and various circuits replicating the human brain. In deep learning, neural networks tend to design systems, dynamics, or circuits (LeCun, Bengio & Hinton, 2015). Two recent developments have emerged within machine learning that creates an opportunity to connect these perspectives. They are: 1) Structured architectures: These include systems for attention, recursion, and various forms of short- and long-term memory storage (Kashyap, R., 2019a). 2) The cost functions and training procedures have become more complex and are varied across layers and over time. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism.

56

Deep Learning in Computational Neuroscience

Neuroscience is the study of the nervous system, which includes the brain, spinal cord, and nerves. The brain controls every aspect of the body, from emotion and memory to necessary bodily activities such as movement, breathing, and managing the heartbeat. It is also concerned with the causes and treatment of nervous system disorders (Kashyap, R., 2019b). There are many branches of neuroscience. Since the brain is a complex organ that carries out a variety of functions and the nerves of the rest of the nervous system extend throughout the entire body, a plethora of neuroscience topics exist. The following are some of the main branches of neuroscience (Hassabis, D., Kumaran, D., Summerfield, C. and Botvinick, M., 2017): 1) Affective neuroscience: It is the study of the neural mechanisms involved in emotion, typically through experimentation on animal models. 2) Behavioral neuroscience: It is the application of the principles of biology to the study of genetic, physiological, and developmental mechanisms of behavior in humans and non-human animals. 3) Cellular neuroscience: It is the study of neurons at a cellular level, including morphology and physiological properties. 4) Clinical neuroscience: It is the study of neurons at a cellular level, including morphology and physiological properties. 5) Cognitive neuroscience: It is the study of the biological mechanism underlying cognition. 6) Computational neuroscience: It is the theoretical study of the nervous system. 7) Developmental neuroscience: It studies the processes that generate, shape, and reshapes the nervous system and seeks to describe the cellular basis of neural development to address underlying mechanisms. 8) Neural engineering: It uses engineering techniques to interact with, understand, repair, replace, or enhance neural systems. 9) Systems neuroscience: Systems neuroscience is the study of the function of neural circuits and system. With increasing advancements in technology, neuroscientists can collect data in higher volumes and with more exceptional resolution. Understanding how the brain works are consequently shifting away from the amount and type of data; one can collect data. There has been a growing interest in acquiring this vast volume of data across levels of analysis, measurement techniques, and experimental paradigms to gain more insight into brain function. At multiple stages and levels of neuroscience investigation, deep learning plays an additional role in the analysis of various tools for discovering how the brain works (Ravì, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B. and Yang, G.Z., 2017).

57

Deep Learning in Computational Neuroscience

The artificial neural networks are prominent in machine learning was initially inspired by neuroscience. Neuroscience has continued to play a role in many of the significant developments. These guided by insights into the mathematics of efficient optimization, rather than neuroscientific findings. This field has advanced from simple linear systems to nonlinear networks, to deep and recurrent nets. Methods of training have improved to include momentum terms, better weight initializations, conjugate gradients, and so forth, evolving to the current breed of networks. These developments have a connection to neuroscience. Some of the applications of neuroscience in deep learning are as follows (Haines, D.E. and Mihailoff, G.A.,2017): 1) Detection of various neuronal disorders like Parkinson’s disease, Alzheimer’s diseases, etc. 2) DNA (Deoxyribonucleic Acid) microarray analysis. 3) Immunology. Some of the primary roles of deep learning in neuroscience are: 1) 2) 3) 4)

Creating a solution to engineering problems. Identifying predictive variables. Setting the benchmark for brain models. Serving itself as a model for the brain.

The Figure 11 below demonstrates the roles: Figure 11. Role of learning in neuroscience (“Neuroscience and its applications through deep learning - Google Search,” 2019)

58

Deep Learning in Computational Neuroscience

CONCLUSION Deep learning has transformed machine learning and artificial intelligence expressively. Recently, it moved back into computational neuroscience. Deep learning offers a fascinating novel framework that permits computational neuroscientists to report essential queries about the computation of the brain. In this chapter, we have gone through in-depth learning, its different models, and different Convolutional neural network architectures. Moreover, we have discussed the similarities between human biological brains with in-depth learning architecture. After going through the several deep models, we have concluded that each model has its individuality and can use as per the essential requirements. For example, Multilayer perceptron is appropriate for classification forecast complications wherever inputs are allocated a class or label, whereas CNN is designed to plot image data to an output variable. Further, we have also discussed different architectures of CNN such as Alex Net, VGG16, VGG19 and many more and we have concluded classic CNN architectures were encompassed merely of different stacked convolutional layers, modern designs reconnoiter novel and advanced ways for constructing convolutional layers in a mode which permits for more effective learning.

REFERENCES Aggarwal, C. C., (2018). Training Deep Neural Networks. Neural Networks and Deep Learning, 105-167. doi:10.1007/978-3-319-94463-0_3 Batista, A. P., & DiCarlo, J. J. (2018). Deep learning reaches the motor system. Nature Methods, 15(10), 772–773. doi:10.103841592-018-0152-6 PMID:30275586 CS231n Convolutional Neural Networks for Visual Recognition. (2019). Retrieved from http://cs231n.github.io/convolutional-networks/ Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., & Ecker, A. S. (2019). Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Computational Biology, 15(4), e1006897. doi:10.1371/journal.pcbi.1006897 PMID:31013278 Conduction of Nerve Impulse. (2019). Retrieved from http://simplebiologyy.blogspot. com/2014/08/conduction-of-nerve-impulse.html

59

Deep Learning in Computational Neuroscience

Cui, Z., Yang, J., & Qiao, Y. (2016). Brain MRI segmentation with patch-based CNN approach. 2016 35th Chinese Control Conference (CCC). doi:10.1109/ chicc.2016.7554465 Das, S., & Das, S. (2017, November 16). CNN Architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, and more... Retrieved from https://medium.com/@sidereal/ cnns-architectures-lenet-alexnet-vgg-googlenet-resnet-and-more-666091488df5 Deng, J., Dong, W., Socher, R., Li, L., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. 10.1109/CVPR.2009.5206848 Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S. C., & Pfeiffer, M. (2015, July). Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE. 10.1109/IJCNN.2015.7280696 Duggal, R., Gupta, A., Gupta, R., Wadhwa, M., & Ahuja, C. (2016). Overlapping cell nuclei segmentation in microscopic images using deep belief networks. Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing - ICVGIP 16. 10.1145/3009977.3010043 Georgakopoulos, S. V., Iakovidis, D. K., Vasilakakis, M., Plagianakos, V. P., & Koulaouzidis, A. (2016). Weakly-supervised Convolutional learning for detection of inflammatory gastrointestinal lesions. 2016 IEEE International Conference on Imaging Systems and Techniques (IST). 10.1109/IST.2016.7738279 Grósz, T. (2014). Document Classification with Deep Rectifier Neural Networks and Probabilistic Sampling. Text, Speech, and Dialogue Lecture Notes in Computer Science. doi:10.1007/978-3-319-10816-2_14 Gruber, L. Z., Haruvi, A., Basri, R., & Irani, M. (2018). Perceptual dominance in brief presentations of mixed images: Human perception versus deep neural networks. Frontiers in Computational Neuroscience, 12, 57. doi:10.3389/fncom.2018.00057 PMID:30087604 Guerguiev, J., Lillicrap, T. P., & Richards, B. A. (2017). Towards deep learning with segregated dendrites. eLife, 6, e22901. doi:10.7554/eLife.22901 PMID:29205151 Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., & Webster, D. R. (2016). Development and Validation of a Deep Learning Algorithm for the Detection of Diabetic Retinopathy in Retinal Fundus Photographs. Journal of the American Medical Association, 316(22), 2402. doi:10.1001/jama.2016.17216 PMID:27898976 60

Deep Learning in Computational Neuroscience

Guo, W. W., & Xue, H. (2014). Crop Yield Forecasting Using Artificial Neural Networks: A Comparison between Spatial and Temporal Models. Mathematical Problems in Engineering, 2014, 1–7. doi:10.1155/2014/857865 Haines, D. E., & Mihailoff, G. A. (2017). Fundamental neuroscience for basic and clinical applications. Elsevier Health Sciences. Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245–258. doi:10.1016/j. neuron.2017.06.011 PMID:28728020 Higgins, I., Matthey, L., Glorot, X., Pal, A., Uria, B., Blundell, C., . . . Lerchner, A. (2016). Early visual concept learning with unsupervised deep learning. arXiv preprint arXiv:1606.05579 Kashyap, R. (2019a). Security, Reliability, and Performance Assessment for Healthcare Biometrics. In D. Kisku, P. Gupta, & J. Sing (Eds.), Design and Implementation of Healthcare Biometric Systems (pp. 29-54). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-7525-2.ch002 Kashyap, R. (2019b). The sensation of Deep Learning in Image Processing Applications. In A. Hassanien, A. Darwish, & C. Chowdhary (Eds.), Handbook of Research on Deep Learning Innovations and Trends (pp. 72–96). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-7862-8.ch005 Kashyap, R. (2019c). Big Data Analytics Challenges and Solutions. doi:10.1016/ B978-0-12-818146-1.00002-7 Kashyap, R. (2020). Applications of Wireless Sensor Networks in Healthcare. In P. Mukherjee, P. Pattnaik, & S. Panda (Eds.), IoT and WSN Applications for Modern Agricultural Advancements: Emerging Research and Opportunities (pp. 8–40). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-9004-0.ch002 Kashyap, R., & Piersson, A. D. (2018). Impact of Big Data on Security. In G. Shrivastava, P. Kumar, B. Gupta, S. Bala, & N. Dey (Eds.), Handbook of Research on Network Forensics and Analysis Techniques (pp. 283–299). Hershey, PA: IGI Global. doi:10.4018/978-1-5225-4100-4.ch015 Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. doi:10.1145/3065386

61

Deep Learning in Computational Neuroscience

Le, P., & Zuidema, W. (2015). Compositional Distributional Semantics with Long Short Term Memory. Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics. 10.18653/v1/S15-1002 LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. doi:10.1038/nature14539 PMID:26017442 Ma, Y., Xiang, Z., Du, Q., & Fan, W. (2018). Effects of user-provided photos on hotel review helpfulness: An analytical approach with deep leaning. International Journal of Hospitality Management, 71, 120–131. doi:10.1016/j.ijhm.2017.12.008 Neuroscience and its applications through deep learning - Google Search. (2019). Retrieved from https://www.google.com/search?q=Neuroscience+and+its+applic ations+through+deep+learning&safe Peng, M., Wang, C., Chen, T., & Liu, G. (2016, October 27). NIRFaceNet: A Convolutional Neural Network for Near-Infrared Face Identification. Retrieved from https://www.mdpi.com/2078-2489/7/4/61 Raina, R., Madhavan, A., & Ng, A. Y. (2009). Large-scale deep unsupervised learning using graphics processors. Proceedings of the 26th Annual International Conference on Machine Learning - ICML 09. 10.1145/1553374.1553486 Ravì, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B., & Yang, G. Z. (2017). Deep learning for health informatics. IEEE Journal of Biomedical and Health Informatics, 21(1), 4–21. doi:10.1109/JBHI.2016.2636665 PMID:28055930 Shen, D., Wu, G., & Suk, H. I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19(1), 221–248. doi:10.1146/annurevbioeng-071516-044442 PMID:28301734 Shukla, R., Gupta, R. K., & Kashyap, R. (2019). A multiphase pre-copy strategy for the virtual machine migration in the cloud. In S. Satapathy, V. Bhateja, & S. Das (Eds.), Smart Intelligent Computing and Applications. Smart Innovation, Systems and Technologies (Vol. 104). Singapore: Springer. doi:10.1007/978-981-13-1921-1_43 Tissera, M. D., & McDonnell, M. D. (2016). Deep extreme learning machines: Supervised autoencoding architecture for classification. Neurocomputing, 174, 42–49. doi:10.1016/j.neucom.2015.03.110

62

Deep Learning in Computational Neuroscience

Tiwari, S., Gupta, R. K., & Kashyap, R. (2019). To enhance web response time using agglomerative clustering technique for web navigation recommendation. In H. Behera, J. Nayak, B. Naik, & A. Abraham (Eds.), Computational Intelligence in Data Mining. Advances in Intelligent Systems and Computing (Vol. 711). Singapore: Springer. doi:10.1007/978-981-10-8055-5_59 Ullman, S. (2019). Using neuroscience to develop artificial intelligence. Science, 363(6428), 692–693. doi:10.1126cience.aau6595 PMID:30765552 Waoo, N., Kashyap, R., & Jaiswal, A. (2010). DNA nanoarray analysis using hierarchical quality threshold clustering. In Proceedings of 2010 2nd IEEE International Conference on Information Management and Engineering (pp. 81-85). IEEE. 10.1109/ICIME.2010.5477579

63

64

Chapter 3

Advanced Diagnosis Techniques in Medical Imaging Ramgopal Kashyap https://orcid.org/0000-0002-5352-1286 Amity School of Engineering and Technology, Amity University, Raipur, India

ABSTRACT The Boltzmann distribution was derived in this chapter. The Boltzmann equation was explained next to the main difficulty of this equation, the integral of the collision operator, which was solved by the BGK-approximation where a long-term substitute is essential. The discretization of the Boltzmann comparison with the BGK-approximation was introduced along with the lattice and the different lattice configurations to define the lattice framework where the method is applied. Also, in this framework, the algorithm of the process was described. The boundary conditions were summarised, where one can see that they represent macroscopic conditions acting locally in every node.

INTRODUCTION Accurate segmentation of medical images is the biggest challenge in computer vision, other than the application itself; image segmentation plays a significant role as a base technique in other computer vision problems. Despite the development of the various unsupervised image, segmentation methods limit the performance. Hence, for better practicability, semi-supervised image segmentation methods that are known as interactive image segmentation are gaining popularity in segmenting the object with less human effort (A. Ducournau and A. Bretto, 2014). In an interactive segmentation, the user intervention tightly coupled with an automatic segmentation DOI: 10.4018/978-1-7998-0182-5.ch003 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Advanced Diagnosis Techniques in Medical Imaging

algorithm needs the knowledge and the automated method’s computational capability. Real-time visualisation on the screen enables the user to quickly validate and correct the automatic segmentation results in a subdomain where the variational model’s statistical assumptions do not agree with the user’s expert knowledge. The user intervention mainly includes initialisation of the methods, and horizontal lines, respectively. Checking the accuracy of the results produced by automatic segmentation and corrections to the segmentation results using specialised interactive segmentation tools. Interactions in the segmentation of medical images can broadly classify into three types: pictorial input on an image grid, parameter tuning, and menu option selection (O. Pauplin, P. Caleb-Solly and J. Smith, 2010). The segmentation results obtained with new configurations like mouse clicking or drawing, new parameter values; another menu option visualised on the screen in real-time for further user evaluation. Among all the three types of user interactions, menu option selection is most efficient, but it constrains the degrees of freedom of the user’s choice of selections. Pictorial input is simple, but it could be time-consuming in case the interaction requires a user to draw precisely on an image grid. Parameter tuning is easy to operate, but it may require specific training for an insight into the automatic computational part.

INTERACTIVE IMAGE SEGMENTATION Interactive segmentation techniques are essential for fast and reliable extraction of the ROI. The level of user interaction in different methods varies in terms of the amount and type of information provided by the users because it requires iterative manual inputs, an acceptable interactive segmentation method has to minimise the human effort and produce a result quickly. That is to say, the seeds needed to obtain the desired outcome must be neither too accurately localised nor too numerous, and the time to compute a segmentation must aim to a few seconds (Y. Chen, A. Cremers and Z. Cao, 2014). The user chooses some pixels named seeds and indicators for each of them the element to which it belongs. Features like location, colour, texture, etc. of desired regions deduced by analysing and adding or removing some seeds can improve the produced result, allowing the user to get any desired segmentation results. It generally accepted that segmentation methods could be classified into an edge-based, region-based and energy-based methods. When an interactive segmentation method searches for the boundaries, then the user gives some points of these boundaries. One of the most representative edge-based interactive segmentation methods is Intelligent Scissors (N. Suetake, E. Uchino and K. Hirata, 2007). However, for highly textured images, a lot of boundary pixels are used for detection of the correct object that makes segmentation task complex. 65

Advanced Diagnosis Techniques in Medical Imaging

Regarding most of the region based methods; the user must draw some strokes on the different areas. With this modality of interaction, if picture elements are small when compared to the brush size, sometimes the user selects the wrong initialisation of the seed that gives false processing results. Some methods attempt to deal with this noise needs image preprocessing for getting better results Weiner filter is better for correction of distortion in the images that results can be significantly better.

PROPOSED METHOD The problem of interactive image segmentation is the typical use of small foreground or background seeds from users to extract the foreground region. One major issue of the existing algorithms for such interactive segmentation using their preference of the bounding box that covers the object which is to be segmented. The proposed method will make user-defined bounding boxes with external energy that inspect the segmentation results. To compute the fit box to the ground-truth foreground and slightly dilate it by 5% of the total pixels in all directions and to take it as the bounding box along with energy. It is required to expand this bounding box according to the energy needed in the instructions and after performing collision and streaming the bounding box will give accurate result, selection of the object will not dependent on the position user will get same results from selecting anywhere in the image this is the significant achievement of the proposed method (H. Li, D. Zhou and Y. Wu, 2011). It is for interactive capturing of images and better segmentation results and an interactive captured image for fast processing that done through using computational fluid dynamics techniques for medical image segmentation with some modifications. The lattice Boltzmann method has developed from the cradle of statistical mechanics as a whole alternative numerical approach for modelling physical phenomena in fluid flows (S. Balla-Arabe, Xinbo Gao and Bin Wang (2013), M. Tsutahara (2012). In contrast to the traditional numerical schemes such as the finite difference method (FDM) and spectral element method (SEM) uses macroscopic equations for processing. The first room starts full of particles while the second room starts empty. From the physical point of view, the Boltzmann method can be interpolated as a microscope for fluid mechanics while a telescope for molecular dynamics and it has successfully applied for the coupling of molecular dynamics in the microscopic world and fluid dynamics in the macroscopic (Q. Lou, Z. Guo and B. Shi, 2012). The fundamental idea behind lattice gas automata is that microscopic interactions of artificial particles living on the microscopic lattice can lead to the corresponding macroscopic equations to describe the same fluid flows. During the interaction, which is consisting of collision and propagation or streaming with lattice velocities, lattice symmetry plays a crucial role in conserving mass and momentum as well as 66

Advanced Diagnosis Techniques in Medical Imaging

ensuring the angular momentum conservation. Hardy developed the first lattice gas automata with 4-fold rotational symmetry in a square lattice, de Pazzis and Pomeau achieving only mass and momentum conservation. The hexagonal symmetry was sufficient to conserve the angular energy to retrieve not only diffusion equation, reaction-diffusion equation, but also the Navier-Stokes equations and the proposed two-phase nine velocity model it is much faster than the traditional models with linearization and simplification of the collision operator in Boltzmann equation. The major challenge with the Boltzmann method is it is tough to achieve higher than the second order of accuracy for both temporal and spatial discretisation. Moreover, because there are more distribution functions than the hydrodynamic variables to keep track of, it cannot avoid the relatively intensive computation as a trade-off for simplicity. In the proposed method a fuzzy external force (FEF) (B. Amarapur and P. Kulkarni, 2011) is making the segmentation fast, robust against noise and efficient whatever the position or the shape of the initial contour it can capture objects accurately. It has, first the advantage of the FCM which controls the evolving curve through the membership degree of the current pixels. Second, the benefits of the energy-based method are that it is independent of the topological change of shape, size and orientation of the object and, third, the advantages are it very suitable for parallel programming due to its local and explicit nature. The novelty of the proposed approach lies first in the handling of segmentation. Here made two enhancements to the traditional method: “iterative estimation” and “perfect boundary condition” that decreases the user interaction for a given quality of the result and fast segmentation. The traditional model in which the grid is two-dimensional and square so that each node in the grid has four neighbours. The particles can have four possible velocities, c1 = (1, 0), c2 = (0, 1), c3=(−1,0), and c4=(0,−1) as shown in Figure 1. For each time step, each particle is moved forward one step in the direction of its velocity, and when two or more particles meet at the same node after a time step, a collision occurs. To conserve mass and momentum, the number of particles and the total velocity of all the particles in the node must be the same before and after the collision. When two particles collide head-on, they are thrown out at right angles to their original velocities (M. Benedict, J. Kovács and A. Czirják, 2012), as shown in Figure 2. It conserves momentum, as the sum of the velocities of the two particles is zero in both configurations. When three or four particles meet at the same node, the only setting that satisfies conservation of mass and momentum is the same configuration as before the collision. Therefore, a clash between three or four particles cannot and does not result in a change of setting and the particle stream on as if no collision occurred. The crashes are entirely deterministic, meaning that each crash has one and only one possible result. Because of this, the proposed model has a property 67

Advanced Diagnosis Techniques in Medical Imaging

Figure 1. The velocity vectors of the model

Figure 2. The head of collision rules of the model

called time-reversal invariance, which means that the model can be run in reverse to recover an earlier state. Two rooms separated by a wall with a small opening (O. Utyuzh, G. Wilk and Z. Wodarczyk, 2007). The first room starts full of particles while the second room starts empty. When the simulation begins, particles escape from the first room to the second until the system reaches an equilibrium of roughly equal particle densities. The model is then run in reverse until all particles in the second room have retraced their steps and gone back to their initial positions in the first room. The proposed method has some advantages compared to other numerical methods. The state of each node in the grid can be described entirely by four bits: Bit I represent the presence or absence of one particle moving in the direction ci. It 68

Advanced Diagnosis Techniques in Medical Imaging

Figure 3. Four particles with different directions moving in the model from one time step to the next

means that very little storage space and memory are required. Another advantage is that due to the Boolean nature of the system, no floating-point numbers used in the model, which means that all names are entirely exact, and no round-off errors occur. The precise and deterministic nature of the model also means that it has a property called time-reversal invariance, which means that the system can be run in reverse to reproduce an earlier state perfectly. A third advantage is the inherently parallel nature of the system. The events in each node are not related to the simultaneous developments in another node since the only communication between nodes is through streaming of particles. Also, the nature of particle streaming is such that each particle can only have one origin. It means that several processors can process the collisions or particle streaming in different sections of the grid simultaneously without a need for communication, apart from distributing the work among each other and sending back results (C. Sun, F. Pérot, R. Zhang, D. Freed and H. Chen, 2013). There are also many inherent weaknesses in the traditional model. Due to the little scope of the model, it can never be said to be incomplete equilibrium. The system will never by itself reach a state where the system is identical from one step to the next. It is evident from the model’s property of time-reversal invariance: For the system to be in a state where the future state is identical to the current state, all previous states must have been equal to the current state. Therefore, it is only possible to reach a permanent status by setting them as the initial condition. A trivial example is a system with all possible particles presents everywhere, but this can hardly be said to be a natural equilibrium. For this reason, the microscopic state of the system is always changing. As mentioned earlier, the macroscopic quantities of the system can found by averaging the tiny amounts over space or time. The constant change in the small state, there will always be statistical noise in the macroscopic quantities; this problem can be reduced by broadening the average, but never avoided. Possibly the greatest weakness of the active contour model is that it fails to achieve rotational invariance, which means that its behaviour becomes 69

Advanced Diagnosis Techniques in Medical Imaging

anisotropic (O. Utyuzh, G. Wilk and Z. Wodarczyk, 2007). For instance, in a system with particles evenly spread out, apart from a high concentration in the centre, this high concentration will spread out from the centre in a diamond pattern. As a result of this, active contour systems fail to behave in accordance to the Navier-Stokes equations. This weakness alone is crippling to the point that the HPP model is not useful for fluid simulations. The quantities used in the simulation relate to real numbers through the lattice spacing ∆x and the time step ∆t. For instance, the physical particle speeds ci , p in the lattice is given by

ci , p 

x  ci , l t

(1)

Here say that the vector is in lattice units, which are units normalised by ∆x and ∆t. In lattice units, that = 1 for all i in the active contour model, and that ∆t = 1.For the vector, which is in physical units, that = ∆x/∆t. The particle density in each node can be calculated simply from the number of particles present in the node, or p( x , t ) 

 n (x , t) i i

(2)

where p( x , t ) is the particle density at the node with a position at time t, and is the Boolean occupation number, meaning the number of particles present (0 or 1) at this node with velocity . The quantities And t is in lattice units. Similarly, the total momentum in each node calculated from where u is the mean velocity of the particles at this node. Hard walls can create an active contour model as nodes, which cause incoming particles to reflect. The boundaries of the simulated system can be hard walls, or they can be periodic. Periodic boundaries imply that a particle which exits the system at one edge will re-enter the system at the opposite edge. With hard boundaries, the behaviour of a fluid trapped in a box is simulated, while with periodic boundaries, it is the behaviour of a fluid in a periodic system which simulated. It is naturally possible to combine the two boundaries, for instance, by having hard vertical boundaries on two sides and periodic boundaries on the other two. Force can be simulated in the proposed method randomly changing some particle velocity in the direction of the force with a given probability. With all these rules known, it is possible to perform a simulation. First, certain geometry is created using hard walls, and initial distribution of particles with certain positions and velocities is created (J. Kundu and R. Rajesh, 2015). The system is then left to run for a while until it reaches a sort of equilibrium. Then, the macroscopic particle density can be found by averaging over space (several nodes) and time (several time steps) to find the average number of particles in each area. The macroscopic momentum can found 70

Advanced Diagnosis Techniques in Medical Imaging

by a similar average of the nodes’ momentum. The lattice vectors in the proposed model are c1 = (1, 0), c2 = (1/2, √3/2), c3 = (−1/2, √3/2), c4 = (−1, 0), c5 = (−1/2, −√3/2) and c6 = (1/2, −√3/2) given in Figure 4. A hexagonal lattice allows two possible resolutions for a head-on collision which conserve both mass and momentum, illustrated in Figure 6.5. It is different from the HPP model, where there is only one possible resolution, which is to occur, is chosen randomly for every collision, with equal probability. Due to this stochastic element in the model, it is no longer fully deterministic and does not have the time-reversal property of the four-velocity model. Figure 4. The velocity vector of the model

Figure 5. Common set of two-dimensional vectors

71

Advanced Diagnosis Techniques in Medical Imaging

Figure 6. Evolution of all the vectors of the model

As mentioned, it can derive that the proposed model gives behaviour by the incompressible Navier-Stokes equations [269]. From the derivation, one can show that the kinematic viscosity of the system must be

    x 2  1 1  Ѵ= 3 t    8    2  1    6   

(3)

where ρ is the equilibrium density of the gas, this can found at the average particle density of the entire system. To see from this equation that the viscosity can become arbitrarily large in the limits ρ → 0 and ρ → 6, indicating no particles in the system and a maximum number of particles in the system, respectively. The minimal viscosity for the model found when ρ = 3/2. Varieties exist in the proposed model the additional collisions introduced. The proposed model is essentially the Lattice Boltzmann model (M. Beneš and P. Kučera, 2015) and active contour model (A. Mohamad and A. Kuzmin, 2012). with a change of the lattice and the collision rules. Thus, it retains all the advantages of the traditional model such as small storage space demands, exact dynamics and parallelism. It removes the most critical disadvantage, that of the inherent anisotropy of the conventional model, giving a behaviour consistent with the incompressible 72

Advanced Diagnosis Techniques in Medical Imaging

Figure 7. The head-on collision rule for the proposed model

Navier-Stokes equations. The problem of statistical noise remains. It is a general problem when trying to recover macroscopic quantities from microscopic-scope simulations, as the microscopic system is subject to random fluctuations that disappear in the continuum limit. On the other hand, if the subject of interest is studying such changes in real systems, this statistical noise is a desired property. A valuable property the proposed method retains from its lattice gas ancestors is locality in each step given above, each node is updated separately from the others because of this parallelism can achieve, and the method can use as many processors as there are nodes in the grid. Nothing has been said yet about the problem of initial conditions. Finite volume method subdivided domain into small control volumes that are known as cells through the grid, and the network describes the boundaries of the control volumes while the computational node at the centre of the control volume and the integral conservation is satisfied exactly over the control volume. The net flux through the control volume boundary is the total of the integrals over the four control volume faces (six in 3D). The control volumes do not overlap, and the value of the integrand is not available at the control volume faces and determined by interpolation. The conservation equations play essential roles in CFD for digitisation the species transport equation for constant density, and incompressible flow is given by:

 c   ui , c   y   c  s D   t  xi  x   t 

(4)

73

Advanced Diagnosis Techniques in Medical Imaging

Figure 8. The proposed method collision handling

Figure 9. The boundary and node representation

Figure 10. The computational node and boundary calculation

74

Advanced Diagnosis Techniques in Medical Imaging

Here c is the concentration of the chemical species and D is the diffusion coefficient. S is a source term. It will discretise this equation (convert it to a solvable algebraic form) for the single flow field shown on the right, assuming steady-state conditions. The balance over the control volume is given by:

AeueCe  AwuwCw  AnunCn  As us Cs  c   c   c   c    DAe   sp    DAn    DAs    DAw  xe    xw    yn    ys  

(5)

It contains values at the faces, which need to determine from interpolated from the values at the cell centres. Figure 11. Discretization of node according to velocity

Figure 12. Control volume of the nodes

75

Advanced Diagnosis Techniques in Medical Imaging

Where, AeAsAwAn are the area of the faces CECSCWCN are the concentrations at the cell centres cscscwcn are the concentrations at the faces ueusuwun is the Velocity at the faces uEuSuWuN is the Velocity at the cell centres D is the Diffusion Coefficient Sp is a source in the cell The simplest way to decide the values at the faces done through first-order upwind differencing. Here, let’s assume that the amount on the face is equal to the centre and using that method results in:

Aeu p C p  AwuwCw  Anu p C p  As us Cs  DAe  CE  CP   DAw   CP  CW   DAn  C N  CP   DAs  CP  CS   s p



(6)

This equation rearranged at the centre of cell P as a function of the concentrations in the adjacent cells, the path, and the grid. This equation can now be simplified:

ae c p  ae ce  as cs  awcw  an cn  b

(7)

ae c p  an cn  b

(8)

nb

Here n refers to the neighbouring cells. The coefficients an And b will change according to the domain of the cells. The species concentration field can calculate recalculating cp for all the cells. The Periodic edges act as if they connected to the opposite side of the system. If the left and right sides of a system are periodic, a particle distribution which streams left at the left corner of the order will reappear at the right edge, heading left. In a 2D system, the left, right boundaries and the top-bottom boundaries can be periodic. Figure 12 shows neighbouring nodes across natural barriers. A periodic limit naturally implies that the system is regular, where a cyclic behaviour is studied, such as the propagation of a plane sound wave in a duct, where the length of the system equals the wavelength (J. Wu and H. Guo, 2014).

76

Advanced Diagnosis Techniques in Medical Imaging

Figure 13. Particle streaming paths from a corner node in 3 x 3 nodes

The other simple lattice Boltzmann boundary is the hard wall, which reflects particles and guarantees a non-slip condition with zero velocity at the wall. There are two variations on this method, the on-grid or full-way bounce-back method and the mid-grid or half-way bounce back method (S. Krithivasan, S. Wahal and S. Ansumali, 2014). In these two methods, individual nodes in the grid are marked as walls and are thus not a part of the fluid. The particle distributions in-wall nodes are not relaxed towards equilibrium and do not act by the lattice Boltzmann evolution equation. Figure 14. Particle streaming on the possible paths

77

Advanced Diagnosis Techniques in Medical Imaging

The on-grid method is the simpler of the two methods. In the collision step of the iteration, where the fluid nodes are relaxed, all particles’ directions reversed so that the particles are sent back in the direction they came from, as shown in Figure 14. The collision step modified for the wall nodes, but the streaming level is unchanged. In the mid-grid method, particles which are set to flow into a wall node are reflected instead of streamed, as shown in Figure 15. Here, the streaming step is modified while the collision step remains the same. In both bounce-back methods, the current position of the wall is actually between the wall nodes and the fluid nodes adjacent to it. It can be seen, for instance, by comparing the mid-grid method is somewhat more challenging to implement than the on-grid way, but the mid-grid method gives better accuracy. It is the reason to use mid-grid walls in the process of a system. They are periodic, a particle distribution which streams left at the left edge of the order will reappear at the right edge, heading left [274]. In a 2D mode, the left-right boundaries and the top-bottom boundaries can be periodic and hard wall the proposed method which reflects particles and guarantees a non-slip condition with zero velocity at the wall. There are two variations on this method, the on-grid or full-way bounce-back method and the mid-grid or half-way bounce back method. In these two methods, individual nodes in the grid are marked as walls and are thus not a part of the fluid. The particle distributions in-wall nodes are not relaxed towards equilibrium and do not act by the lattice Boltzmann evolution equation. The on-grid method is the simpler of the two methods. In the collision step of the iteration, where the fluid nodes are relaxed, all particles’ directions reversed so that the particles are sent back in the direction they came from, as shown in Figure 16. The collision step modified for the wall nodes, Figure 15. The on-grid bounce-back method

78

Advanced Diagnosis Techniques in Medical Imaging

Figure 16. The mid-grid bounce-back method

but the streaming level is unchanged. In the mid-grid method, particles which are set to flow into a wall node are reflected instead of streamed, as shown in Figure 16. Here, the streaming step is modified while the collision step remains the same. In both bounce-back methods, the current position of the wall is actually between the wall nodes and the fluid nodes adjacent to it. The distribution f (,,t) rv gives the probability of finding a single a molecule inside a small volume r, around the position with a certain velocity At time t. So in principle, the probability of finding a molecule with a velocity between and +d and at a specific position between and + d at time t is given by the following relation. f (r  dr , v  dv , t )  f (r , v , t )dr dv

(9)

In the supposed situation where collisions do not exist, every molecule is able to move freely as they do not interact with other molecules. Then, a given the external force is introduced, which acts on a molecule of unit mass. Therefore the velocity of the molecule will change to +dt, which can be expressed as  dv v  Fdt  v    dt

  dt  v  dv 

(10)

and the position will change as well from r to r + v dt, which also can be simplified

79

Advanced Diagnosis Techniques in Medical Imaging

 dr  r  vdt  r    dt  r  dr  dt 

(11)

Hence, since there are no collisions allowed, the distribution density before applying the external force F and the distribution after a differential time step dt follows this relation. f(+d,+d,t+dt)dd=f(,,t)dd

(12)

It is because molecules can move freely, that every particle with a given initial state is free to arrive at a final state after is applied. If collisions are considered, the relation changes because molecules are not able to move with complete freedom around the domain. Hence, a collision term must be added about restrain the liberty of the molecules. This term is the rate of change between the number of particles that are between + d and + d after a time differential dt. The rate of change can express as:

df = Ω dt

(13)

This equation shows that the total rate of change in the distribution function is equal to the rate of collisions, plus the fluxes of particles entering and leaving the volume of study. Eq.(6.7) implies that the total rate of change of the distribution function is equal to the speed of collisions. It is not easy to solve the Boltzmann equation due to the collision term. The Chapman-Enskog expansion, collision term can be related to the viscosity when deriving the NS equation. An approximation without introducing significant error in the result. This approximation by Bhatnagar, Gross and Krook (BGK) as a simplification of the collision operator. Ω = ω(− f) = (− f)

(14)

Where ω is the collision frequency, and its inverse τ is the relaxation time. The term Denotes the distribution function value at the local equilibrium. The relaxation time is related to the viscosity in this framework, νlb, by the following formula

1 1  lb     3 2

80

(15)

Advanced Diagnosis Techniques in Medical Imaging

This factor takes a significant role in the numerical stability of the method. The modified Boltzmann equation with the BGK approximation results

f 1  v f= f eq  f t 





(16)

Moreover, this equation works on a more fundamental level than NS equations, so it is possible to derive NS equations from the Boltzmann equation. In this discrete form of the Boltzmann equation, basic steps in the LBM.

fi (r  eit , t  t )  fi (r , t ) 

t eq [ fi (r , t )  fi (r , t )] + 

fi (r  eit , t  t )  fi (r , t ) is streaming step and

(17)

t eq [ fi (r , t )  fi (r , t )] 

is the collision step. The beauty of this equation is its simplicity. Thus, it can be easily modified to introduce different simulation properties. The algorithm is Initialise; for number of iterations do Apply boundary conditions; Streaming Step; Compute macroscopic quantities; Compute f eq; Collision Step; if there are obstacles then Obstacle treatment; end end Store data;

BOUNDARY CONDITIONS Considering the problem simulates, only solid boundaries are implemented to delimit the domain. The stable boundary conditions used in the proposed method based on the so-called bounce-back boundary condition. That is, each particle that collides with a solid border has its velocities vi reversed (vi = −vi) and therefore bounces back from the edge. It is physically appropriate because solid walls have a sufficient 81

Advanced Diagnosis Techniques in Medical Imaging

roughness to prevent any fluid motion parallel to the wall. Also, the peripheral cells that delimit the domain marked as boundary cells with the bounce-back condition. During their update, these cells still require access to cells that are outside of the area, which is the so-called ghost cells. Special treatment for those cells, another layer of boundary cells put around the area (X. Zhuang, X. Ji and Z. Wei, 2013). However, the additional layer of boundary cells not processed in a lattice Boltzmann step; they only exist to avoid a special treatment for the actual boundary cells. Therefore, all the LBM computations can efficiently accelerate on the GPU; body forces can be added to the LBM as an external input to control the flow behaviour.

EXPERIMENTS AND RESULT ANALYSIS The experimental section described in two parts; the first part shows the accuracy and the usefulness of the proposed method by comparing it with the traditional energybased methods (K. Zhang, H. Song and L. Zhang, 2010), (Y. Wang, C. Shu and L. Yang, 2016), (H. Kang, B. Kim, J. Lee, J. Shin and Y. Shin, 2014), (K. Zhang, H. Song and L. Zhang, 2010). The second part exemplifies the ability of the proposed method concerning speed and efficiency. In the implementation of the proposed approach, the value of the fuzzy parameter set to three, and Convection coefficient kept 0.08. All the techniques have been implemented using Matlab R2014b installed on a PC Pentium(R) Dual-Core CPU E5500 processor with a clock speed of 2.80 GHz and 2 GB of RAM.

Comparison in Terms of Effectiveness and Accuracy In this part, the contrast of the proposed method with four segmentation methods given, the first one is active contours without edges introduced by T. Chan and L. Vese et al for image segmentation submitted by Kaihua Zhang et al.((K. Zhang, H. Song and L. Zhang, 2010) provided by Huihui Song et al. (H. Kang, B. Kim, J. Lee, J. Shin and Y. Shin, 2014), and the last one submitted by Chunming Li et al. (Chunming Li, Rui Huang, Zhaohua Ding, J. Gatenby, D. Metaxas and J. Gore, 2011). Figure 6.17 is showing other methods and the proposed method result. To examine that the proposed method provides the best segmentation results, it gives discontinue and efficient object boundary. The method by Chen et al. and the method by Li et al. both are not succeeding because due to the identification of the weak edges. The method by Huihui Song et al. spend more time in the iterations. Furthermore, the resulting contours present many discontinuities. The process by Li et al. gives over segmented results and takes more time; it is, therefore, less robust than the proposed method, and furthermore, the quality of the segmentation using the proposed method is robust to noise and gives the best results. 82

Advanced Diagnosis Techniques in Medical Imaging

Figure 17. (a) Shows the segmentation results using the active contour method without edges, fig. (b) shows the segmentation results using the improved chan vese method, fig. (c) shows the segmentation results using the local image fitting energy method, fig. (d) shows the segmentation results using a level set method for image segmentation in the presence of intensity inhomogeneities, fig. (e) shows the segmentation results using the proposed method

83

Advanced Diagnosis Techniques in Medical Imaging

Figure 18. Shows the segmentation result of mri image of the brain corrupted by salt and pepper noise (noise density value d=0.02)

Table 1. Pearson correlation coefficient comparison of proposed method with geodesic active contour, globally optimal geodesic active contour, local binary fitting model, local region model and local intensity fitting model Pearson Correlation Coefficient

S.N.

Method

1

Local Intensity Fitting

30059.00372

2

Geodesic Active Contour

29641.67067

3

Local Binary Fitting

29743.21324

Original Image vs Noisy Image

4

GOGAC

29777.78693

5

Local Region Model

29852.18299

6

Proposed Method

30200.33511

Comparison in Terms of Speed and Efficiency In this part, comparison of the proposed method with the well-known CV method, improved CV, local image fitting method, improved level set method with bias correction and the proposed method, all the CPU times displayed in Table 1. It can be seen that the proposed method gives better result than the Chan Vese, improved Chan Vese, local image fitting energy method and level set method, whatever the shape and the position of the initial contour the result changes with the initial outline and one should run the algorithm several times in order to choose the best result. 84

Advanced Diagnosis Techniques in Medical Imaging

Table 2. CPU times shown in the experiment   CPU Time(s)   Types of Images

  Methods   Type 1

  Type 2

  Type 3

  Type 4

  Proposed Method

6.073

5.220

5.668

  11.747

  Chan Vese

9.179

15.453

18.038

  16.496

  Improved Chan Vese

8.270

4.525

14.556

  12.723

  Local image fitting

9.851

17.364

36.987

  32.966

  Level Set Method

10.531

10.796

49.453

  49.750

Figure 19. Shows the segmentation results of medical MRI and CT images of the human body. the results of the proposed method are accurate and clear; the resulting contours are closed and present no discontinuities. figures are showing segmentation results fig (a) using the active contour method without edges, fig. (b) using the improved Chan Vese method, fig. (c) using the active contour method by local image fitting, fig. (d) shows the segmentation result using the level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI and fig. (e) shows the segmentation result using the proposed method

85

Advanced Diagnosis Techniques in Medical Imaging

Thus, these methods cannot be suitable for accurate and proper segmentation, C-V model uses region information and considered as one of the most widely used models for segmentation. One of the prominent advantages of the C-V model is that it performs well on the images with fuzzy or even without edges. However, as a limitation, the C-V model always supposes the image with intensity homogeneity, the local image fitting method gives correct results, but the proposed method is much faster. Here the observation of the proposed method provides the most excellent segmentation results; whatever the form and the location of the initial contour and the resultant contours are smooth and continuous. Figure 21 shows the local image fitting method where the initial contour position affects the final result of segmentation incorrect result downs the accuracy of the process. Figure 20. (a, c, e, g) Shows the initial contour and (b, d, f, h) shows the segmentation results by local image fitting

86

Advanced Diagnosis Techniques in Medical Imaging

Figure 21. Shows the initial contours (a-i) and the (j-k) shows the segmentation results of the proposed method

CONCLUSION The proposed approach to interactively segment the foreground object of interest from an image is to annotate a bounding box that covers the foreground object. Then, double labelling is performed to achieve a refined segmentation. The lattice Boltzmann method, therefore, is a new and promising alternative to the traditional Navier– Stokes solvers in the domain of Computational Fluid Dynamics (CFD). Because of the great need for memory and computing time, most CFD problems on a single 87

Advanced Diagnosis Techniques in Medical Imaging

computer insufficient time. Therefore the solvers have to be parallelised. Because of the implicit parallelism of the proposed method, this promises a good speedup. Numerical simulation of the PDEs usually requires high-intensity computation and large consumption of computational resources. To discretise the PDEs that lead to solving a sparse linear system. The parallelisms of computer algorithms are becoming increasingly important today, as the last years’ trend in processors was to put more CPUs on each processor instead of drastically improving the speed of each CPU. Also, the emergence of general-purpose graphics processing units, which can look at as massively parallel processors, offers exciting new opportunities for rapid calculation of parallelizable problems. It is a term for issues which are particularly simple to parallelise, and where the speed up in the computation is nearly linear with the number of processor cores used. The reason for the lattice Boltzmann’s straightforward parallelisation is that operations on the grid are local so that each node can be updated independently of others. It should not be surprising that the lattice Boltzmann method gives behaviour, according to the wave equation, and that it can be used to simulate acoustics. After all, the compressible Navier-Stokes equation can be simulated using the lattice Boltzmann method and the wave equation can be derived from compressible Navier-Stokes. Of course, for pure wave equation simulations, there will probably be more effective numerical methods. The lattice Boltzmann method’s strength is being a full Navier Stokes solver, which means that it can be used to simulate non-linear acoustics in complex flows.

SUMMARY In this chapter, the Boltzmann distribution was derived, which is the distribution that holds the statistical treatment of the method. The Boltzmann equation explained next to the main difficulty of this equation, the integral of the collision operator solved by the BGK-Approximation where a long term substitutes this essential. The discretisation of the Boltzmann comparison with the BGK-Approximation introduced, all along with the lattice and the different lattice configurations, to define the lattice framework where the method is applied. Also, in this framework, the algorithm of the process was described. The boundary conditions summarised in this chapter, where one can see that they represent macroscopic conditions acting locally in every node.

88

Advanced Diagnosis Techniques in Medical Imaging

REFERENCES Ducournau, A., & Bretto, A. (2014). Random walks in directed hypergraphs and application to semi-supervised image segmentation. Computer Vision and Image Understanding, 120, 91–102. doi:10.1016/j.cviu.2013.10.012 Pauplin, O., Caleb-Solly, P., & Smith, J. (2010). User-centric image segmentation using an interactive parameter adaptation tool. Pattern Recognition, 43(2), 519–529. doi:10.1016/j.patcog.2009.03.007 Chen, Y., Cremers, A., & Cao, Z. (2014). Interactive color image segmentation via iterative evidential labeling. Information Fusion, 20, 292–304. doi:10.1016/j. inffus.2014.03.007 Suetake, Uchino, & Hirata. (2007). Separability-Based Intelligent Scissors for Interactive Image Segmentation. IEICE Transactions on Information and Systems, 90(1), 137-144. Li, H., Zhou, D., & Wu, Y. (2011). Collision detection algorithm based on mixed bounding box. Jisuanji Yingyong, 30(12), 3304–3306. doi:10.3724/ SP.J.1087.2010.03304 Balla-Arabe, S., Gao, X., & Wang, B. (2013). A Fast and Robust Level Set Method for Image Segmentation Using Fuzzy Clustering and Lattice Boltzmann Method. IEEE Transactions on Cybernetics, 43(3), 910–920. doi:10.1109/TSMCB.2012.2218233 PMID:23076068 Tsutahara, M. (2012). The finite-difference lattice Boltzmann method and its application in computational aero-acoustics. Fluid Dynamics Research, 44(4), 045507. doi:10.1088/0169-5983/44/4/045507 Lou, Q., Guo, Z., & Shi, B. (2012). Effects of force discretization on mass conservation in lattice Boltzmann equation for two-phase flows. EPL, 99(6), 64005. doi:10.1209/0295-5075/99/64005 Amarapur, B., & Kulkarni, P. (2011). External Force for Deformable Models in Medical Image Segmentation: A Survey. Signal and Image Processing: an International Journal, 2(2), 82–101. doi:10.5121ipij.2011.2208 Benedict, M., Kovács, J., & Czirják, A. (2012). Time dependence of quantum entanglement in the collision of two particles. Journal of Physics. A, Mathematical and Theoretical, 45(8), 085304. doi:10.1088/1751-8113/45/8/085304 Utyuzh, O., Wilk, G., & Wodarczyk, Z. (2007). Numerical symmetrization of state of identical particles. Brazilian Journal of Physics, 37(2). 89

Advanced Diagnosis Techniques in Medical Imaging

Sun, C., Pérot, F., Zhang, R., Freed, D., & Chen, H. (2013). Impedance Boundary Condition for Lattice Boltzmann Model. Communications in Computational Physics, 13(03), 757–768. doi:10.4208/cicp.421011.260112s Utyuzh, O., Wilk, G., & Wodarczyk, Z. (2007). Numerical symmetrization of state of identical particles. Brazilian Journal of Physics, 37(2). Kundu, J., & Rajesh, R. (2015). Asymptotic behavior of the isotropic-nematic and nematic-columnar phase boundaries for the system of hard rectangles on a square lattice. Physical Review. E, 91(1), 012105. doi:10.1103/PhysRevE.91.012105 PMID:25679568 Beneš, M., & Kučera, P. (2015). Solutions to the Navier-Stokes equations with mixed boundary conditions in two-dimensional bounded domains. Mathematische Nachrichten, 289(2-3), 194–212. Mohamad & Kuzmin. (2012). The Soret Effect with the D1Q2 and D2Q4 Lattice Boltzmann Model. International Journal of Nonlinear Sciences and Numerical Simulation, 13(3-4). Wu, J., & Guo, H. (2014). Sonar Image Segmentation Based on an Improved Selection of Initial Contour of Active Contour Model. AMM, 709, 447–450. doi:10.4028/ www.scientific.net/AMM.709.447 Krithivasan, S., Wahal, S., & Ansumali, S. (2014). Diffused bounce-back condition and refill algorithm for the lattice Boltzmann method. Physical Review. E, 89(3), 033313. doi:10.1103/PhysRevE.89.033313 PMID:24730973 Zhuang, X., Ji, X., & Wei, Z. (2013). A Novel Deformable Grid Method for Image Segmentation. AMM, 310, 624–628. doi:10.4028/www.scientific.net/AMM.310.624 Wang, Y., Shu, C., & Yang, L. (2016). Boundary condition-enforced immersed boundary-lattice Boltzmann flux solver for thermal flows with Neumann boundary conditions. Journal of Computational Physics, 306, 237–252. doi:10.1016/j. jcp.2015.11.046 Kang, H., Kim, B., Lee, J., Shin, J., & Shin, Y. (2014). Automatic left and right heart segmentation using power watershed and active contour model without edge. Biomedical Engineering Letters, 4(4), 355–361. doi:10.100713534-014-0164-9 Zhang, K., Song, H., & Zhang, L. (2010). Active contours driven by local image fitting energy. Pattern Recognition, 43(4), 1199–1206. doi:10.1016/j.patcog.2009.10.010

90

Advanced Diagnosis Techniques in Medical Imaging

Li, C., Huang, R., Ding, Z., Gatenby, J., Metaxas, D., & Gore, J. (2011). A Level Set Method for Image Segmentation in the Presence of Intensity Inhomogeneities With Application to MRI. IEEE Transactions on Image Processing, 20(7), 2007–2016. doi:10.1109/TIP.2011.2146190 PMID:21518662 Wang, L., He, L., Mishra, A., & Li, C. (2009). Active contours driven by local Gaussian distribution fitting energy. Signal Processing, 89(12), 2435–2447. doi:10.1016/j. sigpro.2009.03.014

91

92

Chapter 4

Challenges of Applying Deep Learning in RealWorld Applications Amit Kumar Tyagi https://orcid.org/0000-0003-2657-8700 School of Computing Science and Engineering, Vellore Institute of Technology, Chennai, India G. Rekha Department of Computer Science and Engineering, Koneru Lakshmaiah Educational Foundation, Hyderabad, India

ABSTRACT Due to development in technology, millions of devices (internet of things: IoTs) are generating a large amount of data (which is called as big data). This data is required for analysis processes or analytics tools or techniques. In the past several decades, a lot of research has been using data mining, machine learning, and deep learning techniques. Here, machine learning is a subset of artificial intelligence and deep learning is a subset of machine leaning. Deep learning is more efficient than machine learning technique (in terms of providing result accurate) because in this, it uses perceptron and neuron or back propagation method (i.e., in these techniques, solve a problem by learning by itself [with being programmed by a human being]). In several applications like healthcare, retails, etc. (or any real-world problems), deep learning is used. But, using deep learning techniques in such applications creates several problems and raises several critical issues and challenges, which are need to be overcome to determine accurate results.

DOI: 10.4018/978-1-7998-0182-5.ch004 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Challenges of Applying Deep Learning in Real-World Applications

INTRODUCTION ABOUT DEEP LEARNING Defining Multi-Platform The definition of Artificial Intelligence (AI) is quite long and easy to understand, i.e., it refers to the simulation of intelligent behavior by computer systems. Deep Learning is a subset of Machine Learning, i.e., used to produces prediction from a large collection of data (using analysis techniques or methods, and learning from its own), whereas Machine learning is a subset of Artificial Intelligence, to produces predictions (using human being help in analysis). Machine Learning is defined as “ML is a field of study that gives computers the capability to learn without being explicitly programmed” (Tyagi, A. K. (2019), Samuel, A. L. (2000)), and Awad, M., & Khanna, R. (2015). Machine learning uses algorithms like supervised (regression, decision tree, random forest, classification) and unsupervised (clustering, association analysis, Hidden Markov Model, etc.). Here, supervised, unsupervised and reenforcement learning are three types of machine learning. Such behavior could encompass any range of tasks for which human intelligence is normally necessary, such as visual pattern recognition, language translations, and decision-making (datafloq.com, (2018)). On the other hand, Deep Learning uses Artificial Neural Networks (ANN) inspired by knowledge of human brain biology, from which emerge algorithms which are highly efficient in solving classification problems. In general terms, deep learning is an aspect of AI that uses complex hierarchical neural networks and lots of precise data to make machines capable of learning things automatically by its own (like a human being learns). Today’s the market value of deep learning software is growing at a higher rate, i.e., from $20 million (in 2018) to $930 million (by 2025). Deep Learning is the spearhead of Artificial Intelligence (see figure 1), and it is one of the most exciting technologies of the recent/ past decade. Now days, this kind of learning techniques are being used in several applications/ areas like such as recognizing speech or detecting cancer, etc. Deep Learning is often compared to the mechanisms that underlie the human mind, and some experts believe that it will continue to advance at an unexpected growth and conquer many more fields/ areas (bdtechtalks.com, (2018)). In some cases, there is fear that deep learning might threaten the very social and economic fabrics that hold our societies together, by either driving humans into unemployment or slavery. There is no doubt that machine learning and deep learning are superefficient for many tasks (while are interrelated to each other (refer figure 1)). However, there are not universal techniques (even Deep Learning) which can solve all problems and override all previous technologies. This technologies/ learning technique (in many cases) have several/ its distinct limits and challenges which prevent it from competing with the mind of a human being. Human beings can 93

Challenges of Applying Deep Learning in Real-World Applications

learn abstract, broad relationships (Seymour, V. (2016)) between different concepts and make decisions with little information. In brief, deep learning algorithms are narrow in their capabilities and need precise information to do their job/ analysis information. Note that as discussed above, Deep Learning is a subset of Machine Learning (Tyagi, A. K. (2019)). that achieves great power and flexibility by learning, to represent the world as nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones, cost, mistakes. A deep learning technique learn categories incrementally through its hidden layer architecture, defining low-level to high level categories like letters to words then to sentences, for example, in image recognition it means identifying light/dark areas before categorizing lines and then shapes to allow face recognition. In deep learning, each neuron or node in the network represents one aspect of the whole and together they provide a full representation of the image. Each node or hidden layer is given a weight that represents the strength of its relationship with the output and as the model develops the weights are adjusted. Today’s popular example of Deep Learning (in case of learning itself) is ‘Ex-Machina’ (a Hollywood movie, released in 2014). In this movie, a machine (a robot: machine with intelligence) tries to attack on human being (with thinking), also fall attached with a human. In near future, this kind of future will be interesting when both machine and people are working together, with understanding each other (without doing any malicious activity). Apart that, Deep learning exciting process are: Face recognition, image classification, speech recognition, text-to-speech generation, Figure 1. Relation of deep learning with machine leaning with artificial intelligence

94

Challenges of Applying Deep Learning in Real-World Applications

handwriting transcription, machine translation, medical diagnosis (Cao, C., Liu, F., et.al (2018), & Litjens, G., Kooi, T., et.al (2017), cars (drivable area, lane keeping), digital assistants, Ads, search, social recommendations, game playing with deep RL. Hence now history of Deep Learning is investigated (year-wise) as: • • • • • • • • • • • •

1943: Neural Networks 1957: Perceptron 1974-86: Back-propagation, RBM, RNN 1989-98: CNN, MNIST, LSTM, Bi-directional RNN 2006: “Deep Learning”, DBN 2009: ImageNet 2012: AlexNet, Dropout 2014: GANs 2014: DeepFace 2016: AlphaGo 2017: AlphaZero, Capsule Networks 2018: BERT Further, the history of Deep Learning Tools (year-wise) is included as:

• • • • • • • • • • •

Mark 1 perceptron – 1960 Torch – 2002 CUDA - 2007 Theano – 2008 Caffe – 2014 DistBelief – 2011 TensorFlow 0.1- 2015 PyTorch 0.1 – 2017 TensorFlow 1.0 – 2017 PyTorch 1.0 – 2017 TensorFlow 2.0 - 2019

Hence, now each section will discusses essential topics related to deep learning (in detail).

Deep Learning vs. Machine Learning Now-a-days, the software industry moving towards/ implementation of machine intelligence in their business, i.e., to boost their product. Generally, Machine Learning is being used in industry to make machine automotive and intelligent. In 95

Challenges of Applying Deep Learning in Real-World Applications

a simpler way, Machine Learning is set of algorithms that parse data, learn from them, and then apply what they have learned to make intelligent decisions. For example, it is how Netflix knows which shows we need/ want to watch next or how Facebook recognises your friend’s face in a digital photo. These videos on Netflix are analysed deeply with collection of data of various users, i.e., at which instance user forwards and at which moment users repeated the scenes. In the past decade, traditional Machine Learning algorithms and data mining techniques were unable to solve such complex problems (due to a lot of data, in different- different format). They require a lot of skilled people and efficient people (in domain expertise), i.e., human being only capable of what they are designed/ skilled for doing a task/work. In an Artificial Intelligence world (or we use it by users/ its manufacturers), deep learning role is much important in several domains like image classification, speech recognition, handwriting transcription, machine translation, etc. Deep learning is far different from machine learning in case of working and producing promises or outputs. Some difference between DL and ML can be included as: 1. Data Dependencies: Performance is the main key difference between both algorithms. Although, when the data is small, Deep Learning algorithms do not perform well. This is the only reason Deep Learning (DL) algorithms need a large amount of data to understand it perfectly. 2. Hardware Dependencies: Generally, Deep Learning depends on high-end machines while traditional learning depends on low-end machines. Thus, Deep Learning requirement includes Graphics Processing Units (GPUs). That is an integral part of it is working. They also do a large amount of matrix multiplication operations. 3. Feature Engineering: In this general process, domain knowledge is kept into the creation of feature extractors to reduce the complexity of the data and make patterns more visible to learn the algorithm working. Although, it is very difficult to process. Hence, it is time consuming and expertise. 4. Problem Solving Approach: Generally, we use the traditional algorithm to solve problems. However, it needs to break a problem into different parts to solve them individually. To get a result, combine them all. For Example, we have a task of multiple object detection. In this task, we have to identify what the object is and where is it present in the image. In a Machine Learning approach, we have to divide the problem into two steps: object detection and object recognition. First, we use the Grabcut algorithm to skim through the image and find all the possible objects. Then, of all the recognized objects, we would use an object recognition algorithm like Support Vector Machine (SVM) with Histogram of Oriented Gradients (HOG) to recognize relevant objects. 96

Challenges of Applying Deep Learning in Real-World Applications

5. Execution Time: Usually, Deep Learning takes more time to train as compared to Machine Learning. There are so many parameters in a Deep Learning algorithm, whereas Machine Learning takes much less time to train, ranging from a few seconds to a few hours. 6. Interpretability: We have interpretability as a factor for comparison of both learning techniques. Although, Deep Learning is still thought 10 times before its use in industry/ organisations. Figure 2 shows the difference of working structure of Machine Learning and Deep Learning Techniques. In summary in machine learning, supervised learning is used to predict value based on labelled data (e.g., weather-predicting AI), whereas unsupervised learning predicts value based on unlabelled data (e.g., behavior-predicting AI for an e-commerce website), while re-enforcement learning learns from its own or work on feedback process. Re-enforcement learning is also called reward-based learning. ML algorithm typically consists of two phases: training and testing. Training data is used to develop a model whereas testing phase is used to validate model. Basically, Machine Learning used to solve business problems like detecting spam, image recognition, product recommendation and predictive maintenance, etc. While, in Deep Learning solves problems using input layer, output layers and hidden layers. As much hidden layers are there in deep learning, the more complex problems can be solved. Also, as much small data set will be used in deep learning for processing, the result will be similarly inaccurate. In summary, deep learning make prediction using neural network.

Figure 2. Process of machine learning vs deep learning (Sambit Mahapatra, towardsdatascience.com, (2018))

97

Challenges of Applying Deep Learning in Real-World Applications

Distinctive Features of Deep Learning Generally, Deep Learning is used to extract meaningful data from data (collected from smart things/ internet of things internet connected things) a machine learning method. In general, deep learning is a subset of machine –learning techniques. Initially, concept of deep learning was taken from machine learning. As biggest advantage of deep learning and its popularity is: it works with large amounts of data, while machine learning techniques does not. The “Big Data Era” of technology will provide huge amounts of opportunities for new innovations in deep learning. According to a member of Google-Brain Project, “The analogy to deep learning is that the rocket engine is the deep learning models and the fuel is the huge amounts of data we need to feed to these (existing) algorithms” (Sambit Mahapatra, towardsdatascience.com, (2018)). Deep Learning requires high-end machines contrary to traditional Machine Learning algorithms. Graphical Processing Unit (GPU) has become an essential part in execution of a Deep Learning algorithm. GPU is used to handle large amount of data, in which processor works in parallel. In traditional Machine learning techniques, most of the applied features need to be identified by a domain expert in order to reduce the complexity of the data and make patterns more visible to learning algorithms to work. On another side, the biggest advantage Deep Learning algorithm also is that they try to learn high-level features from data in an incremental manner. This eliminates the need of domain expertise (skilled people) and hard-core feature extraction, i.e., in this extraction of meaningful information is done automatically by reward based learning (without human-intervention).

Why do we Require Deep Learning? We used deep learning to process a large amount of data (big data), especially images, prediction of prices, stock prices, etc. This algorithm is used to solve complex problems using neurons or a concept of hidden layers in its working. Apart from above importance, it is far better than machine learning techniques in producing results, i.e., produce results in minimum times. But, a problem with deep learning is that, it does not tell the way of output, i.e., “How it produced”, whereas in machine learning, we know “How an output is produced” with having every step of inputs or processing. Another major difference between Deep Learning and Machine Learning technique is the problem-solving approach. Deep Learning techniques tend to solve the problem end to end, whereas Machine learning techniques need the problem statements to break down to different parts to be solved first and then their results to be combine at final stage. For example, for a multiple object detection problem, Deep Learning techniques like Yolo net take the image as input and provide the location and name of objects at output. But, in usual Machine Learning algorithms 98

Challenges of Applying Deep Learning in Real-World Applications

like SVM, a bounding box object detection algorithm is required first to identify all possible objects to have the Histogram of Oriented Gradients (HOG) as input to the learning algorithm in order to recognize relevant objects. Usually, a Deep Learning algorithm takes a long time to train due to large number of parameters. Popular Residential Network (ResNet) algorithm takes about two weeks to train completely from scratch, whereas traditional Machine Learning algorithms take few seconds to few hours to train. The scenario is completely reverse in testing phase. And at testing time, Deep Learning algorithm takes much less time to run. Whereas, if we compare it with k-nearest neighbors (a type of machine learning algorithm), test time increases on increasing the size of data. Although this is not applicable on all machine learning algorithms, as some of them have small testing times too.

Strengths of Deep Learning As discussed above, Deep learning techniques are used with neural network to predict prices of a stock- market, price variation of an airline. In such applications, where prices are dynamic and keep changing everyday (also every hour) based on availability of trade shares/ available seats. Hence in such similar scenario, deep learning has following several strengths over other learning techniques, which are included here as: 1. No Need for Feature Engineering: Feature Engineering is the process of extracting features from raw data to explain better the selected problem. It is a work of machine learning technique, which improves model accuracy till a certain level. This process requires domain knowledge about a given/ identified problem, for example, in the real- estate business, the location of a land/ house has a significant impact on the selling price. Prices of houses/ land varied according to locations by locations (according to the population density). For a house/ land’s location, two parameters like latitude and longitude are essential. With these parameters (in putting together), we can reach to respective location. Note that single parameter has no use at all. Hence, combining the latitude and the longitude to identify a location is an example (feature) of feature engineering. In deep learning, feature engineering is done on its own (by machine) while other learning technique do not support this. A deep learning algorithm will scan the data to search for features that correlate and combine them to enable faster learning without being explicitly told to do so. With this ability, researchers/ data scientists can save a lot of times (months/ weeks of work). Moreover this, neural networks (a deep learning algorithm) is made of can uncover new, more complex features that human miss usually.

99

Challenges of Applying Deep Learning in Real-World Applications

2. No Need for Labeling of data: Getting good-quality training data is one of the biggest problems in machine learning because data labeling can be a tedious and expensive job. Sometimes, the data labeling process is simple, but timeconsuming. For example, labeling photos “dog” or “muffin” is an easy task, but an algorithm needs thousands of pictures to tell the difference. Other times, data labeling may require the judgments of highly skilled industry experts. So, for some examples, getting high-quality training data can be very expensive. For example, Microsoft’s project InnerEye is a tool that uses computer vision to analyze radiological images. To make correct, autonomous decisions, the algorithm requires thousands of well-annotated images where different physical anomalies of the human body are clearly labeled. Such work needs to be done by a radiologist with experience, and a trained eye. Given that around 4-5 images can be analyzed per hours, proper labeling of all images will be expensive (according to Glassdoor, an average base salary for a radiologist is $290.000 a year, i.e., $200 per hour). With deep learning, the need for welllabeled data is made obsolete as deep learning algorithms excel at learning without guidelines. Other forms of machine learning are not successful with this type of learning. In the respective example, a deep learning algorithm would be able to detect physical anomalies of the human body, even at earlier stages than human doctors. 3. Best results with Unstructured Data: According to research from Gartner, up to 80% of a company’s data is unstructured because most of it exists in different formats such as texts, pictures, pdf files and more. Unstructured data is hard to analyze for most machine learning algorithms, which means it is also going unutilized. In this area, deep learning can help a lot. Some examples of structured and unstructured data can be discussed in table 1. Deep learning algorithms can be trained using different data formats, and still derive insights, that are relevant to the purpose of its training. For example, a deep learning algorithm can uncover any existing relations between pictures, social media chatter, industry analysis, weather forecast, and predicting future stock prices of a company, etc. 4. Efficient at delivering high-quality results: Humans need as much as rest (because of having laziness) and fuel free of cost/ at lower price. They get tired or hungry and make silly mistakes (time to time, again and again). This case of tiredness, laziness, or feeling hungry is not for neural networks. Once trained correctly, a deep learning brain can perform thousands of repetitive, routine tasks within a shorter period of time than it would take a human being. The quality of its work never diminishes, unless the training data includes raw data that does not represent the problem we are trying to solve.

100

Challenges of Applying Deep Learning in Real-World Applications

Table 1. Comparison of used structured data and unstructured data in deep learning Structured data

Unstructured data

Characteristics

Pre-defined data models, usually text only, easy to search

No Pre-Defined Data Model, May Be Text, Images, Sound, Video or Other Formats, Difficult to Search

Resides in

Relational Databases, Data Warehouses

Application, NoSQL Databases, Data Warehouses, Data Lakes

Generated by

Human or Machines

Human or Machines

Typical applications

Airline Reservation Systems, Inventory Control, CRM systems, ERP systems

Word Processing, Presentation Software, Email Clients, Tools for Viewing or Editing Media

Examples

Dates, Phone Numbers, Social Security Numbers, Credit Card Numbers, Customer Names, Addresses, Product Names and Numbers, Transaction Information.

Test files, Reports, Email Messages, Audio Files, Video Files, Images, Surveillance Imagery.

In summary, there are some advantages of deep leaning like it (deep learning) gives good performance with problems which require several solutions in multiple domains. This includes speech, language, vision, playing games like Go, etc. It is useful in reducing the need for feature engineering, one of the most time-consuming parts of machine learning practice. In last, it is used an architecture that can be adapted to new problems relatively easily, for example, Vision, Time series, Language etc., are using techniques like Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), etc.

Weaknesses of Deep Learning Deep learning (also known as deep structured learning or hierarchical learning) is a family of artificial intelligence to produce intelligent decisions using supervised, semi-supervised or unsupervised learning techniques. It works far better than machine learning techniques. But as discussed above and in (Grm, K., et.al (2017), & Najafabadi, M. M., et.al (2015)], some weaknesses of deep learning are, i.e., it does not work efficiently with bad quality of pictures/ images (having less colours in images), with partial information/ data, small amount of data. Apart that, there are some other/ several disadvantages of deep leaning techniques like •

Deep Learning requires a large amount of data, if we only have thousands of samples, deep learning is unlikely to outperform other approaches.

101

Challenges of Applying Deep Learning in Real-World Applications

• • • •

Deep Learning is extremely computationally expensive to train. The most complex models take weeks to train using hundreds of machines equipped with expensive GPUs (Graphics Processing Unit). Deep Learning does not require strong theoretical foundation. Deep Learning determines the topology/ training method/hyper-parameters (Lorenzo, P. R., et.al (2017, July)) for training/ learning purpose, i.e., it is a black art without a theory. In Deep Learning, what is learned is not easy to understand/ implement. In this, other classifiers (e.g. decision trees, logistic regression, etc.) make it much easier to understand (in terms of explanation of a process).

Deep Learning means “how much a machine can think deeply (like a human) and can produce efficient results”. Using this technique, machine learns by example, i.e., reward or feedback based learning, but still we find several weaknesses of this learning technique, which are discussed in this section in brief. Now, next section of this chapter will discuss several limitations of deep learning techniques.

The Limitations of Deep Learning The use of deep learning in several applications is nearly infinite. Many applications are unreachable for current deep learning techniques which had large amounts of human-or machine generated data, for example, we collect a dataset of billions of English language descriptions of the features of a software product (written by a product manager/corresponding source code developed by a team of engineers to meet these requirements). But, with this collected data, we cannot train a deep learning model efficiently, i.e., which can easily read a product description and generate the appropriate codebase. Basically, reasoning method like programming, or applying the scientific method, i.e., long-term planning, and algorithmic-like data manipulation also do not work properly for deep learning models, i.e., it does not matter that how much data we collected or generated (or produced) for those models. Even getting explanation of a sorting algorithm with the help of deep neural network is really a complex process. Now the question is “why a deep learning model is just a chain of simple, continuous geometric transformations mapping one vector space into another”? Generally, deep learning works on Deep Neural Network and on its training and testing of data-set (re-sampling of data again and again). A Deep Neural Network (DNN) can solve a task, even if it training less but accuracy of the algorithm may differ than more trained data. Hence, now some limitations of deep learning are included as:

102

Challenges of Applying Deep Learning in Real-World Applications

•

•

•

•

Scaling Deep Learning Techniques: Scaling Deep Learning techniques by stacking more layers and using more training data create several issues. It will not solve the more fundamental problem that deep learning models are very limited in what they can represent, and all programs which have been learnt, cannot be denoted for simpler (continuous) representation. Scaling of deep learning techniques (for determining expected output) always is a complex task. Risk of Anthropomorphizing Machine Learning Models: One very real risk with contemporary Artificial Intelligence is that of misinterpreting what deep learning models do, and overestimating their abilities. A fundamental feature of the human mind is our “theory of mind”, our tendency to project intentions, beliefs and knowledge on the things around us. Drawing a smiley face on a rock suddenly makes it “happy” in our minds. Applied to deep learning, this means that when we are able to somewhat successfully train a model to generate captions to describe pictures, for example, we are led to believe that the model “understands” the contents of the pictures, as well as the captions it generates. We then proceed to be very surprised when any slight departure from the sort of images present in the training data causes the model to start generating completely absurd captions. In particular, this is highlighted by “adversarial examples”, which are input samples to a deep learning network that are designed to trick the model into misclassifying them. Through gradient ascent, one can slightly modify an image in order to maximize the class prediction for a given class. By taking a picture of a panda and adding to it a “gibbon” gradient, we can get a neural network to classify this panda as a gibbon. This evidence both the brittleness of these models, and the deep difference between the input-to-output mapping that they operate and our own human perception. Interpretability: Interpretability is the main issue why many sectors using other Machine Learning techniques over Deep Learning. For example, suppose we use deep learning to calculate the relevance score of a document. The performance it gives is quite excellent and is near human performance. But there is an issue, i.e., it does not reveal why and how it has given that score (called black box problem). Indeed, mathematically we can find out which nodes of a deep neural network were activated, but we do not know what their neurons were supposed to model and what these layers of neurons were doing collectively. So, we fail to interpret the results. Which is not in case of Machine Learning algorithms like decision trees, logistic regression etc.? Standards: No unique standards are available to use deep learning technique on different types of data (collected from many devices, enabled in various 103

Challenges of Applying Deep Learning in Real-World Applications

applications). As we know, deep learning is also applied on images, and audio based content. So “What types of standards of images or audio should be available for applying this concept/ deep learning on collected images/ audio data-sets”? For such issues, we need to find efficient solutions from several research communities in near future. In simple words, deep learning models do not have any understanding of their input, at least not in any human sense. Our own understanding of images, sounds, and language, is grounded in our sensorimotor experience as humans as embodied earthly creatures. Machine learning models have no access to such experiences and thus cannot “understand” their inputs in any human-relatable way. By annotating large numbers of training examples to (as)input/ feed into our models, we get them to learn a geometric transform that maps data to human concepts on this specific set of examples, but this mapping is just a simplistic sketch of the original model in our minds, the one developed from our experience as embodied agents, it is like an image with low pixel in a mirror. A machine learning practitioner (and keeping this in our mind) should never fall into the trap of believing that neural networks understand the task they perform. They (deep learning models) were trained on a different, far narrower task than the one we wanted to teach them, that of merely mapping training inputs to training targets (point by point). If we show them anything that deviates from their training data, and they will break in the most absurd ways, which is a major limitation of Deep Learning. Hence, this section discusses about limitation of deep learning in detail. Now next section will deal with workingphenomena of deep learning.

Working of Deep Learning Deep Learning is a technique for classifying information through layered Neural Networks (NN). The working structure of this learning technique is quite similar to human, i.e., it works like how a human brain works. In this process, Neural Networks (NN) has a set of input units, where raw data is given/ fed as input (like pictures, or sound samples, or written text). Then inputs are mapped with the output nodes, which determine the category to which the input information belongs. For example, it can determine that the input picture contains a cat, or that the small sound sample was the word “Hello”. Deep Neural Networks (DNN) consist power of deep learning algorithms. It also has several hidden layers between the input and an output node, which makes them capable of making much more complicated classifications of data (see figure 3). Deep learning algorithms need to be trained with large sets of labelled data (for getting higher accuracy of results). For example, we put in/ give it thousands of pictures of cats or dogs before it can start classifying new cat pictures 104

Challenges of Applying Deep Learning in Real-World Applications

with relative accuracy. The larger the training data set, the better the performance of the used algorithm/ method will be. Also, with using more hidden layers in a Deep Neural Network (DNN), more difficult/ complex problems can be solved. Today’s big firms/ companies or Multinational Companies are using more and more data (large amount of data), and are willing to offer their (maximum) services for free of cost (i.e., in exchange for accessing user data). The more classified information about their users they have, the better they will be able to train or implement their deep learning algorithms. With that, firms/ organisations can produce some efficient decision for themselves, i.e., to fight against their market competitors. It is gamechanging process, i.e., changing the competition scenario of a market (i.e., related to one product). This will in turn make their services more efficient than those of their competitors and bring them more customers (some of whom will pay for their premium services). Hence, now when and where we can use Deep Learning, with respect to that, some basic facts are discussed as: • • • • •

Deep Learning outperforms other techniques if the data size is large. But with small data size, traditional Machine Learning algorithms are preferable. Deep Learning techniques need to have high end infrastructure to train in reasonable time. When there is lack of domain understanding for feature introspection, Deep Learning techniques outshines others as you have to worry less about feature engineering. Deep Learning really shines when it comes to complex problems such as image classification, natural language processing, and speech recognition. Deep Learning outperforms other techniques if quality of images in high definition.

Deep Learning With Neural Network The neurons are grouped into three different types of layers: Input Layer, Hidden Layer(s) and Output Layer. With these three layers, we can understand the process of figure 3 with one example. We need to identify first an image from a collection of images. Using deep learning models/ techniques, an image can be pre-processed using three phases, i.e., in identifying pattern of local contrasts, face features and in last respective faces (for example: face recognition). Hence here, the input layer receives input data. In our case, we have four neurons in the input layer: Pictures, local Contrast in a picture, face features and faces. The input layer passes the inputs to the first hidden layer. The hidden layers perform mathematical computations on our inputs. One of the challenges in creating neural networks is deciding the number of hidden layers, as well as the number of neurons 105

Challenges of Applying Deep Learning in Real-World Applications

Figure 3. Deep learning with neural network

(Credit: Edureka Videos and Introduction on Deep Learning)

for each layer. Note that the word “Deep” in Deep Learning refers to having more than one hidden layer. And in last, the output layer returns the output data. In our case, it gives us the face recognition with respect to some given features. Hence in summary, we can say that the more hidden layer in a deep learning are being used, the accurate results will be received. Hence, this section discusses the main working of deep learning, in which we found that hidden layers play an important role in processing of deep learning. Now, several problems or issues raised in deep learning will be discussed in next section.

Problems/Issues With Deep Learning We have discussed that deep learning technique is too useful (beneficial)in producing some predictions/ decisions based on some data-sets than its other family’s fields like machine learning, and artificial intelligence. But it has also raised several issues in it time to time. Apart that, if some algorithms are unable to understand a data then even collected meaningful/ cleaned data also can give inaccurate results. Several problems have been investigated like this problem in deep learning, which will be discussed here. We need to look over such issues in near future. 1. Deep Learning is data hungry: In a world with infinite (i.e., huge) data, and infinite computational resources, we always require some efficient techniques to handle it (Tyagi, A. K., & Reddy, V. K. (2019)). But, were always fail to deliver appropriate tool or methods, because this data is growing everyday (by billions of internet of things or internet connected things). The tools proposed by several researchers do not provide every possible labelled sample of a problem space 106

Challenges of Applying Deep Learning in Real-World Applications

to a deep learning algorithm. Therefore, it will have to generalize or interpolate between its previous samples in order to classify data it has never seen before (i.e., a new image or sound that is not contained in its dataset). Currently, Deep Learning lacks a mechanism for learning abstractions through explicit, verbal definition, and works best when there are thousands, millions or even billions of training examples Marcus, G. (2018). Soon essential questions raised here, “what happens when deep learning algorithm does not have enough quality training data”? It can fail instantly, such as mistaking a rifle for a helicopter, or humans for gorillas. The heavy reliance on precise and abundance of data also makes deep learning algorithms vulnerable to spoofing. Testament to the fact are many crazy stories such as deep learning algorithms mistaking stop signs for speed limit signs with a little defacing, or British police software not being able to distinguish sand dunes from nudes (Ben Dickson, bdtechtalks. com, (2018)). 2. Deep Learning is Shallow: Deep Learning algorithms are very good at mapping inputs to outputs but not so much at understanding the context of the data they are handling. In fact, the word “deep” in deep learning is much more a reference to the architecture of the technology and the number of hidden layers it contains rather than an allusion to its deep understanding of what it does. For example, naturally apply to abstract concepts like ‘justice,’ ‘democracy’ or ‘meddling’” (Ben Dickson, bdtechtalks.com, 2018). In an example like gaming, deep learning algorithms can become very good at playing games, and they can eventually beat the best human players in both video and board games. However, this does not mean that Artificial Intelligence (AI) algorithm has the same understanding as humans in the different elements of the game. It has learned through trial and error that making those specific moves will prevent it from losing. For example, Google DeepMind’s mastering of the Atari game. According to the designers of this algorithm (used in respective game), the game realized after 240 minutes that the best way to beat the game is to dig a tunnel in the wall. But it neither knows what a tunnel or a wall is. It has made itself expert through millions of trials and errors combinations, to beat another player (playing, and winning a game with high points in the possible shortest time). 3. Deep Learning is Opaque: While decisions made by rule-based software can be traced back to the last, but in case of machine learning and deep learning algorithms, this facility is not available. This missing feature (of transparency) in deep learning is called as “black box” problem. Deep learning algorithms shift through millions of data points to find patterns and correlations that often go unnoticed to human experts. The decisions they make based on these findings often confound even the engineers who created them. This might not 107

Challenges of Applying Deep Learning in Real-World Applications

be a problem when deep learning is performing a trivial task where a wrong decision will cause little or no damage. But, when it is deciding the fate of a defendant in court or the medical treatment of a patient, mistakes can have more serious repercussions. Today’s the transparency issue is yet to be solved. It is a potential liability when using deep learning for problem domains like financial trades or medical diagnosis (Cao, C., et.al (2018)), in which human users might like to understand how a given system made a given decision (Ben Dickson, bdtechtalks.com (2018), Hagendorff, T. (2019)). Algorithmic bias as one of the problems stemming from the opacity of deep learning algorithms. Machine learning algorithms often inherit the biases of the training data the ingest, such as preferring to show higher paying job ads to men rather than women, or preferring white skin over dark in adjudicating beauty contests. These problems are hard to debug in development phase and often result in controversial news headlines when the powered software (included deep learning) goes into production. 4. Is Deep Learning doomed to fail? No, but it is bound for a reality check. In general, deep learning is “a perfectly fine way of optimizing a complex system for representing a mapping between inputs and outputs, given a sufficiently large data set” (Ben Dickson, bdtechtalks.com (2018), Marcus, G. (2018)). Deep learning must be acknowledged for what it is, a highly efficient technique for solving classification problems, which will perform well when it has enough training data and a test set that closely resembles the training data set. Note that if we do not have enough training data, or when our test data differs greatly from our training data, or when we are not solving a classification problem, then deep learning becomes a square peg slammed into a round hole, a crude approximation when there must be a solution elsewhere (Marcus, G. (2018)). Deep learning technique need to be combined with other technologies (i.e., with plain-old rule-based programming/ other AI techniques such as reinforcement learning). In (www.wired.com, (2018)), author proposes neuroscience as the key to creating real AI that will be able to achieve human-like problem solving. 5. Deep Learning Needs Enough Quality Data: Deep learning works when it is performed on large amount of data and it improves as the collected data increases. However, when enough quality data simply is not fed into a deep learning system, it can fail badly. In 2017, researchers fooled Google deep learning systems into making errors after altering the available data by adding “noise”. While the errors unearthed by the researchers related to trivial matters (i.e., mistaking rifles for turtles in image recognition algorithms). This discusses the dependency of deep learning on the right quantity and quality of data for it (to work accurately). With small input variations in data quality having such profound results on outcomes and predictions, there is a real need to ensure 108

Challenges of Applying Deep Learning in Real-World Applications

6.

7.

8.

9.

greater stability and accuracy in deep learning. Furthermore, in some firms/ industries like industrial applications, sufficient data might not be available, it limits deep learning’s adoption. Artificial Intelligence and its Expectations: There is a difference between the layman’s expectations of Artificial Intelligence (AI) technologies and the actual uses of AI in several applications/ domains. The perception which printmedia/ media want to report is one of the super-smart computers with cognitive capabilities which will replace many jobs (done by human beings). But the computer and data science industries have a challenge to address that AI is a tool to enhance productivity with maximum human roles (with providing high accuracy with high productivity). AI requires minimum cases of errors in its process/ execution. In general, AI can do work efficiently with automation of mundane tasks, data-driven predictions, and optimizations, etc. In summary, AI cannot replace a human being, i.e., what a human brain thinks time to time and how he thinks (to do/ solve critical tasks). Becoming Production-Ready: Today’s people/ firms are investing in area of Artificial Intelligence (AI), to build more efficient artificial intelligence/ machine, which can work faster, smarter and in a secure way. Providing security to all internets of things, and machines is really a critical task (also biggest challenges). An AI is expected to solve real-life problems/ any worthwhile problem. For that, AI capabilities should be upgraded with new technology infrastructure (to address security concerns). With this, security of the AI platforms, and delivering of solutions to respective (notified) problems will be available instantly (e.g., deliver results, predictions, etc., when needed). A critical goal of AI in the enterprise for machines to assist executives and key stakeholders in making important decisions; both strategic and tactical. Deep Learning does not Understand Context Very Well: The word “deep” in deep learning denotes to its architecture than the level of understanding that these algorithms are currently capable of producing. For example, a deep learning algorithm might become highly proficient at mastering a video game to the point it can easily defeat human players. However, based on a different game, we require to change the setting of neural network to train it again, because to new game or its context it will not understand in first time. For any application, we need to train a neural network once. Similarly, in Internet of Things (IoTs), everyday new data is being generated, for that setting or training neural network is not an easy task. Deep Learning Security: Deep Learning networks require security component in several applications (Najafabadi, M. M., et.al (2015)), i.e., cyber security is needed to be implement in several application (on network). However, giving inputs to these deep learning models and expected outputs from these models 109

Challenges of Applying Deep Learning in Real-World Applications

(also networks themselves) have a possibility of altering in input, which may affect output (produced one) completely. Note that deep learning networks may be vulnerable to malicious attacks. For example, self-driving vehicles are partly powered by deep learning. Here, if one was to access the deep learning model and make input alterations, vehicle behaviour could potentially be controlled in a malicious manner. In past, a black-box attack on several deep learning networks has been attacks, i.e., which misclassify the results (making results irrelevant for any future use). Hence, there are several problems have been investigated with respect to deep learning in the past decade. And many more are still to be count/ focus. Here, we should not forget that deep learning era is an infancy stage too, so much solutions are not available in current. Note that training a Neural Network is hardest part of Deep Learning, because for that we need a large data set. Also, we need a large amount of computational power. In next section, this chapter will discuss about several challenges in deep learning.

Challenges in Deep Learning Deep Learning is largely responsible for today’s growth (also using Artificial Intelligence (AI)). The technology has given computers a lot of powers in most of the areas, for example, ability to recognize natural language/ speech of a human being, medicine, finance, marketing, etc. Deep Learning has transformed computer vision and improved machine translation. Several areas for Deep learning are still uncovered, but in the existing one, several challenges have been mitigated which are being addressed here as: 1. Lots and Lots of data: Deep Learning algorithms are trained to learn so fast (in minimum time) using data (learning for producing predication/ decision from collected data). Large data sets are required to make sure that the machine delivers desired and accurate results (to respective user/ firms/ organisations). As human brain needs a lot of time, experiences to learn and extract information (or facts), similarly analogous Artificial Neural Network (ANN) requires large amount of data. The more powerful machine/ system we want to build, the more parameters (also features) need to be considered and more parameters require more data. For example, for speech recognition, machine would require data from multiple dialects, demographics and time scales. For that, researchers feed terabytes of data for the algorithm (in a machine) to learn a single (natural) language, which is a time-consuming (also requires tremendous data processing capabilities). Hence using deep learning, we can 110

Challenges of Applying Deep Learning in Real-World Applications

solve a problem with providing large amount of data with enough training and testing of respective data-sets. Note that the complexity of a neural network algorithm can be represented with using number of parameters. In deep neural networks, used parameters can be in millions, tens of millions or hundreds of millions (depend on cases/ problems). Remember that if a user’s requirements change, then data will change, and then outcomes of an algorithm will change surely. Now here, a question is arising that “How much data is required to train an algorithm efficiently”? For neural networks, the amount of data needed for training will be much higher compared to other machine learning algorithms, because task of a deep learning algorithm is two-folded. First, it needs to learn about the domain, and only then solve the problem. When the training begins, the algorithm starts from scratch. To learn about a given domain, the algorithm needs a huge number of parameters (also require a lot of experiences to learn and collect information about the world around us) to tune and “play around with”. With that, a deep learning algorithm can think much like as a human brain (not completely). On reaching such level, deep learning can produce some useful decisions. For example, we know that gas is hot by putting out finger on it, or snow melts at warm temperature when we try to bring it home or in presence of air. Similarly, a neural network learns (with learning patterns, experiences, etc.) about a stock price of a given company, i.e., which one stock will be increased or decreased by end of the day. Note that for getting higher accuracy with respect to stock price, we require large amount of data-sets (i.e., training and testing on it again and again). 2. Overfitting in Neural Networks/ the model: As discussed above, a model is typically trained by maximizing its performance on a particular training data set. The model memorizes the training examples but does not learn to generalize to new situations and data set. In general, Overfitting refers to an algorithm that models the “training data” too well (i.e., one that overtrains the model). Overfitting occurs in complex models when having too many parameters relative to the number of observations. Overfitting occurs when an algorithm learns the detail and noise in the training data to the extent that negatively impacts the performance of the model in real-life scenarios. It is a major problem in neural networks, also in deep learning (because modern network share very large numbers of parameters and thereby a lot of “noise”). Now here, some questions are arising, i.e., “How do we know if our model is overtrained” and “When the accuracy stops improving after a certain number of epochs”? We need to provide answer to such questions, if we are working with neural networks/deep learning.

111

Challenges of Applying Deep Learning in Real-World Applications

3. Hyperparameter Optimization: Hyperparameters are the parameters which are defined prior to the starting of a learning process (Lorenzo, P. R., et.al (2017, July)). Changing the value of such parameters by a small amount can create a major change in the performance of our implemented model. Depending on the default parameters and not performing Hyperparameter Optimization can have useful impact on the model performance. Also, having too few hyperparameters and hand tuning them rather than optimizing through proven methods is also a performance driving aspect. 4. Requires High-Performance Hardware: Training a data set for a Deep Learning solution needs lots of data. To solve real world problems using deep learning, the machine needs to be equipped with adequate processing power (e.g., GPU). To ensure better efficiency and less time consumption, researchers move to multi-core high performing GPUs and similar processing units. These processing units are costly and consume a lot of power. Industry level Deep Learning systems require high-end data centers while smart devices such as drones, robots, mobile devices, etc., require small, and efficient processing units. Using deep learning algorithms to solve real world problems is a costlier and power consuming affair. 5. Neural Networks are essentially a Blackbox: Neural networks are essentially Black-boxes and researchers do not know that how they receive results/ conclusions with using neural networks. Basically, we know which data we have feed to machine or deep learning algorithm, but we cannot say anything about outcomes, i.e., how we have received that (operations are largely invisible to humans). Note that we should remember that deep learning uses neural network concept to run itself and do not require human intervention to predict decision. It is one of popular limitations of deep learning, i.e., we do not know that how a neural network reaches at a particular solution (refer section 1.2). It is impossible to look inside of it to see how it works (called black box problem). The lack of ability of neural networks makes it difficult to implement for highlevel cognitive functions. Similar to a human brain, the reasoning of a neural network is done with the behavior of thousands of simulated neurons, then it arranged into dozens/ hundreds of intricately interconnected layers. With several hidden layers processing (where inputs are sent from one level to the next level until an overall output is produced), we produce an output. Also, deep learning uses a process known as back-propagation to tweaks individual neurons in such a way that network learn to produce the desired output faster. Hence, lack of transparency, interpretability fails deep learning algorithms (makes it hard to predict), even some time neural networks produce great results. For example, Deep Patient, a deep learning program that was applied to patient records of more than 7,00,000 individuals at Mount Sinai Hospital in New York. After 112

Challenges of Applying Deep Learning in Real-World Applications

a long training period, Deep Patient was able to detect certain illnesses better than human doctors. On one hand, this was great news. On the other, if a tool like Deep Patient is actually going to be helpful to medical personnel, it needs to provide the reasoning for its prediction, to reassure their accuracy and to justify a change in someone’s treatment. Without the justification, it is difficult to gain the trust of patients or learn why any mistakes in diagnosis were made. This issue is very critical and require a spark/ light to find justification for a received output (by neural network). 6. Lack of Flexibility and Multitasking: Deep Learning models, once trained, can deliver efficient and accurate solution to a real-world problem. However, the neural network architectures are highly specialized to specific domains of application. To solve a problem, we require retraining and reassessment, i.e., deep learning models which can multitask without the need of reworking on the whole architecture. In (Manor, R., Mishali, L., & Geva, A. B. (2016)), researchers presented a paper on Multi Model, i.e., a neural network architecture that extract from the success of vision, language and audio networks to simultaneously solve a number of problems spanning multiple domains, including image recognition, translation and speech recognition. Hence, Deep Learning is one of the primary research areas for Artificial Intelligence (AI)/ of using AI. But, while exploring new, or less explored territories of cognitive technology, it is very difficult to solve raised hurdles and difficulties. Today’s the future requires the answer for a question “Is Deep Learning our best solution towards real AI”? In summary, researchers must come with some efficient solutions to overcome above raised challenges (with respecting to adopting/ using deep learning). In summary, deep learning challenges are five-folded, i.e., First, we require to process large amount of data-sets for training using modern tools/ hardware like GPUs (Graphics Processing Units). Emerging techniques like transfer learning and generative adversarial networks show some promise with regard to overcoming this challenge. Second, deep learning works very well, because it uses large number of interconnected neurons, or many parameters (for better accuracy), which allow for capturing subtle nuances and variations in data. However, this also means that it is harder to identify hyper-parameters, parameters whose values need to be fixed before training. As discussed above, there is also a danger of over-fitting data (i.e., when many parameters being used/ when the number of parameters greatly exceeds the number of independent observations).Third, deep learning networks require a lot of time for training, thereby making it very hard to quickly retrain models on the edge using newly available information, i.e., issues like latency and privacy will be lost in era of edge computing. Fourth, due to the large number of layers, nodes, and 113

Challenges of Applying Deep Learning in Real-World Applications

connections, it is very difficult to understand that “how deep learning networks arrive at particular results”? This issue may not be essential in applications like tagging photos on online social networking, but it is very important in mission-critical applications like predictive maintenance or clinical decision-making. Finally, deeplearning networks are highly susceptible to the butterfly effect small variations in the input data can lead to drastically different results, making them inherently unstable. Hence, for developing an efficient deep learning algorithm (for tuning and assessing the performance of these algorithm based on empirical data), we require: 1. To measure the complexity of the models, as a function of the number and type of computational units, model topology/structure, model generalization, and learning efficiency; 2. To allow the definition of theoretically grounded strategies for tuning and assessing the performance of models learned from empirical data; 3. To develop regularization schemes. A specific framework for assessment of unsupervised learning is also needed. Note that computational efficiency is an essential issue in deep learning. In current, deep learning models require a large amount of data to give efficient performances. But this data is growing at a rapid pace every day, so sufficient and efficient model require for handling (analysing) this data (with properly training, with parallel and in distributed computational infrastructures).

SUMMARY AND DISCUSSION Generally, Deep learning is used to extract useful patterns from data. It is a sub-field of machine learning (also Artificial Intelligence). It is work with the powers like a human- being (using learning structures based on artificial neural network). Deep learning algorithms with a properly training and testing of large amount of datasets (available in different data formats) are much useful for various use cases. For example, Deep learning algorithm uncover existing relations between pictures, social media chatter, industry analysis, weather forecast, predict future stock prices of a given company, ticket prices of a journey (in airlines/ airplane ticket price estimator, etc. Hence, Deep learning is summarized with seven essential points: • •

114

Deep Learning uses a Neural Network to imitate animal intelligence. There are three types of layers of neurons in a neural network: The Input Layer, the Hidden Layer(s), and the Output Layer.

Challenges of Applying Deep Learning in Real-World Applications

• • • • •

Connections between neurons are associated with a weight, dictating the importance of the input value. Neurons apply an Activation Function on the data to “standardize” the output coming out of the neuron. To train a Neural Network (to get accurate results), we require a large data-set. Iterating through the data set and comparing the outputs will produce a Cost Function, indicating how much the AI is off from the real outputs. After every iteration through the data set, the weights between neurons are adjusted using Gradient Descent to reduce the cost function.

As discussed above, there three major drivers which caused the breakthrough of (deep) neural networks are: the availability of huge amounts of training data, powerful computational infrastructure, and advances in academia. Today’s deep learning systems have started to outperform not only classical methods, but also with other areas like identification of human-benchmarks (i.e., image classification or face recognition). Deep learning has opened a clear road for research community to do their research work or to solve real-world problems. Hence, to make deep learning powerful and more useful in coming era, we need to overcome above discussed challenges like computational efficiency, transparency, interpretability, latency, privacy. In deep learning, most of big questions of intelligence have not been answered nor properly formulated, which is a major issue in this learning technique. Deep learning is an evolving technology. In spite of promising results stated so far, there is a need for much to be developed. We have seen success of deep learning in the fields of speech, image and language recognition. But in the tasks with nontrivial representations like information retrieval and natural language processing much research need to be carried off. Recent research work showed a vast improvement need in optimization techniques for the architecture of deep learning (Martens, J., & Sutskever, I. (2011), Bengio, Y., et.al (2013, May), Sutskever, I. (2013), Dean, J., (2012), Le, Q. V., et.al (2011, June)). In (Coates, A., (2013, February) experimental results show the need for extending the training of deep features when very large amounts of training data are available. The scope is to find whether full set of parameters are needed in deep learning or not. One important issue to address is to develop scalable and effective parallel algorithms for training deep architecture with massive datasets at the Web scale. Recent advances in developing asynchronous stochastic gradient learning using large-scale CPU clusters (e.g., (Dean, J., (2012), Le, Q. V., et.al (2011, June)) and GPU clusters (Coates, A., (2013, February) shows promising results. To make deep learning techniques scalable to very large training data, theoretically sound parallel learning algorithms or more effective architectures than the existing ones need to be further developed (e.g., Bengio, Y., et.al (2013, May), Sutskever, I. (2013), Dean, J., 115

Challenges of Applying Deep Learning in Real-World Applications

(2012)). One major barrier to the application of DNNs and related deep models is that it currently requires considerable skills and experience to choose sensible values for hyper-parameters such as the learning rate schedule, the strength of the regularized, the number of layers and the number of units per layer, etc. A solid theoretical foundation of deep learning needs to be established to learn in unsupervised mode. Moreover this, the interest and enthusiasm of deep learning is growing and already too popular now days in several real-world applications, for example, consumerrelated applications like helping voices of Siri and Cortana, Google Photos’ tagging feature, Grammarly’s proof reader, and Spotify’s music recommendations are some example of deep learning algorithm. Also, a bigger impact/ usage of deep learning is also found in the business world, for example, customer data in CRM systems, social media and other online data to better segment clients, predict churn and detect fraud. The financial industry is depending a lot on deep learning algorithms to deliver stock price predictions and execute trades at the right time. In the healthcare industry deep learning networks are exploring the possibility of repurposing known and tested drugs for use against new diseases to shorten the time before the drugs are made available to the general public. Governmental institutions are also turning to deep learning for help to get real-time insights into metric like food production and energy infrastructure by analyzing satellite imagery. There is a long list (of usecases of deep learning algorithm) to continue. Hence in near future, we can expect large investments to be made to further perfect this technology (deep learning), and more and more of the current challenges to be solved in the future. In last, for solving real world problems and making human life easier, we should say thanks to Deep Learning and Artificial Intelligence for a Bright Future.

REFERENCES Awad, M., & Khanna, R. (2015). Efficient learning machines: theories, concepts, and applications for engineers and system designers. Apress; doi:10.1007/978-14302-5990-9 Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013, May). Advances in optimizing recurrent networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 8624-8628). IEEE. doi:10.1109/ ICASSP.2013.6639349 Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., ... Xie, Z. (2018). Deep learning and its applications in biomedicine. [PubMed]. Genomics, Proteomics & Bioinformatics, 16(1), 17–32. doi:10.1016/j.gpb.2017.07.003

116

Challenges of Applying Deep Learning in Real-World Applications

Coates, A., Huval, B., Wang, T., Wu, D., Catanzaro, B., & Andrew, N. (2013, February). Deep learning with COTS HPC systems. In International conference on machine learning (pp. 1337-1345). Academic Press. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., & Ng, A. Y. (2012). Large scale distributed deep networks. In Advances in neural information processing systems (pp. 1223–1231). Academic Press. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., . . . Ng, A. Y. (2012). Large scale distributed deep networks. In Advances in neural information processing systems (pp. 1223-1231). Academic Press. Coates Deng, L., & Yu, D. (2014). Deep learning: methods and applications. Foundations and Trends® in Signal Processing, 7(3–4), 197-387. Grm, K., Štruc, V., Artiges, A., Caron, M., & Ekenel, H. K. (2017). Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biometrics, 7(1), 81–89. doi:10.1049/iet-bmt.2017.0083 Hagendorff, T. (2019). The Ethics of AI Ethics--An Evaluation of Guidelines. arXiv preprint arXiv:1903.03425 Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A. R., Jaitly, N., ... Sainath, T. (2012). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29. Huval, B., Wang, T., Wu, D., Catanzaro, B., & Andrew, N. (2013, February). Deep learning with COTS HPC systems. In International conference on machine learning (pp. 1337-1345). Academic Press. Le, Q. V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., & Ng, A. Y. (2011, June). On optimization methods for deep learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning (pp. 265-272). Omnipress. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., ... Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. [PubMed]. Medical Image Analysis, 42, 60–88. doi:10.1016/j.media.2017.07.005 Lorenzo, P. R., Nalepa, J., Ramos, L. S., & Pastor, J. R. (2017, July). Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (pp. 1864-1871). ACM. doi:10.1145/3067695.3084211

117

Challenges of Applying Deep Learning in Real-World Applications

Manor, R., Mishali, L., & Geva, A. B. (2016). Multimodal neural network for rapid serial visual presentation brain computer interface. [PubMed]. Frontiers in Computational Neuroscience, 10, 130. doi:10.3389/fncom.2016.00130 Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631 Martens, J. (2010, June). Deep learning via hessian-free optimization. ICML, 27, 735–742. Martens, J., & Sutskever, I. (2011). Learning recurrent neural networks with hessianfree optimization. In Proceedings of the 28th International Conference on Machine Learning (ICML-11) (pp. 1033-1040). Academic Press. Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015). Deep learning applications and challenges in big data analytics. Journal of Big Data, 2(1), 1. doi:10.118640537-014-0007-7 Samuel, A. L. (2000). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 44(1.2), 206-226. Seymour, V. (2016). The human–nature relationship and its impact on health: A critical review. [PubMed]. Frontiers in Public Health, 4, 260. doi:10.3389/fpubh.2016.00260 Sutskever, I. (2013). Training recurrent neural networks. Toronto, Canada: University of Toronto. Tyagi, A. K. (2019). Machine Learning with Big Data. Retrieved from https:// towardsdatascience.com/why-deep-learning-is-needed-over-traditional-machinelearning-1b6a99177063 Tyagi, A. K., & Reddy, V. K. (2019). Performance Analysis of Under-Sampling and Over-Sampling Techniques for Solving Class Imbalance Problem. Retrieved from https://www.edureka.co/blog/what-is-deep-learning

118

119

Chapter 5

Challenges and Applications for Implementing Machine Learning in Computer Vision: Machine Learning Applications and Approaches Hiral R. Patel Ganpat University, India Ajay M Patel AMPICS, India Satyen M. Parikh FCA, India

ABSTRACT The chapter introduces machine learning and why it is important. Machine learning is generally used to find knowledge from unknown data. There are many approaches and algorithms available for performing machine learning. Different kinds of algorithms are available to find different patterns from the data. This chapter focuses on different approaches with different usage.

INTRODUCTION The world is seeing the ongoing stream of a wide range of organized and unstructured information from internet based life, correspondence, transportation, sensors, and gadgets. World wide Data Corporation (IDC) is gauges that 180 zettabytes of information will be produced by 2025. This blast of information has offered ascend to DOI: 10.4018/978-1-7998-0182-5.ch005 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Challenges and Applications for Implementing Machine Learning

another economy known as the Data Economy or Importance. Information is the new oil that is valuable yet helpful just when cleaned and prepared. There is a consistent fight for responsibility for between endeavors to get profits by it. The information economy with its tremendous supply is empowering extraordinary development in information sciences, the field which manages extricating helpful data and bits of knowledge from the accessible information. Information science is going toward another worldview where one can instruct machines to gain from information and infer an assortment of helpful experiences. This is known as Artificial Intelligence. Man-made reasoning alludes to knowledge shown by machines that reenact human and creature insight. Computer based intelligence is utilized generally. The following related task is performed by Artificial Intelligence. • • • • • •

Self-driving vehicles Applications like Siri that comprehend and react to human discourse Google’s AlphaGo AI has vanquished many Go champions, for example, Ke Jie Actualizing AI in chess Amazon ECHO item (home control chatbot gadget) Hilton utilizing Connie – attendant robot from IBM Watson

The general example of AI that is Amazon pulls in information from its client database to prescribe items to clients. This usefulness gets more clients. More clients produce significantly more information that assistance improves the proposals considerably further. Alpavdin characterizes Machine Learning as- “Upgrading an execution standard utilizing precedent information and past experience”. The term Machine Learning was begat by Arthur Samuel in 1959, an American pioneer in the field of PC gaming and man-made brainpower and expressed that “it enables PCs to learn without being unambiguously modified”. Also, in 1997, Tom Mitchell gave an “all around presented” scientific and social definition that “A PC program is said to gain as a matter of fact E as for some undertaking T and some execution measure P, if its execution on T, as estimated by P, enhances with experience E. Machine Learning is a most recent trendy expression coasting around. It has the right to, as it is a standout amongst the most intriguing subfield of Computer Science. Data is the key idea of machine learning. Analyst can apply its calculations on information to distinguish concealed examples and addition bits of knowledge. These examples and picked up information help frameworks to learn and improve their execution.

120

Challenges and Applications for Implementing Machine Learning

Machine learning innovation includes the two insights and software engineering. Measurements enable one to draw deductions from the given information. To actualize proficient calculations we can likewise utilize software engineering. It speaks to the required model, and assesses the execution of the model. Machine learning includes some progressed factual ideas, for example, demonstrating and enhancement. Displaying alludes to the conditions or likelihood dissemination for the given example information. Improvement additionally incorporates procedures used to locate the most fitting parameters for the given arrangement of information. The information causes frameworks to learn and improve their execution. We can utilize Modern Learning innovation in a few zones, for example, counterfeit neural systems, information mining, web positioning and so forth. In Big Data Analytics, Data mining and machine learning are the two most regularly utilized strategies. So people can get befuddled between the two however they are two distinct methodologies utilized for two unique purposes. Here just the sight view of difference between Data Mining and Machine Learning. Data mining is the way toward distinguishing designs in a lot of information to investigate valuable data from those examples. It might incorporate methods of computerized reasoning, machine learning, neural systems, and measurements. The premise of information mining is genuine information. It might have taken motivation and methods from machine learning and measurements yet is put to various finishes. An individual completes information mining in a particular circumstance on a specific informational collection. The objective is to use the intensity of the different example acknowledgment procedures of machine learning. In any case, machine learning process is a way to deal with creating man-made reasoning. We use Machine Learning calculation to build up the new calculations and strategies. These enable the machine to gain from the examined information or with experience. Most errands that need insight must have a capacity to initiate new information from encounters. Accordingly, a huge region inside AI is machine learning. This includes the investigation of calculations that can separate data without on-line human direction. Machine Learning identifies with the investigation, structure, and improvement of the calculations. These give PCs the ability to learn without being expressly modified. Information Mining begins with unstructured information and attempts to separate learning or intriguing examples. Amid this procedure, we use machine Learning calculations. The ability of Artificial Intelligence frameworks to take in by removing designs from information is known as Machine Learning. Machine Learning is a plan to gain from precedents and experience, without being unequivocally modified. Rather than composing code, you feed information to the nonexclusive calculation, and it fabricates rationale dependent on the information given. 121

Challenges and Applications for Implementing Machine Learning

Machine Learning Benefits Give us a chance to take a gander at a portion of the advantages in this Machine Learning instructional exercise. • • • • • • •

Amazing Processing Better Decision Making and Prediction Faster Processing Precise Reasonable Data Management Reasonable Dissecting Complex Big Data

Highlights of Machine Learning Give us a chance to take a gander at a portion of the highlights given beneath in this Machine Learning instructional exercise. Machine Learning is registering escalated and for the most part requires a lot of preparing information. It includes tedious preparing to enhance the learning and basic leadership of calculations. As more information gets included, Machine Learning preparing can be mechanized for adapting new information designs and adjusting its calculation. Machine Learning is a field which is raised out of Artificial Intelligence (AI). Applying AI, we needed to assemble better and wise machines. In any case, with the exception of couple of simple errands, for example, finding the most brief way between point A and B, we were not able program increasingly mind boggling and always advancing challenges. There was an acknowledgment that the best way to almost certainly accomplish this assignment was to give machine a chance to gain from itself. This sounds like a youngster gaining from its self. So machine learning was produced as another capacity for PCs. What’s more, presently machine realizing is available in such huge numbers of sections of innovation, that we don’t understand it while utilizing it. Discovering designs in information on planet earth is conceivable just for human cerebrums. The information being extremely enormous, the time taken to register is expanded, and this is the place Machine Learning comes without hesitation, to help individuals with huge information in least time. In the event that huge information and distributed computing are picking up significance for their commitments, machine learning as innovation breaks down those enormous lumps of information, facilitating the undertaking of information researchers in a robotized procedure and increasing equivalent significance and acknowledgment. 122

Challenges and Applications for Implementing Machine Learning

The procedures we use for information digging have been around for a long time, however they were not compelling as they didn’t have the focused capacity to run the calculations. On the off chance that you run profound learning with access to better information, the yield we get will prompt sensational leaps forward which is machine learning. This infers the assignments in which machine learning is concerned offers an essentially operational definition instead of characterizing the field in subjective terms. This pursues Alan Turing’s proposition in his paper “Registering Machinery and Intelligence”, in which the inquiry “Can machines believe?” is supplanted with the inquiry “Can machines do what we (as speculation elements) can do?” Inside the field of information investigation, machine learning is utilized to devise complex models and calculations that loan themselves to forecast; in business use, this is known as prescient examination. These investigative models permit specialists, information researchers, designers, and examiners to “produce solid, repeatable choices and results” and reveal “concealed experiences” through gaining from authentic connections and patterns in the information set(input). So on the off chance that you need your program to anticipate, for instance, traffic designs at a bustling convergence (errand T), you can run it through a machine learning calculation with information about past traffic designs (experience E) and, on the off chance that it has effectively “educated”, it will at that point improve the situation at foreseeing future traffic designs (execution measure P). The very unpredictable nature of some genuine issues, however, regularly implies that creating particular calculations that will tackle them superbly every time is unreasonable, if certainly feasible. Instances of machine learning issues incorporate, “Is this malignant growth?”, “Which of these individuals are great companions with one another?”, “Will this individual like this motion picture?” such issues are astounding focuses for Machine Learning, and in actuality machine learning has been connected such issues with extraordinary achievement.

Kinds of Machine Learning There are three common taxonomies of Machine Learning Algorithm based on the result conclusion of the algorithm (Figure 1). The Figure 2 is also the categories of Ml Algorithms using learning Style. The different methodologies under each category are mentioned in figure 2.

123

Challenges and Applications for Implementing Machine Learning

Figure 1. Kinds of ML & Categorization of ML Algorithms

Figure 2. Categorization of ML Algorithms

1. Supervised Learning In regulated learning, we are given an informational index and as of now comprehend what our right yield should resemble, having the possibility that there is a connection between the information and the yield. Directed learning issues are additionally sorted into relapse and arrangement issues. To portray the directed learning issue somewhat more formally, our objective is, given a preparation set, to get familiar with a capacity h: X → Y so that h(x) is a “decent” indicator for the comparing estimation of y. For authentic reasons, this capacity h is known as a theory. Seen pictorially, the procedure is thusly similar to what is shown in Figure 3. At the point when the objective variable that we’re endeavoring to foresee is consistent, for example, in our lodging model, we consider the learning issue a relapse issue. At the point when y can take on just few discrete qualities, (for example, if, given the living territory, we needed to anticipate if a home is a house or a condo, state), we consider it an arrangement issue. The figure 4 shows the algorithms which are under the Supervised Learning Category of Machine Learning.

124

Challenges and Applications for Implementing Machine Learning

Figure 3. Supervised Learning Style

2. Unsupervised Learning Unsupervised learning works by breaking down the information without its marks for the shrouded structures inside it, and through deciding the relationships, and for highlights that really correspond two information things. It is being utilized for bunching, dimensionality decrease, include learning, thickness estimation, and so forth.

Figure 4. Supervised Learning Algorithms

125

Challenges and Applications for Implementing Machine Learning

The shrouded structure once in a while called include vector, speaks to the info information such a way, that on the off chance that a similar component vector is being use to recreate the info, at that point one can do that with some satisfactory misfortune. The change in two component vectors of two data sources specifically relative to the difference in the information sources itself. Along these lines this shrouded structure or highlight vector just speaks to highlights in the information that really offer refinement to it. In this Unsupervised Machine Learning, input information isn’t marked and does not have a known outcome. Figure 5. Unsupervised Learning Algorithms

126

Challenges and Applications for Implementing Machine Learning

We need to get ready model by deriving structures present in the info information. This might be to extricate general guidelines. It might be through a scientific procedure to lessen excess. Model issues are bunching, dimensionality decrease, and affiliation rule learning. Precedent calculations incorporate the Apriori calculation and k-Means.

3. Reinforcement Learning Reinforcement learning is a region of Machine Learning. Reinforcement: It is tied in with making reasonable move to expand compensate in a specific circumstance. It is utilized by different programming and machines to locate the most ideal conduct or way it should take in a particular circumstance. Fortification taking in contrasts from the directed learning in a route that in regulated learning the preparation information has the appropriate response key with it so the model is prepared with the right answer itself while in Reinforcement learning, there is no answer yet the fortification specialist chooses what to do to play out the given errand. Without preparing dataset, it will undoubtedly gain from its experience. Precedent: The issue is as per the following: We have a specialist and a reward, with numerous obstacles in the middle. The specialist should locate the most ideal way to achieve the reward. The accompanying issue clarifies the issue all the more effectively. The figure 6 indicates robot, precious stone and flame. The objective of the robot is to get the reward that is the jewel and dodge the obstacles that is fire. The robot learns by attempting all the conceivable ways and after that picking the way which gives him the reward with the least obstacles. Each correct advance will give the robot a reward and each wrong advance will subtract the reward of the robot. The absolute reward will be determined when it achieves the last reward that is the precious stone. Figure 6. Reinforcement Learning Example

127

Challenges and Applications for Implementing Machine Learning

Main Points for Reinforcement Learning Input: The information ought to be an underlying state from which the model will begin Outcome: There are numerous conceivable output as there are assortment of answer for a specific issue Training/Preparing: The preparation depends on the information, The model will restore a state and the client will choose to remunerate or rebuff the model dependent on its yield. The model keeps on learning. The best arrangement of solution is chosen dependent on the most extreme reward.

Reinforcement Learning vs. Supervised Learning •

• •

Reinforcement Learning is tied in with settling on choices successively. In basic words we can say that the out relies upon the condition of the present information and the following information relies upon the yield of the past input where as In Supervised learning the choice is made on the underlying information or the information given toward the begin. In Reinforcement learning choice is reliant, So we offer names to groupings of ward decisions where as Supervised learning the choices are free of one another so names are given to every choice. Playing Chess is the example of Reinforcement Learning and Object Recognition is the example of Supervised Learning.

Applications of Machine Learning One of the cutting edge developments we’ve seen is the making of Machine Learning. This fantastic type of man-made consciousness is as of now being utilized in different ventures and callings. For Example, Image and Speech Recognition, Medical Diagnosis, Prediction, Classification, Learning Associations, Statistical Arbitrage, Extraction, Regression. These are this present reality Machine Learning Applications; we should see them one by one-

128

Challenges and Applications for Implementing Machine Learning

1. Picture Recognition It is a standout amongst the most widely recognized AI applications. There are numerous circumstances where you can arrange the item as an advanced picture. For advanced pictures, the estimations depict the yields of every pixel in the picture. On account of a highly contrasting picture, the power of every pixel fills in as one estimation. So if a high contrast picture has N*N pixels, the all out number of pixels and thus estimation is N2. How about we examine ANN in Machine Learning: In the hued picture, every pixel considered as giving 3 estimations of the forces of 3 principle shading segments ie RGB. So N*N shaded picture there are 3 N2 estimations. For face location – The classes may be face versus no face present. There may be a different classification for every individual in a database of a few people. For character acknowledgment – We can fragment a bit of composing into littler pictures, each containing a solitary character. The classifications may comprise of the 26 letters of the English letter set, the 10 digits, and some unique characters.

2. Discourse Recognition Discourse acknowledgment (SR) is the interpretation of verbally expressed words into content. It is otherwise called “programmed discourse acknowledgment” (ASR), “PC discourse acknowledgment”, or “discourse to content” (STT). In discourse acknowledgment, a product application perceives verbally expressed words. The estimations in this Machine Learning application may be a lot of numbers that speak to the discourse flag. We can fragment the flag into parts that contain particular words or phonemes. In each portion, we can speak to the discourse motion by the forces or vitality in various time-recurrence groups. Despite the fact that the subtleties of flag portrayal are outside the extent of this program, we can speak to the flag by a lot of genuine qualities. Do you think about Artificial Neural Network Model Discourse acknowledgment, Machine Learning applications incorporate voice UIs. Voice UIs are, for example, voice dialing, call steering, domotic apparatus control. It can likewise use as straightforward information passage, arrangement of organized records, discourse to-content preparing, and plane.

129

Challenges and Applications for Implementing Machine Learning

3. Medicinal Diagnosis ML gives strategies, procedures, and devices that can help in tackling symptomatic and prognostic issues in an assortment of therapeutic areas. It is being utilized for the examination of the significance of clinical parameters and of their mixes for visualization, for example expectation of ailment movement, for the extraction of medicinal information for results inquire about, for treatment arranging and support, and for in general patient administration. ML is likewise being utilized for information examination, for example, discovery of regularities in the information by properly managing blemished information, translation of consistent information utilized in the Intensive Care Unit, and for savvy disturbing bringing about successful and proficient checking. It is contended that the effective execution of ML techniques can help the mix of PC based frameworks in the social insurance condition giving chances to encourage and upgrade crafted by therapeutic specialists and eventually to improve the effectiveness and nature of restorative consideration. We should take a voyage through Neural Network Algorithms In restorative conclusion, the principle intrigue is in setting up the presence of an illness pursued by its exact distinguishing proof. There is a different classification for every malady under thought and one class for situations where no illness is available. Here, AI improves the precision of restorative analysis by breaking down information of patients. The estimations in this Machine Learning applications are ordinarily the aftereffects of certain restorative tests (precedent circulatory strain, temperature and different blood tests) or therapeutic diagnostics, (for example, medicinal pictures), nearness/ nonappearance/force of different manifestations and essential physical data about the patient(age, sex, weight and so on). Based on the consequences of these estimations, the specialists slender down on the ailment exacting the patient.

4. Measurable Arbitrage In money, measurable exchange alludes to mechanized exchanging procedures that are regular of a present moment and include countless. In such procedures, the client endeavors to execute an exchanging calculation for a lot of securities based on amounts, for example, verifiable connections and general monetary factors. These estimations can be given a role as an arrangement or estimation issue. The fundamental presumption is that costs will move towards an authentic normal.

130

Challenges and Applications for Implementing Machine Learning

Do you think about Kernel Functions We apply AI strategies to acquire a file exchange methodology. Specifically, we utilize direct relapse and bolster vector relapse (SVR) onto the costs of a trade exchanged store and a surge of stocks. By utilizing foremost part examination (PCA) in lessening the component of highlight space, we watch the advantage and note the issues in the use of SVR. To create exchanging signals, we show the residuals from the past relapse as a mean returning procedure. On account of arrangement, the classes may be sold, purchase or do nothing for every security. I the instance of estimation one may endeavor to foresee the normal return of every security over a future time skyline. For this situation, one normally needs to utilize the assessments of the normal come back to make an exchanging decision(buy, sell, and so forth.)

5. Learning Associations Learning affiliation is the way toward forming bits of knowledge into different relationship between items. A genuine model is the way apparently inconsequential items may uncover a relationship to each other. At the point when examined in connection to purchasing practices of clients. How about we talk about Deep learning and Neural Networks in Machine Learning One utilization of AI Often concentrating the relationship between the items individuals purchase, which is otherwise called bin examination. On the off chance that a purchaser purchases ‘X’, would the person in question power to purchase ‘Y’ in view of a relationship that can recognize between them? This prompts the relationship that exists among fish sticks and french fries and so on when new items dispatch in the market a Knowing these connections it builds up another relationship. Realizing these connections could help in proposing the related item to the client. For a higher probability of the client getting it, It can likewise help in packaging items for a superior bundle. This learning of relationship between items by a machine is learning affiliations. When we found a relationship by inspecting a lot of offers information, Big Data experts. It can build up a standard to infer a likelihood test in learning a contingent likelihood.

131

Challenges and Applications for Implementing Machine Learning

6. Arrangement Arrangement is a procedure of putting every person from the populace under examination in numerous classes. This is recognized as autonomous factors. View Convolutional Neural Networks Architecture Grouping causes examiners to utilize estimations of an article to recognize the classification to which that object has a place. To set up a productive standard, investigators use information. Information comprises of numerous instances of articles with their right characterization. For instance, before a bank chooses to dispense an advance, it evaluates clients on their capacity to reimburse the advance. By considering variables, for example, client’s acquiring, age, reserve funds and monetary history we can do it. This data is taken from the past information of the credit. Henceforth, Seeker uses to make a connection between client qualities and related dangers.

7. Expectation Consider the case of a bank registering the likelihood of any of credit candidates blaming the advance reimbursement. To figure the likelihood of the blame, the framework will initially need to order the accessible information in specific gatherings. It is portrayed by a lot of tenets recommended by the investigators. How about we reconsider Recurrent Neural Networks When we do the characterization, according to require we can figure the likelihood. These likelihood calculations can figure over all parts for fluctuated purposes The present forecast is one of the most smoking AI calculations. How about we take a case of retail, prior we had the capacity to get bits of knowledge like deals report a month ago/year/5-years/Diwali/Christmas. These sort of announcing is called as verifiable detailing. In any case, as of now business is progressively keen on discovering what will be my deals one month from now/year/Diwali, and so on. With the goal that business can take a required choice (identified with obtainment, stocks, and so on.) on time.

132

Challenges and Applications for Implementing Machine Learning

8. Extraction Data Extraction (IE) is another use of AI. It is the way toward removing organized data from unstructured information. For instance pages, articles, web journals, business reports, and messages. The social database keeps up the yield delivered by the data extraction. The procedure of extraction accepts contribution as a lot of records and delivers an organized information. This yield is in a condensed structure, for example, an exceed expectations sheet and table in a social database. These days extraction is turning into a key in the huge information industry. As we realize that the immense volume of information is getting created out of which the majority of the information is unstructured. The primary key test is taking care of unstructured information. Presently change of unstructured information to organized structure dependent on some example with the goal that the equivalent can put away in RDBMS. Aside from this in current days information gathering instrument is likewise getting change. Prior we gathered information in clumps like End-of-Day (EOD)

CONCLUSION Machine Learning is assuming control over the world-and with that, there is a developing need among organizations for experts to know the intricate details of ML. ML is figuring escalated and by and large requires a lot of preparing information. It includes dull preparing to improve the learning and basic leadership of calculations. As more information gets included, Machine Learning preparing can be computerized for adapting new information designs and adjusting its calculation.

REFERENCES Awad, W., & Elseuofi, S. (2011). Machine Learning methods for E-mail Classification. International Journal of Computers and Applications, 16(1), 39–45. doi:10.5120/1974-2646 Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., ... Xie, Z. (2018). Deep Learning and Its Applications in Biomedicine. [PubMed]. Genomics, Proteomics & Bioinformatics, 16(1), 17–32. doi:10.1016/j.gpb.2017.07.003 133

Challenges and Applications for Implementing Machine Learning

Deng & Yu. (2013). Deep Learning: Methods and Applications, Foundations and Trends R. Signal Processing, 7(3-4), 197–387. doi:10.1561/2000000039 Grm, K., Štruc, V., Artiges, A., Caron, M., & Ekenel, H. (2018). Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biometrics, 7(1), 81–89. doi:10.1049/iet-bmt.2017.0083 Kashyap, R. (2019a). Big Data and Global Software Engineering. In M. Rehman, A. Amin, A. Gilal, & M. Hashmani (Eds.), Human Factors in Global Software Engineering (pp. 131–163). Hershey, PA: IGI Global; doi:10.4018/978-1-52259448-2.ch006 Kashyap, R. (2019b). Sensation of Deep Learning in Image Processing Applications. In A. Hassanien, A. Darwish, & C. Chowdhary (Eds.), Handbook of Research on Deep Learning Innovations and Trends (pp. 72–96). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7862-8.ch005 Kashyap, R. (2019c). Big Data Analytics Challenges and Solutions. doi:10.1016/ B978-0-12-818146-1.00002-7 Kashyap, R. (2019d). Computational Healthcare System With Image Analysis. In C. Chen & S. Cheung (Eds.), Computational Models for Biomedical Reasoning and Problem Solving (pp. 89–127). Hershey, PA: IGI Global; doi:10.4018/978-15225-7467-5.ch004 Kashyap, R. (2019e). Systematic Model for Decision Support System. In A. Mukherjee & A. Krishna (Eds.), Interdisciplinary Approaches to Information Systems and Software Engineering (pp. 62–98). Hershey, PA: IGI Global; doi:10.4018/978-15225-7784-3.ch004 Kashyap, R. (2019f). Miracles of Healthcare With Internet of Things. In J. Rodrigues, A. Gawanmeh, K. Saleem, & S. Parvin (Eds.), Smart Devices, Applications, and Protocols for the IoT (pp. 120–164). Hershey, PA: IGI Global; doi:10.4018/978-15225-7811-6.ch007 Kashyap, R. (2019g). Deep Learning: An Application in Internet of Things. In H. Purnomo (Ed.), Computational Intelligence in the Internet of Things (pp. 130–158). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7955-7.ch006 Kashyap, R. (2019h). Machine Learning, Data Mining for IoT-Based Systems. In G. Kaur & P. Tomar (Eds.), Handbook of Research on Big Data and the IoT (pp. 314–338). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7432-3.ch018

134

Challenges and Applications for Implementing Machine Learning

Kashyap, R. (2019i). Decision Support Systems in Aeronautics and Aerospace Industries. In T. Shmelova, Y. Sikirda, N. Rizun, D. Kucherov, & K. Dergachov (Eds.), Automated Systems in the Aviation and Aerospace Industries (pp. 138–165). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7709-6.ch005 Kashyap, R. (2019j). Medical Image Segmentation and Analysis. In C. Chakraborty (Ed.), Advanced Classification Techniques for Healthcare Analysis (pp. 132–160). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7796-6.ch007 Kashyap, R. (2019k). Big Data and High-Performance Analyses and Processes. In A. Voghera & L. La Riccia (Eds.), Spatial Planning in the Big Data Revolution (pp. 45–83). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7927-4.ch003 Kashyap, R. (2019l). Artificial Intelligence Systems in Aviation. In T. Shmelova, Y. Sikirda, N. Rizun, & D. Kucherov (Eds.), Cases on Modern Computer Systems in Aviation (pp. 1–26). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7588-7.ch001 Kashyap, R. (2020). Applications of Wireless Sensor Networks in Healthcare. In P. Mukherjee, P. Pattnaik, & S. Panda (Eds.), IoT and WSN Applications for Modern Agricultural Advancements: Emerging Research and Opportunities (pp. 8–40). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-9004-0.ch002 Kashyap, R., & Rahamatkar, S. (2019b). Healthcare Informatics Using Modern Image Processing Approaches. In B. Singh, B. Saini, D. Singh, & A. Pandey (Eds.), Medical Data Security for Bioengineers (pp. 254–277). Hershey, PA: IGI Global; doi:10.4018/978-1-5225-7952-6.ch013 Litjens, G., Kooi, T., Bejnordi, B., Setio, A., Ciompi, F., Ghafoorian, M., ... Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. [PubMed]. Medical Image Analysis, 42, 60–88. doi:10.1016/j.media.2017.07.005 Manor, R., Mishali, L., & Geva, A. (2016). Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface. [PubMed]. Frontiers in Computational Neuroscience, 10. doi:10.3389/fncom.2016.00130 Najafabadi, M., Villanustre, F., Khoshgoftaar, T., Seliya, N., Wald, R., & Muharemagic, E. (2015). Deep learning applications and challenges in big data analytics. Journal Of Big Data, 2(1), 1. doi:10.118640537-014-0007-7

135

136

Chapter 6

Medical Imaging Importance in the Real World Ramgopal Kashyap https://orcid.org/0000-0002-5352-1286 Amity School of Engineering and Technology, Amity University, Raipur, India

ABSTRACT In the medical image resolution, automatic segmentation is a challenging task, and it’s still an unsolved problem for most medical applications due to the wide variety connected with image modalities, encoding parameters, and organic variability. In this chapter, a review and critique of medical image segmentation using clustering, compression, histogram, edge detection, parametric, variational model. and level set-based methods is presented. Modes of segmentation like manual, semi-automatic, interactive, and automatic are also discussed. To present current challenges, aim and motivation for doing fast, interactive and correct segmentation, the medical image modalities X-ray, CT, MRI, and PET are discussed in this chapter.

INTRODUCTION In the medical image resolution, automatic segmentation is a challenging task, and it’s still an unsolved problem for most medical applications due to the wide variety connected with image modalities, encoding parameters and organic variability. Manual segmentation is time-consuming and frequently not applicable in clinical routine. Semi-automatic segmentation methods which required-user interaction can use in instances where automatic algorithms fail. A wide variety of semi-automatic segmentation methods exist that will roughly classify into voxel-based approaches, where the end-user draws seed things to define fore and background voxels and DOI: 10.4018/978-1-7998-0182-5.ch006 Copyright © 2020, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Medical Imaging Importance in the Real World

surface-based means, where the shape of a functional object is reconstructed depending on contours or subject models. Image segmentation can smoothly proceed on three other ways, manually, interactive, semi-automatic and automatic. Segmentation is the process of partitioning an image into semantically interpretable regions. The purpose of segmentation is to decompose the image into parts that are meaningful concerning a particular application. Image segmentation is typically used to locate objects and boundaries like lines, curves in images. The result of image segmentation is a set of regions that collectively cover the entire image or a set of contours extracted from the image. Each of the pixels in a region is similar concerning any characteristic or computed property, such as colour, intensity, or texture. Adjacent areas are significantly different concerning the same characteristic. Segmentation subdivides an image into its constituent regions or objects. That is, it partitions an image into distinct areas that are meant to correlate strongly with objects or features of interest in the image. Segmentation can also be regarded as a process of grouping together pixels that have similar attributes. The level to which the subdivision carried depends on the problem solved (Anderson, C. W., 1987). That is, segmentation should stop when the objects of interest in an application have isolated. There is no point in taking segmentation past the level of detail required to identify those elements. It is the process that partitions the image pixels into nonoverlapping regions such that: Each region is homogeneous, i.e., uniform in terms of the pixel attributes such as intensity, colour, range, or texture. To understand the concept using mathematical representation here {Ri} is a segmentation of an entire image R if: n

1. R= ∪Rj the union of all regions covers entire R j =1

2.

Ri ∩ Rj For all i and j, I≠j there is no overlap of the regions 3. P (Ri) for i = 1, 2... n, P is the logical uniformity predicate defined over the points in the set Ri 4. P (Ri ∪ Rj ) =false, for I≠j and Ri and Rj are neighbouring regions. 5. Ri is a connected region, i = 1, 2... n All pixels must be assigned to regions. Each pixel must belong to a single region only. Each region must be uniform. Any merged pair of adjacent regions must be non-uniform. Each region must be a connected set of pixels.

Several Predicate Examples 1. P(R) =True, if |g(x1, y1) – g(x2, y2)| Confidence intervals.

257

Machine Learning for Health Data Analytics

•

Enter the mean and std. error in cii dialog box as shown in figure 3 for calculating the normal CI calculation. Hence, the value of crude birth rate urban is 95% CI = 13.5 to 24.71,

mean = 19.13 and SE = 2.17; The assessed value of crude birth rate rural - 95% CI = 15.50 to 27.12 mean = 21.31 and SE = 2.25. The evaluated value of crude mortality rate rural – 95% CI 5.97 to 8.18, mean = 7.08 and SE = 0.43. Finally, the calculated value of crude mortality rate urban - 95% CI = 3.90 to 6.46, mean = 5.18 and SE =0 .49 shown in Table 9. From the graph in figure 4, the results of total six states for crude birth rate mean = 21.22, SE = 2.27 and 95% CI = 15.38 to 27.07; The results of crude birth rate rural mean = 20.78, SE = 2.44 and 95% CI = 14.51 to 27.05 and crude birth rate urban mean = 19.90, SE = 2.41 and 95% CI = 13.69 to 26.11. The results of total six states for crude mortality rate mean =6.3, SE = 0.58 and 95% CI= 4.81 to 7.78; The results of crude mortality rate urban mean = 5.29, SE = 0.60 and 95% CI = 3.73 to 6.85. The crude mortality rate rural mean = 6.93, SE = 0.49 with 95% CI = 5.66 to 8.21.

258

Machine Learning for Health Data Analytics

Figure 4. Graph representing crude birth rate and crude mortality rate in urban and rural

The peak crude birth rate can be seen in Dadra and Nagar Haveli with 27.7 and lowest total crude mortality rate was found be 4 in Dadra and Nagar Haveli. In figure 5, the graph shows the highest crude birth rate in the year 1984 with 45.9 in Dadra and Nagar Haveli and minimum in the year 2007 in Pondicherry.

Figure 5. Graph representing crude birth rate in six different states

259

Machine Learning for Health Data Analytics

Table 10. Confidence interval, mean and standard error for crude birth rate in India (1981-2011)

260

Year

Mean

Standard Error

[95% Conf. Interval]

1981

32.53

1.59

29.19 to 35.86

1982

34.32

1.59

30.99 - 37.65

1983

33.85

1.54

30.63 - 37.06

1984

35.65

1.71

32.07 - 39.21

1985

32.58

1.54

29.35 - 35.80

1986

34.03

1.77

30.32 – 37.72

1987

32.04

1.41

29.10 - 34.98

1988

32.34

1.49

29.21 - 35.45

1989

30.66

1.48

27.55 - 33.75

1990

30.54

1.55

27.30 - 33.77

1991

28.98

1.49

25.87 - 32.09

1992

30.65

1.67

27.15 - 34.15

1993

28.27

1.90

24.30 - 32.24

1994

28.86

1.63

25.45 - 32.25

1995

28.12

1.26

25.48 - 30.76

1996

26.85

1.37

23.99 - 29.72

1997

26.66

1.30

23.94 - 29.38

1998

27.6

1.45

24.56 - 30.63

1999

27.49

1.53

24.29 - 30.69

2000

27.70

1.56

24.43 - 30.97

2001

26.25

1.29

23.55 - 28.95

2002

26.23

1.29

23.53 - 28.92

2003

25.96

1.31

23.23 - 28.70

2004

25.13

1.28

22.44 - 27.81

2005

24.90

1.35

22.08 - 27.73

2006

24.30

1.33

21.52 - 27.09

2007

23.83

1.34

21.02 - 26.62

2008

23.81

1.17

21.37 - 26.24

2009

23.68

1.14

21.31 - 26.05

2010

23.45

1.08

21.19 - 25.71

2011

22.95

1.09

20.67 - 25.23

Machine Learning for Health Data Analytics

Table 11. Confidence interval, mean and standard error for crude mortality rate in India (1981-2011) Year

Mean

SE

95% Conf. Interval

1981

13

1.42

9.32 to 16.67

1982

12.12

1.31

8.74 - 15.49

1983

12.53

1.11

9.67 - 15.39

1984

13.33

1.36

9.81 - 16.85

1985

12.1

1.24

8.89 - 15.30

1986

11.4

1.03

8.72 - 14.07

1987

11.43

0.94

8.99 - 13.87

1988

11.2

0.96

8.72 - 13.67

1989

10.31

0.84

8.14 - 12.49

1990

10

0.94

7.58 - 12.41

1991

10.71

0.98

8.19 - 13.24

1992

10.58

0.95

8.13 - 13.03

1993

10.25

0.99

7.70 - 12.79

1994

9.5

0.64

7.86 - 11.13

1995

9.21

0.56

7.76 - 10.66

1996

9.28

0.57

7.80 - 10.76

1997

9.28

0.52

7.94 - 10.62

1998

9.36

0.57

7.88 - 10.84

1999

8.71

0.70

6.89 - 10.53

2000

8.78

0.62

7.17 - 10.39

2001

8.58

0.64

6.91 - 10.25

2002

8.38

0.57

6.91 - 9.85

2003

8.13

0.66

6.43 - 9.83

2004

7.83

0.61

6.24 - 9.41

2005

7.65

0.60

6.09 - 9.20

2006

7.6

0.63

5.97 - 9.22

2007

7.61

0.60

6.06 - 9.16

2008

7.66

0.49

6.38 - 8.94

2009

7.41

0.57

5.94 - 8.88

2010

7.38

0.55

5.95 - 8.81

2011

7.23

0.54

5.82 - 8.63

261

Machine Learning for Health Data Analytics

The spacing between children in the rural and urban areas implies that about half of the birth should have spacing of 36 months and above. Most of rural and urban areas now have 70 percent of births which have birth interval of about 24 months.

6. DECISION TREES: A CASE STUDY IN HEALTH CARE Decision tree is the form of a tree structure which is built on regression or classification models. Decision Tree is a type of very powerful ML model capable of achieving high accuracy in many tasks while being highly interpretable. What makes decision trees special in the realm of ML models is really their clarity of information representation. The “knowledge” learned by a decision tree through training is directly formulated into a hierarchical structure. This structure holds and displays the knowledge in such a way that it can easily be understood, even by non-experts. Decision trees are typically a set of inputs used to create a tree with new cases. It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. Mujtaba & Irshad (2017) did a Comparative study of existing techniques for heart diseases prediction using data mining approach. On the contrary, decision tree is widely used and practically useful method for reliable inference. For approximating any typical discrete valued target function, learned function is represented by a decision tree.

Decision Tree for Brain Tumour For this study data in excel format is taken from Gandhi Hospital and Omega hospital. The decision tree is experimented on the unseen datasets that is highlighted by the prediction and application which decision making does. Decision trees work on huge training set and are built in parallel tractable size training datasets. Decision tree will reduce a set of rules and resolve conflict rules, and the resultant is merged into a single unit. Each case is defined by a set of attributes / features. Each training case is given the class name, and each node of the decision tree contains a conditional test that divides the training cases, and the test has two branches (‘Yes’ or ‘No’) which the result has to follow. These properties make decision trees a valuable and standard tool for classification and all it follows is the basic divide and conquer algorithm (Steven, 1994). The construction for a decision tree method adopted is below [Quinlan, 1993]:

262

Machine Learning for Health Data Analytics

If S represents training set and R classes denoted by {P1, P2... Pl}, then • • •

if S has one or more objects where all objects belong to a single class Pj, the decision tree is a leaf identifying class Pj. if S has no objects, the decision tree is a leaf itself (S). if S has objects that belong to a mixture of classes, then a test is chosen, based on single attribute, that has one or more mutually exclusive outcomes {O1, O2,..., On}. S is partitioned into subsets S1, S2,..., Sn, where Si contains all the objects in S that have outcome Oi of the chosen test. The same method is applied recursively to each subset of training objects (Podgorelec V et al, 2002).

Figure 6. Decision tree diagram on brain tumor obtained through Python Application [ See Appendix 1 and 2]

263

Machine Learning for Health Data Analytics

With huge number of inputs widely known as features tend the decision trees to overfit. In high-dimensional space, a tree with a few samples is likely to overfit, and the right ratio for samples to the number of features is neccesary. As the number of patient cases increases, the branches in the decision tree increases as shown in figure 6. The figure 6 is an output of the decision tree using python code. The leaf nodes of the tree show that it cannot be divided further and contains an output variable which is used to make a prediction. In this method, from the large training sets the dataset is broken into n subsets (Abhishek, R. 2011). A decision tree is generated on each of the subsets, and rules are generated from the decision tree. This rule sets are combined into a single rule set with conflicts among rules resolved. Decision trees are found to be reliable and scalable with its property to handle huge and large datasets in this study as it gave satisfactory result in diagnosing and predicting brain tumors. Decision tree is found to be one of the successful data mining techniques used in the disease diagnosis as per the literature and in the present case of application.

CONCLUSION •

•

•

•

• •

264

In the light of the few case studies reported as part of literature survey and specific exercises carried out for this paper, it is possible to say that machine learning provides new opportunities for automatically learning in expressive models. The demanding health services by the ever-growing population put the health sector to adopt the computer- based services. This made the entry of machine learning and artificial intelligence, and data mining techniques to unimaginable domain of predictive modelling. Regression Models including Multiple and Multivariate Regression techniques are suitable for modeling Air Pollution and heart disease prediction. The applicability of STATA and R packages for multiple linear regression and predictive modelling for Crude Birth Rate and Crude Mortality Rate is well established in the present study carried out using the data from data.gov.in. Decision Tree is a class of very powerful Machine Learning model is applied for Brain Tumor. Machine learning is maturing promptly and requires efficient, well-knowledge and skilled persons on supervised as well as on unsupervised machine learning. In simple terms, machine learning techniques and data mining techniques go hand-in-hand as the data analytics tools used in the study for prediction, data modelling, and decision making. The health analytics and unpredictable growth of health databases require integration of the conventional data analysis to be paired with methods

Machine Learning for Health Data Analytics

•

for efficient computer-assisted analysis. There is need for validation of computational models that are composed of multiple processing layers, which form part of deep learning efforts and are not part of this paper. Deep learning (DL), a technique in artificial neural networks is emerging as a powerful tool for machine learning, promising to reshape the future of artificial intelligence. Electronic health records are the source of patient information that include medical history details such as diagnosis, diagnostic exams, medications and treatment plans, immunization records, radiology images, electroencephalogram EEG, laboratory and test lab reports. As such it requires standards for maintenance of these records so as to serve as meaningful and reliable database.

REFERENCES Abhishek, R. (2011). Computer aided detection of brain tumor in magnetic resonance images. IACSIT International Journal of Engineering and Technology, 3(5), 523–532. doi:10.7763/IJET.2011.V3.280 Agosta, L. (2004, August). The future of data mining -- predictive analytics. DM Review. Bernard, M. (2015). Big Data: 20 Mind-Boggling Facts Everyone Must Read. Retrieved September 30, 2015 from https://www.forbes.com Brooks, R., & Dahlke, K. (2017). Artificial Intelligence vs. Machine Learning vs. Data Mining 101 – What’s the Big Difference? Retrieved from October 6, 2017 from https://www.guavus.com Charles, R. B. (2005). Predictive analysis is data mining’s future. Retrieved June 5, 2005 from http://www.bio-itworld.com Data Mining- Machine Learning. (n.d.). Retrieved from August 25, 2014. https:// doublebyteblog.wordpress.com Data Mining Tutorial: Process, Techniques, Tools & Applications. (n.d.). Retrieved from https://www.guru99.com Jason, B. (2018). Confidence-intervals-for-machine-learning. Retrieved from May 28,2018. https://machinelearningmastery.com LogisticRegression. (n.d.). Retrieved from http://ufldl.stanford.edu

265

Machine Learning for Health Data Analytics

Machine_learning. Relation_to_data_mining. (n.d.). Retrieved from https:// en.wikipedia.org/wiki/ Mateus, H., Monica, B., & Marie, H. (2015). Land use regression as method to model air pollution. Previous results for Gothenburg/Sweden. ScienceDirect Procedia Engineering, 115, 21–28. doi:10.1016/j.proeng.2015.07.350 McNulty, E. (2015). Whats-the-difference-between-supervised-and-unsupervisedlearning. Retrieved from January 8, 2015 http://dataconomy.com Mujtaba, A. Q., & Irshad, A. M. (2017). Comparative Study of Existing Techniques for Heart Diseases Prediction Using Data Mining Approach. Asian Journal of Computer Science and Information Technology, 7(3), 50–56. Podgorelec, V., Kokol, P., Stiglic, B., & Rozman, I. (2002). Decision trees: An overview and their use in medicine. [PubMed]. Journal of Medical Systems, 26(5), 445–463. doi:10.1023/A:1016409317640 Polaraju, K., & Durgaprasad, D. (2017). Prediction of Heart Disease using Multiple Linear Regression Model. International Journal of Engineering Development and Research, 5(4), 1419–1425. Quinlan, R. J. (1993). C4.5: programs for machine learning. San Francisco, CA: Morgan Kaufmann Publishers Inc. Schneider, A., Hommel, G., & Blettner, M. (2010). Linear regression analysis. In a series of part 14 on evaluation of scientific publications. [PubMed]. Deutsches Ärzteblatt International, 107(44), 776–782. Souhila, S. (2013). What is the difference between machine learning and data mining? Retrieved October 23, 2013 from https://www.researchgate.net Steven, L. S. (1994). Book Review: C4.5: programs for machine learning by Ross Quinlan J., Machine Learning (16). Boston: Kluwer Academic Publishers. Xavier, A. (2016). What’s the relationship between machine learning and data mining? Retrieved January 14, 2016 from https://medium.com

266

Machine Learning for Health Data Analytics

APPENDIX 1 Algorithm for Computation of Information Gain and Entropy of Class Attribute Understanding which attribute to be placed at the root level from the dataset consisting of ‘n’ attributes is a challenging step. Selecting randomly a node to be the root does not solve the problem. Information gain and entropy are the standards which calculate values for every attribute. The attribute with the highest value is placed at the root. Information is the measure of purity, and entropy is the impurity measure. Information gain is the expected reduction in entropy caused by the splitting of attribute. The entropy of class attribute is calculated by the standard rule Entropy = -P/(P+N)log2(P/(P+N)) – N/(P+N)log2(N/(P+N)) where P and N represent the positive and negative outcome of the class attribute (predicted column). The standard rule for information gain and entropy of an attribute is given below. I(Pi,Ni) = - P/(P+N)log2(P/(P+N)) – N/(P+N)log2(N/(P+N)) Entropy attribute = Σ(Pi+Ni)/(P+N)∙I(Pi,Ni) Gain = Entropy class – Entropy attribute The gain is calculated to find which attribute has the highest value. The one with the highest value will be the root node. The analysis was carried out using python on 101 patients’ reports which were collected from Gandhi Hospital (Secunderabad) and Omega Hospital (Hyderabad). Using the standards discussed above the entropy of class attribute (brain tumor) = 2.9736. Calculating for sex attribute. I(Pi,Ni) female=0.6235. I(Pi,Ni) male=0.2076. Entropy of the sex=0.3929. Gain of sex=2.5807; similarly, calculate for the age; it can be split into three groups 1. Age≥25 and age