Deep Learning Approaches for Early Diagnosis of Neurodegenerative Diseases 9798369312810, 9798369312827, 9798369332184, 9798369304488, 9781668492413, 9798369302606, 9781668468555

Citation preview

Deep Learning Approaches for Early Diagnosis of Neurodegenerative Diseases Raul Villamarin Rodriguez Woxsen University, India Hemachandran Kannan Woxsen University, India Revathi T. Woxsen University, India Khalid Shaikh Pgrognica Labs, UAE Sreelekshmi Bekal Pgrognica Labs, UAE

A volume in the Advances in Medical Diagnosis, Treatment, and Care (AMDTC) Book Series

Published in the United States of America by IGI Global Medical Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA, USA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2024 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Names: Rodriguez, Raul Villamarin, editor. | Kannan, Hemachandran, 1985editor. | T., Revathi, (Theerthagiri) 1989- editor. | Shaikh, Khālid (Of Prognica Labs), editor. | Bekal, Sreelekshmi, 1991- editor. Title: Deep learning approaches for early diagnosis of neurodegenerative diseases / edited by Raul Villamarin Rodriguez, Hemachandran Kannan, Revathi T., Khalid Shaikh, Sreelekshmi Bekal. Description: Hershey, PA : Medical Information Science Reference, [2024] | Includes bibliographical references and index. | Summary: “The primary objective is to provide a comprehensive resource that explores the integration of deep learning methodologies with neuroscience for the early detection of neurodegenerative disorders”-- Provided by publisher. Identifiers: LCCN 2023046352 (print) | LCCN 2023046353 (ebook) | ISBN 9798369312810 (hardcover) | ISBN 9798369312827 (ebook) Subjects: MESH: Neurodegenerative Diseases--diagnosis | Deep Learning | Early Diagnosis Classification: LCC RC376.5 (print) | LCC RC376.5 (ebook) | NLM WL 358.5 | DDC 616.8/3--dc23/eng/20231207 LC record available at https://lccn.loc.gov/2023046352 LC ebook record available at https://lccn.loc.gov/2023046353 This book is published in the IGI Global book series Advances in Medical Diagnosis, Treatment, and Care (AMDTC) (ISSN: 2475-6628; eISSN: 2475-6636) British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. For electronic access to this publication, please contact: [email protected].

Advances in Medical Diagnosis, Treatment, and Care (AMDTC) Book Series ISSN:2475-6628 EISSN:2475-6636

Mission

Advancements in medicine have prolonged the life expectancy of individuals all over the world. Once life-threatening conditions have become significantly easier to treat and even cure in many cases. Continued research in the medical field will further improve the quality of life, longevity, and wellbeing of individuals. The Advances in Medical Diagnosis, Treatment, and Care (AMDTC) book series seeks to highlight publications on innovative treatment methodologies, diagnosis tools and techniques, and best practices for patient care. Comprised of comprehensive resources aimed to assist professionals in the medical field apply the latest innovations in the identification and management of medical conditions as well as patient care and interaction, the books within the AMDTC series are relevant to the research and practical needs of medical practitioners, researchers, students, and hospital administrators. Coverage • Medical Procedures

• Medical Testing • Experimental Medicine • Emergency Medicine • Disease prevention • Patient Interaction • Patient-Centered Care • Disease Management • Internal Medicine • Cancer Treatment

IGI Global is currently accepting manuscripts for publication within this series. To submit a proposal for a volume in this series, please contact our Acquisition Editors at [email protected] or visit: http://www.igi-global.com/publish/.

The Advances in Medical Diagnosis, Treatment, and Care (AMDTC) Book Series (ISSN 2475-6628) is published by IGI Global, 701 E. Chocolate Avenue, Hershey, PA 17033-1240, USA, www.igi-global.com. This series is composed of titles available for purchase individually; each title is edited to be contextually exclusive from any other title within the series. For pricing and ordering information please visit http://www.igi-global.com/book-series/advances-medical-diagnosistreatment-care/129618. Postmaster: Send all address changes to above address. Copyright © 2024 IGI Global. All rights, including translation in other languages reserved by the publisher. No part of this series may be reproduced or used in any form or by any means – graphics, electronic, or mechanical, including photocopying, recording, taping, or information and retrieval systems – without written permission from the publisher, except for non commercial, educational use, including classroom teaching purposes. The views expressed in this series are those of the authors, but not necessarily of IGI Global.

Titles in this Series

For a list of additional titles in this series, please visit: http://www.igi-global.com/book-series/advances-medical-diagnosis-treatment-care/129618

AI-Driven Innovations in Digital Healthcare Emerging Trends, Challenges, and Applications Alex Khang (Global Research Institute of Technology and Engineering, USA) Medical Information Science Reference • © 2024 • 400pp • H/C (ISBN: 9798369332184) • US $545.00 Cutting-Edge Applications of Nanomaterials in Biomedical Sciences Pranav Kumar Prabhakar (Lovely Professional University, India) and Ajit Prakash (University of North Carolina, USA) Medical Information Science Reference • © 2024 • 605pp • H/C (ISBN: 9798369304488) • US $285.00 Improving Predictive Detection of Leukemia Using Critical Thinking Models Kuntal Barua (SAGE University, Indore, India) and Celestine Iwendi (University of Bolton, United Kingdom) Medical Information Science Reference • © 2024 • 330pp • H/C (ISBN: 9781668492413) • US $480.00 Geriatric Dentistry in the Age of Digital Technology Dachel Martínez Asanza (University of Medical Sciences of Havana, Cuba) Medical Information Science Reference • © 2024 • 347pp • H/C (ISBN: 9798369302606) • US $360.00 Clinical Practice and Post-Infection Care for COVID-19 Patients Gunda Varaprasad Rao (NMH Heart Care Center, Nashik, India) and Sangeeta DhamdhereRao (Modern College of Arts, Science, and Commerce, Pune, India) Medical Information Science Reference • © 2024 • 262pp • H/C (ISBN: 9781668468555) • US $435.00 For an entire list of titles in this series, please visit: http://www.igi-global.com/book-series/advances-medical-diagnosis-treatment-care/129618

701 East Chocolate Avenue, Hershey, PA 17033, USA Tel: 717-533-8845 x100 • Fax: 717-533-8661 E-Mail: [email protected] • www.igi-global.com

Table of Contents

Preface................................................................................................................ xvii Chapter 1 Fundamentals of Deep Learning.............................................................................1 P. V. Chandrika, Prin L.N. Welingkar Institute of Management, Development, and Research, India Sandeep Madhusudhan Kelkar, Prin L.N. Welingkar Institute of Management, Development, and Research, India Chapter 2 Introduction to Neurodegenerative Diseases........................................................25 Reihaneh Seyedebrahimi, Anatomy Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran & Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Piao Yang, Department of Molecular Genetics, College of Arts and Sciences, The Ohio State University, Columbus, USA Maryam Azimzadeh, Department of Medical Laboratory Sciences, Khomein University of Medical Sciences, Khomein, Iran & Molecular and Medicine Research Center, Khomein University of Medical Sciences, Khomein, Iran Mohsen Eslami Farsani, Anatomy Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran & Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Shima Ababzadeh, Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran & Tissue Engineering Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran Naser Kalhor, Department of Mesenchymal Stem Cells, Academic Center for Education, Culture, and Research (ACECR), Iran



Mohsen Sheykhhasan, Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Chapter 3 Human Brain Imaging for Cognitive Neuroscience: Data Acquisition and Preprocessing........................................................................................................59 Affaan Shaikh, Prognica Labs, UAE Chapter 4 A Comprehensive Survey of Deep Learning Approaches in Neurodegenerative Disease Diagnosis and Prediction..........................................73 Pruthvi Boda, Woxsen University, India Sumanth Munari, Woxsen University, India K. Sai Rama Prasanth, Woxsen University, India Shahid Mohammad Ganie, AI Research Centre, School of Business, Woxsen University, Inda Chapter 5 Deep Learning Techniques for Alzheimer’s Disease Detection: A Comprehensive Study...........................................................................................91 Bazila Farooq, Lovely Professional University, India Shahid Mohammad Ganie, AI Research Centre, School of Business , Woxsen University, India Chapter 6 Deep Neural Networks for Early Diagnosis of Neurodegenerative Diseases.....112 K. Suneetha, Jain University, India Karthik Kovuri, Kaziranga University, Assam, India Chengamma Chitteti, Mohan Babu University, India J. Avanija, Mohan Babu University, India K. Reddy Madhavi, Mohan Babu University, India Naresh Tangadu, Aditya Institute of Technology and Management, India Chapter 7 Deep Learning Approaches in the Early Diagnosis of Parkinson’s Disease.......128 Bilal El-Mansoury, Faculty of Sciences, Chouaib Doukkali University, Morocco Youssef Ait Hamdan, Higher Normal School, Cadi Ayyad University, Morocco Kamal Smimih, Faculty of Sciences and Techniques, Sultan Moulay Slimane University, Beni Mellal, Morocco Abdelaati El Khiat, Higher Institute of Nursing Professions and Health



Techniques, Ouarzazate, Morocco Ahmed Draoui, Faculty of Science Semlalia, Cadi Ayyad University, Morocco Mohamed Hammani, Faculty of Letters and Human Sciences, Cadi Ayyad University, Morocco Arumugam Jayakumar, Miller School of Medicine, University of Miami, USA Omar El Hiba, Faculty of Sciences, Chouaib Doukkali University, Morocco Chapter 8 Automated Neurological Brain Disease Detection in Magnetic Resonance Imaging Using Deep Learning Approaches........................................................150 S. Thilagavathi, Department of Computer Science, Sri Krishna Adithya College of Arts and Science, Coimbatore, India D. Sridhar, School of Computer Science, Dr.Vishwanath Karad MIT World Peace University, Pune, India S. Jawahar, School of Sciences, CHRIST Deemed to be University, Ghaziabad, India Chapter 9 Automatic Diagnosis of Parkinson’s Disease Based on Deep Learning Models and Multimodal Data.............................................................................179 Ling Li, Zhoushan Hospital, Wenzhou Medical University, China Fangyu Dai, Zhoushan Hospital, Wenzhou Medical University, China Songbin He, Zhoushan Hospital, Wenzhou Medical University, China Hao Yu, The Second Affiliated Hospital, Zhejiang University School of Medicine, China Haipeng Liu, Coventry University, UK Chapter 10 Advancements in Healthcare: Harnessing Machine Learning for Medical Devices................................................................................................................201 Dwith Chenna, Magic Leap Inc., USA Chapter 11 Ethical Considerations and Challenges in Neurodegenerative Diseases Using Machine Learning...............................................................................................234 Moushumi Das, Institute of Engineering and Technology, Chitkara University, India Hitakshi Hitakshi, Institute of Engineering and Technology, Chitkara University, India



Vandana Mohindru Sood, Institute of Engineering and Technology, Chitkara University, India Kamal Deep Garg, Institute of Engineering and Technology, Chitkara University, India Sushil Kumar Narang, Institute of Engineering and Technology, Chitkara University, India Chapter 12 Future Directions and Emerging Trends.............................................................257 Revanth Vemireddy, Woxsen University, India Harish Kakaraparthi, Woxsen University, India Naveen Kumar Challakolusu, Woxsen University, India Compilation of References............................................................................... 270 About the Contributors.................................................................................... 320 Index................................................................................................................... 323

Detailed Table of Contents

Preface................................................................................................................ xvii Chapter 1 Fundamentals of Deep Learning.............................................................................1 P. V. Chandrika, Prin L.N. Welingkar Institute of Management, Development, and Research, India Sandeep Madhusudhan Kelkar, Prin L.N. Welingkar Institute of Management, Development, and Research, India This chapter deals with the basic understanding of the deep learning model. It starts with a view of artificial intelligence, machine learning, and deep learning. It also represents how these three concepts are related to each other. The entire chapter is divided into three sections where Section 1 represents the evolution of deep learning models, Section 2 talks about the components of neural network, and Section 3 captures working of neural network models and different types. The chapter starts with introduction to Neurodegenerative Diseases and lastly the chapter gives a thorough implementation of neural network model by considering a Parkinson’s case study along with the Python code. It also clearly interprets the metrics of evaluation based on the classification neural network model.



Chapter 2 Introduction to Neurodegenerative Diseases........................................................25 Reihaneh Seyedebrahimi, Anatomy Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran & Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Piao Yang, Department of Molecular Genetics, College of Arts and Sciences, The Ohio State University, Columbus, USA Maryam Azimzadeh, Department of Medical Laboratory Sciences, Khomein University of Medical Sciences, Khomein, Iran & Molecular and Medicine Research Center, Khomein University of Medical Sciences, Khomein, Iran Mohsen Eslami Farsani, Anatomy Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran & Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Shima Ababzadeh, Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran & Tissue Engineering Department, Faculty of Medicine, Qom University of Medical Sciences, Qom, Iran Naser Kalhor, Department of Mesenchymal Stem Cells, Academic Center for Education, Culture, and Research (ACECR), Iran Mohsen Sheykhhasan, Cellular and Molecular Research Center, Qom University of Medical Sciences, Qom, Iran Neurons are vital for brain function and communication. Neurodegeneration, the irreversible loss of neurons, disrupts brain-body interactions, causing diseases like Alzheimer’s disease (AD), amyotrophic lateral sclerosis (ALS), Huntington’s disease (HD), multiple sclerosis (MS), and Parkinson’s disease (PD). Factors like aging, genetics, and environment contribute to these disorders. They affect various neurons, leading to speech, movement, sensory, and balance issues. Alzheimer’s features amyloid plaques affecting memory. Parkinson’s stems from midbrain dopaminergic neuron loss, causing tremors and mobility problems. Huntington’s, a basal ganglia disorder, results from a gene mutation, inducing involuntary movements. MS involves neuron demyelination, causing diverse complications. ALS entails motor neuron degeneration, resulting in muscle weakness and paralysis. This chapter aims to provide a brief overview of neurodegenerative diseases and an introduction to some of its key characteristics.



Chapter 3 Human Brain Imaging for Cognitive Neuroscience: Data Acquisition and Preprocessing........................................................................................................59 Affaan Shaikh, Prognica Labs, UAE In this chapter, a brief background of neuroimaging a human brain by data acquisition and preprocessing is provided. Neuroimaging is a medical imaging process that uses various cutting-edge technologies with artificial intelligence and machine learning to produce a clear and specific image of the brain in a non-invasive manner. Neuroimaging methods such as EEG, CT, and MRI allow researchers to directly observe brain activities from different perspectives. Data acquisition and preprocessing are essential steps in the data analysis and machine learning pipeline. They involve collecting, cleaning, and preparing raw data for further analysis or modeling. These steps are used in noise reduction, sharpening, or brightening an image, and contrast enhancement, color correction, makes it easier to identify the key features. By combining functional brain imaging with sophisticated experimental designs, data analysis methods and machine learning algorithms, functions of brain regions and their interactions can be examined and further how the neurodegenerative diseases are diagnosed. Chapter 4 A Comprehensive Survey of Deep Learning Approaches in Neurodegenerative Disease Diagnosis and Prediction..........................................73 Pruthvi Boda, Woxsen University, India Sumanth Munari, Woxsen University, India K. Sai Rama Prasanth, Woxsen University, India Shahid Mohammad Ganie, AI Research Centre, School of Business, Woxsen University, Inda In recent years, there has been an adverse rise in neurodegenerative disorders which are commonly found in older people. There has been a lot of research conducted to characterize and diagnose these diseases. Many computational methods, particularly deep learning (DL) models, are being used to diagnose these diseases. In this chapter, the authors conduct an extensive literature survey on neurodegenerative diseases including Alzheimer’s, Parkinson’s, and amyotrophic lateral sclerosis using deep learning models. Also, they performed a comparative analysis of DL models such as CNN, RNN, ResNet50, DenseNet, etc., which have shown some groundbreaking results for neurodegenerative disease detection and classification using medical image computing including MRI and PET scans. In addition, they have discussed how these DL models are used for feature extraction and to identify disease-relevant biomarkers from high-dimensional data. Furthermore, this study emphasizes the potential of deep learning techniques to revolutionize the diagnosis of neurodegenerative diseases to collaborate in clinical practice.



Chapter 5 Deep Learning Techniques for Alzheimer’s Disease Detection: A Comprehensive Study...........................................................................................91 Bazila Farooq, Lovely Professional University, India Shahid Mohammad Ganie, AI Research Centre, School of Business , Woxsen University, India Alzheimer’s disease (AD) is a common chronic disorder with a high incidence rate that disproportionately affects elderly people. Deep learning (DL) has been increasingly popular in recent years, resulting in notable developments and innovations in medical imaging. Consequently, deep learning has emerged as the preferred approach for analyzing medical visuals, particularly in the realm of Alzheimer’s disease detection. In this chapter, the authors performed a comparative analysis of various DL models such as convolutional neural networks, DenseNet, ResNet, EfficientNet, etc., which have shown some groundbreaking results for AD disease detection. Also, they focused on investigating data collection and feature extraction techniques pertinent to AD. In addition, they further discussed briefly the deep learning models for AD detection. This not only increases hope for the advancement of AD research and therapy but also highlights how deep learning can revolutionize the field of medical image analysis and illness identification. Chapter 6 Deep Neural Networks for Early Diagnosis of Neurodegenerative Diseases.....112 K. Suneetha, Jain University, India Karthik Kovuri, Kaziranga University, Assam, India Chengamma Chitteti, Mohan Babu University, India J. Avanija, Mohan Babu University, India K. Reddy Madhavi, Mohan Babu University, India Naresh Tangadu, Aditya Institute of Technology and Management, India Early diagnosis of neurodegenerative diseases plays a remarkable role in providing timely treatment for the affected person and reduces the mortality rate. Neurodegenerative diseases can affect the mental and physical health of a person and can impact decision-making. Deep neural networks help in automating disease diagnosis by identifying biomarkers from complex patterns of a large dataset. This chapter discusses the impact of using deep neural networks in neurodegenerative disease diagnosis. The initial section of the chapter focuses on the basics of neurodegenerative diseases and their impact on the well-being of the individual. The chapter also outlines the importance of transfer learning and multimodal fusion in early disease diagnosis. Finally, the chapter explores ethical considerations to be followed during the deployment of a deep learning model used in prediction of neurodegenerative disease. Thus, this chapter provides a remarkable role of deep learning architectures in neurodegenerative disease diagnosis.



Chapter 7 Deep Learning Approaches in the Early Diagnosis of Parkinson’s Disease.......128 Bilal El-Mansoury, Faculty of Sciences, Chouaib Doukkali University, Morocco Youssef Ait Hamdan, Higher Normal School, Cadi Ayyad University, Morocco Kamal Smimih, Faculty of Sciences and Techniques, Sultan Moulay Slimane University, Beni Mellal, Morocco Abdelaati El Khiat, Higher Institute of Nursing Professions and Health Techniques, Ouarzazate, Morocco Ahmed Draoui, Faculty of Science Semlalia, Cadi Ayyad University, Morocco Mohamed Hammani, Faculty of Letters and Human Sciences, Cadi Ayyad University, Morocco Arumugam Jayakumar, Miller School of Medicine, University of Miami, USA Omar El Hiba, Faculty of Sciences, Chouaib Doukkali University, Morocco Parkinson’s disease (PD) is the second most common neurodegenerative disease worldwide. PD is characterized by motor and non-motor symptoms. It is highly established that PD is mainly caused by the degeneration of dopamine (DA) producing neurons in the substantia nigra pars compacta of the midbrain leading to nigro-striatal pathway dysregulation. The diagnosis of PD is difficult since its symptoms are quite similar to those of other disorders and current assessments of symptoms have many limitations. Moreover, there are currently no effective biomarkers for diagnosing this condition or tracking its progression. Recently, digital technologies including artificial intelligence (AI) methods have emerged. Indeed, machine learning and deep learning models can help in the diagnosis and management of PD. Deep learning models have shown promising results in the diagnosis of PD even at the early stages of the disease. This chapter will discuss the potential role of deep learning methods in the early diagnosis of PD. Chapter 8 Automated Neurological Brain Disease Detection in Magnetic Resonance Imaging Using Deep Learning Approaches........................................................150 S. Thilagavathi, Department of Computer Science, Sri Krishna Adithya College of Arts and Science, Coimbatore, India D. Sridhar, School of Computer Science, Dr.Vishwanath Karad MIT World Peace University, Pune, India S. Jawahar, School of Sciences, CHRIST Deemed to be University, Ghaziabad, India A neurological type of brain disease called multiple sclerosis (MS) impairs how well the nervous system is able to function efficiently and causes people to experience visual, sensory, and problems with movement. Multiple methods of detection have been proposed so far for diagnosing MS; among them, magnetic resonance imaging (MRI) has drawn a lot of interest from healthcare providers. The ability to quickly



diagnose lesions related to MS depends on a fundamental understanding of the anatomy and workings of the brain that MRI technology provides doctors. Using an MRI for diagnosing MS is tedious, time-consuming, and prone to human error. In the present investigation, lesion activity involves preprocessing and segmentation of the MS images from two time points using deep learning approaches. Chapter 9 Automatic Diagnosis of Parkinson’s Disease Based on Deep Learning Models and Multimodal Data.............................................................................179 Ling Li, Zhoushan Hospital, Wenzhou Medical University, China Fangyu Dai, Zhoushan Hospital, Wenzhou Medical University, China Songbin He, Zhoushan Hospital, Wenzhou Medical University, China Hao Yu, The Second Affiliated Hospital, Zhejiang University School of Medicine, China Haipeng Liu, Coventry University, UK Parkinson’s disease (PD) is a common age-related neurodegenerative disorder in the aging society. Early diagnosis of PD is particularly important for efficient intervention. Currently, the diagnosis of PD is mainly made by neurologists who assess the abnormalities of the patient’s motor system and evaluate the severity according to established criteria, which is highly dependent on the neurologists’ expertise and often unsatisfactory. Artificial intelligence provides new potential for automatic and reliable diagnosis of PD based on multimodal data analysis. Some deep learning models have been developed for automatic detection of PD based on diverse biomarkers such as brain imaging images, electroencephalograms, walking postures, speech, handwriting, etc., with promising accuracy. This chapter summarizes the state-of-the-art, technical advancements, unmet research gaps, and future directions of deep learning models for PD detection. It provides a reference for biomedical engineers, data scientists, and health professionals. Chapter 10 Advancements in Healthcare: Harnessing Machine Learning for Medical Devices................................................................................................................201 Dwith Chenna, Magic Leap Inc., USA This chapter aims to provide an in-depth exploration of the integration of machine learning techniques into medical devices, revolutionizing the landscape of healthcare. Machine learning has demonstrated its potential to enhance the accuracy, efficiency, and diagnostic capabilities of medical devices, leading to improved patient care and outcomes. Through this chapter, the authors intend to delve into the various applications, challenges, and future prospects of machine learning-enabled medical devices, highlighting their impact on modern healthcare.



Chapter 11 Ethical Considerations and Challenges in Neurodegenerative Diseases Using Machine Learning...............................................................................................234 Moushumi Das, Institute of Engineering and Technology, Chitkara University, India Hitakshi Hitakshi, Institute of Engineering and Technology, Chitkara University, India Vandana Mohindru Sood, Institute of Engineering and Technology, Chitkara University, India Kamal Deep Garg, Institute of Engineering and Technology, Chitkara University, India Sushil Kumar Narang, Institute of Engineering and Technology, Chitkara University, India Neurodegenerative illnesses, some of the most common diseases affecting public health, are affecting an increasing number of nations on a daily basis. Alzheimer’s disease (AD) and other neurodegenerative disorders, such as Parkinson’s disease (PD) and Huntington’s disease (HD), cause gradual declines in cognitive, motor, emotional, and functional abilities and have a significant impact on activities of daily life (ADL) and quality of life. Because of major improvements in digital technology over the last 10 years, digital endpoints may now be integrated into clinical trials to change how neurodegenerative symptoms are diagnosed and tracked. A few additional challenges must be considered to protect those interests. In this chapter, the moral and legal difficulties surrounding the use of technology-assisted treatment for neurodegenerative disorders will be discussed along with a new technique using machine learning algorithms that would work upon the prevailing challenges and come out with better results. Chapter 12 Future Directions and Emerging Trends.............................................................257 Revanth Vemireddy, Woxsen University, India Harish Kakaraparthi, Woxsen University, India Naveen Kumar Challakolusu, Woxsen University, India In this chapter, the authors talk about how AI and deep learning are rapidly transforming the study of neurodegenerative diseases. This chapter highlights the significant advances in neuro-imaging technology and motor function study, thanks to the integration of ML methods in research. The authors also discuss the “labelfree identification of neurodegenerative-disease-associated aggregates” technique which is a method used in DL. It is focused on studying diseases like Huntington’s disease. The authors also continue to talk about a few more ML methods, like support vector machines, random forests, and CNNs. These techniques can help predict these diseases and also treat conditions such as Alzheimer’ s disease or ALS. The



pressing need to add more AI and ML technologies in this difficult research area is clearly shown by the authors. While the techniques discussed are promising, there’s still a long way to go. The authors talk about future directions of this technology, the challenges, overall impact on diagnostics, treatment, patient care, and the future of this technology itself. Compilation of References............................................................................... 270 About the Contributors.................................................................................... 320 Index................................................................................................................... 323

xvii

Preface

It is with great pleasure and enthusiasm that we present Deep Learning Approaches for Early Diagnosis of Neurodegenerative Diseases, a pioneering reference book that delves into the intricate intersection of artificial intelligence and neuroscience. This collaborative effort brings together the expertise of distinguished professionals in the realms of deep learning, neurology, and healthcare. Our esteemed contributors, Raul Rodriguez, Hemachandran Kannan, Revathi Theerthagiri, Khalid Shaikh, and Sreelekshmi Bekal, have collectively crafted a comprehensive resource that seeks to revolutionize the landscape of early diagnosis and management of neurodegenerative disorders. Neurodegenerative diseases, such as Alzheimer’s and Parkinson’s, present formidable challenges to global health. Recognizing the pivotal role of early diagnosis in effective intervention, this book endeavors to explore cutting-edge techniques and advancements in the realm of deep learning applied to neurodegenerative disease diagnosis. The primary objective of this volume is to provide a holistic resource that integrates deep learning methodologies with neuroscience, offering valuable insights into the early detection of neurodegenerative disorders. Through a fusion of innovative research and practical applications, the book aims to: • • • • •

Present state-of-the-art deep learning techniques tailored to neurodegenerative disease diagnosis. Bridge the gap between AI experts and neurologists, fostering interdisciplinary collaboration. Offer insights into the development of accurate, non-invasive, and costeffective diagnostic tools. Showcase practical applications of deep learning in clinical settings, enhancing disease management. Contribute to current research by promoting novel approaches and potential breakthroughs, ultimately advancing the field of early diagnosis for neurodegenerative diseases.

Preface

The content of this reference book is organized into thematic chapters, each addressing key aspects of the integration between deep learning and neurodegenerative diseases. From the fundamentals of deep learning to the ethical considerations and challenges in its application, the book encompasses a diverse array of topics, including neuroimaging data acquisition, feature extraction, multimodal data integration, evaluation and validation of models, translational applications, clinical implementations, and future directions. Our target audience includes researchers, clinicians, and professionals in the fields of neurology, artificial intelligence, machine learning, and biomedical engineering. Additionally, graduate students, postdoctoral researchers, industry professionals, policymakers, and healthcare administrators focused on improving diagnostic practices and patient care for neurodegenerative disorders will find this publication informative and thought-provoking. We extend our gratitude to the contributors for their invaluable insights and dedication to advancing knowledge in this critical intersection of fields. May this book serve as a catalyst for transformative solutions, pushing the boundaries of early diagnosis and paving the way for a future where neurodegenerative diseases can be detected and managed more effectively. Chapter 1: Fundamentals of Deep Learning: This chapter, authored by Chandrika PV and Sandeep Kelkar, provides a foundational understanding of deep learning models. It commences with an overview of artificial intelligence, machine learning, and deep learning, emphasizing their interconnectedness. The chapter is structured into three sections, tracing the evolution of deep learning models, elucidating the components of neural networks, and comprehensively detailing their functioning. A practical application is demonstrated through a Parkinson’s case study, complete with Python code and an interpretation of evaluation metrics for a classification neural network model. Chapter 2: Introduction to Neurodegenerative Diseases: Authored by Reihaneh Seyedebrahimi, Piao Yang, Maryam Azimzadeh, Mohsen Eslami Farsani, Shima Ababzadeh, Naser Kalhor, and Mohsen Sheykhhasan, this chapter offers a succinct overview of various neurodegenerative diseases. It delves into the importance of neurons for brain function, the impact of neurodegeneration on brain-body interactions, and the factors contributing to disorders such as Alzheimer’s, Parkinson’s, ALS, HD, MS, and more. The aim is to provide a comprehensive introduction to the key characteristics of neurodegenerative diseases. Chapter 3: Human Brain Imaging for Cognitive Neuroscience – Data Acquisition and Preprocessing: Authored by Affaan Shaikh, this chapter provides a background on neuroimaging for cognitive neuroscience. It covers the non-invasive imaging of the human brain using technologies such as EEG, CT, and MRI. The focus is on the crucial steps of data acquisition and preprocessing in the context xviii

Preface

of neuroimaging, highlighting their significance in analyzing brain activities and aiding in the diagnosis of neurodegenerative diseases. Chapter 4: A Comprehensive Survey of Deep Learning Approaches in Neurodegenerative Disease Diagnosis and Prediction: In this collaborative effort by Pruthvi Boda, Sumanth Munari, K Sai Prasanth, and Shahid Ganie, the chapter conducts an extensive literature survey on neurodegenerative diseases. It emphasizes the use of deep learning models, including CNN, RNN, ResNet50, DenseNet, etc., for the detection and classification of diseases like Alzheimer’s, Parkinson’s, and ALS. The survey encompasses the use of these models for feature extraction and identification of disease-relevant biomarkers from high-dimensional data. Chapter 5: Deep Learning Techniques for Alzheimer’s Disease Detection – A Comprehensive Study: Authored by Bazila Farooq and Shahid Ganie, this chapter focuses on the application of deep learning techniques for the detection of Alzheimer’s disease. It presents a comparative analysis of various deep learning models, including CNN, DenseNet, ResNet, EfficientNet, etc., showcasing their groundbreaking results in AD detection. The chapter also explores data collection and feature extraction techniques relevant to Alzheimer’s, emphasizing the potential of deep learning in medical image analysis. Chapter 6: Deep Neural Networks for Early Diagnosis of Neurodegenerative Diseases: Authored by Suneetha K, Karthik Kovuri, Chengamma Chitteti, Avanija J, Reddy Madhavi K, and Naresh Tangadu, this chapter highlights the role of deep neural networks in early diagnosis. It covers the basics of neurodegenerative diseases, the importance of transfer learning and multimodal fusion, and ethical considerations in deploying deep learning models for disease prediction. Chapter 7: Deep Learning Approaches in the Early Diagnosis of Parkinson’s Disease: This chapter, co-authored by Bilal El-Mansoury, Youssef Ait Hamdan, Kamal Smimih, Abdelaati El Khiat, Ahmed Draoui, Mohamed Hammani, Arumugam Jayakumar, and Omar El Hiba, focuses on the early diagnosis of Parkinson’s Disease using deep learning approaches. It discusses the challenges in diagnosing PD, the role of dopamine-producing neuron degeneration, and how deep learning models, including Machine Learning, can aid in the diagnosis and management of PD, particularly at early stages. Chapter 8: Automated Neurological Brain Disease Detection in Magnetic Resonance Imaging Using Deep Learning Approaches: Authored by THILAGAVATHI S MAHENDRARAJAN K and Sridhar. D, this chapter explores the use of deep learning approaches for automated neurological brain disease detection, with a specific focus on multiple sclerosis (MS). The chapter discusses the challenges associated with using MRI for diagnosing MS, emphasizing the role of deep learning in preprocessing and segmentation of MS images.

xix

Preface

Chapter 9: Automatic diagnosis of Parkinson’s Disease Based on Deep Learning Models: This chapter, authored by Ling Li, Fangyu Dai, Songbin He, Hao Yu, and Haipeng Liu, addresses the automatic diagnosis of Parkinson’s disease using deep learning models. It emphasizes the importance of early diagnosis for efficient intervention in PD and provides an overview of deep learning models analyzing diverse biomarkers such as brain imaging images, electroencephalograms, walking postures, speech, handwriting, etc. Chapter 10: Advancements in Healthcare – Harnessing Machine Learning for Medical Devices: Authored by Dwith Chenna, this chapter explores the integration of machine learning techniques into medical devices. It delves into the applications, challenges, and future prospects of machine learning-enabled medical devices, showcasing their potential to enhance accuracy, efficiency, and diagnostic capabilities in healthcare. Chapter 11: Ethical Considerations and Challenges in Neurodegenerative Diseases Using Machine Learning: In this chapter by Moushumi Das, Hitakshi Hitakshi, Vandana Sood, Kamal Garg, and Sushil Narang, the ethical considerations and challenges associated with using machine learning for neurodegenerative diseases are discussed. The chapter explores the moral and legal difficulties surrounding technology-assisted treatment and proposes machine learning algorithms as a means to address prevailing challenges. Chapter 12: Future Directions and Emerging Trends: Co-authored by Revanth Vemireddy, Harish Kakaraparthi, and Naveen Kumar Challakolusu, this chapter delves into the rapidly transforming landscape of neurodegenerative disease study through AI and deep learning. It highlights advances in neuro-imaging technology, motor function study, and various machine learning methods, emphasizing their potential impact on diagnostics, treatment, patient care, and the future of technology in the field. In the culmination of Deep Learning Approaches for Early Diagnosis of Neurodegenerative Diseases, we are honored to present a groundbreaking reference book that marries the realms of artificial intelligence and neuroscience. The collaborative efforts of our esteemed contributors, Raul Rodriguez, Hemachandran Kannan, Revathi Theerthagiri, Khalid Shaikh, and Sreelekshmi Bekal, have resulted in a comprehensive resource poised to revolutionize early diagnosis and management of neurodegenerative disorders. Neurodegenerative diseases, with their formidable impact on global health, necessitate innovative solutions. Recognizing the pivotal role of early diagnosis, this book explores cutting-edge techniques and advancements in deep learning applied to neurodegenerative disease diagnosis. Our primary objective is to provide a holistic resource that integrates deep learning methodologies with neuroscience, offering valuable insights into the early detection of neurodegenerative disorders. xx

Preface

The thematic chapters are meticulously organized, covering diverse aspects from the fundamentals of deep learning to ethical considerations and challenges in its application. From neuroimaging data acquisition to feature extraction, multimodal data integration, model evaluation, translational applications, clinical implementations, and future directions—the book provides a comprehensive journey through this critical intersection of fields. Our intended audience spans researchers, clinicians, professionals in neurology, artificial intelligence, machine learning, and biomedical engineering. Graduate students, postdoctoral researchers, industry professionals, policymakers, and healthcare administrators focused on improving diagnostic practices and patient care for neurodegenerative disorders will find this publication both informative and thought-provoking. We extend our deepest gratitude to our contributors for their invaluable insights and dedication. May this book serve as a catalyst for transformative solutions, pushing the boundaries of early diagnosis and paving the way for a future where neurodegenerative diseases can be detected and managed more effectively. Raul Villamarin Rodriguez Woxsen University, India Hemachandran Kannan Woxsen University, India Revathi T. Woxsen University, India Khalid Shaikh Prognica Labs, UAE Sreelekshmi Bekal Prognica Labs, UAE

xxi

1

Chapter 1

Fundamentals of Deep Learning P. V. Chandrika Prin L.N. Welingkar Institute of Management, Development, and Research, India Sandeep Madhusudhan Kelkar Prin L.N. Welingkar Institute of Management, Development, and Research, India

ABSTRACT This chapter deals with the basic understanding of the deep learning model. It starts with a view of artificial intelligence, machine learning, and deep learning. It also represents how these three concepts are related to each other. The entire chapter is divided into three sections where Section 1 represents the evolution of deep learning models, Section 2 talks about the components of neural network, and Section 3 captures working of neural network models and different types. The chapter starts with introduction to Neurodegenerative Diseases and lastly the chapter gives a thorough implementation of neural network model by considering a Parkinson’s case study along with the Python code. It also clearly interprets the metrics of evaluation based on the classification neural network model.

INTRODUCTION Introduction to Neurodegenerative Disease Neurodegenerative disorders are basically characterized by neuron loss. It is a slow and progressive loss of neuronal cells specifically in brain. When these neuronal disorders are observed in brain neurons, it leads to many dysfunctionalities in the

DOI: 10.4018/979-8-3693-1281-0.ch001 Copyright © 2024, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Fundamentals of Deep Learning

human body. These are permanent and incurable. The main conditions of degenerative brain diseases include 1. 2. 3. 4. 5.

Dementia Demyelinating diseases Parkinson’s disease Motor Neuron diseases Prion diseases

Dementia: These cause confusion, memory loss, trouble thinking or concentrating, and behaviour changes. Demyelinating disease: This includes numbness, pain, muscle spasms, weakness and paralysis. Coordination and fatigue issues. Parkinson’s disease: This includes slowed movements, shaking and tremors, balance problems and hunched postures. Motor Neuron diseases: This affects the muscles of brain and nervous system. The neurons in those areas die and lose control. All these conditions arose because of age and observed after the age of 35 years. This is because as the functionality of the brain starts decreasing with the increase in age, the disorders also increase.

Causes of Neurodegenerative Diseases Some of the reasons for causing neurodegenerative diseases include age, genetics, environment, medical history and habits, routines and choices. Complications of neurodegenerative diseases: The complications of this disease remain common in most of the neurodegenerative diseases as it effects the functioning of brain neurons. Some of them include • • • •

Movement disorders that affect the strength, flexibility and agility. With the decrease of this it leads to increase of fractures. Paralysis is another complication of this disease. When this affects muscles that control breathing, it increases the risk of pneumonia and other respiratory conditions. They also affect memory which eventually leads for memory loss and no judgmental thinking. Because of all these complications the patient cannot live independently. Statistics representing neurodegenerative disorders:

2

Fundamentals of Deep Learning

Figure 1. Prevalence of neurodegenerative disorders in USA

Diagnosis and Tests: The diagnosis of neurodegenerative diseases depends on the condition. Based on the condition of the patient appropriate diagnosis or test is suggested. Some of the tests that are considered for diagnosing the disease include 1. Laboratory Test 2. Imaging Scan 3. Histopathology after death Laboratory Test: Laboratory tests include blood test and genetic analysis. Imaging Scan: Images of CT Scan reports and MRI scan reports are suggested for diagnosing the disease. Through these reports the health providers try to analyze the brain functionality. Sometimes other image scans are also suggested to find the root cause of the disease and to rule out certain other causes. Histopathology after death: This is also called as microscopic tissue analysis. This is conducted after the death of the patient. Some neurodegenerative conditions, such as Pick’s disease or chronic traumatic encephalopathy, aren’t diagnosable while you’re alive. Healthcare providers can suspect this condition while you’re alive, but the only way to confirm the diagnosis for certain is to look at samples of your brain under a microscope after an autopsy. 3

Fundamentals of Deep Learning

Role of AI in detecting neurodegenerative diseases: With the availability of newer technology, the health care sector is trying to predict the disorders before so that earlier diagnosis helps in preventing the worsening the conditions. In order to know the extent of application of this technology a systematic review of literature is carried on.

RELATED WORK Aguayo, et. al. (2023) in their research tried to predict the risk factors that help in implementing the preventing measures. They constructed different neural network models, a machine learning model and a cox regression transformation taking 5433 participants with 91 epidemiological and clinical baseline variables. The demographics of the participants include both men and women aged 60 years and above. The data is captured at every 2 years period and is termed as waves viz., 2004 to 2005 (wave 2) to 2016-2017 (wave 8). After training this data in all the five models that is DNN algorithm: Feedforward, Tab Transformer, Dense Convolution, Cox models: Elastic Net Regularization and Selected features. When compared with the performance of all these models Tab Transformer performed well on heterogenous tabular dataset with numerous features. This model can also work on censored data. Kriti Raj et al. (2022) developed a classification model for neurodegenerative diseases. This research paper presents fully automated early screening system based on the capsule network for the classification of neurodegenerative diseases. Capsule network model is used to classify the patient into three classifications viz., Alzheimer class or Parkinson Class and Healthy control. Along with the capsule classification model a max pooling convolutional neural network model is used for image classification. To develop these models the datasets are extracted from Alzheimer disease and Parkinson progression dataset. The proposed neurodegenerative disease caps systems deliver accuracies of 97.81%, 98% and 96.81% for Alzheimer, Parkinson and Healthy control. Arafa et al. (2022) used two deep neural networks to discover the symptoms of Alzheimer disease. The primary objective of this research is to present a complete framework of deep learning approaches and convolutional neural networks. Under the deep learning approaches a simple convolution neural network is considered and under the deep learning model VGG16 model is selected. Based on the experimental studies, both the proposed model represented high accuracy. The paper also finds that when the same models are used on the published data, the results were not convincing. Ashan Bin Tufali et al. (2021) developed 3D Convolution Neural Network model to classify the features towards Alzheimer Disease or Parkinson Disease or Normal 4

Fundamentals of Deep Learning

Class classes. To develop this model, data related to positron emission tomography and single photon emission computed tomography is collected. Along with the 3D convolution neural network model, discrete cosine transformation, random weak gaussian blurring and zooming augmentation methods are developed. To validate this models 5-fold, 10-fold cross validation approaches are considered. On evaluating these models, the best performing model is found to be random weak Gaussian blurred augmentation with 5-fold. The proposed model is found to be performing better because of minimal correlation between dementia types on the clinical pathological between Alzheimer disease and Parkinson disease. Liu et al. (2018) in their research paper implemented multi-modality cascaded convolutional neural networks for Alzheimer’s disease diagnosis. This paper proposes to construct a cascade convolutional neural network by considering the features of magnetic resonance images (MRI) and positron emission tomography (PET) brain images. A multiple deep 3D – CNNs and a high level 2D – CNNs with SoftMax layer is developed to classify the features. The models are developed on 397 subjects which include 93 Alzheimer’s Disease and 204 mild cognitive impairment and 100 normal controls. Through this experimental method the accuracy is observed to be 93.26% in classifying the Alzheimer’s disease. Therefore, through the review of literature it can be clearly summarized that the application of deep learning models is majorly applied in predicting the early symptoms of Alzheimer’s and Parkinson disease, but the limitation is that it considers the images of positron emission tomography and MRI images. Most of the literature represents the processing of images using deep neural network models. This chapter tries to give a thorough conceptual understanding of these deep learning models from the processing of the data to the model building along with the interpretation. In this the data related to the images is not considered whereas the other variables like voice measures are considered.

INTRODUCTION TO DEEP LEARNING Deep Learning is one of the modern techniques used in health care to process the large volumes of the data. This has the capability of processing the images and helping to arrive at accurate diagnosis. The techniques of deep learning are used widely in cancer detection through image processing, medical transcription using Natural Language Processing and cost of the treatment estimation. These models are also used to identify the significant features based on the data provided. Having done the research in the area of deep learning the author tries to implement these techniques in diagnosing neurodegenerative diseases. Based on the past studies is found that the implementation of these techniques related to neurodegenerative 5

Fundamentals of Deep Learning

disease and patient care is at nascent stage. Taking this as a gap this chapter provides a complete understanding of Deep Learning Algorithms. This chapter is developed based on the case study, talking about the fundamental challenges faced by the health care industry and how technology like Artificial Intelligence and techniques like deep learning has helped to solve the problems quickly. Each section of the chapter gives the python code associated along with the data captured through secondary sources. At the end of this chapter, learner gets a clear and in depth understanding of implementation of neural network on python. At the end of this the readers will be knowing how and when these techniques can be implemented to fasten the diagnosis process and improve patient care. To develop Artificial intelligence, a set of algorithms are required which helps in processing quantitative and qualitative data. Quantitative data is called as Structured data and Qualitative data is called as Unstructured data. In health care the data generated is captured in both the forms. For example, Quantitative data like electronic health records, clinical data, pre-medical history of the patient etc., and Qualitative data like CT Scan images, X-rays, discharge summary etc., So, it is required to analyze both the forms of data to derive a proper medical procedure with correct diagnosis. To analyze both structured (quantitative) and unstructured (qualitative) data there requires technological support. The emerging technology which helps us to deal with both structured and unstructured data is Artificial Intelligence. To develop AI there are set of algorithms on which the data can be trained. Those algorithms fall under two categories: Machine Learning and Deep Learning. Machine Learning algorithms and Deep Learning algorithms are said to be the subsets of Artificial intelligence.

6

Fundamentals of Deep Learning

Figure 2. Subset of artificial intelligence

EVOLUTION OF DEEP LEARNING Deep Learning algorithms are evolved based on the biological function of the human brain. Human brain has neurons which are interconnected and stores the information in the form of memories. Figure 3. Evolution of DL algorithms Source: Wikipedia

7

Fundamentals of Deep Learning

Axons in the neuron collect the information through sensory organs and store it in nucleus. The dendrites of the nucleus help to connect to other nucleus and communicate through chemicals which are called as neuro transmitters. All these connected together form a network. As the neurons are connected to each other this is called as Neural Network (NN). The same mechanism is adapted in the Deep Learning algorithms. Every DL algorithm has three layers which include input layer, hidden layer and output layer. They adopt the concept of NN where the INPUT is trained on multiple hidden layers with different nodes by generating the weights. NN has the ability to extract the patterns and trends through complex networks. The INPUT is processed for multiple times and process to get the output. Some of the examples which adapt deep learning include image processing, language translation, speech recognition, robotic surgery, simulation, autonomous driving etc. These deep learning models are capable of learning and representing intricate patterns and features in data.

COMPONENTS OF NEURAL NETWORK NN are data driven models. They are capable to discover relationship and association with the input and output variable. DL Models are developed based on neural network evolved from biological function of ‘human brain’ which is very complex and interconnected with neurons. They are designed based on the analogy of human brain. Neural Network models are proven to be robust in processing the INPUT and exhibiting the OUPUT which is continuous or discrete in nature. The INPUT variable is matched with OUPUT variable by applying an activation function. Each INPUT which is processed will be taken through multiple dense layers. They also have the capability of storing the information in the Memory to process. Neural networks consist of organized layers of neurons or artificial neurons (interconnected nodes). These networks are capable of learning and generalizing from data, making them a fundamental component of deep learning and machine learning systems.

8

Fundamentals of Deep Learning

Figure 4. Neural network architecture

Parts of Neural Network A typical Neural Network (NN) model consists of three parts: 1. Input layer 2. Hidden layer and 3. Output layer Input Layer is first layer in NN which is fed with Input data for processing. Hidden Layer: Hidden Layer is the second layer of the NN which consists of neurons that takes input data for processing. Number of neurons to be considered in NN is not limited similarly number of hidden layers is also not limited in NN. It can also be noted that number of neurons in each hidden layer need not be the same as the count of neurons on each hidden layer. Every Input fed in NN is taken to the hidden layer and is processed on all neurons in a hidden layer. While processing the layer generates weights to each Input data. There are different ways of connecting the input layer to the neurons in the hidden layer. Those include 1. 2. 3. 4.

Fully Connected Layer 2D Convolution Layer LSTM Layer Attention Layer

When each input data is connected to all the neurons in the next layer then it is said to be fully connected. 9

Fundamentals of Deep Learning

The fully connected NN processes the data by generating weights, processed data with high weights is taken to the next layer, while the rest of the processed data is fed back to NN as the input data. The data is iterated for multiple times in the network till the learning happens. In order to generate logged weights, the connections within the NN are to be evaluated. Some NN are basically trained with back propagation algorithm. The mathematics behind the back-propagation algorithm considers basic differential calculus and chain rule of differentiation. Every layer is trained on Activation Function to generate weights. After the requisite iterations the NN stops learning by dropping out certain weights which can be given as INPUT again. Output Nodes: The processed INPUT from hidden layer is fed as to OUPUT layer. The OUPUT layer is further trained with optimization function to generate the requisite OUPUT. Figure 5. Deep learning neural network

Every NN when the input is fed to the first hidden layer generates weights at each node, these weights are then taken to form a mathematical function called Activation Functions. Later the OUPUT signal is compared with the INPUT response. If the INPUT response matches with the OUPUT signal, then the error signal or the accuracy signal will be generated. If the INPUT response doesn’t match the OUPUT 10

Fundamentals of Deep Learning

signal, then the data is fed again to the dense layer and the process gets repeated. Hence neural network adapts iterative process.

Activation Functions To activate the weights in the NN activation functions are used. These activation functions help in extracting the relevant information and send the processed data again as INPUT through back propagation technique. Activation function is used in every layer of the NN to increase the learning rate by defining the drop out ratio. Activation functions used in the model constitutes processing of the data by using different mathematical functions in the NN. The basic activation function used ∝ (x) = ∑ wixi which is weighted average. Y = Activation (∑ (Weight * INPUT) + bias) (1) The most commonly used activation functions include: 1. Sigmoid function:

f a 

1 1  e a

(2)

This function is used when OUPUT is bounded between 0 & 1. This activation function is used in classification model to generate Boolean numbers as OUPUT. The generated OUPUT is interpreted as the class probability. It is commonly called logistic function, classification model using NN. 2. Hyperbolic Tangent

f a 

e a  e  a 1  e 2 a  e a  e  a 1  e 2 a

(3)

This is used when the OUPUT is to be falling either towards 0 to +1. This is popularly called as ‘tanh’. 3. Bipolar Sigmoid

f a 

1 1  e a 1   1  e a 1  e a

(4)

4. Rectified Linear Unit (ReLu):

11

Fundamentals of Deep Learning

a, ifa  0 , This is a mix of identity and threshold function. f a   0, ifa  0 It is popularly represented as ‘ReLu’ this is always an increasing function.

TYPES OF NEURAL NETWORKS The various types of NN that are available include Multi-Layer Perceptron, Artificial Neural Network (ArNN), Convolution Neural Network (CoNN) and Recurrent Neural Network with Long Short-Term Memory (RNNLSTM).

Artificial Neural Network (ArNN) ArNN are the extension of Multiple Layer perceptron with back propagation. The data is fed into the INPUT layer and the data is processed through the hidden layers by generating the weights. Artificial Neural Network contains hidden layers and each hidden layer will have a set of neurons alike basic neural network. Every layer is processed with learning rate and dropout function. After processing the INPUT information by generating the weights the dropped-out data is fed back as INPUT to the network model. Logs of highest generated weights are then taken on to the activation function. Selection of the activation function depends on the desired outcome. After processing through the activation function then the processed data with requisite weights is optimized by using optimization functions. Then the last layer which is the OUPUT layer helps in classifying the data towards the right class. Figure 6. Artificial neural network: Architecture

12

Fundamentals of Deep Learning

ArNN is fed with input data and is processed through dense layers. Each dense layer is activated with an activation function and the dropout layer. Once the INPUT is fed to the first dense layer, weights are generated and generated weights are processed with Activation function of ‘sigmoid’ and a drop out ratio to initiate the learning process. Figure 7. Working of artificial neural network

Totally three dense layers are used when training Artificial neural network. The first three dense layers are trained using ‘tanh’ activation function with drop out of 0.1. The last dense layer is activated with sigmoid function with a dropout of 0.1. After processing through first three dense layers with ‘tanh’ activation function the INPUT data is fed to fourth dense layer which uses ‘sigmoid’ activation function and then classifies processed INPUT towards zero or one. The learning parameters considered while training Artificial Neural Network • • • •

Training data = 80 percent of the data. Test data = 20 percent of the data. Units of Dense = [100,3] which represents 100 neurons in each dense layers and with 3 dense layers. Activation Function: ‘tanh’ for first two layers and ‘sigmoid’ at last dense layer since it is a classification problem.

However, ArNNs have several limitations, because of difficulty in setting the weights and biases ArNN approximates the target data but can be taken care by using different activation function. ArNNs also leads for over-fitting, which means that it memorizes the data and fails to give generalized OUPUT. This is due to the fact 13

Fundamentals of Deep Learning

that NN has to be tuned on number of parameters. when the number of parameters is large over fitting is more likely to occur.

Convolution Neural Network CoNN is a type of Deep Neural Network (DNN) which is commonly used in image processing, computer vision and Natural Language Processing. The architecture of CoNN adopted in this study is as follow Figure 8. Convolution neural network architecture

CoNN is generally used for processing high-dimensional data. Major applications of Convolution Neural Network include image processing, Natural Language Processing and hand write recognition. Every Convolution Neural Network involves three components: 1. Convolution Layer: This layer takes the INPUT by applying filters. This INPUT is processed across filters to get convolved OUPUT. 2. Pooled Dense Layer: The convolved OUPUT is then processed onto the pooled dense layers, to extract the correct features of the INPUT data. This adapts max pooling function to weigh the INPUTs. 3. Fully connected layer: This layer connects the entire convolved processed data and extracts the OUPUT by using activation function. The convolution layers are initially processed using ‘tanh’ activation function and later at the last layer it is processed with ‘sigmoid’ activation function to estimate the direction of the index. 14

Fundamentals of Deep Learning

The Learning Parameters of CoNN: • • • • • • •

Training data ~ 80 percent of the dataset around 1840 data points Test data ~ 20 percent of the datasets 460 data points Units of Dense = [175,6] which represents 175 neurons in each dense layers and with 6 dense layers. No. of Filters: [112, 64, 32, 16] the dense layers have been processed with these filters. Pool Size: [6,1] in respective dense layers. Kernel Size = 6 Activation Function: First five layers are activated with ‘tanh’ whereas last layer is activated with ‘Sigmoid’ to classify processed INPUT.

Recurrent Neural Network and Long ShortTerm Memory (RNNLSTM) RNN are good with sequential data by defining the occurrence of relation over variables across the time. These are found to be good in processing the sequential data. RNN have the capability of storing the data in their memory cell. Recurrent NN apply time series data and the OUPUT generated through the layers at time, t is taken as the INPUT to the other units of time t+1.

st  f  st 1 , xt 

(5)

Where St = State at step ‘t’ St-1= State at previous step. Xt = Current step. In RNN each state depends on all previous computations. RNNs can be trained on storing memory over time. When the INPUT is given data is processed based on state ‘S’ containing the information of the previous steps. Since RNNs consist memory gate they have the capacity to store information for long period. But practically they need batch size to process and store. INPUT given to RNNs can be trained on different relations like one-to-one, one-to-many, many-to-one, many-to-many to generate appropriate functions. The fundamental working of RNN considers two parameters 1. INPUT weight ‘U’ and 2. recurrence weight ‘W’. Let ‘Y’ be the OUPUT state then the RNN is defined as function of:

st  st 1W  xtU

(6) 15

Fundamentals of Deep Learning

RNN calculate weights and generates weight matrix. This weight matrix is used as a computational function at all states. Weights generated are updated at every time step. But while training RNNs, there is a difficulty in dealing with vanishing gradient and exploding gradient. These gradients are learning estimates of NN model. When NN is trained there is a chance that either NN may not learn or learns at the first iteration itself leading to zero cost function. Exploding Gradient is one where values jump out of the parameter values and show 100 percent accuracy. The exploding and vanishing gradient are expressed as:

z 1 z L L yÆ zn     i  i i yÆ zn zh 1 zi w1

(7)

Source: The Vanishing Gradient Problem by Harini Suresh • •

If |w| > 1 the gradient grows exponentially. If |w|