Artificial Intelligence and Internet of Things: Applications in Smart Healthcare (Innovations in Big Data and Machine Learning) [1 ed.] 0367562944, 9780367562946

Citation preview

Artificial Intelligence and Internet of Things

Innovations in Big Data and Machine Learning Series Editors: Rashmi Agrawal and Neha Gupta

This series will include reference books and handbooks that will provide the conceptual and advanced reference materials that cover building and promoting the field of Big Data and Machine Learning which will include theoretical foundations, algorithms and models, evaluation and experiments, applications and systems, case studies, and applied analytics in specific domains or on specific issues. Artificial Intelligence and Internet of Things Applications in Smart Healthcare Edited by Lalit Mohan Goyal, Tanzila Saba, Amjad Rehman, and Souad Larabi For more information on this series, please visit: https://www.routledge.com/ Innovations-in-Big-Data-and-Machine-Learning/book-series/CRCIBDML

Artificial Intelligence and Internet of Things Applications in Smart Healthcare

Edited by

Lalit Mohan Goyal, Tanzila Saba, Amjad Rehman, and Souad Larabi-Marie-Sainte

First edition published 2021 by CRC Press 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742 and by CRC Press 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN © 2021 selection and editorial matter, Lalit Mohan Goyal, Tanzila Saba, Amjad Rehman, and Souad Larabi-Marie-Sainte; individual chapters, the contributors CRC Press is an imprint of Taylor & Francis Group, LLC The right of Lalit Mohan Goyal, Tanzila Saba, Amjad Rehman, and Souad Larabi to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978750-8400. For works that are not available on CCC please contact [email protected] Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Goyal, Lalit Mohan, editor. Title: Artificial intelligence and Internet of Things : applications in smart healthcare / edited by Lalit Mohan Goyal, Tanzila Saba, Amjad Rehman, and Souad Larabi. Description: First edition. | Boca Raton : CRC Press, 2021. | Series: Innovations in big data and machine learning | Includes bibliographical references and index. Identifiers: LCCN 2020053220 (print) | LCCN 2020053221 (ebook) | ISBN 9780367562946 (hardback) | ISBN 9781003097204 (ebook) Subjects: LCSH: Medical technology. | Wireless communication systems in medical care. | Medical care—Technological innovations. | Artificial intelligence. | Internet of things. Classification: LCC R855.3 .A78 2021 (print) | LCC R855.3 (ebook) | DDC 610.285—dc23 LC record available at https://lccn.loc.gov/2020053220 LC ebook record available at https://lccn.loc.gov/2020053221 ISBN: 978-0-367-56294-6 (hbk) ISBN: 978-0-367-56295-3 (pbk) ISBN: 978-1-003-09720-4 (ebk) Typeset in Times by KnowledgeWorks Global Ltd.

Contents Preface......................................................................................................................vii Acknowledgments......................................................................................................ix Editor Biographies.....................................................................................................xi

PART I Chapter 1 AI Technologies in Health-care Applications....................................... 3 Sajid Iqbal, Mehreen Tariq, Hareem Ayesha, Noor Ayesha Chapter 2 Two-Level Breast Cancer Staging Diagnosis...................................... 45 Sahar Bayoumi, Sanaa Ghouzali, Souad Larabi-Marie-Sainte, Hanaa Kamel Chapter 3 Breast Cancer Detection and Diagnostic with Convolutional Neural Networks................................................................................. 65 Muhammad Kashif, Amjad Rehman, Tariq Sadad, Zahid Mehmood Chapter 4 Automated Medical Image Analysis in Digital Mammography......... 85 Mohsen Karimi, Majid Harouni, Shadi Rafieipour Chapter 5 Precise Segmentation Techniques in Various Medical Images........ 117 Majid Harouni, Mohsen Karimi, Shadi Rafieipour Chapter 6 Lung Cancer Detection and Diagnosis with Deep Learning Models Evaluation............................................................. 167 Tanzila Saba, Muhammad Kashif, Hind Alaskar, Erum Afzal Chapter 7 Image-Based Glioblastoma Segmentation—Traditional versus Deep Learning Approaches: Analysis, Comparisons and Future Look.................................................................................................. 189 Amjad Rehman

v

vi

Contents

Chapter 8 Artificial Intelligence Techniques for Glaucoma Detection Through Retinal Images—State of the Art......................209 Ayesha Shoukat and Shahzad Akbar Chapter 9 Artificial Intelligence in Brain Tumor Detection through MRI Scans—Advancements and Challenges................................... 241 Sahar Gull and Shahzad Akbar

PART II Chapter 10 An Empirical Study of Domain, Design and Security of Remote Health Monitoring Cyber-Physical Systems ................... 279 Albatool Al-katrangi, Shahed Al-kharsa, Einaas Kharsah, Anees Ara Chapter 11 IoT Security and Privacy Issues—A Game of Catch-Up.................. 301 Atheer Almogbil Chapter 12 Internet of Things for Mitigating Climate Change Impacts on Health............................................................................. 317 Rehab A. Rayan, Imran Zafar, Christos Tsagkaris, Iryna Romash Chapter 13 Automated Hybrid Recommender System for Cardiovascular Disease with Applications in Smart Healthcare............................... 331 Zahid Mehmood, Fouzia Jabeen, Muhammad Tahir, Rehan Mehmood Yousaf, Noor Ayesha Chapter 14 Virtual Reality—Robotic Improved Surgical Precision Using AI Techniques......................................................................... 353 Aditi Sharma and Aman Dureja Chapter 15 Lung Nodule Detection and Classification using 2D and 3D Convolution Neural Networks (CNNs)............................................. 365 Hikmat Yar, Naveed Abbas, Tariq Sadad, Sajid Iqbal Index���������������������������������������������������������������������������������������������������������������������� 387

Preface Smart healthcare is nowadays one of the major challenges for most of the health-care organizations. The need to embrace innovation by introducing and implementing new technology is critical in such an important and diverse industry as healthcare. The main objective of this book is to provide a complete insight into the state of the art and recent advancement of artificial intelligence (AI) and Internet of things (IoT) applications in smart healthcare. The trade-offs between smart health systems, cost and performance have become key challenges that must be addressed – particularly the enrich combination of AI and IoT that require further research on emerging smart equipment, monitoring systems, early diagnosis, and remote caring centers. This book offers a guided tour of the state of the arts and the emerging applications of AI and the IoT in smart healthcare. It provides comprehensive information on the fundamental concepts, applications, algorithms, protocols, new trend and challenges, and research results in the area of AI and IoT in smart healthcare. The powerful combination of AI and IoT technology helps to avoid unplanned downtime, increase operating efficiency, enable new products and services, and enhance risk management. Cutting-edge AI subjects such as deep learning, reinforcement learning, architecture design security and privacy issues of IoT, and remote-healthmonitoring systems are thoroughly presented. It is an invaluable resource giving knowledge on the core and specialized issues in the field, making it highly suitable for both the new and experienced researcher in the field of smart healthcare. Current analysis and comparisons on benchmark datasets in different areas of healthcare such as brain tumor, lung cancer, cardiovascular, breast cancer, and mental behavior are also presented. Research gaps highlighted to be filled by the future researchers. The book consists of two main parts: Part I: This part focuses on the applications of AI in healthcare such as diseases diagnosis with AI techniques, mining and managing medical data for analysis and comparisons. Part II: This part deals with IoT applications in healthcare such as smart healthcare applications, cloud computing, and service models for smart healthcare, remote medical assistance using cloud services, security and privacy issues of IoT.

vii

Acknowledgments This work was supported by Artificial Intelligence and Data Analytics Lab (AIDA), CCIS, Prince Sultan University, Riyadh, Saudi Arabia. Editors are thankful for this support.

ix

Editor Biographies Lalit Mohan Goyal  received BTech (Hons.) in computer engineering from Kurukshetra University, Kurukshetra; MTech (Hons.) in information technology from Guru Gobind Singh Indraprastha University, New Delhi; and PhD in computer engineering from Jamia Millia Islamia, New Delhi. He has 16 years of teaching experience in the area of theory of computation, parallel and random algorithms, distributed data mining, and cloud computing. He is currently with the Department of Computer Engineering, J.C. Bose University of Science and Technology, YMCA, Faridabad, India. He is currently working on the project sponsored by the Indian Council of Medical Research, Delhi. He has published research papers in SCI indexed and Scopus indexed journals and conferences. He is a reviewer of many reputed journals and conferences. Tanzila Saba earned her PhD in document information security and management from Faculty of Computing, Universiti Teknologi Malaysia (UTM), Malaysia in 2012. She won the best student award in the Faculty of Computing, UTM for 2012. Currently, she is serving as an Associate Chair of Information Systems Department in the College of Computer and Information Sciences, Prince Sultan University Riyadh, Saudi Arabia. Her primary focuses of research focuses in recent years are medical imaging, pattern recognition, data mining, MRI analysis, and soft-computing. She has above 200 publications that have above 5000 citations with h-index 46. Her mostly publications are in biomedical research published in ISI/SCIE indexed. Due to her excellent research achievement, she is included in Marquis Who’s Who (S & T) 2012. Currently, she is an editor and reviewer of reputed journals and on the panel of TPC of international conferences. She has full command of a variety of subjects and taught several courses at the graduate and postgraduate levels. On the accreditation side, she is a skilled lady with ABET and NCAAA quality assurance. She is the senior member of IEEE. Dr. Tanzila is the leader of the Artificial Intelligence and Data Analytics Research Lab at PSU and active professional members of ACM, AIS, and IAENG organizations. She is the PSU WiDS (Women in Data Science) ambassador at Stanford University and Global Women Tech Conference. She earned the best researcher award at PSU for consecutive 4 years. She has been nominated as a Research Professor at PSU since September 2019. Amjad Rehman is a Senior Researcher in the Artificial Intelligence and Data Analytics Lab, Prince Sultan University, Riyadh, Saudi Arabia. He received his PhD and Postdoc from Faculty of Computing, Universiti Teknologi Malaysia with specialization in Forensic Documents Analysis and Security with honors in 2010 and 2011, respectively. He received the Rector Award 2010 for best student in the university. Currently, he is PI in several funded projects and also completed projects funded from MOHE, Malaysia, Saudi Arabia. His keen interests are in data mining, health informatics, and pattern recognition. He is author of more than 200 ISI journal papers, conferences and is a senior member of IEEE with h-index 44. xi

xii

Editor Biographies

Souad Larabi-Marie-Sainte earned her Ph. D degree in Artificial Intelligence from the Computer Science Department, Toulouse1 University Capitole, France, in 2011. She earned three degrees, Engineer in Operation Research from the University of Sciences and Technology Houari Boumediene, Algeria, and a dual M.Sc. degree in mathematics, decision and organization from Paris Dauphine University, France, and computing and applications from Sorbonne Paris1 University, France. She was an Associate Researcher with the Department of Computer Science, Toulouse1 Capitole University, France. Then, an Assistant Professor with the College of Computer and Information Sciences, King Saud University. She is currently an Associate Professor with the Department of Computer Science, Associate Director of postgraduate programs at the College of Computer and Information Sciences, and Vice-Chair of the ACM Professional Chapter at Prince Sultan University. She taught several courses at the graduate and postgraduate levels. She published several articles in ISI/Scopus indexed and attended various specialized international conferences. Currently, she is an editorial board member and reviewer of reputed journals and on the panel of TPC of international conferences. Her research interests include Metaheuristics, Artificial Intelligence, Healthcare, Machine and Deep Learning, Natural Language Processing, Educational Data Mining, Pattern recognition, and Software Engineering.

Part I AI Applications in Healthcare

1

AI Technologies in Health-care Applications Sajid Iqbal, Mehreen Tariq, Hareem Ayesha, Noor Ayesha

CONTENTS 1.1 Artificial Intelligence (AI) in Medical Care......................................................4 1.1.1 Advantages of AI in Healthcare............................................................5 1.1.2 Risks and Limitations of AI in Healthcare............................................6 1.1.3 Applications of AI in Health-care Industry........................................... 7 1.2 Type of Data and AI Systems............................................................................ 8 1.2.1 Numeric Data.........................................................................................9 1.2.2 Textual Data......................................................................................... 10 1.2.3 Imaging Data....................................................................................... 10 1.2.4 Genetic Data........................................................................................ 10 1.2.5 Sound and Analog Data....................................................................... 11 1.2.6 Synthetic and Surrogate Data.............................................................. 11 1.3 AI Technologies Used in Medical Processes................................................... 11 1.3.1 Symbolic AI (S-AI)............................................................................. 11 1.3.2 Knowledge-based AI (KB-AI)............................................................ 11 1.3.2.1 Advantages of S-AI Systems................................................ 12 1.3.2.2 Disadvantages and Limitations of S-AI Systems................. 13 1.3.3 Fuzzy Logic System (FLS).................................................................. 13 1.3.3.1 Advantages of Fuzzy Logic Systems.................................... 14 1.3.3.2 Disadvantages and Limitations of S-AI Systems................. 14 1.3.4 Machine Learning............................................................................... 15 1.3.4.1 Advantages of Machine Learning........................................ 15 1.3.4.2 Disadvantages and Limitations of Machine Learning......... 16 1.3.5 Artificial Neural Networks (ANNs).................................................... 17 1.3.5.1 Deep Learning (DL)............................................................. 17 1.3.5.2 Types of Deep Learning....................................................... 18 1.3.5.3 Advantages of Deep Learning.............................................. 19 1.3.5.4 Disadvantages and Limitations of Deep Learning............... 19 1.3.6 Intelligent Multi-Agent Systems (IA)..................................................20 1.3.6.1 Advantages of Intelligent Agents..........................................20 1.3.6.2 Disadvantages and Limitations of Intelligent Agents........... 21 1.3.7 Hybrid Intelligence (HA)..................................................................... 21

3

4

Artificial Intelligence and Internet of Things

1.4 Computer-Aided Diagnosis (Cad) Using AI Technologies.............................. 21 1.4.1 Cardiovascular Diseases...................................................................... 21 1.4.1.1 Diagnosis through Risk Factors (Clinical Data)................... 21 1.4.1.2 Diagnosis through ECG (Heart Signals)............................... 22 1.4.1.3 Echocardiogram (Live Heart Video).................................... 22 1.4.2 Neurological Disease........................................................................... 23 1.4.2.1 Diagnosis through Imaging Data.......................................... 23 1.4.2.2 Diagnosis through EEG (Brain Signals)...............................24 1.4.3 Cancer..................................................................................................24 1.4.3.1 Lungs Cancer........................................................................24 1.4.3.2 Breast Cancer........................................................................25 1.4.3.3 Colorectal Cancer.................................................................25 1.4.3.4 Prostate Cancer.....................................................................26 1.4.3.5 Others.................................................................................... 27 1.4.4 Ocular Disease..................................................................................... 27 1.4.5 Diabetes Mellitus.................................................................................28 1.4.5.1 Diagnosis through ECG (Heart Signals)...............................28 1.4.5.2 Diagnosis through Clinical Data (Risk Factors)................... 28 1.4.6 Bacterial and Viral Infections.............................................................28 1.4.6.1 Lower Respiratory Infections...............................................28 1.4.6.2 Hepatitis Viral Infection....................................................... 29 1.4.7 Other Diseases..................................................................................... 30 1.5 Some Other AI Technologies for Disease Diagnosis and Healthcare............. 30 1.5.1 Virtual Assistants (Chatbots)............................................................... 30 1.5.2 Disease Monitoring Devices (Wearable)............................................. 31 1.5.3 Cloud-based/Web-based Diagnosis and Prediction (Remote Diagnosis)............................................................................................ 31 1.5.4 Benefits of AI in Medical Diagnosis................................................... 32 1.5.5 Challenges of AI in Medical Diagnosis............................................... 33 1.6 AI in Health-Care Research and Industry....................................................... 33 1.6.1 Research Community.......................................................................... 33 1.6.2 Technology Providers..........................................................................34 1.6.3 Service Providers................................................................................. 35 1.6.4 End Users............................................................................................. 35 References................................................................................................................. 36

1.1  ARTIFICIAL INTELLIGENCE (AI) IN MEDICAL CARE There is no universal definition of the term artificial intelligence (AI) and it has different meanings depending on the type of AI method applied to solve the problem (Younus et al., 2015). According to Oxford English Dictionary, AI is formally defined as “the study and development of computer systems that can copy intelligent human behavior”. In medical, AI applies a set of algorithms over a significant volume of medical data to learn its features and assists the medical practitioners and patients through extracted information. In recent years, because of the increase in the availability of a large amount of medical data, AI techniques are being widely

AI Technologies in Health-care Applications

5

used in medical care (Liaqat et al., 2020). AI is revolutionizing whether to find relationships between genomic codes, to develop virtual robots to perform surgery or to maximize the efficiency of a bigger health-care unit, i.e. hospital. This vast use of AI in medical care even fuels the debate whether AI doctors will finally replace the human doctors in the future. AI is not meant to replace human doctors but to automate the diagnosis procedure and to assist human experts in decision-making. Technologies built through combining AI and human intelligence are developed to support and enhance human processing rather than to replace humans.

1.1.1  Advantages of AI in Healthcare If we talk about the motivation of applying AI systems in medical care, they provide many advantages, some of which are listed in the following: • Efficient processing: AI-based machines can learn, predict, prescript and diagnose in a quick and cost-effective manner (Ejaz et al., 2018). • Experience-based reasoning: AI systems can be used to analyze the disease symptoms and can recommend treatment based on the previous cases having similar symptoms (Fahad et al., 2018; Husham et al., 2016). • Enhanced accuracy: Humans are prone to errors and variability but AI systems can generate precise and more accurate results (Iftikhar et al., 2017; Jamal et al., 2017; Javed et al., 2019a,b). • Automatic and self-learning: AI systems can help and streamline diagnosis process by extracting valuable information from unstructured data of a large number of patients. Extracted information can provide better insight in the situations where immediate health prediction is required (Javed et al., 2020a,b; Khan et al., 2019a,b,c,d,e, 2020a,b). • Useful predictions: Methods based on AI can warn the patients about their future condition based on their learning of symptoms and corresponding actions (Khan et al., 2017). • Early disease detection: Detection of disease at later stages can lead to severe and fatal results. AI systems can easily detect different types of diseases at early stages that could be cured at right time (Marie-Sainte et al., 2019a,b; Mittal et al., 2020). • Processing difficult data: Human experts have a number of limitations that confine their professional capabilities. One of them is the analysis of complex and large data. AI-based algorithms break such bounds and can see beyond human level. • Complex and deep analysis: Analyzing given medical data from multiple aspects is very complex and tiresome job that is mostly avoided by the medical practitioners in usual clinical trials. However, artificially intelligent machines using state-of-the-art hardware and processing methods can easily perform such tasks, i.e. creating detailed textual reports of patients. • Decision support system: In addition to disease analysis and diagnosis, health-care units need to make their processes intelligent, optimized and efficient in short amount of time. For example, managing patient flow and

6

Artificial Intelligence and Internet of Things

resource scheduling for them is one of such areas where decision support systems can play their role (Nazir et al., 2019; Norouzi et al., 2014; Saba et al., 2012). • Unmanned operations: There are a number of operations in healthcare where right human expert is not available. AI can fill the gap using intelligent agent’s technology in which using self-learning can work as society to achieve the goal (Iqbal et al., 2017, 2018). • AI for AI: New and innovative algorithms are being developed to increase the efficiency of different medical tasks being solved through AI (Rehman et al., 2018; 2020a,2020b; Saba et al., 2020a,b). This list is not exhaustive in any context. AI-based systems are helping and even guiding the human experts in the whole medical care industry, including practitioners, patients, hospital administrators, drug discovery, medical equipment design and development and related businesses.

1.1.2 Risks and Limitations of AI in Healthcare Like any other technology, AI too is not free of dark sides that must be considered before applying this technology. Here is the non-exhaustive list of risks and limitations associated AI-based systems in medical domain. • Data security and privacy: AI systems require a large amount of data for their training and evaluation. Improper use of data or inappropriate access to such data can put patient life to risk. Each new type of experiment suffers from this factor and must be dealt with care and proper data safety protocols. Although health-care providers try to secure their data, data breaches are increasing. The use of AI can create new vulnerabilities in data management, transmission and sharing (https://portswigger.net/daily-swig/thelatest-healthcare-data-breaches) (Rahim et al., 2013; Hassan et al., 2019). • Biased data: Machine learning (ML) algorithms are trained on data which is sometimes not true representative of the domain from which it is taken. Such type of data creates inherent biasness in the ML methods and even well-designed ML method can create biased results when trained on such type of data (Tahir et al., 2019). • Data interoperability: Dataset formatted for one platform differs from data format used by another platform. This interoperability is addressed by ontology design for data representation (Ullah et al., 2019). • Medical error: This is a real problem caused due to poorly designed AI in medicine and medical practices. The inclusion of AI in medical systems requires specialized programmers who have knowledge of both domains. At present, the availability of such experts is scarce. Perhaps, the introduction of specialized education programs covering both domains can cover the gap (Hussain et al., 2020; Yousaf et al., 2019a,b). • Malicious autonomous agents and biased algorithms: The software components that can learn automatically without human supervision can act in

AI Technologies in Health-care Applications

•

•

• •

•

•

•

7

unexpected ways. They may operate against social norms due to the use of insufficient number of parameters in their design. Such deviation may cause unrecoverable loss; hence constant human monitoring is required (Lung et al., 2014). Performance standards: There is a requirement to formulate well-defined standards to measure the performance quantitatively of AI systems. There exists fear among medical communities to be replaced by AI systems. Reassurance is required that AI systems are there to help, not to replace them (Majid et al., 2020). Slow adaptation: There is low enthusiasm among medical practitioners to adopt the AI systems. This is due to a lack of confidence and knowledge about the potentials of AI systems (https://hbr.org/2019/10/ adopting-ai-in-health-care-will-be-slow-and-difficult). Lack of compulsory ethical standards: Due to a lack of data protection and privacy standards and ethical protocols, people in general are reluctant to allow researchers to use their data for the use and test of AI methods. Rush in AI deployment: The industry has observed a greedy aptitude toward the deployment of AI-based systems, without its rigorous testing and validation, to earn more revenue. This rush is causing failure of systems (Khan et al., 2020a,b). Professional realignment: Although enough training is provided to medical professional to use automated tools and software, the AI is improving at fast pace and its users in medical field are not updated accordingly. There is a great need to keep them updated with recent developments. Patient awareness: It is very required to educate the patients to make them supportive toward their data usage in AI-based expert systems (ESs). Due to lack of awareness, the patients are usually reluctant to allow their data for use in technology. Costly solutions: The price of developing AI-based systems is much higher as compared to standard software. This cost is usually divided into three phases; preparatory phase, prototyping phase and final system development phase. The high price of system may lead to failure of AI system.

When a technology shows success, the policy makers and other stakeholders compare the new option to perfection rather than a limited support. AI in healthcare faces risks and challenges; therefore, it must be used with care and attention.

1.1.3  Applications of AI in Health-care Industry There are numerous applications of intelligent systems in health-care industry. A few of these applications are listed in the following: • Symptom checking and triage: Automatic disease diagnosis, disease prevention, disease prediction, outbreak of diseases and finding the chances of a person to become a patient are major areas where AI is being applied. Timely availability of such information can lead to better disease and relevant resource planning.

8

Artificial Intelligence and Internet of Things

• Robot-assisted surgery: Using existing patient medical records and realtime data, robots can assist and even do the surgery independently. This application is considered to be one of greatest revenue-generated fields. • Virtual nurses: It is the processing of accessing the symptoms of patients’ remotely using medical gadgets like heartbeat trackers and pulse rate detection. This saves patients from unnecessary visits of the hospitals and also saves time of staff. • Treatment plans and medication management: If real-time and history data of a patient is available, it can better help the consultant to plan the treatment for the patient in advance which results in keeping lives safe from severe conditions. • Precision medicine: It is an emerging application area of disease treatment and prevention using individual variability in genes, environment and lifestyle for each patient. • Health monitoring: It is the process of tracking any aspect of patient health by precisely measuring health data and doing its analysis. It also deals with simulations used to make predictions and decisions about the condition of a patient (Saba et al., 2018a,b). • Health-care system analysis: This analysis is based on industrial and human factors and use of engineering tools that contribute toward patient safety and treatment. It helps the clinicians and administrators to access the performance of health-care units.

1.2  TYPE OF DATA AND AI SYSTEMS Data is critical for AI systems because they have to learn the characteristics/patterns from data for precise and accurate results. AI systems have the ability to understand the meaningful relationships in medical data of both structured and unstructured nature and can take optimal decisions using extracted information. Recent AI methods utilize a large volume of medical data to understand the patterns in data and then test data which is not used in training of algorithms. If the performance of trained methods is found to be acceptable, the systems are deployed in real medical applications. Due to advancement in medical care technologies (such as wearable devices) and computer-based data management tools, heterogeneous medical data in large amount is available which is increasing at a fast pace. In the current era, there exist a number of sources from where a large amount of medical data about patients can be generated that is specifically used in the diagnosis procedure. These sources include, but not limited to, physical notes, electronic records, laboratory results and diagnostic images obtained through different imaging modalities (Food and Drug Administration, 2013). Medical data can be grouped into two major categories: structured and unstructured data. Structured data is well organized and can be directly analyzed either manually or using conventional computational methods like databases and data mining. Mostly, this data is available in the form of databases with definite relationships defined by human experts. On the other hand, there is unstructured data that is neither standardized nor directly analyzable due to multiple shortcomings.

AI Technologies in Health-care Applications

9

FIGURE 1.1  Medical data categorization

Noise, incompleteness, impreciseness and inconsistency are a few of such issues. There is a variety of types of unstructured data that include unstructured and random diagnosis and treatment reports produced in textual form, unannotated medical imagery and sound-based data. Normally, AI methods such as natural language processing (NLP) and computer vision apply different preprocessing techniques to transform data into uniform and processable form before applying AI methods. Availability of medical data in multiple languages and its varying representation for medical systems is another variation in data format. There is another type of data that overlaps both structured and unstructured categories. This category has mixed features of both, i.e. it is unstructured data what shows structured organization to a limited extent. We organize the medical data based on its type in Figure 1.1.

1.2.1 Numeric Data Numeric data basically includes patient’s attributes, i.e. gender, age and numerical measurements; vital signs (temperature, pulse and respiration rate etc.); and lab examination results. The data may be used for varying purposes like risk prediction of particular disease, patient disease diagnosis and health-care unit resource allocation. There are multiple ways to collect this data that may include patient-doctor dialogue, laboratory reports and patient disease history. Gadgets such as smartphones, smart watches and trackers are also being used to collect medical data directly from patient body instead of verbal inquiry. For example, heart rate, pulse rate, temperature, blood pressure and sugar level can be recorded in real time using wearable or mobile devices. Usually, this type of data is present in structured form like databases or files; however, there is also abundance of semi-structured and unstructured numerical data.

10

Artificial Intelligence and Internet of Things

1.2.2 Textual Data Textual data consists of electronic records in the form of textual reports, i.e. physical examination outcomes, description of disease symptoms, operative notes, laboratory reports, progress reports and discharge summaries that assist doctors in making different treatment and diagnosis decisions (Afza et al., 2019). Usually, this textual data is present either in semi-structured or unstructured form. At present, most of this data is being produced by medical practitioners and, therefore, there is higher probability to have errors, inconsistencies and duplications in this type of data. Specialized NLP-based methods of ML are required to analyze this data. A recent breakthrough is the data-to-text technology (a sub-domain of NLP) in AI which automatically generates textual descriptions by looking at given data (the data may be in any format, i.e. numbers, images or sound). There are a number of NLP-based ML methods that have been developed to deal with textual data (Yala et al., 2017; Ayesha et al., 2021).

1.2.3  Imaging Data A pictorial view of human internal organs or tissues can produce a lot of information that cannot be collected through other means. Imaging data encompasses screened images obtained from different modalities such as radiology images (computed tomography (CT), magnetic resonance imaging (MRI), X-rays and ultrasound (US), dermoscopy images, fundus and eye screening images, pathology, optical images and many more). Imaging data makes the disease diagnosis process easier and faster (Gillies et al., 2016). Imaging results are either produced in 2D form or in 3D volumetric form. The image capture rate and resolution has been improved a lot than past both in spatial and temporal domains. Patient breathing, motion and other dynamic processes can produce more useful information which is usually recorded in volumetric forms. At present, computer vision is the major application area of AI weather it is medical-related data or not (Rahim et al., 2012;). However, with this improved quality and wide availability of medical data, new and improved computational methods are required.

1.2.4 Genetic Data In recent years, genomics and genetic data have become an area of interest for researchers. The data size is increasing at fast pace and there are many data banks that contain the gene-related data. GenBank of National Center for biotechnology information, DNA Data Bank of Japan from the National Institute of Genetics and EMBL from the European Bioinformatics Institute are a few of such examples. These databases collect genome sequences, annotate and analyze them and also provide public access. Genetic data is used for the diagnosis of several complex diseases such as cancers, Down’s syndrome and infectious disease. Accurately interpreting the genetic data is a key factor to find differences across persons and processing the gene data can accelerate precision medicines (Weir 1990). However, the use of this new type of data has its own challenges.

AI Technologies in Health-care Applications

11

1.2.5 Sound and Analog Data Sound data includes electronic records or signals generated from medical gadgets or devices such as heartbeat in the form of electrocardiogram (ECG) and electroencephalography (EEG) (Istrate et al., 2006). Sound data like other data can be used for various purposes like disease detection and improving health environment. Compared to other data types, sound data has low availability; however, the researchers are working to produce more databases for this type of data. The data is usually categorized as unstructured data.

1.2.6 Synthetic and Surrogate Data Accessing realistic patient medical data is often difficult because of cost, privacy and other medicolegal issues. The data which is similar to real data and generated by an algorithm instead of taking directly from real patients is called the synthetic data. This real-like artificial data can address all medicolegal and other patient-related issues. This can also speed the initiation, testing and system prototype development with the availability of as much data as required by the application. There are a number of software (online and offline) which can generate synthetic data like Generative Adversarial Networks (https://ai.googleblog.com/2020/02/generating-diverse-syntheticmedical.html), Synthea (https://synthetichealth.github.io/synthea/), SynSys and Statice (https://www.statice.ai/ how-statice-works). However, the use of this data has its own complications and limitations (Goncalves et al., 2020).

1.3  AI TECHNOLOGIES USED IN MEDICAL PROCESSES AI is a big umbrella covering various technologies where each technology consists of intelligent approaches designed in different ways. Some of these AI technologies used in medical processes are described in Figure 1.2.

1.3.1 Symbolic AI (S-AI) This AI is based on the approach that intelligence can be achieved by representing intelligence factors as symbols and manipulating them, for example ESs. These, also known as rule-based systems, use a collection of interconnected production rules which connect the symbols (variables and constants) by defining their relationship like if-then-else statements of programming languages. Based on given input, the ES searches and follows a set of rules from given rule base to deduce the results. Using simple and interpretable rules, symbolic AI (S-AI), also known as “good old-fashioned AI (GOFAI)”, can produce human-like basic intelligence. In 1974, an ES named MYCIN was developed at Stanford University for medical diagnosis but it remained as laboratory product. Chatbots are one of the recent medical applications that use S-AI.

1.3.2 Knowledge-based AI (KB-AI) A knowledge-based ES is a set of computer algorithms that use reasoning, heuristics and facts to make decision like a human expert. These AI systems are used to solve

12

Artificial Intelligence and Internet of Things

FIGURE 1.2  AI technologies

complex problems. There are two principal components of a knowledge-based ES. The first one is a knowledge base that is a repository containing domain-specific heuristic and factual knowledge acquired from knowledge engineers and human experts. The second component is an inference engine that acquires and manipulates knowledge base and applies its knowledge to input data for reasoning and to infer new facts. Knowledge-based systems can be categorized into three types; rule based, case based and model based. In rule-based ESs, simple “IF-ELSE” rules or decision trees are used to express the knowledge. In case-based approach, a list of cases is stored in knowledge base in the form of problem and solution pairs. To solve new problems, inference engine searches the knowledge base for similar past successful cases. Finally, in model-based systems, a set of models (biophysical or biochemical) of physical world are stored in the knowledge base. System developed using this approach can be considered as an extension of S-AI. S-AI uses symbols (small data items) and rules to deduce the output, and knowledge-based AI (KB-AI) uses a database consisting of knowledge called knowledge base and an inference engine (a set of rules). The main difference is the size of input and number of rules that are much larger than simple S-AI systems. In medical care applications, ESs are being used for medical image analysis and recommending medication based on patient’s medical data and in the diagnosis of various diseases, including, but not limited to, heart, liver intestinal, infectious and endocrine diseases. These ESs reduce decision-making time and increase diagnosis accuracy. Another application of ESs is in Web-based diagnosis in which ES is implemented as knowledge server. This application is giving economically costeffective and feasible diagnosis to large Internet community globally. Examples of these systems include Quick Medical Reference (QMR) and DXplain. 1.3.2.1  Advantages of S-AI Systems • Accuracy: The systems are accurate in terms of end results based on the rules and knowledge provided to the system. However, the coverage of relevant scope is small.

AI Technologies in Health-care Applications

13

• Cost-efficient: Comparatively, small cost is incurred by these systems. • Speed: The system has to check the rules with little computations; hence faster results could be produced. • Reduced Risk: As the accuracy is higher, the risk is reduced. • Interpretability: It is the process to understand the internal working of system. Interpretability of these systems is highest. 1.3.2.2  Disadvantages and Limitations of S-AI Systems • High level of expertise: S-AI system requires deep knowledge of the domain from the programmers and software engineers. • Complex rule base: For complex systems, the rule base becomes very large where thousands of rules may be included which is very challenging task. Usually, a rule base does not scale and even with the induction of new rules, the rule base becomes a confusing repository with overlapping rules, conflicting rules, incomplete rules etc. • Learning capability: As all the rules are built into the system so the system has just to execute them showing no learning at all. There may be systems that deduce new rules and add them into rule base; such systems have some learning capability. • Strict well-defined scope: Such systems have very strict scope and if a data item is not covered by any rule, it is just ignored. • Fixed level of pattern identification: These systems can only identify the patterns that are given in rule base. For example, if a rule checks for a value of variable to two decimal digits, it would not be able to decide properly about three or more digits.

1.3.3 Fuzzy Logic System (FLS) Another technology of AI is fuzzy logic that is one of the most efficient qualitative computational methods. While rules in knowledge-based ESs are based on Boolean logic – {0, 1} OR {true, false} OR {Yes, No}, in fuzzy logic system (FLS), Yes, No and multiple in-between values (such as certainly yes, possibly yes, certainly no and possibly no) form the fuzzy sets for the rule base. Thus, reasoning of Fuzzy systems deals with uncertain, incomplete, vague and imprecise data (Nodehi et al., 2014; Sadad et al., 2018). Basic components of a FLS include knowledge base in which knowledge and fuzzy rules are stored by experts; fuzzifier that converts input variables into fuzzy sets based on degree of being truth relationship; inference engine that applies rules of knowledge base on the fuzzy input by determining the relationship of input against each rule; defuzzifier that converts fuzzy output of inference engine into variables useful and understandable in real world. Diagnostic decisions of medical practitioners depend on their knowledge, expertise, experience and their ability to deal with uncertainty and vagueness. With increasing complexity of encountering problems, chances of doing mistakes in diagnosis also increase. To handle this situation, FLSs or fuzzy ESs have been proved an effective and helpful method to make clear diagnostic decisions. In medical, FLS generally deals with patients’ monitoring, disease detection, diagnosis procedure and treatment prescriptions.

14

Artificial Intelligence and Internet of Things

1.3.3.1  Advantages of Fuzzy Logic Systems • Input insensitive: These systems can accommodate forgotten rules, vague, noisy or distorted inputs. • Robust: Since these systems do not require precise inputs and are insensitive to changing atmosphere, these are robust systems that have the ability to work on imprecise or erroneous data. • Easy to construct: FLSs are easy to construct because the structure of these systems is simpler. • Easy to understand: These systems are based on simple mathematical concepts and linguistic reasoning which are easy to understand. • Deals with uncertainty: These systems have the ability to deal with uncertainty. • Flexibility: FLSs are flexible and allow contradiction and modifications (addition and deletion) in knowledge base to improve or adjust the system’s performance. • Human-like thinking: As these systems are similar to human’s ability of thinking and making decisions, they can deal with ambiguous inputs and provide efficient solutions to complicated problems. • Less memory consumption: Algorithms of FLS can be built using small amount of data so they do not require large memory space. • Rapid development: Algorithms used in FLS systems can be developed in shorter time than compared to conventional approaches. • Less computing power: They do not involve huge mathematical calculations and consume less computing power because the reasoning development is simpler compared to other computationally accurate systems. • Inexpensive: These systems are cheaper because they use less expensive sensors and hardware. 1.3.3.2  Disadvantages and Limitations of S-AI Systems • Less accurate: Inputs and data given to these systems are mostly inaccurate. They produce less accurate results that are accepted based on assumptions. They are appropriate for only those problems that do not require high accuracy. • Confused outputs: FLS systems are not based on any systematic approach and a problem is solved using fuzzy logic that produces several solutions for a specific problem causing confusion in selecting one. • Fewer acceptances: Since they produce fuzzy results, they are not widely accepted. • Highly dependent on human knowledge: Major limitation of these systems is that their accuracy is fully dependent on human knowledge and expertise. • Demands regular updating: For maintenance, it requires regular updates in knowledge base. • No machine learning: These systems lack the ability of ML and neural networks.

AI Technologies in Health-care Applications

15

• Extensive testing: FLSs require extensive testing for the verification and validation of their knowledge base. • Difficult to form rules: Determining so many fuzzy rules and tuning member functions are difficult and tedious tasks and require human experts.

1.3.4 Machine Learning ML is the broad technology of AI based on statistical methods. It is a combination of mathematical and statistical techniques in which using medical data can learn the patterns present in given dataset. ML is big leap from conventional decision-making methods based on rule-based algorithms (S-AI). Rule-based algorithms are programmed by the programmer, who defines the patterns; however, in ML algorithms, data is provided as an input and patterns are extracted by the machine automatically. The methods are also called nonsymbolic methods (Sharif et al., 2017). As the relationships between different aspects of data are learned automatically, such methods can easily scale to large data. The AI model when learns the relationships is named trained model. More the data used for training, better the generalization of the model on unseen data. A trained model is then used to make predictions on test dataset, the dataset that is not used in training and normally provided in system production environment. At present, almost all of the AI has been shifted to statistical methods (Meethongjan et al., 2013). In medical applications, usually, the data is converted into numerical form even if it is categorical or textual, sound or image. The output generated by these systems is also in the form of numbers which are then converted to human-understandable form. There are two major tasks that are performed by these statistical methods, (1) classification and (2) regression. Successful applications of ML methods include disease prediction, disease stage prediction and patient survival expectancy prediction. Other application areas may include drug discovery and manufacturing, information extraction from medical images, personalized medicines, smart health records, patient behavioral analysis, clinical trials, crowdsource-based medical data collection and pandemic prediction (Al-Ameen et al., 2015; Amin et al., 2018). Figure 1.3 shows the overview of ML methods workflow. Support vector machines (SVM), decision trees and Naïve Bayes (NB) classifiers are a few of the example methods in this domain. Human experts, programming the medical task based on AI, define the types of patterns and relationships to be analyzed which limits the method insight into data. This is the major limitation of these methods (Khan et al., 2020a,b). 1.3.4.1  Advantages of Machine Learning • Automatic rule search: ML methods are versatile and capable enough to automatically extract the rule base from given dataset that is big leap from manual rule base. Such automatically searched rule base can cover larger context and also avoids with confusing rules. Even the rules that are not understandable by human experts are extracted by the algorithms (Rahim et al., 2017a,b).

16

Artificial Intelligence and Internet of Things

FIGURE 1.3  Machine learning process

• Scale of generalization: ML methods can deal with a large amount of data due to automatic search of relationships between the data items. • Self-learning: As the new data is seen by the system, the system automatically extracts new rules and updates itself. This self-learning provides another high edge to ML methods over rule-based systems. • Handling the complex data: The ML-based systems can handle the complex data that is multidimensional and multivariable. • Better accuracy: Although the results produced by ML-AI are not perfect, i.e. 100% accurate, the abovementioned advantages make it superior to the rule-based systems which miss all those aspects that are not covered by rule base. 1.3.4.2  Disadvantages and Limitations of Machine Learning • Manual feature selection: The data items among which relationships are to be found must be decided by human experts. ML methods can not automatically determine the features. • Large size data: ML methods require massive amount of data to work with and acquiring such dataset required a large amount of human effort. • Data dependency: As the rules are automatically extracted from given dataset, the rules are very specific to dataset. It means that ML trained model is very specific to patterns seen during training. If such system is employed to predict on the data which is not from the same “statistical distribution”, the system performance will not be good even not acceptable. • Time and resources: ML-based AI requires a large amount of time to train and a large amount of other storage and computational resources.

AI Technologies in Health-care Applications

17

• Interpretability: Different ML methods have different levels of interpretability; however, in general, the interpretability of ML methods is very low than S-AI. • High error susceptibility: If the dataset is unbalanced or training sample is not a true representative of the domain, results produced by trained model on test dataset will show higher error.

1.3.5  Artificial Neural Networks (ANNs) Artificial neural networks (ANNs) are a more complex type of ML methods and inspired by the manner a human brain works. ANNs are used for modeling nonlinear and complex relationships between input and output features. These also automatically find the weights of features that depict the association between input and output. An ANN consists of a series of artificial neurons or nodes that are arranged in layers, including an input layer, one or two hidden layers and an output layer (Sharif et al., 2017). If there are more than two hidden layers in a neural network, it is referred to as deep ANN (DANN) or just deep neural network (DNN). Each node of a layer transfers data to each node of the next layer through weighted links. Here, the goal is to find optimal weights (parameters) that give minimum error rate between predicted and target output. There are multiple variants of ANNs that are being used in research and development. Examples of these variants are ANNs, convolutional neural networks (CNNs), recurrent neural networks (RNNs) and restricted Boltzmann machines (RBMs) (Saba et al., 2012;). At present, ANNs are being used in many medical applications such as disease detection, medical image analysis, signal analysis and interpretation, biochemical analysis, drug development, and many more and are assisting doctors to model, analyze and understand complex medical data. Currently, disease diagnosis is the most active application of ANNs in medical care industry, for example, ANN’s are being widely used for diagnosing cancer and heart problems. There are two major tasks for which ANNs are used, (1) classification and (2) regression. In classification, input data is assigned a class from predefined set of classes, whereas in regression, based on input data, an output data (usually a number) is predicted. 1.3.5.1  Deep Learning (DL) Deep learning (DL) is the most complex type of ML in which DNN models are used. In functionality, DL is the same as ML (both are statistical models) but they are different in their capabilities. As DL consists of many hidden layers, more complex patterns can be explored in the data using DL methods. DL methods process large volumes of complex data and for this, massive computer resources such as large amount of RAM and specialized devices such as graphical processing units (GPUs) are required. The key difference between DL and ML lies in their learning ability. In ML, designed algorithms learn from their training on the manually provided structured/labeled data, where features are specified by the programmer, and then use their training to give results on unseen data. However, if the output is wrong, ML methods need a human expert to teach them, while in DL, it is not necessary for data to be structured or restructured. DL methods go a step ahead from ML methods and can determine the

18

Artificial Intelligence and Internet of Things

features among which relationship is to be found. A DL method can train itself to extract features from provided data of any type. They can learn from their own errors and make intelligent results themselves using an optimization algorithm. As DL can extract patterns from high-dimensional data, it is being widely used in many medical care applications, including biomedicine, health informatics, diagnosis, image analysis, segmentation, classification, prediction and detection of various diseases. 1.3.5.2  Types of Deep Learning 1.3.5.2.1  Supervised Learning It is a type of learning in which algorithm is trained over learning the mapping from labeled inputs to desired output. During training, algorithm predicts the output of labeled input. It compares its prediction with actual output to find the error and modify its mapping accordingly. The goal is to find the optimal mapping function, which can predict the output against unlabeled new data. Supervised learning is the major part of DL which performs two tasks mainly: classification and regression. A classification algorithm identifies the class of given input from a set of already defined classes. For classification, the prediction of the system is nominal, categorical, symbolic or discrete. A regression algorithm produces numeric output that is usually continuous by looking at given values. Problems like the prediction of tomorrow’s temperature or prediction of stock price in next week can be solved using regression. Most of the algorithms being designed for medical domain lie in this area. 1.3.5.2.2  Unsupervised Learning In this type of learning, algorithm is trained over unlabeled data to find the commonality or difference among given input data. As there is no target data provided during training, the goal of algorithm is to learn or discover the representation or structure of given data by its own. Unsupervised learning is further categorized into two types: clustering and association finding. In clustering, a set of data points is grouped in such a way that points in the same group (called a cluster) are more similar to other data points in same groups (clusters). The task of Association attempts to find the linkage among different data items. For example, if you have seen a video on YouTube, what type of video you will like to watch and based on this observation, the recommender system provides with list of videos that may interest you. This is comparatively hard problem to solve and an active research is being done to find better algorithms to find medical solutions using raw (unannotated) medical data. 1.3.5.2.3  Semi-Supervised Learning This learning method uses both labeled and unlabeled data and hence falls between supervised and unsupervised learning. Usually, a small portion of data is labeled and a large portion is unlabeled. This learning method attempts to find results with minimal use of guidance. Just like supervised learning, researchers are proposing methods to solve different health-care problems. 1.3.5.2.4  Reinforcement Learning In reinforcement learning, the algorithm performs actions to interact with environment and discovers errors or rewards. Trial and error search and delayed rewards

AI Technologies in Health-care Applications

19

are some of the common features of reinforcement learning. This method allows machines and software agents to automatically determine the ideal behavior within a specific context in order to maximize its performance. Medical robots are major users of such algorithms. 1.3.5.2.5  Transfer Learning Transfer learning is the process of transferring knowledge from a source domain having a large scale of labeled data to a different but related target domain having small scale of labeled data. These algorithms use trained DL models to solve the similar problem, the trained model could be any, supervised, unsupervised or semi-supervised. However, at present, most of the transfer learning is obtained using already supervised-learning-based trained models. 1.3.5.3  Advantages of Deep Learning In this discussion, we assume that there are two distinct classes of statistical methods: (1) ML methods like SVM, NB and (2) DL methods like neural networks. Most of the features of DL are similar to ML because it is the branch of ML. However, this domain has added advantages: • Automatic rule and feature search: Big leap of AI is the formulation of methods that can automatically find the correct data items from given dataset and their relationships. This mechanism gives a shape of black box to DL methods where internal working of the algorithm remains unexplainable to major extent. It utilizes unstructured data to maximum extent. • Scale of generalization: ML methods can deal with a large amount of data; however, DL methods can scale to larger extent than conventional ML methods. In other words, these are more scalable than ML methods. • Enhanced self-learning: DL methods can learn deeper than other statistical methods. These can find the discrete as well as continuous features from the data. Nonlinearity is intrinsic component of learning which gives more power to learning algorithms. • Handling the complex data: Higher level of complexity of data can easily be handled in DL methods. It is because we just input the data and rest of the task is done by the algorithm to find the relevant output. • Higher accuracy: The results produced by DL methods show higher accuracy than ML methods • Support for advanced analysis: The ML methods only deal with labeled data whereas DL can support more learning mechanisms like unsupervised and semi-supervised. 1.3.5.4  Disadvantages and Limitations of Deep Learning • Large size data: DL methods require huge amount of data to work with and acquiring such dataset required a large amount of human effort. • Data dependency: Like ML methods, DL methods also rely upon the dataset use for training and suffer from the same problems.

20

Artificial Intelligence and Internet of Things

• Time and resources: DL methods require more time and computational power than ML methods. Depending upon the size of data and size of neural networks, the training may require weeks. Although CPU-based systems can work for DL methods, most of the models can only be trained using special devices like GPUs. • Interpretability: DL models are difficult to impossible to interpret. A DL model is incapable of providing arguments why it has reached a particular result. As the size of network increases, the way to understand becomes more and more difficult. • Error susceptibility: DL methods show low error susceptibility than ML methods and can deal with unbalanced datasets. However, the error probability is higher if dataset for testing is not taken from the same data distribution from where training data is taken. • Model overfitting: Neural networks usually deal with millions of parameters resulting in more noise and as a result, it can easily overfit the data used in training. This mechanism leads neural networks toward lower performance.

1.3.6  Intelligent Multi-Agent Systems (IA) Intelligent agents are the software components that operate autonomously in the environment, in which they are deployed, get perception from environment and perform actions to achieve their design goals. We can consider them as basis for robotics and hence reinforcement learning is the right AI domain for them. It is to note that IA systems could be either pure software parts or a combination of software and hardware. There are numerous applications of intelligent agents in healthcare. A comprehensive review is given in Iqbal et al. (2016). As software systems, these could be used in ESs, self-care systems, resource management system and data mining systems. Similarly, there are multiple roles like assisting in surgeries, dispensing medication, disinfecting rooms and making routine medical procedures less costly for patients. Multi-agent systems can easily model the real-world medical problems in cyberworld. Multiple intelligent agents can also form the society of autonomous agents that through collaboration can find the solution to complex problems and can solve the problem in distributive way. 1.3.6.1  Advantages of Intelligent Agents • Intelligence: An IA bases its decision on the intelligence mechanism provided to it. These can collect the relevant data, enhance its experience and improve its intelligence. The intelligence engine could be rule based, fuzzy, ML or even DL. However, the use of DL is still finding its ways in IA (Nodehi et al., 2014). • Goal oriented and adaptability: Most of the time, the design goals are built-in intelligent agent; however, an intelligent agent has the capability to accept the goal statement from the user and attempt to reach those goals (Ahmad, et al., 2014). • Mobility: An IA can move from machine to machine to achieve its designated goals. • Independence: An IA can operate independently. However, it has the capability to form the society of agents collaborating with each other.

AI Technologies in Health-care Applications

21

• Multiple tasks: An intelligent agent can perform multiple tasks at the same time which makes it versatile for varying applications. • Homogeneous versus heterogeneous: If agents are working in a society, the agents with similar goals can adapt with the passage of time and become heterogeneous where each agent is performing a separate task to meet the collective goal. 1.3.6.2  Disadvantages and Limitations of Intelligent Agents The versatile use of multi-agent systems has posed new challenges, a few of which are mentioned in the following: • Communication protocols: There is a need to define the uniform communication protocols. An agent developed by one company may not be able to talk to an agent developed by other companies. • Data representation: Different agents can represent data differently which becomes bottleneck when communicating with other agents. • Data security: This is another issue faced by agents. A data sent by one agent may be intercepted by unauthorized agent that may use it in a negative way.

1.3.7  Hybrid Intelligence (HA) There are cases where one type of intelligence cannot meet the goals. In those situations, multiple intelligence methods are used in parallel to meet the requirements. It is believed that every natural intelligent system is hybrid because it performs operations on both the symbolic and sub-symbolic levels. Therefore, the use of mixed intelligence methods can produce better results. Ensemble models are proof of this to some extent. Any medical ES inherently employs hybrid intelligence because the use of if-else rules to minimal level even incorporates S-AI (Rehman et al., 2020a,b).

1.4 COMPUTER-AIDED DIAGNOSIS (CAD) USING AI TECHNOLOGIES Computer-aided diagnosis is the process of using different AI technologies (described in the previous section) to detect or classify disease in the form of computer output with improved accuracy. Produced results are used by radiologists as “second opinion” while the final decision is made by human experts. We have discussed a few of human diseases that are being diagnosed by using CAD systems developed using AI algorithms.

1.4.1 Cardiovascular Diseases 1.4.1.1  Diagnosis through Risk Factors (Clinical Data) According to the World Health Organization (WHO), ischemic heart disease is a major cause of mortality worldwide. One of the most prevalent heart diseases is coronary artery disease (CAD) (Acharya et al., 2018a) that is developed due to the collection of plaque in arteries which blocks oxygenated blood to reach heart. For the automated diagnosis of CAD, a hybrid approach is proposed by

22

Artificial Intelligence and Internet of Things

Verma et al. (2016) based upon a proprietary dataset consisting of risk factors of 335 patients. Four supervised ML classifiers, multilayer perceptron (MLP), fuzzy unordered rule induction algorithm (FURIA), multinomial logistic regression (MLR) and C4.5, are trained over filtered risk factors using particle swarm optimization (PSO), correlation-based feature subset (CFS) selection and K-means clustering algorithms, while achieved accuracy of 88.4%, 84.11%, 82.8% and 80.68%, respectively. Similarly in Terrada et al. (2018), a fuzzy ES has been proposed for early diagnosis of heart illness using publicly available heart risk factors dataset (Cleveland heart diseases Database). For the construction of rule base, rules are extracted from decision tree induced using C45 algorithm. Using the same openaccess dataset (Cleveland heart diseases Database), another heart disease diagnosis system based on decision tree and NB supervised classifiers is proposed in Venkatalakshmi et al. (2014), which achieved an accuracy of 85.01% and 84.03%, respectively. A decision support system is presented in Yan et al. (2003) for the diagnosis of five different cardiovascular diseases using clinical data. The system is mainly based upon a three-layer MLP, which is trained over a proprietary dataset of 352 instances and has achieved classification accuracy in the range of 63.6%–82.9% over different diseases. 1.4.1.2  Diagnosis through ECG (Heart Signals) A major test for the inspection of heart activity is ECG, which assists in prognosis of different cardiovascular diseases. Several sophisticated algorithms have been proposed for automated diagnosis of different heart diseases using ECG data, i.e. in Acharya et al. (2017a), a CNN is proposed for CAD diagnosis, while the proposed model is trained over data taken from two different PhysioNet databases (Goldberger et al., 2000), which is first sampled at different sampling rates and then segmented in 2- and 5-second time intervals that are subsequently fed to proposed CNN and achieved an accuracy of 94.95% and 95.11%, respectively. Another CNN has been proposed in Acharya et al. (2017b) for the automated detection of myocardial infarction (MI), which is trained over diagnostic ECG dataset (with noise and without noise) of 200 subjects. From dataset, noise is removed using Daubechies wavelet with six mother wavelet functions (Singh et al., 2006), while signals are segmented using Pan-Tompkins algorithm (Pan et al., 1985) before feeding to the network, and the proposed methodology has achieved an accuracy of 93.53% and 95.22% over segmented data (i.e. with and without noise, respectively). 1.4.1.3  Echocardiogram (Live Heart Video) In addition to automated systems trained over clinical and sound (ECG) data, imaging data extracted from echocardiogram (live heart video) can also be used for several cardiac disease diagnosis such as in Zhang et al. (2018), a fully automated system has been designed for the diagnosis of several heart diseases by utilizing deep CNN for identification of views from echocardiogram, segmentation of five major views to locate heart chambers and for the detection of disease (i.e. cardiac amyloidosis, hypertrophic cardiomyopathy (HCM) and pulmonary arterial hypertension (PAH)).

AI Technologies in Health-care Applications

23

1.4.2 Neurological Disease There are more than 600 neurological diseases, including brain tumor, stroke, Alzheimer’s disease (AD), Parkinson’s disease (PD), epilepsy, multiple sclerosis (MS), dementia, headache disorders (Qureshi et al., 2020; Ramzan et al., 2020a,b). Among which stroke is the second, while Alzheimer and other dementias are fifth major causes of death worldwide (WHO) (https://www.who.int/news-room/factsheets/detail/the-top-10-causes-of-death). A plenty of systems have been designed for computer-aided diagnosis of these chronic diseases, some of them are mentioned in the following sections. 1.4.2.1  Diagnosis through Imaging Data The presence of hyperdense middle cerebral artery (MCA) dot sign is an important early marker of acute ischemic stroke (AIS), which is difficult to interpret correctly by scrutiny of CT-brain images. A DL-based system has been designed in You et al. (2019a) for automated detection and segmentation of MCA using non-contrast CT-brain images dataset of 150 patients. Firstly, hemisphere regions (ROIs) are extracted and preprocessed to improve contrast of extracted regions, while the resultant patches are fed to propose fully convolutional network (FCN) architecture for segmentation that achieved dice similarity coefficient of 0.686. Another major factor that plays an important role in diagnosis of AIS is large vessel occlusion (LVO), in which a three-level hierarchal architecture has been proposed in You et al. (2019b); at level-1, model has utilized some basic features (i.e. age, gender, weakness), at level-2, clinical features (i.e. diabetes mellitus (DM), hypertension, smoking) and at level-3, image features extracted from segmented MCA dot sign CT image are utilized, where segmentation of CT images is done using the same methodology as in You et al. (2019a). T-test is applied on extracted features and resultant features are then fed to extreme gradient boosting model (XGBoost) for classification. Stroke lesion or tissue area damaged by stroke mainly divided into two parts, i.e. infarct core (area that cannot be recovered) and penumbra (that still can be recovered). Correct localization of these areas is of great importance for further treatment. Therefore, Liu et al. (2018) proposed a residual-structured FCN (Res-FCN) for automated semantic segmentation of stroke lesion while utilizing 212 multispectral MRI images (i.e. apparent diffusion coefficient (ADC), diffusion-weighted image and T2-weighted (T2w) image) extracted from a proprietary dataset. For features extraction, pretrained ResNet-50 is used, which is followed by Res-blocks, global convolutional network (GCN) blocks and boundary refinement (BR) blocks for the refinement of extracted features. The proposed segmentation methodology achieved a mean dice coefficient of 0.645. In their work, Lee et al. (2015) presented an automated approach for the diagnosis of different stages of AD (i.e. mild cognitive impairment (MCI), progressive MCI (pMCI), stable MCI (sMCI)) using publically available ADNI data (http://adni.loni.usc. edu/data-samples/access-data/) consisting of MRI and PET brain images. The main focus of this study is the extraction of most effective deep features and fusion of features extracted from different modalities, for which complementary features from concatenated original low-level features and features extracted from stacked auto-encoder (SAE) model are used for classification using multi-kernel SVM. Meanwhile, deep

24

Artificial Intelligence and Internet of Things

Boltzmann machine has also been used for features extraction by fusing information of both modalities, while resultant features are fed to linear SVM for classification. Another automated system for early diagnosis of five different stages of AD has been proposed in Islam et al. (2018), which is based upon an ensemble of three deep CNNs trained over a publicly available brain MRI dataset (i.e. OASIS dataset). Before feeding the data to the network 3D-MRI, data is converted to 2D by extracting patches from three different MRI views and data augmentation technique is applied, where the proposed model has achieved a final accuracy of 93.18%. 1.4.2.2  Diagnosis through EEG (Brain Signals) A major chronic brain disorder is epilepsy, in which a person experiences recurrent seizures (short occurrences of involuntary movement) (WHO). A common test for its prediction is EEG; however, correct interpretation of EEG signals is time-consuming and requires high-level expertise. For the automated diagnosis of epilepsy (i.e. normal, interictal and ictal), an ensemble of K-nearest neighbor (KNN), nonlinear SVM and NB has been proposed in Abdulhay et al. (2017), which are trained over wavelet-based entropy, higher order spectra and nonlinear features, respectively, and extracted from 500 EEG signals dataset provided by Bonn University, while the final results are predicted using a meta-learning classifier, Stacking and Correspondence Analysis and Nearest Neighbor (SCANN) that achieved 98.5% of accuracy. In his study, Acharya et al. (2018b) proposed a 13-layer deep CNN for the automated diagnosis of seizure (i.e. normal, preictal and seizure) that is trained over EEG signals dataset provided by Bonn University and has achieved an accuracy of 88.67%. In addition to the above-discussed diseases, considerable amount of work has been done for brain tumor detection from MRI images (Iqbal et al., 2019; Amin et al., 2019a,b,c,d) and diagnosis of other neurological diseases using AI-based technologies such as computer-aided diagnosis of MS lesions using brain MRI images, PD using dopamine transporter imaging and brain tumor.

1.4.3 Cancer Cancer is one of the major health issues worldwide. Globally, about one in six deaths are due to cancer, which makes it the second leading cause of death throughout the world (WHO). It is originated due to abnormal growth of cells in different body organs. An abundance of AI-based systems has been designed for the automated prognosis of different categories of cancer; some of them are mentioned in the following sections. 1.4.3.1  Lungs Cancer Among other kinds of cancer, lung’s cancer is the most ubiquitous and major cause of death around the globe (WHO) (https://www.who.int/news-room/fact-sheets/detail/ the-top-10-causes-of-death). Its proper diagnosis primarily depends upon early detection of pulmonary nodules (PN) (abnormal growth in lungs) using low-dose CT scans (Saba, 2019). A DL-based framework for automated detection of PN has been proposed in Dou et al. (2017). The author has used 3D-FCN for candidate PN extraction that is trained over hard samples extracted using an online sample filtering scheme. FCN results annotated candidate regions together with probability

AI Technologies in Health-care Applications

25

prediction, which is subsequently fed to the proposed hybrid-loss residual network for false-positive reduction. The proposed architecture is trained over LUNA16 dataset containing 888 low-dose CT scans. Another state-of-the-art architecture for LN segmentation has been proposed in Huang et al. (2019) in which faster RCNN is employed for candidate regions extraction, where pretrained VGG-16 model is used as a features extractor. At the second stage, detection results belonging to same nodule are merged and subsequently fed to propose CNN (i.e. three-convolution and three-pooling layers) for false-positive reduction, while at the last stage, a modified de-CNN with pretrained VGG-16 as a backbone is exploited for pixel-wise segmentation of nodules. The model is trained over LUNA16 CT scans dataset and has achieved a final dice coefficient of 0.793. 1.4.3.2  Breast Cancer After lung cancer, the second most widespread type of cancer is breast cancer (WHO) (https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death). The detection of this catastrophic disease at an early stage is crucial to improve survival rate. In a recent study, Abdar et al. (2020) has proposed a two-layer nested ensemble approach for the classification of breast tumors (i.e. benign or malign) by training the proposed model over Wisconsin dataset (https://archive.ics.uci.edu/ml/machinelearning-databases/breast-cancer-wisconsin/) consisting of breast tumor features of 569 subjects. The proposed architecture consists of four ensembles arranged in two layers and consists of six base classifiers, including NB, Bayesian network (BN), J48, stochastic gradient descent (SGD), logistic model trees (LMT) and reduced error pruning tree (REPTree), which achieved accuracy of 98.07% (Mughal et al., 2017, 2018a,b). A plenty of researchers have done remarkable work for the automated segmentation and classification of breast lesions and also on metastases detection using medical images of different modalities (i.e. X-ray, US, MRI, CT scan and histopathology) (Tariq et al., 2020). In a recent study, Ahmed et al. (2020), a DL-based model is presented to assists clinicians in segmentation as well as classification of breast abnormalities in mammograms. Principally, the proposed methodology is based upon two deep architectures, including DeepLabv3 and Mask RCNN with Xception65 and ResNet-104, respectively, as backbone pretrained networks used for features extraction. For the training and fine-tuning of proposed methodology, two publically available breast images datasets are used, i.e. MIAS and CBIS-DDSM. Model has achieved area under the curve (AUC) of 0.98 and 0.95 on two mentioned architectures. In another study by Jazayeri et al. (2020), DNA methylation gene expression or genomic data (series GSE32393 NCBI dataset) of 137 patients with 27,578 features is utilized for automated diagnosis of breast cancer. To tackle high dimensionality problem, author has employed nonnegative matrix factorization (NMF) and also proposed a new features reduction method, namely “column-splitting”. After dimensionality reduction, resultant features are then fed to extreme learning machine and SVM classifier, while the best-achieved prediction result is at zero error rate. 1.4.3.3  Colorectal Cancer Colorectal or colon cancer is the third most common type of cancer worldwide (WHO). The major cause of its occurrence is anomalous growth of tissue called

26

Artificial Intelligence and Internet of Things

polyp. Usually, colonel CT scan is done as its diagnostic test. However, for the correct interpretation of these CT images, the removal of noise (i.e. lungs, small intestine) is crucial. An automated polyp detection approach has been proposed in Bardhi et al. (2017), in which author has utilized an encoder-decoder architecture, namely SegNet, and trained it over three publically available datasets, including ETISLaribPolypDB, CVC-ClinicDB and CVCColonDB, and achieved accuracies of 0.967, 0.951 and 0.937, respectively. A hierarchal approach has been presented in Bernal et al. (2012), for automated polyp detection and for the diagnosis of colon cancer. The proposed methodology is evaluated over TCIA dataset of 825 CT colonography images. At first, air-filled colonel region is extracted using fixed thresholding, from which patches are extracted and subsequently categorized into three categories (i.e. ascending and descending colon, traversal and sigmoidal colon, noise). Extracted colonel ROIs are then fed to proposed CNN for polyp detection. The main limitation in such conventional computer-aided diagnosis (CADe) systems for colorectal cancer is lack of specificity (i.e. large amount of false positives). To tackle this problem, a deep ensemble learning scheme has been proposed in Umehara et al. (2017), using virtual endoluminal images calculated by applying nine transfer functions on polyp candidates that are extracted using a conventional CADe system that involves the following stages: colon segmentation, polyp candidates detection and false-positive reduction using AdaBoost classifier. The proposed ensemble architecture is based over three pretrained models, including GoogLeNet, AlexNet15 and CaffeNet14, which are fine-tuned over transformed endoluminal images, while the final decision is done using random forest as a meta-classifier. 1.4.3.4  Prostate Cancer Another major category of cancer is prostate melanoma, causing significant amount of deaths among men (WHO). For the automated diagnosis of clinically significant prostate cancer, an optimized DL approach has been proposed in Wang et al. (2018), using multiparameter MRI (mp-MRI), including T2w and ADCs prostate images. Specifically, proposed framework incorporates two sub-nets, a tissue deformation network (TDN) and modified pretrained GoogLeNet. TDN is intended for image registration task and for the detection of control points for detection and cropping of prostate region. This sub-net is optimized by calculating three types of losses. Secondly, a dual-path multimodal truncated GoogLeNet is utilized for the probability prediction of extracted patches to be cancerous and for the generation of class response map (segmentation). Similarly, for the localization of prostate cancer, a radiomics-driven architecture has been proposed in Khalvati et al. (2018), using six different mp-MRI modalities (including diffusion-weighted imaging (DWI) for different b-values, ADC, T2w, CHB-DWI, relative-ADC and CDI). At first, stage of proposed methodology, suspicious prostate tissue candidate regions are selected based on statistical texture distinctiveness (i.e. by extracting voxel-based texture features). Secondly, from the selected tumor candidates, features depicting their asymmetry, morphology, size and physiology are extracted, from which the best set of features is selected and fed to SVM for the categorization of cancerous and noncancerous regions. Boundary of the resultant selected candidates is then refined by employing a conditional random field framework based upon ADC features. The proposed methodology achieved an accuracy of 86%.

AI Technologies in Health-care Applications

27

1.4.3.5 Others In addition to abovementioned automated predictive models delineated for most prevalent cancer types, a plethora of work has been done for computer-aided diagnosis of other chronic cancer categories such as skin, stomach or gastric cancers; throat melanoma; thyroid nodules; bone cancer; kidney (renal cell carcinoma); liver lesions diagnosis; blood cancer or leukemia and ovarian cancer (Hussain et al., 2020; Saba et al., 2020a,b).

1.4.4 Ocular Disease The majority of the people are unaware about their ocular disease due to its asymptotic progression and having good visual acuity which may turn to severity if not diagnosed and treated early. Such situations could be prohibited by the regular eye screening tests; however, that would significantly increase the burden over limited clinical resources (Zhang et al., 2014). With the advent of AI, many computer-aided diagnosis systems have been designed to automate the whole task, i.e. for early detection, precise diagnoses of different ocular diseases (i.e. glaucoma, cataract, diabetic retinopathy (DR)). Uncontrolled blood glucose level causes a major ocular disorder known as DR that originates significant visual loss among adults specifically aged between 20 and 74 (Fong et al., 2004). For the automated diagnosis of DR, a DL residual architecture for deep features extraction has been proposed in Gargeya et al. (2017) that is trained over EyePACS publically available color fundus imaging dataset. Afterward, three metadata attributes are appended with extracted deep features and subsequently fed to decision tree classifier for binary classification of these images (i.e. healthy and abnormal) and achieved AUC of 0.97. Cataract is another major cause of visual impairment and a leading cause of vision loss worldwide (Abner et al., 2012). Basically, it is clouding or dullness of eye lens due to clustering of protein on it, which blocks course of light. For computer-aided diagnosis and grading of cataract by means of fundus images, a hybrid methodology is proposed in Zhang et al. (2019). Both high- and low-level features extracted through pretrained ResNet-18 CNN and grey-level co-occurrence matrix (GLCM), respectively, are fused for distinguishing different grades of cataract. Two different SVM classifiers are trained over two categories of extracted features, while the final decision is done by a fully connected meta-classifier, which achieved best accuracy of 94.75%. Furthermore, hybrid architecture has been proposed in Al-Bander et al. (2017) for automated glaucoma diagnosis, which is another major eye disease that causes degeneration of optic nerve fibers (source of visual communication between brain and retina) due to increased intraocular pressure (Nayak et al., 2009). Globally, it is the second leading cause of blindness (i.e. after cataract) (Abner et al., 2012). In proposed methodology, a pretrained deep CNN, namely ALexNet, is employed for deep features extraction from retinal fundus images taken from a publically available dataset known as RIMONE, while the extracted features are fed to SVM for classification of images (i.e. normal and abnormal) and achieved an accuracy of 88.2%. In addition to abovementioned ocular diseases, a plethora of work has been done for the advancements in automated AI-based diagnosis system for the early identification of other eye diseases, i.e. age-related macular degeneration (AMD), pathological myopia (PM), corneal opacity (CO) and trachoma.

28

Artificial Intelligence and Internet of Things

1.4.5 Diabetes Mellitus 1.4.5.1  Diagnosis through ECG (Heart Signals) DM is a persistent metabolic disorder, caused by an excessive rise in blood glucose level, which may lead to other chronic diseases (i.e. damage in eyes, heart, kidney) if not properly treated at early stages (Perveen et al., 2020). According to the 2016 statistics of WHO, it is the seventh most common cause of mortality worldwide. A hybrid DL-based framework has been proposed in Swapna, Soman, and Vinayakumar (2018), for diagnosis of DM by the examination of heart rate variability (HRV) signal acquired from the ECG signals dataset. The proposed hybrid network particularly incorporates 14 layers (i.e. 5 convolutions, 5 max-pooling, 1 LSTM layer consisting of 70 memory units, 1 dropout followed by 1 FC layer) and has achieved the finest accuracy of 95.1% by training over fivefolds. An amendment of the above hybrid architecture has been presented in Swapna, Vinayakumar, and Soman (2018), in which features extracted from hybrid CNN-LSTM are subsequently fed to SVM for classification, which depicted an improvement in performance by 0.06%. 1.4.5.2  Diagnosis through Clinical Data (Risk Factors) A comparative analysis has been done in Kandhasamy et al. (2015), to assess the performance of four different statistical ML classifiers (including J48, KNN, SVM and random forest) for automated diagnosis of DM using clinical data acquired from Pima Indian Diabetes (PID) database. Accuracies of 86.46%, 100%, 77.73% and 100%, respectively, have been achieved over the mentioned classifiers. Another promising approach has been proposed in Santhanam et al. (2015) for automated diagnosis of DM, in which, firstly, K-means clustering algorithm is employed for noise removal (i.e. for removal of outliers). Secondly, genetic algorithm is applied for the extraction of optimal set of features, while the final set of extracted features is then fed to SVM for classification, which attained an average accuracy of 98.79%.

1.4.6 Bacterial and Viral Infections 1.4.6.1  Lower Respiratory Infections Tuberculosis (TB) is the apex infectious disease causing deaths worldwide (WHO). It is caused by bacteria called “Mycobacterium tuberculosis” that affects lung. For the computerized diagnosis of this widespread disease, a memory-efficient 25-layered deep CNN has been proposed in Pasa et al. (2019), that is trained over two publically available TB Chest-X-ray dataset (i.e. NIH TB chest X-ray dataset (Jaeger et al., 2014), Belarus TB Portal dataset – available at http://tuberculosis.by), and attained an AUC of 0.925. Furthermore, saliency maps and gradient class activation maps (gradCAMs) have are been generated for the visualization of TB. In another research (Mithra et al., 2019), an automated approach has been proposed for the detection and counting of bacilli (a type of bacteria) by the inspection of stained microscopic sputum images. A channel area thresholding (CAT) approach has been proposed for the segmentation of bacillus. From segmented images, two types of local intensity features (i.e. location-oriented histogram and speed up robust feature) are extracted

AI Technologies in Health-care Applications

29

and fed to deep belief network (simulate RBM layers) to count the total number of bacilli occurrences for grading of disease severity. The proposed approach has achieved an accuracy of 97.55%. Pneumonia is a type of acute respiratory infection that damages lungs and most probably causes deaths in children (WHO). Different infectious agents, i.e. viruses, bacteria and fungi, could originate these infections. A computer-aided diagnostic approach for pneumonia is proposed in Aziz et al. (2019), that is based upon pulmonary auscultations signals. For the assessment of proposed approach, a lung sound proprietary dataset has been utilized, which is first preprocessed during which signals are decomposed into its base constituents called intrinsic mode functions (IMF) by employing empirical mode decomposition (EMD) method, and subsequently, noisy and redundant IMFs are discarded and resultant signals are reconstructed. From the preprocessed signal, 89 different features (i.e. time domain and Mel-frequency cepstral coefficients (MFCC features)) are extracted and fed to SVM classifier for the categorization of signals into normal and pneumonia, and attained 99.7% of accuracy. Another such diagnostic system has been presented by O’Quinn et al. (2019), in which chest X-ray radiography dataset acquired from the Radiological Society of North America (RSNA) Pneumonia Detection Challenge has been utilized for the fine-tuning of transfer-learning-based pretrained CNN, namely AlexNet for pneumonia detection, which obtained a moderate accuracy of 72% due to the presence of noise and low-quality radiological images in dataset. Coronavirus disease (COVID-19) is an epidemic infectious disease that mainly causes respiratory illness. It could be fetal for older people and for those who are already facing some major medical problems. In a recent study by Apostolopoulos et al. (2020), a chest X-ray-based automated COVID-19 diagnosis system has been designed, in which several transfer-learning-based pretrained CNNs (including VGG19, MobileNetV2, Inception, Xception and Inception ResNetv2) have been evaluated by utilizing three publically available chest radiography datasets (i.e. containing normal, viral and bacterial pneumonia and COVID-19-positive chest X-ray images) that are acquired from Cohen (2020), Kermany et al. (2018) and https://www.kaggle.com/ andrewmvd/convid19-X-rays. Results depict that among all of the mentioned CNNs, MobileNet has performed well and achieved a maximum accuracy of 96.78%. Precise segmentation of infectious lung regions could assist in quantification of this disease, for which a novel DL architecture has been proposed in Chen et al. (2020), that is based upon traditional encoder-decoder U-Net architecture (Ronneberger et al., 2015). In the proposed model, encoder layers of U-Net are replaced by ResNetXt blocks for precise multi-class segmentation and to avoid problems like network degradation and network complexity. Moreover, for the extraction of complex features while decoding, a soft attention method has been employed. The proposed architecture is evaluated over 110 chest CT images collected from Italian Society of Medical and Interventional Radiology (SIRM) (Shan et al., 2020) and obtained an accuracy of 89%. 1.4.6.2  Hepatitis Viral Infection Hepatitis is another major viral disease that originates inflammation of liver and causes a significant number of deaths worldwide (WHO). For the automated

30

Artificial Intelligence and Internet of Things

diagnosis of hepatitis, author in Abdullah et al. (2018) has evaluated the proficiency of three statistical ML classifiers, KNN, SVM and MLP by training them over publically available hepatitis risk factors data repository (https://archive.ics.uci.edu/ml/ datasets/Hepatitis), which attained maximum accuracies of 100%, 97.87% and 97.40%, respectively. By utilizing the same data repository, an ensemble learning architecture has been proposed in Nilashi et al. (2019). In the proposed methodology, for dealing with missing values and correlated attributes of dataset, Nonlinear Iterative Partial Least Squares (NIPALS) technique is employed, which also improves clustering results done using an unsupervised neural network-based technique self-organizing map (SOM). For the extraction of the most significant features from clustered data, decision tree (J48) is used, while the final dataset is fed to the ensemble of adaptive neuro-fuzzy inference system (ANFIS) for the prediction of hepatitis and achieved ROC of 93.06%. In addition to abovementioned disease, a plethora of studies has been done for automating diagnostic task of other infectious diseases, i.e. for the diagnosis of parasitic infection malaria, using blood cells image data (Shen et al., 2016); for the risk assessment of HIV/AIDS by processing clinical notes through NLP (Feller et al., 2018); for the diagnosis of deadly sepsis infection by utilizing electronic health records (EHR) (Lauritsen et al., 2020); and for the classification of different skin infections (Saba et al., 2019a,b,c).

1.4.7 Other Diseases Several admirable AI-based systems that assist medical practitioners in automated indication or diagnosis of different human diseases have been explored earlier. Majority of these diseases are either ranked in the top 10 causes of death worldwide by WHO or originate major disabilities in humans. However, in addition to these, an abundance of research has also been done to automate diagnostic task of other diseases such as in detection of liver disease, kidney illnesses, behavioral disorders, oral disease, orthodontic disease and orthopedic disease.

1.5 SOME OTHER AI TECHNOLOGIES FOR DISEASE DIAGNOSIS AND HEALTHCARE 1.5.1 Virtual Assistants (Chatbots) The increasing demand for health-care services, shortage of medical professionals or physicians and high cost of medical facilities had made it challenging to provide people with the most essential health-care facilities timely. AI-based chatbots or conversational agents could prove useful in overcoming these barriers by dispensing patients with low-cost health services without any time or place anticipation. These interactive agents are a kind of computer programs that serve as a human clone and provide symptom-based diagnosis and also give immediate feedback for questions asked about general health issues, either through voice conversation or in the form of textual messages. They are being utilized in tackling different types of human illnesses such as assisting patients with different neurological disorders,

AI Technologies in Health-care Applications

31

encouraging people to perform healthy activities (i.e. exercise) for their mental and physical fitness, guiding people who are enduring nutritional and metabolic disorders (i.e. diabetes, allergies, obesity), intervening drug-addicted patients, in diagnosis of cardiovascular disorder, aphasia, HIV/AIDS, syphilis etc. by monitoring several bio-signals, including blood pressure, heart rate, physical activities, voice intonations, facial expressions, body temperature and by sentimental analysis of chat-based answer feedback (Pereira et al., 2019).

1.5.2 Disease Monitoring Devices (Wearable) Wearable devices are the personal portable equipment that assist long-term and real-time monitoring of patients, especially those who are suffering from some chronic disease and those who are gone through a surgery. These devices are not only assisting in improvement of patient’s health status, but also playing a significant role in advancement of medical technology by collecting huge amount of medical data related to patient’s health information (Lou et al., 2020). Furthermore, these devices also serve in surveillance of elderly people living alone at home, to know about their medical condition. Typically, their monitoring is based upon several human physiological health parameters and biomarkers such as motion monitoring of different body organs can detect sudden tremors in hands that could be a pre-symptom of AD, diabetes or PD, inspection of body temperature that assists in knowing general body health and its abnormal change could be a symptom of cardiovascular problems, scrutiny of respiratory rate could assist in diagnosing several disease like asthma, chronic obstructive pulmonary disease, anemia and sleep apnea, examination of blood pressure and heart rate for assessing cardiovascular health, electrophysiological signals (i.e. ECG, EEG, EMG) could support in predicting neuromuscular, cardiovascular and brain disease, metabolism monitoring (i.e. sweat, saliva, tears) could help in predicting overall body health status etc. (Lou et al., 2020).

1.5.3 Cloud-based/Web-based Diagnosis and Prediction (Remote Diagnosis) With the increasing cost of medical amenities and rapid occurrences of disease worldwide, it is becoming crucial to transfer health-care facilities from hospitals to person’s centric environment (Verma et al., 2018). Typically, patients are also interested in getting health-care facilities in comfort of their home. Moreover, people living in far rural areas have lack of basic health facilities, which make it more crucial to provide them with essential medical services remotely. Internet of things (IoT) and cloud computing are a promising technological intervention and a powerful platform such that (1) it facilitates patients with remote health monitoring, (2) provides real-time information to medical experts about patient’s health (i.e. by physiological signals, including ECG, blood glucose, blood pressure, blood oxygen and other patient’s activities) that assists them in timely diagnosis of fetal disease and (3) provides virtual unlimited resources and capabilities for large-scale storage of EHRs and their massive processing. A plenty of researches have been done for designing

32

Artificial Intelligence and Internet of Things

such cloud- and IoT-based frameworks for the sake of patient’s monitoring and to diagnose different diseases in them (Verma et al., 2018).

1.5.4 Benefits of AI in Medical Diagnosis AI is dominating health-care industry in numerous ways, i.e. assisting medical experts in regular check-ups, in management of patient’s health records, in automated diagnosis of fetal disease, in robotic surgeries and many more. Some of its beneficial usages in medical diagnosis include: • Assistance of AI in medical care saves a lot of expert doctor’s time and effort that they serve in manual inspection of medical tests outcomes (i.e. radiological images, ECG signals, blood test reports) for detection of different abnormalities to diagnose specific disease. • Early diagnosis of several life-threatening diseases (i.e. cancer) is crucial, to avoid mortality by providing early suitable treatment, where it is more fault prone to diagnose disease at early stage manually (i.e. diagnosing benign breast tumors). However, computer-aided ML- or DL-based systems could assist in precise early-stage diagnosis and staging of disease. • There is high chance of subjectivity and visual misinterpretation while observing complex medical images (i.e. radiological brain or breast images), which can be avoided using DL-based techniques. • Diagnosing and treatment cost could be significantly abridged by using computer-aided diagnosis systems. • Application of statistical ML classifiers had made early risk prediction of disease more advanced, i.e. based on certain risk factors found in a person, it became easier to predict whether a person will be diagnosed by some disease or not. • Based on patient’s clinical attributes and medical history, computer-aided AI-based systems could forecast that what treatment protocol would be best for the patient. • For patients suffering from particular life-threatening disease or those who get recovered from such disease, survival prediction (i.e. how long a patient will survive after getting specific treatment) could be done using automated AI-based techniques. • Using cloud-based health-care systems, patients can be provided by realtime monitoring, expert’s advice and timely diagnosis of fetal disease remotely, regardless of time and location barriers. • AI-based personal assistants could alert patients at early stage of disease by observing different physiological biomarkers. • AI-based robotic assistants could provide timely first-aid in case of emergencies. • Moreover, robots are also being used in medical surgeries and can also be employed in epidemic situations, when doctors are also at high risk of disease. • AI is also being used in preparation of new medicines.

AI Technologies in Health-care Applications

33

1.5.5 Challenges of AI in Medical Diagnosis In addition to a large amount of beneficial AI applications in healthcare, several challenges are also being faced in it, discussed in the following: • Automated AI-based systems need a huge amount of labeled training data for precise learning, which is usually not publically available. • Development of diagnosis systems for rare disease becomes more challenging due to scarcity of data. • Machine diagnosis of critical illness always remains doubt prone. • A major issue is the threat to the privacy of healthcare-related data that sometimes patients do not want to share. • Complicated AI-algorithms are incapable to provide transparency to patients, i.e. if a certain image is diagnosed with cancer and patient wants to know why, then it is difficult to virtually explain typical AI image analysis techniques specifically deep networks. • Biasness or subjectivity of algorithms could lead to incorrect results. • For cloud-based real-time systems, there is a crucial need of high-speed Internet connection that is usually not available in rural areas. • High charges of wearable sensors make remote healthcare costly.

1.6  AI IN HEALTH-CARE RESEARCH AND INDUSTRY Due to the successful implementation of AI in multiple health-care sectors, a number of new products and businesses are emerging. The range of businesses can be viewed as a pyramid in Figure 1.4.

1.6.1 Research Community The basic job of researchers is to create new technology by providing and proving new ideas and research directions. There are a number of research labs in the world investigating one or other aspects of solving health-care problems using computation. There are multiple sites that provide listing of these labs. Research laboratories are being established by academia and industries as well. A simple Internet search on medical research laboratories can generate long lists. All major IT companies are also having their sections conducting research on health and related sectors. Google, Microsoft, Apple, IBM are giants in this field. There are a number of research journals and publications that continuously publish this area. There are a number of sites that can provide the ranked lists of research journals. The examples may include scimagojr.com, IEEE.org, elsvier.com, springer.com and mdpi.com. Most of the researchers are now making their code available for evaluation and enhancement to community using online platforms. GitHub (github.com) is one of such sites where one can find code to start with. Availability of data is highly required for ML and DL methods. There are multiple platforms that are now providing standard medical data for researchers and developers either free of cost or on paid basis. Examples of these platforms include,

34

Artificial Intelligence and Internet of Things

FIGURE 1.4  Business pyramid

but not limited to, IEEE dataport (www.ieee-dataport.org), Kaggle.com, MICCAI. org, datahub.io, data.world and https://www.ncbi.nlm.nih.gov/datasets/.

1.6.2 Technology Providers Sate of the art research in AI is either produced as theoretical concepts or practical applications in the form of algorithms and software codes. Successful ideas are made available to other researchers and developers in the form of software and libraries. The following table provides an overview of such libraries which not exhaustive.

Sl. No. Library/Tool Name Scientific Computing Libraries 1 Numpy

Purpose

Supports numerical and scientific computations 2 Pandas Python data analysis library 3 SciPy Scientific computation library Natural Language Processing Libraries 4 NLTK Natural language processing toolkit Machine Learning Libraries 5 Scikit-Learn Python-based most popular machine learning library 6 Weka Java-based machine learning library by university of Waikato New Zealand

Developed By https://numpy.org/ https://pandas.pydata.org/ https://scipy.org/

https://www.nltk.org/

https://scikit-learn.org/

https://www.cs.waikato. ac.nz/ml/weka/

35

AI Technologies in Health-care Applications Sl. No. Library/Tool Name Data Visualization Libraries 7 Matplotlib Mixed Libraries 8 Matlab 9 TensorFlow

10

PyTorch

11

Keras

12 13

Nvidia CUDA Theano

Purpose

Developed By

Supports the drawing of multiple charts and plots

https://matplotlib.org/

Popular machine learning and deep learning library by Google A popular deep learning library built by Facebook A popular high-level deep learning library that uses various low-level libraries like TensorFlow, CNTK or Theano on the backend. Fast DL training on GPU A well-known scientific computing library for CPU and GPU

https://www.tensorflow. org/ https://pytorch.org/ https://keras.io/

Nvidia.com http://deeplearning.net/ software /theano/

In addition to these tools, there are a number of tools available to perform specialized tasks in medical applications like medical image preprocessing and annotation tools. A big list of these libraries can be found at http://deeplearning.net/ software_links/.

1.6.3 Service Providers There are many companies that are providing services for applying AI in healthcare, for medical image annotation for supervised learning, TrainingData.io, LionBridge AI and ITK-SNAP. Medical Imaging SDK Technology by LEADTOOLS, MedSeg by medseg.ai for medical image segmentation, Biomedisa online biomedical image segmentation Application and WebMD symptoms checker are few examples from a big list. Among giant companies, Google Health (https://health.google/), health and related services by IBM (https://www.ibm.com/watson-health), health-related products and services by Microsoft (https://www.microsoft.com/en-us/industry/health), solutions by Apple (https://www.apple.com/healthcare/) and Samsung Health-care applications are few examples.

1.6.4 End Users An end user is one who finally uses the tool or technology. Although each phase of our pyramid has different end users, we will consider only medical professional here as end users. There are a number of tools developed by researchers, technology

36

Artificial Intelligence and Internet of Things

providers and service providers to assist medical practitioners in their tasks. We list here a few of tasks and their relevant software. Sl. No. 1

Task Pathology

2

Symptoms checker

3

Radiology diagnoses

4

Electronic health records management

Software Pathai A technology that assists pathologists in making rapid and accurate diagnoses for every patient, every time BUOY HEALTH AI-based symptom and cure checker that uses algorithms to diagnose and treat illness ENLITIC A deep-learning-based medical tools to streamline radiology diagnoses MEDITECH A company providing EHR solutions

Provider Company Pathai.com

https://www.buoyhealth. com/

https://www.enlitic.com/

https://ehr.meditech.com/

A detailed list of few projects can also be found at https://builtin.com/ artificial-intelligence/artificial-intelligence-healthcare.

REFERENCES Abdar, M., Zomorodi-Moghadam, M., Zhou, X., Gururajan, R., Tao, X., Barua, P. D., & Gururajan, R. (2020). A new nested ensemble technique for automated diagnosis of breast cancer. Pattern Recognition Letters, 132, 123–131. Abdulhay, E., Elamaran, V., Chandrasekar, M., Balaji, V. S., & Narasimhan, K. (2017). Automated diagnosis of epilepsy from EEG signals using ensemble learning approach. Pattern Recognition Letters, 139, 174–181. Abdullah, H. N., Abd, B. H., & Muhi, S. H. (2018, December). High-resolution systems for automated diagnosis of hepatitis. In 2018 Third Scientific Conference of Electrical Engineering (SCEE) (pp. 39–44). IEEE. doi: 10.1109/SCEE.2018.8684154. Abner, G. H., Lahm, E. A., Islam, J., Mario, S. P., Mary, F., & Siu, D. A. Y. (2012). EFA global monitoring report regional overview-monitoring the education for all goals: subSaharan Africa. Journal of Visual Impairment & Blindness, 1(2), 1–14. Acharya, U. R., Fujita, H., Lih, O. S., Adam, M., Tan, J. H., & Chua, C. K. (2017a). Automated detection of coronary artery disease using different durations of ECG segments with convolutional neural network. Knowledge-Based Systems, 132, 62–71. Acharya, U. R., Fujita, H., Oh, S. L., Hagiwara, Y., Tan, J. H., & Adam, M. (2017b). Application of deep convolutional neural network for automated detection of myocardial infarction using ECG signals. Information Sciences, 415, 190–198. Acharya, U. R., Hagiwara, Y., Koh, J. E. W., Oh, S. L., Tan, J. H., Adam, M., & San Tan, R. (2018a). Entropies for automated detection of coronary artery disease using ECG signals: a review. Biocybernetics and Biomedical Engineering, 38(2), 373–384.

AI Technologies in Health-care Applications

37

Acharya, U. R., Oh, S. L., Hagiwara, Y., Tan, J. H., & Adeli, H. (2018b). Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals. Computers in Biology and Medicine, 100, 270–278. Afza, F., Khan, M.A. Sharif, M., & Rehman, A. (2019). Microscopic skin laceration segmentation and classification: a framework of statistical normal distribution and optimal feature selection. Microscopy Research and Technique, https://doi.org/10.1002/jemt.23301 Ahmad, A. M., Sulong, G., Rehman, A., Alkawaz, M. H., & Saba, T. (2014). Data hiding based on improved exploiting modification direction method and Huffman coding. Journal of Intelligent Systems, 23(4), 451–459, doi:10.1515/jisys-2014-0007. Ahmed, L., Iqbal, M. M., Aldabbas, H., Khalid, S., Saleem, Y., & Saeed, S. (2020). Images data practices for Semantic Segmentation of Breast Cancer using Deep Neural Network. Journal of Ambient Intelligence and Humanized Computing, 1–17. https:// doi.org/10.1007/s12652-020-01680-1 Al-Ameen, Z., Sulong, G., Rehman, A., Al-Dhelaan, A., Saba, T., & Al-Rodhaan, M. (2015). An innovative technique for contrast enhancement of computed tomography images using normalized gamma-corrected contrast-limited adaptive histogram equalization. EURASIP Journal on Advances in Signal Processing, 32, doi:10.1186/ s13634-015-0214-1. Al-Bander, B., Al-Nuaimy, W., Al-Taee, M. A., & Zheng, Y. (2017, March). Automated glaucoma diagnosis using deep learning approach. In 2017 14th International MultiConference on Systems, Signals & Devices (SSD) (pp. 207–210). IEEE. doi: 10.1109/ SSD.2017.8166974. Alkawaz, M. H., Sulong, G., Saba, T., Almazyad, A.S., & Rehman, A. (2017). Concise analysis of current text automation and watermarking approaches. Security and Communication Networks, 9 (18), 6365–6378 Amin, J., Sharif, M., Rehman, A., Raza, M., & Mufti, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy Research and Technique, 81(9), 990–996. Amin, J., Sharif, M., Raza, M., Saba, T., & Rehman, A. (2019a). Brain tumor classification: feature fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. doi: 10.1109/ICCISci.2019.8716449. Amin, J., Sharif, M., Raza, M., Saba, T., & Anjum, M. A. (2019b). Brain tumor detection using statistical and machine learning method. Computer Methods and Programs in Biomedicine, 177, 69–79. Amin, J., Sharif, M., Raza, M., Saba, T., Sial, R., & Shad, S. A. (2019c). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 1–9. https://doi.org/10.1007/s00521-019-04650-7 Amin, J., Sharif, M., Yasmin, M., T Saba, & M Raza (2019d). Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-019-7324-y Apostolopoulos, I. D., & Mpesiana, T. A. (2020). COVID-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine, 43(2), pp.635–640. Ayesha, H., Iqbal, S., Tariq, M., Abrar, M., Sanaullah, M., Abbas, I., ... & Hussain, S. (2021). Automatic medical image interpretation: State of the art and future directions. Pattern Recognition, 107856. Aziz, S., Khan, M. U., Shakeel, M., Mushtaq, Z., & Khan, A. Z. (2019). An automated system towards diagnosis of pneumonia using pulmonary auscultations. In 2019 13th International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS) (pp. 1–7). IEEE. doi:10.1109/MACS48846.2019.9024789. Bardhi, O., Sierra-Sosa, D., Garcia-Zapirain, B., & Elmaghraby, A. (2017). Automatic colon polyp detection using convolutional encoder-decoder model. In 2017 IEEE

38

Artificial Intelligence and Internet of Things

International Symposium on Signal Processing and Information Technology (ISSPIT) (pp. 445–448). IEEE. doi:10.1109/ISSPIT.2017.8388684. Bernal, J., Sánchez, J., & Vilarino, F. (2012). Towards automatic polyp detection with a polyp appearance model. Pattern Recognition, 45(9), 3166–3182. Chen, X., Yao, L., & Zhang, Y. (2020). Residual attention U-net for automated multi-class segmentation of COVID-19 chest CT images. arXiv preprint arXiv:2004.05645. Cohen, J. P. (2020). COVID-19 image data collection. https://github.com/ieee8023/ covid-chestxray-dataset Dou, Q., Chen, H., Jin, Y., Lin, H., Qin, J., & Heng, P. A. (2017, September). Automated pulmonary nodule detection via 3d convnets with online sample filtering and hybridloss residual learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 630–638). Springer, Cham. Ejaz, K., Rahim, M. S. M., Rehman, A., Chaudhry, H., Saba, T., Ejaz, A., & Ej, C. F. (2018). segmentation method for pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), 394–401. Fahad, H. M., Khan, M. U. G, Saba, T, Rehman, A., & Iqbal, S. (2018). Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microscopy Research and Technique 81(5):449–457, doi: 10.1002/jemt.22998. Feller, D. J., Zucker, J., Yin, M. T., Gordon, P., & Elhadad, N. (2018). Using clinical notes and natural language processing for automated HIV risk assessment. Journal of Acquired Immune Deficiency Syndromes (1999), 77(2), 160. Fong, D. S., Aiello, L., Gardner, T. W., King, G. L., Blankenship, G., Cavallerano, J. D., …, & Klein, R. (2004). Retinopathy in diabetes. Diabetes Care, 27(Suppl 1), s84–s87. Food and Drug Administration. Electronic Source Data in Clinical Investigations (Guidance for Industry). (2013). http://www.fda.gov/downloads/Drugs/ GuidanceComplianceRegulatoryInformation/Guidances/UCM328691.pdf [accessed 2016-12-05] [WebCite Cache ID 6mXWgJsut]. Gargeya, R., & Leng, T. (2017). Automated identification of diabetic retinopathy using deep learning. Ophthalmology, 124(7), 962–969. Gillies, R. J., Kinahan, P. E., & Hricak, H. (2016). Radiomics: images are more than pictures, they are data. Radiology, 278(2), 563–577. Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., … & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation, 101(23), e215–e220. Goncalves, A., Ray, P., Soper, B., Stevens, J., Coyle, L., & Sales, A. P. (2020). Generation and evaluation of synthetic patient data. BMC Medical Research Methodology, 20, 1–40. Hassan, M. K., El Desouky, A. I., Elghamrawy, S. M., & Sarhan, A. M. (2019). Big data challenges and opportunities in healthcare informatics and smart hospitals. In Security in smart cities: Models, applications, and challenges (pp. 3-26). Springer, Cham. Huang, X., Sun, W., Tseng, T. L. B., Li, C., & Qian, W. (2019). Fast and fully-automated detection and segmentation of pulmonary nodules in thoracic CT scans using deep convolutional neural networks. Computerized Medical Imaging and Graphics, 74, 25–36. Husham, A., Alkawaz, M. H., Saba, T., Rehman, A., & Alghamdi, J. S. (2016). Automated nuclei segmentation of malignant using level sets. Microscopy Research and Technique, 79(10), 993–997, doi:10.1002/jemt.22733. Hussain, N., Khan, M. A., Sharif, M., Khan, S. A., Albesher, A. A., Saba, T., & Armaghan, A. (2020). A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications, https:// doi. org/10.1007/s11042-020-08852-3. Iftikhar, S. Fatima, K. Rehman, A. Almazyad, A.S., & Saba, T. (2017). An evolution based hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research, 28(8), 3451–3455.

AI Technologies in Health-care Applications

39

Iqbal, S., Altaf, W., Aslam, M., Mahmood, W., & Khan, M. U. G. (2016). Application of intelligent agents in health-care. Artificial Intelligence Review, 46(1), 83–112. Iqbal, S., Khan, M. U. G., Saba, T., Rehman, A. (2017). Computer assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, 8(1), 5–28, doi:10.1007/s13534-017-0050-3. Iqbal, S., Ghani, M. U., Saba, T., & Rehman, A. (2018). Brain tumor segmentation in multispectral MRI using convolutional neural networks (CNN). Microscopy Research and Technique, 81(4), 419–427, doi: 10.1002/jemt.22994. Iqbal, S., Khan, M. U. G., Saba, T., Mehmood, Z., Javaid, N., Rehman, A., & Abbasi, R. (2019). Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microscopy Research and Technique, https://doi.org/10.1002/jemt.23281 Islam, J., & Zhang, Y. (2018). Brain MRI analysis for Alzheimer’s disease diagnosis using an ensemble system of deep convolutional neural networks. Brain Informatics, 5(2), 2. Istrate, D., Castelli, E., Vacher, M., Besacier, L., & Serignat, J. F. (2006). Information extraction from sound for medical telemonitoring. IEEE Transactions on Information Technology in Biomedicine, 10(2), 264–274. Jaeger, S., Candemir, S., Antani, S., Wáng, Y. X. J., Lu, P. X., & Thoma, G. (2014). Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quantitative Imaging in Medicine and Surgery, 4(6), 475. Jamal, A, Hazim Alkawaz, M, Rehman, A, & Saba, T. (2017) Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17),799–811. https://doi. org/10.1002/jemt. Javed, R., Rahim, M. S. M., & Saba, T. (2019a) An improved framework by mapping salient features for skin lesion detection and classification using the optimized hybrid features. International Journal of Advanced Trends in Computer Science and Engineering, 8(1), 95–101. Javed, R., Rahim, M. S. M., Saba, T., & Rashid, M. (2019b). Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), 1–10. Javed, R., Rahim, MSM, Saba, T., & Rehman, A. (2020a). A comparative study of features selection for skin lesion detection from dermoscopic images. Network Modeling Analysis in Health Informatics and Bioinformatics, 9(1), 4. Javed, R., Saba, T., Shafry, M., & Rahim, M. (2020b). An intelligent saliency segmentation technique and classification of low contrast skin lesion dermoscopic images based on histogram decision. In 2019 12th International Conference on Developments in eSystems Engineering (DeSE) (pp. 164–169). doi:10.1109/DeSE.2019.00039. Jazayeri, N., & Sajedi, H. (2020). Breast cancer diagnosis based on genomic data and extreme learning machine. SN Applied Sciences, 2(1), 3. Kandhasamy, J. P., & Balamurali, S. (2015). Performance analysis of classifier models to predict diabetes mellitus. Procedia Computer Science, 47, 45–51. Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C., Liang, H., Baxter, S. L., …, & Dong, J. (2018). Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 172(5), 1122–1131. Khalvati, F., Zhang, J., Chung, A. G., Shafiee, M. J., Wong, A., & Haider, M. A. (2018). MPCaD: a multi-scale radiomics-driven framework for automated prostate cancer localization and detection. BMC Medical Imaging, 18(1), 16. Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., & Rehman, A. (2019a). Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–7). IEEE. doi: 10.1109/ICCISci.2019.8716400. Khan, M. A., Sharif, M. I., Raza, M., Anjum, A., Saba, T., & Shad, S. A. (2019b). Skin lesion segmentation and classification: a unified framework of deep neural network features fusion and selection. Expert Systems, e12497. https://doi.org/10.1111/exsy.12497

40

Artificial Intelligence and Internet of Things

Khan, M.A., Lali, I.U., Rehman, A., Ishaq, M., Sharif, M., Saba, T., Zahoor, S., & Akram, T. (2019c). Brain tumor detection and classification: a framework of marker-based watershed algorithm and multilevel priority features selection. Microscopy Research and Technique, https://doi.org/10.1002/jemt.23238 Khan, M.A., Akram, T., Sharif, M., Saba, T., Javed, K., Lali, I. U., Tanik, U. J., & Rehman, A. (2019d). Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion. Microscopy Research and Technique, http://doi. org/10.1002/jemt.23220 Khan, M. A., Akram, T., Sharif, M., Javed, K., Raza, M., & Saba, T. (2020a). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, 79(25-26), 1–30. Khan, M. A, Ashraf, I., Alhaisoni, M., Damaševičius, R., Scherer, R., Rehman, A., & Bukhari, S. A. C. (2020b). Multimodal brain tumor classification using deep learning and robust feature selection: a machine learning application for radiologists. Diagnostics, 10, 565. Khan, M. W., Sharif, M., Yasmin, M., & Saba, T. (2017). CDR based glaucoma detection using fundus images: a review. International Journal of Applied Pattern Recognition, 4(3), 261–306. Khan, S. A., Nazir, M., Khan, M. A., Saba, T., Javed, K., Rehman, A., … & Awais, M. (2019e). Lungs nodule detection framework from computed tomography images using support vector machine. Microscopy research and technique, 82(8), 1256–1266. Lauritsen, S. M., Kalør, M. E., Kongsgaard, E. L., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020). Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artificial Intelligence in Medicine, 104, 101820. Lee, S. W., Bülthoff, H. H., & Müller, K. R. (Eds.). (2015). Recent Progress in Brain and Cognitive Engineering. Springer, The Netherlands. Liaqat, A., Khan, M. A., Sharif, M., Mittal, M., Saba, T., Manic, K. S., & Al Attar, F. N. H. (2020). Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Current Medical Imaging, 16, 1229–1242. Liu, Z., Cao, C., Ding, S., Liu, Z., Han, T., & Liu, S. (2018). Towards clinical diagnosis: automated stroke lesion segmentation on multi-spectral MR image using convolutional neural network. IEEE Access, 6, 57006–57016. Lou, Z., Wang, L., Jiang, K., Wei, Z., & Shen, G. (2020). Reviews of wearable healthcare systems: materials, devices and system integration. Materials Science and Engineering: R: Reports, 140, 100523. Lung, J. W. J., Salam, M. S. H, Rehman, A., Rahim, M. S. M., & Saba, T. (2014). Fuzzy phoneme classification using multi-speaker vocal tract length normalization. IETE Technical Review, 31(2), 128–136, doi:10.1080/02564602.2014.892669. Majid, A., Khan, M. A., Yasmin, M., Rehman, A., Yousafzai, A., & Tariq, U. (2020). Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microscopy Research and Technique, 83(5), 562–576. Marie-Sainte, S. L., Aburahmah, L., Almohaini, R., & Saba, T. (2019a). Current techniques for diabetes prediction: review and case study. Applied Sciences, 9(21), 4604. Marie-Sainte, S. L., Saba, T., Alsaleh, D., Alotaibi, A., & Bin, M. (2019b). An improved strategy for predicting diagnosis, survivability, and recurrence of breast cancer. Journal of Computational and Theoretical Nanoscience, 16(9), 3705–3711. Meethongjan, K., Dzulkifli, M., Rehman, A., Altameem, A., & Saba, T. (2013). An intelligent fused approach for face recognition. Journal of Intelligent Systems, 22(2), 197–212. Mithra, K. S., & Emmanuel, W. S. (2019). Automated identification of mycobacterium bacillus from sputum images for tuberculosis diagnosis. Signal, Image and Video Processing, 13(8), 1585–1592.

AI Technologies in Health-care Applications

41

Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A., & Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), 1068. Mughal, B., Muhammad, N., Sharif, M., Saba, T., & Rehman, A. (2017). Extraction of breast border and removal of pectoral muscle in wavelet domain. Biomedical Research, 28(11), 5041–5043. Mughal, B., Muhammad, N., Sharif, M., Rehman, A., & Saba, T. (2018a). Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC Cancer, 18(1), 778. Mughal, B., Sharif, M., Muhammad, N., & Saba, T. (2018b). A novel classification scheme to decline the mortality rate among women due to breast tumor. Microscopy Research and Technique, 81(2), 171–180. Nayak, J., Acharya, R., Bhat, P. S., Shetty, N., & Lim, T. C. (2009). Automated diagnosis of glaucoma using digital fundus images. Journal of Medical Systems, 33(5), 337. Nazir, M., Khan, M. A., Saba, T., & Rehman, A. (2019). Brain tumor detection from MRI images using multi-level wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). doi:10.1109/ICCISci.2019.8716413. Nilashi, M., Ahmadi, H., Shahmoradi, L., Ibrahim, O., & Akbari, E. (2019). A predictive method for hepatitis disease diagnosis using ensembles of neuro-fuzzy technique. Journal of Infection and Public Health, 12(1), 13–20. Nodehi, A., Sulong, G., Al-Rodhaan, M., Al-Dhelaan, A., Rehman, A., & Saba, T. (2014). Intelligent fuzzy approach for fast fractal image compression. EURASIP Journal on Advances in Signal Processing, 2014(1), 112, doi:10.1186/1687-6180-2014-112. Norouzi, A., Rahim, M. S. M, Altameem, A., Saba, T., Rada, A. E., Rehman, A., & Uddin, M. (2014). Medical image segmentation methods, algorithms, and applications. IETE Technical Review, 31(3), 199–213, doi:10.1080/02564602.2014.906861. O’Quinn, W., Haddad, R. J., & Moore, D. L. (2019, January). Pneumonia radiograph diagnosis utilizing deep learning network. In 2019 IEEE 2nd International Conference on Electronic Information and Communication Technology (ICEICT) (pp. 763–767). IEEE. doi: 10.1109/ICEICT.2019.8846438. Pan, J., & Tompkins, W. J. (1985). Pan Tomkins 1985 – QRS detection.pdf. IEEE Transactions on Biomedical Engineering, 32(3), 230–236, doi: 10.1109/TBME.1985.325532. Pasa, F., Golkov, V., Pfeiffer, F., Cremers, D., & Pfeiffer, D. (2019). Efficient deep network architectures for fast chest X-ray tuberculosis screening and visualization. Scientific Reports, 9(1), 1–9. Pereira, J., & Díaz, Ó. (2019). Using health chatbots for behavior change: a mapping study. Journal of Medical Systems, 43(5), 135. Perveen, S., Shahbaz, M., Saba, T., Keshavjee, K., Rehman, A., & Guergachi, A. (2020). Handling irregularly sampled longitudinal data and prognostic modeling of diabetes using machine learning technique. IEEE Access, 8, 21875–21885. Qureshi, I., Khan, M. A., Sharif, M., Saba, T., & Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images. International Journal of Intelligent Systems Technologies and Applications, 19(1), 1–16, doi:10.1504/IJISTA.2020.105172. Rahim, M. S. M, Rehman, A., Kumoi, R., Abdullah, N., & Saba, T. (2012). FiLeDI framework for measuring fish length from digital images. International Journal of Physical Sciences, 7(4), 607–618. Rahim, M. S. M, Norouzi, A., Rehman, A., & Saba, T. (2017a). 3D bones segmentation based on CT images visualization. Biomedical Research, 28(8), 3641–3644 Rahim, M. S. M., Rehman, A., Kurniawan, F., & Saba, T. (2017b). Ear biometrics for human classification based on region features mining. Biomedical Research, 28(10), 4660–4664. Rahim, F. A., Ismail, Z., & Samy, G. N. (2013, November). Information privacy concerns in electronic healthcare records: A systematic literature review. In 2013 International

42

Artificial Intelligence and Internet of Things

Conference on Research and Innovation in Information Systems (ICRIIS) (pp. 504–509). IEEE. Ramzan, F., Khan, M. U. G., Rehmat, A., Iqbal, S., Saba, T., Rehman, A., & Mehmood, Z. (2020a). A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), 37. Ramzan, F., Khan, M. U. G., Iqbal, S., Saba, T., & Rehman, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, 103697–103709. Rehman, A. (2020a). An ensemble of neural networks for nonlinear segmentation of overlapped cursive script. International Journal of Computational Vision and Robotics, 10(4), 275–288 Rehman, A., Abbas N., Saba, T., Mahmood, T., & Kolivand, H. (2018). Rouleaux red blood cells splitting in microscopic thin blood smear images via local maxima, circles drawing, and mapping with original RBCs. Microscopic Research and Technique, 81(7):737–744, doi: 10.1002/jemt.23030 Rehman, A., Khan, M. A., Mehmood, Z., Saba, T., Sardaraz, M., & Rashid, M. (2020b). Microscopic melanoma detection and classification: a framework of pixel-based fusion and multilevel features reduction. Microscopy Research and Technique, doi: 10.1002/ jemt.23429. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Springer, Cham. Saba, T. (2019) Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique, 2019, 1–9, https://doi.org/10.1002/jemt.23326 Saba, T. (2020). Recent advancement in cancer detection using machine learning: systematic survey of decades, comparisons and challenges. Journal of Infection and Public Health, https://doi.org/10.1016/j.jiph.2020.06.033 Saba, T., Al-Zahrani, S., & Rehman, A. (2012). Expert system for offline clinical guidelines and treatment. Life Sciences Journal, 9(4), 2639–2658. Saba, T., Rehman, A., Mehmood, Z., Kolivand, H., & Sharif, M. (2018a). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), doi:10.2174/1573405613666170912164546. Saba, T., Bokhari, S. T. F., Sharif, M., Yasmin, M., & Raza, M. (2018b). Fundus image classification methods for the detection of glaucoma: a review. Microscopy Research and Technique, doi: 10.1002/jemt.23094. Saba, T., Khan, M. A., Rehman, A., & Marie-Sainte, S. L. (2019a). Region extraction and classification of skin cancer: a heterogeneous framework of Deep CNN features fusion and reduction. Journal of Medical Systems, 43, 289. https://doi.org/10.1007/ s10916-019-1413-3 Saba, T., Sameh, A., Khan, F., Shad, S. A., & Sharif, M. (2019b). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43(12), 332. Saba, T., Khan, S. U., Islam, N., Abbas, N., Rehman, A., Javaid, N., & Anjum, A. (2019c). Cloud-based decision support system for the detection and classification of malignant cells in breast cancer using breast cytology images. Microscopy Research and Technique, 82(6), 775–785. Saba, T. Mohamed, AS., El-Affendi, M., Amin, J., & Sharif, M (2020a) Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, 221–230 Saba, T., Haseeb, K., Ahmed, I., & Rehman, A. (2020b). Secure and energy-efficient framework using Internet of Medical Things for e-healthcare. Journal of Infection and Public Health, 13, 1567–1575.

AI Technologies in Health-care Applications

43

Sadad, T., Munir, A., Saba, T., & Hussain, A. (2018). Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. Journal of Computational Science, 29, 34–45. Santhanam, T., & Padmavathi, M. S. (2015). Application of K-means and genetic algorithms for dimension reduction by integrating SVM for diabetes diagnosis. Procedia Computer Science, 47, 76–83. Shan, F., Gao, Y., Wang, J., Shi, W., Shi, N., Han, M., … & Shi, Y. (2020). Lung infection quantification of COVID-19 in CT images with deep learning. arXiv preprint arXiv:2003.04655. Sharif, M., Khan, M. A., Akram, T., Javed, M. Y., Saba, T., & Rehman, A. (2017). A framework of human detection and action recognition based on uniform segmentation and combination of Euclidean distance and joint entropy-based features selection. EURASIP Journal on Image and Video Processing, 2017(1), 89, 1–18. Shen, H., Pan, W. D., Dong, Y., & Alim, M. (2016, December). Lossless compression of curated erythrocyte images using deep autoencoders for malaria infection diagnosis. In 2016 Picture Coding Symposium (PCS) (pp. 1–5). IEEE. doi: 10.1109/PCS.2016.7906393. Singh, B. N., & Tiwari, A. K. (2006). Optimal selection of wavelet basis function applied to ECG signal denoising. Digital Signal Processing, 16(3), 275–287. Swapna, G., Vinayakumar, R., & Soman, K. P. (2018). Diabetes detection using deep learning algorithms. ICT express, 4(4), 243–246. Swapna, G., Kp, S., & Vinayakumar, R. (2018). Automated detection of diabetes using CNN and CNN-LSTM network and heart rate signals. Procedia Computer Science, 132, 1253–1262. Tahir, B., Iqbal, S., Khan. M. U. G., Saba, T., Mehmood, Z., Anjum, A., & Mahmood, T. (2019). Feature enhancement framework for brain tumor segmentation and classification. Microscopy Research & Technique. https://doi.org/10.1002/jemt.23224 Tariq, M., Iqbal, S., Ayesha, H., Abbas, I., Ahmad, K. T., & Niazi, M. F. K. (2020). Medical Image based Breast Cancer Diagnosis: State of the Art and Future Directions. Expert Systems with Applications, 114095. Terrada, O., Cherradi, B., Raihani, A., & Bouattane, O. (2018, December). A fuzzy medical diagnostic support system for cardiovascular diseases diagnosis using risk factors. In 2018 International Conference on Electronics, Control, Optimization and Computer Science (ICECOCS) (pp. 1–6). IEEE. doi:10.1109/ICECOCS.2018.8610649. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z., & Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique, https:// doi.org/10.1002/jemt.23178 Umehara, K., Näppi, J. J., Hironaka, T., Regge, D., Ishida, T., & Yoshida, H. (2017, March). Deep ensemble learning of virtual endoluminal views for polyp detection in CT colonography. In Medical Imaging 2017: Computer-Aided Diagnosis (Vol. 10134, p. 101340G). International Society for Optics and Photonics. https://doi.org/10.1117/12.2255606 Venkatalakshmi, B., & Shivsankar, M. V. (2014). Heart disease diagnosis using predictive data mining. International Journal of Innovative Research in Science, Engineering and Technology, 3(3), 1873–1877. Verma, L., Srivastava, S., & Negi, P. C. (2016). A hybrid data mining model to predict coronary artery disease cases using non-invasive clinical data. Journal of Medical Systems, 40(7), 178. Verma, P., & Sood, S. K. (2018). Cloud-centric IoT based disease diagnosis healthcare framework. Journal of Parallel and Distributed Computing, 116, 27–38. Wang, Z., Liu, C., Cheng, D., Wang, L., Yang, X., & Cheng, K. T. (2018). Automated detection of clinically significant prostate cancer in mp-MRI images based on an end-to-end deep neural network. IEEE Transactions on Medical Imaging, 37(5), 1127–1139.

44

Artificial Intelligence and Internet of Things

Weir, B. S. (1990). Genetic Data Analysis. Methods for Discrete Population Genetic Data. Sinauer Associates, Inc. Publishers, Sunderland, MA. Yala, A., Barzilay, R., Salama, L., Griffin, M., Sollender, G., Bardia, A., ... & Hughes, K. S. (2017). Using machine learning to parse breast pathology reports. Breast cancer research and treatment, 161(2), 203–211. Yan, H., Zheng, J., Jiang, Y., Peng, C., & Li, Q. (2003, May). Development of a decision support system for heart disease diagnosis using multilayer perceptron. In Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS’03 (Vol. 5, pp. V-V). IEEE. doi:10.1109/ISCAS.2003.1206411. You, J., Yu, P. L., Tsang, A. C., Tsui, E. L., Woo, P. P., & Leung, G. K. (2019a). Automated segmentation for hyperdense middle cerebral artery sign of acute ischemic stroke on non-contrast CT images. arXiv preprint arXiv:1905.09049. You, J., Yu, P. L., Tsang, A. C., Tsui, E. L., Woo, P. P., & Leung, G. K. (2019b). Automated computer evaluation of acute ischemic stroke and large vessel occlusion. arXiv preprint arXiv:1906.08059. Younus, Z. S., Mohamad, D., Saba, T., Alkawaz, M. H., Rehman, A., Al-Rodhaan, M., & Al-Dhelaan, A. (2015). Content-based image retrieval using PSO and K-means clustering algorithm. Arabian Journal of Geosciences, 8(8), 6211–6224, doi:10.1007/ s12517-014-1584-7. Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A. M., Alharbey, R., & Rashid, M. (2019a). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: a survey. BioMed Research International, 2019, 1–26 Yousaf, K., Mehmood, Z., Awan, I. A., Saba, T., Alharbey, R., Qadah, T., & Alrige, M. A. (2019b). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, 1–23. doi:10.1007/s10729-019-09486-0. Zhang, H., Niu, K., Xiong, Y., Yang, W., He, Z., & Song, H. (2019). Automatic cataract grading methods based on deep learning. Computer Methods and Programs in Biomedicine, 182, 104978. Zhang, J., Gajjala, S., Agrawal, P., Tison, G. H., Hallock, L. A., Beussink-Nelson, L., … & Fleischmann, K. E. (2018). Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation, 138(16), 1623–1635. Zhang, Z., Srivastava, R., Liu, H., Chen, X., Duan, L., Wong, D. W. K., … & Liu, J. (2014). A survey on computer aided diagnosis for ocular diseases. BMC medical informatics and decision making, 14(1), 80.

2

Two-Level Breast Cancer Staging Diagnosis Sahar Bayoumi, Sanaa Ghouzali, Souad Larabi-Marie-Sainte, Hanaa Kamel

CONTENTS 2.1 Introduction..................................................................................................... 45 2.2 Related Works.................................................................................................. 47 2.3 Mammographic Diagnostic Assessment Categories....................................... 49 2.4 Proposed Approach......................................................................................... 51 2.4.1 Overall Framework Description.......................................................... 51 2.4.2 Framework Components Description.................................................. 52 2.4.2.1 ROI Segmentation................................................................. 52 2.4.2.2 Feature Extraction................................................................. 52 2.4.2.3 Classification......................................................................... 53 2.5 Results and Discussion.................................................................................... 54 2.5.1 Mammogram Database....................................................................... 54 2.5.2 Implementation.................................................................................... 55 2.5.3 Experiment 1: First-Level Classification............................................. 56 2.5.4 Experiment 2: Second-Level Classification......................................... 59 2.6 Conclusion....................................................................................................... 61 References................................................................................................................. 62

2.1 INTRODUCTION Breast cancer is among the most prevalent diseases impacting women’s health, with about 2.1 million new infections every year worldwide. The breast cancer rates have been increasing since the last decade in almost every country but, more specifically, in the developed ones. It is considered as a second leading cause of cancer deaths after lung cancer; one out of nine women develops breast cancer in their lifetime (Institute of Medicine (US) and National Research Council (US) Committee on the Early Detection of Breast Cancer, 2001). The estimation of death caused by breast cancer, as measured by the World Health Organization (WHO) in 2018, is approximately 15% of all cancer deaths among women. The causes of breast cancer are still unknown, but there are multiple factors such as lifestyle factors, environmental factors etc. Since the prevention is impossible, early detection of a tumour in the breast is widely believed to save lives by facilitating intervention early in the course of the disease. Early detection and diagnosis can be made through many screening tools that prevent the death rate from increasing 45

46

Artificial Intelligence and Internet of Things

worldwide by identifying the disease in the early stages. X-ray film mammography has been the mainstay screening mechanism for the early detection of breast cancer and is currently recommended every 1–2 years for women over age 40 (Institute of Medicine (US) and National Research Council (US) Committee on the Early Detection of Breast Cancer, 2001). However, to provide accurate and correct detection and diagnosis of the tumour, the same mammogram image needs to be analysed by two different radiologists at different times, which might not be a viable option in practice as it can be costly and very time-consuming. Henceforth, many researchers and scientists were interested in developing automated systems using computing technology to help radiologists make decisions related to the breast cancer detection and diagnosis. Computer technology and continuous technological developments are increasing in the healthcare field to provide high-quality services and assist caregivers in decision-making. In medical imaging and diagnostic radiology, a topic currently receiving extensive research attention is a computer-aided detection/diagnosis (CAD). Many CAD systems have aided doctors in identifying breast cancer at an early stage. The majority of such systems rely on images generated through mammography or other means as input. CAD gives the same consideration to the input of doctors and computers, and computers do not need to perform better than doctors, but only for their performance to complement doctors’ performance (Katzen & Dodelzon, 2018). Indeed, CAD does not improve the diagnostic accuracy of mammography and may result in missed cancers (Lehman et al., 2015). A CAD system can be classified as CADe, which detects and locates abnormalities in breast screening images, or CADx, which determines the level of abnormalities as benign or malignant. More generally, the CAD systems comprise several steps for detecting and diagnosing breast cancer, including image preprocessing or enhancement, region of interest (ROI) segmentation, feature extraction and classification (Jalalian et al., 2013; Ramadan, 2020). Many studies have been conducted for developing approaches for the different phases of the CAD system to improve accuracy and reduce errors (Cheng et al., 2003, 2006). First, image preprocessing and segmentation of ROI play a significant role in isolating areas that can be subject to tumours, which can drastically influence the accuracy of the CAD system. Image preprocessing can be achieved through noise removal and contrast enhancement. Region growing, adaptive thresholding and active contours are well-established techniques for ROI segmentation. Next, the extraction of pertinent features is considered very important in the overall performance of breast cancer detection and diagnosis. Relevant features can be extracted from the ROI to depict valuable information in the mammograms. The widely used feature extraction methods in the literature are grey-level co-occurrence matrix (GLCM), Haralick’s texture features and local binary patterns (LBPs). Moreover, the features can be extracted from spatial, fractal and multi-resolution domains. Finally, the classification of the breast tissues as normal versus abnormal or malignant versus benign is considered challenging to accomplish because of the complexity of the features. Different machine learning approaches have been used in the literature to classify breast tissues in mammograms, including Naive Bayesian (NB), decision tree (DT), K-nearest neighbors (KNN), artificial neural network (ANN) and support

Two-Level Breast Cancer Staging Diagnosis

47

vector machines (SVMs). The reader is referred to Cheng et al. (2003, 2006) for a detailed review. Recently, deep learning has also shown good performance in the detection and diagnosis of breast cancer (Tsochatzidis et al., 2019). The existing research studies are mainly based on classifying breast cancer into normal/abnormal or benign/malignant cases. Although the early diagnosis of the severity stage for the malignant level would save lives, it requires advanced laboratory investigations or experienced pathologists due to the slight mass change in mammograms. However, no studies investigate the severity stage of the malignancy level. This study aims to build a new model based on image processing and machine learning approaches to identify breast cancer in mammogram images to assist pathologists in their decision-making. The ROI classification will comprise two levels; the first one will decide whether the breast tissue is benign or malignant. In a cancerous tumour, the second level of the classification will determine whether the malignancy stage is early or advanced. To validate the proposed approach, the Mammographic Image Analysis Society (MIAS) dataset is used along with two classifiers, SVM and KNN. The remainder of this chapter is as follows. Section 2.2 presents the recent state of the art studies. Section 2.3 addresses the categories of the mammographic diagnostic assessment. Section 2.4 describes the methodology followed in this study. Section 2.5 discusses the experimental results. Finally, Section 2.6 concludes the present research work.

2.2  RELATED WORKS In the literature review, we explored some previous studies concerning the detection and diagnosis of breast cancer in mammograms. The primary objective of these studies was to develop efficient algorithms to enhance accuracy and reduce CAD systems’ errors. However, the main difference between these studies lies in the mammogram-based feature analysis domain, either spatial or multi-resolution. In Eddaoudi et al. (2011), the authors proposed a framework with three main stages; first, the ROI is segmented using pectoral muscle removal and hard density zone detection. Second, the texture analysis of ROI is performed by calculating the Haralick texture features from the generated co-occurrence matrix. Finally, the extracted features are classified into normal versus abnormal, and benign versus malignant is performed by using the SVM classifier. The experimental results obtained from the MIAS database showed a success rate in detecting masses, achieving 95% on average. In Setiawan et al. (2015), the authors proposed a three-phase approach, including image preprocessing, feature extraction and classification. For preprocessing, they cropped the mammogram images into a smaller size to increase processing speed. Then they have extracted features from the mammogram images like shape, colour and texture. Texture features are extracted using Law’s Texture Energy Measure (LAWS). In the final phase, they used two ANN classifiers to classify the extracted feature into (1) normal versus abnormal classes and (2) benign versus malignant classes. They used mammogram images from MIAS dataset to validate their proposed method, which achieved an accuracy rate of 93.90% for normal-abnormal classification and 83.30% for benign-malignant classification.

48

Artificial Intelligence and Internet of Things

Several studies in breast detection and diagnosis have demonstrated that the use of multi-resolution analysis provides an efficient representation for mammograms and improves the results of the diagnosis (e.g. Ferreira & Borges, 2003; Liu et al., 2001). In Beura et al. (2015), the authors proposed a mammogram classification scheme using multi-resolution texture features. First, the manually cropped ROI is transformed into the second level of the wavelet domain, and the co-occurrence matrix of all the detailed coefficients is generated using the GLCM. The t-test and F-test are then investigated to select the relevant texture features extracted from the GLCM matrices. For classification, the relevant features are fed to a backpropagation neural network (BPNN) classifier. The experimental results obtained using MIAS and the Digital Database for Screening Mammography (DDSM) databases showed that the t-test could select relevant features than F-test concerning performance accuracy. Moreover, the accuracy measures, computed concerning normal versus abnormal and benign versus malignant, achieved 98.0% and 94.2% for the MIAS database, and 98.8% and 97.4% for the DDSM database. In Kaushik et al. (2020), the authors proposed two breast cancer classification schemes using LBP and its variants such as local quinary pattern (LQP) and local ternary pattern (LTP) to extract texture descriptors. In the first scheme, the breast tissues’ classification in mammograms into normal-abnormal is performed using the texture descriptors extracted from the ROIs. The second scheme used discrete wavelet transform (DWT) and LBP variants to classify breast tissues into benign/malignant. The SVM classifier is used to validate the accuracy of both schemes. The accuracy rate achieved 100% with the first scheme and 82.19% with the second scheme. In Moayedi et al. (2010), the authors developed an automated three-stage system for mass classification in mammogram images. In the first stage, they have extracted suspicious areas of breast masses. In the second stage, the ROI is first projected into the contourlet domain, and then textural and geometrical features are derived from the contourlet coefficients. Different textural features such as entropy, energy, correlation and inertia are derived from the co-occurrence matrix. For feature selection, genetic algorithms are applied for dimensionality reduction of the generated feature vectors. In the last stage, the classification is conducted using different variants of classifiers from the SVM family, including successive enhancement learning (SEL)-weighted SVM, kernel SVM and support-vector-based fuzzy neural network. The experimental results obtained from the MIAS database demonstrated the high performance of the SEL-weighted SVM to classify breast masses into benign or malignant cases with an accuracy rate achieving 96.6%. Eltoukhy et al. (2010a) proposed an approach of breast cancer diagnosis in mammograms using multiscale decomposition with the curvelet transform. First, the manually cropped ROI is decomposed using the curvelet transform. The ratios of the most significant curvelet coefficients in each scale decomposition level are used to construct the features vector of the mammogram. These extracted feature vectors are used with the Euclidean distance to classify seven types of tumours, which are microcalcification, speculated mass, circumscribed mass, ill-defined mass, architectural distortion, asymmetry and normal tissues. The experimental results obtained from the MIAS database showed, for normal and abnormal classification, an average classification accuracy rate of 98.59%. In Eltoukhy et al. (2010b), the authors followed up their previous study by providing a comparative analysis between a wavelet-based

Two-Level Breast Cancer Staging Diagnosis

49

system and the proposed curvelet-based system. The experimental results obtained using the same steps in both systems suggested the curvelet transform’s effectiveness over the wavelet transform, which achieved a maximum successful classification accuracy rate of 95.07%. In Eltoukhy et al. (2012), the authors used a feature selection method based on the t-test to rank the wavelet (resp. curvelet) coefficient with regards to its capability to differentiate the different classes. A dynamic threshold method is then used to optimize the features’ number, achieving the maximum classification accuracy rate. SVM classifier is then employed to differentiate between normal and abnormal tissues, and between benign and malignant tissues. The experimental results proved the efficacy of the approach in drastically reducing the feature vector’s size while achieving comparable classification accuracy rates. The maximum accuracy values achieved to classify normal/abnormal and benign/malignant are 95.98% and 97.30%, respectively. In Dong et al. (2017), the authors first used dual contourlet (Dual-CT) transform to decompose the ROI in the mammogram image. Nine textural features are then extracted from the directional sub-band coefficients, including mean, smoothness, standard deviation, skewness, uniformity, entropy, contrast, correlation and homogeneity. Finally, an improved KNN is used to classify these extracted features into normal versus abnormal and benign versus malignant. The authors used the MIAS database in their experimentation. They proved the efficiency of their proposed method to enhance breast cancer detection, which achieved the best accuracy rate of 94.14% and 95.76%. This approach showed high performance compared to the state-of-the-art literature. Several studies have recently demonstrated high performance of deep learning in breast cancer detection and diagnosis. The majority of studies proposed the fine-tuning of pre-trained networks instead of training networks from scratch (Tsochatzidis et al., 2019). In Ragab et al. (2019), the authors proposed a CAD system using a deep convolutional neural network (DCNN) to classify benign and malignant mass tumours in breast mammogram images. After segmentation of the ROIs, the features are extracted using a DCNN. The last layer of the DCNN was connected to the SVM classifier to improve the classification results. The experiments were conducted using the DDSM databases after applying a rotation-based data augmentation to increase the input data’s size. The SVM accuracy achieved 87.2% with an AUC (area under ROC curve) equalling 0.94 (94%). In Larabi-Marie-Sainte et al. (2019), the authors predicted the recurrence, survivability and the type of breast cancer disease using the feedforward neural network. Three experiments were performed using the SEER dataset: the first one consists of the classification into benign/malignant, the second involves the classification into survivability/mortality and the last experiment includes the classification of recurrence/curable. The accuracy results achieved 99.8%, 97.3% and 88.1% from the three experiments, respectively.

2.3  MAMMOGRAPHIC DIAGNOSTIC ASSESSMENT CATEGORIES A screening mammogram consists of two X-ray films, taken from the side (referred to as the “mediolateral oblique view”) and from above (referred to as the “craniocaudal view”), for each breast (Kopans, 2007). A diagnostic mammogram, which may include additional views or magnifications, is usually performed following a

50

Artificial Intelligence and Internet of Things

TABLE 2.1 Mammographic Diagnostic Assessment Categories—BIRADS BIRADS Category 0

Assessment Incomplete

1 2 3

Negative, no findings Benign finding Probably benign

4 5

Suspicious abnormality Highly suggestive

Recommendations Other mammographic views and techniques or ultrasound needed Routine screening Routine screening Short-term follow-up to establish stability Biopsy should be considered Appropriate action should be taken of malignancy

suspicious finding on a screening mammogram or when a woman has a new symptom such as a breast lump (Kopans, 2007). Since breast cancer is an abnormal, uncontrolled cell growth arising in the breast tissue, any area that does not look like normal tissue is a possible cause for concern. The radiologist will look for white, high-density tissue areas and note its size, shape and edges (Siu & on behalf of the US Preventive Services Task Force, 2016). Radiologists will also look for microcalcifications (tiny calcium deposits), architectural distortions, asymmetrical densities, masses and densities developed since the previous mammogram (Siu & on behalf of the US Preventive Services Task Force, 2016). To create a uniform system of assessing mammography results, the American College of Radiology developed the Breast Imaging Reporting and Data System (BIRADS) (Fowler et al., 2013). The BIRADS system includes five categories of assessment with increasing suspicion of malignancy and standard follow-up recommendations for each category as shown in Table 2.1. According to the BIRADS system, a mass is characterized by Fowler et al. (2013): • The shape: round, oval, lobulated or irregular; • The contour: circumscribed, microlobulated, masked, indistinct, spiculated; • The density with respect to normal fibroglandular tissue: high, medium, or low density or containing fat; • The association with other anomalies: micro- or macrocalcifications, skin retraction, skin thickening, architectural distortion, etc.; and • The evolution over time when past mammograms are available. The contour is the most discriminating morphological criterion between benign and malignant masses (Berment et al., 2014). • A circumscribed mass in mammography is a mass where the contour is clearly defined with at least 75% of its surface. Circumscribed masses first indicate benign lesions. In mammography, circumscribed masses of

Two-Level Breast Cancer Staging Diagnosis

51

typically benign appearance, placed in category 2 of the BIRADS system (Berment et al., 2014). Probably benign masses, placed in category 3, as they require short-term monitoring (in general after 4 months, then 1 year, then 2 years). In mammography, they are well-circumscribed, round, oval or lobulated masses and are not calcified. A histological sample should be taken in the case of a morphological modification or an increase in the size of a circumscribed mass (Berment et al., 2014). • Non-circumscribed masses (BIRADS 4 or 5): in mammography, the existence of a non-circumscribed contour, whether microlobulate, masked or indistinct, justifies a biopsy for histological examination (Lakhani et al., 2012). • Spiculated masses (BIRADS 5): in mammography, spiculated (or stellar) masses correspond to opacities formed by a dense centre from which arise multiple linear radial prolongations called spicules. The positive predictive value (PPV) of malignancy in mammography of a spiculated mass is very high, about 96% (Lakhani et al., 2012).

2.4  PROPOSED APPROACH 2.4.1 Overall Framework Description This chapter proposes a two-level approach to classify abnormal breast tissues in mammogram images using two classification algorithms. The proposed approach involves three main phases: ROI segmentation, feature extraction and classification, as shown in Figure 2.1. First, the ROI is located and segmented in the mammogram image of the abnormal breast tissue. Features that best depict different abnormality types, such as microcalcifications and masses, are then extracted from the ROI. Before classification, features are extracted from the ROI in the multi-resolution domain using the wavelet transform. In the first level of classification, the extracted features are classified into benign or malignant breast tissue using SVM and KNN

FIGURE 2.1  The proposed framework

52

Artificial Intelligence and Internet of Things

classifiers. The second-level classification aims to specify the level of severity of malignancy, such as early or advanced stage using the aforementioned classifiers.

2.4.2 Framework Components Description 2.4.2.1  ROI Segmentation A critical stage in a CAD system is ROI segmentation, whereby the image is manipulated and prepared for further analysis. The segmentation procedure aims to detect the location of the ROI in an image. In this study, the ROIs are manually cropped on the lesion areas. The centre of ROI is selected with regards to the centre of the abnormal tissue. Figure 2.2 shows an example of an ROI segmentation. 2.4.2.2  Feature Extraction Masses and microcalcification are significant early signs of possible breast cancers. They can help detect breast cancer at an early stage. Since the probability of malignancy lies in the characteristics of masses and calcifications, the diagnosis efficiency relies on extracting corresponding features to these characteristics. In this study, we used the DWT to decompose the ROI into its detailed coefficients using the three sub-band images: vertical, horizontal and diagonal. Furthermore, different sets of features are extracted using GLCM and LBP on detailed sub-images from the first and the second levels of the wavelet decomposition. The sets are used later as an

FIGURE 2.2  Example of an ROI segmentation

Two-Level Breast Cancer Staging Diagnosis

53

FIGURE 2.3  Directions used in calculating the GLCM

input to the classification stage. In the following, the two proposed feature extraction methods are presented and used in the experimental study. 1. Two-dimensional DWT-based GLCM (2D-DWT-based GLCM): the cooccurrence matrix makes it possible to define the occurrence of two pixels (the reference and its neighbour) separated by a distance d at angle θ  from the horizontal, with d = [1, 2] and θ = [0, 45, 90, 135] as shown in Figure 2.3. Texture descriptors are then extracted from the normalized GLCMs using the 14 Haralick texture features (e.g. entropy, contrast, energy, homogeneity and correlation) (Haralick et al., 1973). These texture features are shown to be very useful in the discrimination between benign and malignant lesions (Cheng et al., 2003, 2006; Jalalian et al., 2013). To avoid the dependency of the directions, for each distance along the four angular, a set of four values for each of the 14 Haralick measures are obtained. The mean and range of each of these measures averaged over the four values form a set of 28 features which can be fed to the classifier. 2. 2D-DWT-based LBPs: LBP is a texture-based approach that depends on labelling each pixel based on the grey-level differences between the centre pixel and its neighbours within a specified radius. In our experiment, we extracted the LBP features with radius = [1, 2] and symmetric neighbour for P = 8 as shown in Figure 2.4. 2.4.2.3 Classification In this research study, the supervised learning was adopted to predict breast cancer to either benign/malignant (first level) or early/advanced (second level). Supervised learning consists of mapping an input to an output according to a labelled dataset

54

Artificial Intelligence and Internet of Things

FIGURE 2.4  Pixel x and its neighbours with radius = [1, 2] and symmetric neighbour P = 8

called a training set. The classification is one kind of supervised learning where the dataset is divided into training and testing sets; the training labelled set is utilized to predict the output of the testing set. In order to predict the results of this study, two classifiers were used, the KNN and the SVM. The reason behind this choice is that both classifiers are recommended for small dataset’s size (Tomar & Nagpal, 2016). 1. SVM: it is characterized by using one or multiple hyperplanes that segregate the different classes (Cortes & Vapnik, 1995). The separation is performed by a kernel function that involves transforming the dataset from the original space into a higher dimensional space to determine the classes. The main well-known kernels are linear, radial, polynomial and sigmoid (Demidova et al., 2016; Huang et al., 2017). These kernels are considered in this study, and the best one was kept based on the best accuracy. Another SVM’s parameter that needs investigation is the cost C, which refers to the SVM model’s extent to fit the data. This parameter is set based on the kernel and the best accuracy. 2. KNN: it is the simplest classifier based on the notion of similarity or distance (Altman, 1992). The classification of a case is performed using the majority poll of its “k” neighbors. The best value of k is set based on the best accuracy.

2.5  RESULTS AND DISCUSSION 2.5.1 Mammogram Database In this study, the freely available MIAS database (Suckling et al., 2015) is used to demonstrate the proposed approach’s efficacy. MIAS database is selected from the United Kingdom National Breast Screening Program; it contains

55

Two-Level Breast Cancer Staging Diagnosis

TABLE 2.2 The Description of the Dataset Based on BIRADS Category

BIRADS Category 2 3 4

5

Assessment Benign

Number of mammograms 69

Early malignant

26 Mdb (IDs): 023, 058, 075, 090, 117, 120, 130, 134, 141, 144, 155, 158, 181, 186, 202, 206, 213, 231, 233, 238, 245, 249, 256, 264, 267, 271, 274 Advanced malignant 23

Number of Number of mammograms mammograms with with no defined wrong defined tumour tumour 1 4 2

0

1

2

Mdb (IDs): 028, 072, 092, 095, 102, 105, 110, 111, 115, 124, 125, 148, 170, 171, 178, 179, 184, 209, 211, 216, 239, 241, 253, 265, 270

330 mammograms of right and left breast, from 161 patients, divided into 54 malignant, 69 benign and 207 normal cases. The different samples of this database are labelled by an expert radiologist based on technical experience and biopsy. Moreover, the centre coordinates and radius of abnormality are given by experts and available in the database. Our expert radiologist Dr Hanaa Kamel determined the stage of malignancy, resulting in 26 early and 23 advanced samples as shown in Table 2.2.

2.5.2  Implementation The implementation has been carried out using MATLAB. As explained in our framework, we are only concerned about benign and malignant mammograms from the MIAS dataset. Figure 2.5 shows a sample of benign, early malignant and advanced malignant. The ROI is segmented using the centroid and radius provided with the MIAS dataset. During ROI segmentation, we found four mammogram images with no defined centroid and radius, in addition to six images with a wrong defined tumour, as shown in Table 2.2. The ROIs for the images with wrong defined tumour are shown in Figure 2.6. These images have been removed from the experimentation.

56

Artificial Intelligence and Internet of Things

FIGURE 2.5  Sample from the MIAS database

After performing the ROI segmentation, the 2D-DWT is computed using symmetric biorthogonal 4.4 at three decomposition levels. GLCM and LBP are applied to the three detailed coefficient matrices (H, V, D) to form the feature vector at each level. Feature vectors were then combined from all detailed sub-images for the three decomposition levels (see Figure 2.7).

2.5.3 Experiment 1: First-Level Classification Our first-level classification experiment consists of classifying mammograms into benign and malignant. The combination of the extracted features is used as an input to KNN and SVM. The dataset is split into 70% training and 30% testing. In the training set, ten-fold cross validation is employed for parameter setting. For SVM (resp. KNN), the kernel (Linear, Polynomial, Radial and Sigmoid) and the cost

FIGURE 2.6  MIAS mammograms with wrong defined tumour. (a)–(d) are benign, and (e) and (f) are advanced malignant

Two-Level Breast Cancer Staging Diagnosis

57

FIGURE 2.7  DWT three levels

(resp. the parameter K) are investigated to select the best values based on the best classification accuracy. After training the classification models and finding the best parameters, the classifiers are applied to the testing set to obtain the approach’s accuracy. Table 2.3 shows the classification accuracy after applying both classifiers to five sets of feature vectors. As displayed in Table 2.3, the first four proposed feature sets yielded a high accuracy with both classifiers due to the Haralick descriptors’ ability to emphasize changes in texture within the detailed sub-images. However, integrating the Haralick descriptors over the four directions yielded the best results by avoiding directional information, which decreased the classifier behaviour for feature vector 5. Besides, achieving 100% for both classifiers using the features’ set 1 indicates that changes in texture between benign and malignant were significant within a close neighbour (D = 1).

58

Artificial Intelligence and Internet of Things

TABLE 2.3 Experimentation Results of the First-Level Classification: Benign versus Malignant Number Features Vector SVM (%) 1 100 Integrated 14 GLCM over 4 directions at D = 1 from DWT L1 (D, H, V) 2 96.97 Integrated 14 GLCM over 4 directions at D = 1 and D = 2 from DWT L1 (D, H, V) 3 96.97 Integrated 14 GLCM over 4 directions at D = 1 from DWT L1 (D, H, V) and D = 2 from DWT L2 (D, H, V) 4 Integrated 5 GLCM over 4 directions at D = 1 from DWT L1 (D, H, 96.97 V) and D = 2 from DWT L2 (D, H, V) 5 60.61 5 GLCM over 4 directions at D = 1 from DWT L1 (D, H, V) and D = 2 from DWT L2 (D, H, V)

KNN (%) 100 87.88 100 93.94 57.58

Although both classifiers’ results are outstanding, SVM has shown consistent results in all the proposed features sets. This result confirms the recommendation proposed in the literature about using SVM along with DWT (Eddaoudi et al., 2011; Eltoukhy et al., 2012; Kaushik et al., 2020; Moayedi et al., 2010). The obtained results are compared with some existing studies in the literature to show the proposed approaches’ efficiency. Table 2.4 summarized the feature

TABLE 2.4 Summary of Comparisons with the State-of-the-Art Approaches over the MIAS Database Reference Beura et al. (2015) Moayedi et al. (2010)

Feature Extraction 2D DWT and GLCM Textural and geometrical features derived from the Contourlet coefficients

Eltoukhy et al. (2012) Setiawan et al. (2015)

Curvelet coefficients Shape, colour and Law’s Texture Energy Measure Features extracted from Improved KNN the dual contourlet, including mean, smoothness, std, skewness, uniformity, entropy, contrast, correlation, homogeneity

Dong et al. (2017)

Classification BPNN SEL-weighted SVM Support-vector-based fuzzy neural network Kernel SVM SVM ANN

Accuracy Rate (%) 94.2 96.6 91.5 82.1 97.30 83.30 95.76

Two-Level Breast Cancer Staging Diagnosis

59

extraction approach, the classification algorithm and the classification accuracy of the state-of-the-art studies discussed in the related work section. These studies used the same dataset MIAS to classify breast cancer images into benign/malignant. The results showed that the proposed feature extraction methods (the first four approaches in Table 2.3) outperformed all the studies presented in Table 2.4 using SVM or any other classifier. Besides, only two proposed methods (see feature vectors 1 and 3 in Table 2.3) using KNN outperformed the results presented in Table 2.4. Our proposed features vector GLCM with DWT obtained outstanding accuracy with both classifiers SVM and KNN. Moreover, our approach using SVM outperformed BPNN classifier with 2.77% compared to the work presented in Beura et al. (2015), which used the same feature vector 4 from Table 2.4 and obtained 94.2%.

2.5.4 Experiment 2: Second-Level Classification In this experiment, the severity of the malignant stage is considered. As discussed previously, two classes are generated (early/advanced) from 49 malignant images. The same process addressed earlier is performed. Both proposed feature extraction approaches are applied with different directions, radius and levels. Then, the classification, using KNN and SVM, is performed on the obtained features sets. The dataset is split into training/testing (70:30). The ten-fold cross validation is applied to the training set to determine both classifiers’ parameters, as explained in Section 2.5.3. Table 2.5 figures out the classification accuracy obtained for each feature vector using SVM and KNN. The best-achieved classification accuracy is 76.92% with SVM and 69.23% with KNN, obtained when using the first features set. SVM performs better than KNN with the first three and the ninth feature vectors, while KNN outperforms SVM only with the fifth and sixth feature vectors. Again, the SVM demonstrates its efficacy when applied to DWT. Moreover, it has been shown that the first feature vector (DWT L3(A)) yields the best result with both classifiers, which indicates that changes between malignant stages were positively reflected in the approximation components instead of the textural features. It is also clear that decomposition from the second level to the third level enhanced the classification results with both classifiers as the approximation components are significantly concentrated. It is worth mentioning that feature vector 3, which includes detailed components from first-level DWT with LBP features, has achieved accuracy similar to using the low-frequency component from the second level for both classifiers. In contrast, the worst result (46.15%) is obtained with the fourth feature vector (DWT L3 (D, H, V) and LBP for R = 1), which can be justified by the fact that detailed components did not express significant change for malignant stages like in the approximation components. Most of the results shown in Table 2.5 are promising. However, no comparison could be performed due to a lack of research investigating the severity of breast cancer malignancy stages. To well explain the obtained results, the feature vector providing the best classification accuracy is investigated. Table 2.6 shows the different classification metrics (precision, recall and F measure) for both classifiers. The

60

Artificial Intelligence and Internet of Things

TABLE 2.5 Experimentation Results of the Second-Level Classification: Early versus Advanced Number  1  2  3  4  5  6  7  8  9

10

Features Vector DWT L3 (A) DWT L2 (A) DWT L1 (D, H, V) and LBP for R = 1 DWT L3 (D, H, V) and LBP for R = 1 DWT L1 (D, H, V) and LBP for R = 2 DWT L3 (D, H, V) and LBP for R = 2 DWT L1 (D, H, V) and LBP for R = 1 and R = 2 DWT L3 (D, H, V) and LBP for R = 1 and R = 2 DWT L1 (D, H, V) and 14 GLCM integrated over directions for D = 1 and DWT L2 (D, H, V) and 14 GLCM integrated over directions for D=2 DWT L1 (D, H, V) and 5 GLCM integrated over directions for D = 1 and DWT L2 (D, H, V) and 5 GLCM integrated over directions for D = 2

SVM (%) 76.92 69.23 69.23 46.15 46.15 53.85 53.85

KNN (%) 69.23 46.15 46.15 46.15 53.85 61.54 53.85

61.54

61.54

61.54

53.85

53.85

53.85

precision computes the accuracy of the positive class, which is the advanced class. As seen, all the cases predicted as advanced, by both classifiers, are indeed advanced (precision = 100%). Recall expresses the capability of the proposed model to classify all the appropriate cases to the associated classes. SVM succeeded in predicting 50% of the actual advanced cases as advanced, contrary to KNN, which predicted only 33% of the actual advanced cases to the advanced class. F measure is considered as a

TABLE 2.6 Classification Measures Using KNN and SVM for the Best Feature Vector (DWT L3(A)) Measures/Classifiers Accuracy Precision Recall F measure

SVM (%) 76.92 100 50 66.66

KNN (%) 69.23 100 33.33 49.62

Two-Level Breast Cancer Staging Diagnosis

61

sort of an average between the precision and the recall. This average reached 67% in SVM and 50% in KNN. Hence, it is clear that SVM outperforms KNN in classifying the malignant stage. The proposed approach successfully predicted the advanced/ early stages for the breast cancer malignant cases.

2.6 CONCLUSION The diagnosis of breast cancer is mainly based on the mammogram, which can considerably reduce the mortality ratio. The manual analysis of the different mammogram images requires time and human efforts. This research work proposed a new framework to help the radiologist achieve efficient work in less time and effort. This study has two-fold contributions—a new proposed framework based on two-level classification. The first-level classification consists of predicting benign/malignant breast cancer, while the second level involves classifying the mammogram images, selected as malignant, into early/advanced stage. Up to our knowledge, the diagnosis of the malignancy stage has not been investigated. Besides, a specialized radiologist proposed two new labelled classes early/ advanced in the MIAS dataset. The presented framework is based on two feature extraction methods that combine DWT to either LBP or GLCM using different sub-images (horizontal, vertical and diagonal), with different radius and levels. The proposed feature vectors are tested using two classifiers, KNN and SVM. Two experimentations are realized; the first experiment focuses on the first-level classification, where five feature vectors are proposed to classify the mammogram images into benign/malignant. The classification accuracy exceeded 96% (resp. 88%) for the first four proposed feature sets and reached 100% for the first feature set (resp. the first and the third feature sets) using SVM (resp. KNN). The best accuracy for both classifiers (100%) is obtained using DWT-based GLCM. Moreover, SVM performed better than KNN. The comparison study showed that proposed methods outperformed five state-of-the-art studies. The second experiment involves the prediction of the malignant cases to early/advanced stages. Ten feature vectors are proposed. The classification accuracy achieved 76.92% for SVM and 69.23% for KNN. The best result is obtained using three-level DWT decomposition. It has been shown that all the predicted advanced cases are checked to be the actual advanced cases (precision 100%) for both classifiers. However, the obtained results using SVM are superior to the results yielded by KNN. Consequently, SVM’s proposed model is efficient in predicting the actual cases into advanced/early classes. The radiologists can utilize the presented framework in classifying the mammogram images for an in-depth diagnosis of breast cancer disease. Even though the results are promising, they could be improved using other features such as the shape, contour and size. Also, some well-known feature reduction methods can be used to enhance the classification accuracy and prevent the curse of dimensionality problem since the size of the database is small. Furthermore, the segmentation phase can also be enhanced by combining traditional segmentation methods with artificial intelligence algorithms such that genetic algorithms or intelligent swarm methods.

62

Artificial Intelligence and Internet of Things

REFERENCES Altman, N. S. (1992). An introduction to kernel and nearest neighbor nonparametric regression. The American Statistician, 46(3), 175–185. https://doi.org/10.1080/00031305.199 2.10475879 Berment, H., Becette, V., Mohallem, M., Ferreira, F., & Chérel, P. (2014). Masses in mammography: What are the underlying anatomopathological lesions? Diagnostic and Interventional Imaging, 95(2), 124–133. https://doi.org/10.1016/j. diii.2013.12.010 Beura, S., Majhi, B., & Dash, R. (2015). Mammogram classification using two dimensional discrete wavelet transform and gray-level co-occurrence matrix for detection of breast cancer. Neurocomputing, 154, 1–14. https://doi.org/10.1016/j.neucom.2014.12.032 Cheng, H. D., Cai, X., Chen, X., Hu, L., & Lou, X. (2003). Computer-aided detection and classification of microcalcifications in mammograms: A survey. Pattern Recognition, 36(12), 2967–2991. https://doi.org/10.1016/S0031-3203(03)00192-4 Cheng, H. D., Shi, X. J., Min, R., Hu, L. M., Cai, X. P., & Du, H. N. (2006). Approaches for automated detection and classification of masses in mammograms. Pattern Recognition, 39(4), 646–668. https://doi.org/10.1016/j.patcog.2005.07.006 Cortes, C., & Vapnik, V. N. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018 Demidova, L, Nikulchev, E, & Sokolova, Y. (2016). The SVM classifier based on the modified particle swarm optimization. ArXiv Preprint ArXiv:1603.08296. Dong, M., Wang, Z., Dong, C., Mu, X., & Ma, Y. (2017). Classification of region of interest in mammograms using dual contourlet transform and improved KNN. Journal of Sensors, 2017, 1–15. https://doi.org/10.1155/2017/3213680 Eddaoudi, F., Regragui, F., Mahmoudi, A., & Lamouri, N. (2011). Masses detection using SVM classifier based on textures analysis. Applied Mathematical Sciences, 5(8), 367–379. Eltoukhy, M. M., Faye, I., & Samir, B. B. (2010a). A comparison of wavelet and curvelet for breast cancer diagnosis in digital mammogram. Computers in Biology and Medicine, 40(4), 384–391. https://doi.org/10.1016/j.compbiomed.2010.02.002 Eltoukhy, M. M., Faye, I., & Samir, B. B. (2010b). Breast cancer diagnosis in digital mammogram using multiscale curvelet transform. Computerized Medical Imaging and Graphics, 34(4), 269–276. https://doi.org/10.1016/j.compmedimag.2009.11.002 Eltoukhy, M. M., Faye, I., & Samir, B. B. (2012). A statistical based feature extraction method for breast cancer diagnosis in digital mammogram using multiresolution representation. Computers in Biology and Medicine, 42(1), 123–128. https://doi.org/10.1016/j. compbiomed.2011.10.016 Ferreira, C. B. R., & Borges, D. L. (2003). Analysis of mammogram classification using a wavelet transform decomposition. Pattern Recognition Letters, 24(7), 973–982. https:// doi.org/10.1016/S0167-8655(02)00221-0 Fowler, E. E., Sellers, T. A., Lu, B., & Heine, J. J. (2013). Breast Imaging Reporting and Data System (BI-RADS) breast composition descriptors: Automated measurement development for full field digital mammography. Medical Physics, 40(11). https://doi. org/10.1118/1.4824319 Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973). Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6), 610–621. https://doi.org/10.1109/TSMC.1973.4309314 Huang, M.-W., Chen, C.-W., Lin, W.-C., Ke, S.-W., & Tsai, C.-F. (2017). SVM and SVM ensembles in breast cancer prediction. PLoS ONE, 12(1). https://doi.org/10.1371/journal. pone.0161501

Two-Level Breast Cancer Staging Diagnosis

63

Institute of Medicine (US) and National Research Council (US) Committee on the Early Detection of Breast Cancer. (2001). Mammography and Beyond: Developing Technologies for the Early Detection of Breast Cancer: A Non-Technical Summary (M. Patlak, S. J. Nass, I. C. Henderson, & J. C. Lashof, Eds.). National Academies Press (US). http://www.ncbi.nlm.nih.gov/books/NBK223391/ Jalalian, A., Mashohor, S. B. T., Mahmud, H. R., Saripan, M. I. B., Ramli, A. R. B., & Karasfi, B. (2013). Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: A review. Clinical Imaging, 37(3), 420–426. https://doi.org/10.1016/j. clinimag.2012.09.024 Katzen, J., & Dodelzon, K. (2018). A review of computer aided detection in mammography. Clinical Imaging, 52, 305–309. https://doi.org/10.1016/j.clinimag.2018.08.014 Kaushik, M. K., Paramkusham, S., & Akhilandeshwari, V. (2020). Breast abnormality detection using LBP variants and discrete wavelet transform. In Advanced Computing and Intelligent Engineering (B. Pati, C. R. Panigrahi, R. Buyya, & K.-C. Li, Eds.) (pp. 433–442). Springer. https://doi.org/10.1007/978-981-15-1081-6_37 Kopans, D. B. (2007). Mammography positioning. In Breast Imaging (D. B. Kopans, Ed.) (3rd ed.). Lippinctt Williams & Wilkins. http://pubs.rsna.org/doi/10.1148/ radiol.2483082524 Lakhani, S. R. (2012). WHO classification of tumours of the breast. In WHO Classification of Tumours (Lakhani, S. R., Ellis, I. O., Schnitt, S. J., Tan, P. H., van de Vijver, M. J. Eds.) (4th ed.). International Agency for Research on Cancer, Lyon, France. Larabi-Marie-Sainte, S., Saba, T., Alsaleh, D., & Alamir Alotaibi, M. B. (2019). An improved strategy for predicting diagnosis, survivability, and recurrence of breast cancer. Journal of Computational and Theoretical Nanoscience, 16(9), 3705–3711. https://doi. org/10.1166/jctn.2019.8238 Lehman, C. D., Wellman, R. D., Buist, D. S. M., Kerlikowske, K., Tosteson, A. N. A., & Miglioretti, D. L. (2015). Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Internal Medicine, 175(11), 1828. https:// doi.org/10.1001/jamainternmed.2015.5231 Liu, S., Babbs, C. F., & Delp, E. J. (2001). Multiresolution detection of spiculated lesions in digital mammograms. IEEE Transactions on Image Processing, 10(6), 874–884. https://doi.org/10.1109/83.923284 Moayedi, F., Azimifar, Z., Boostani, R., & Katebi, S. (2010). Contourlet-based mammography mass classification using the SVM family. Computers in Biology and Medicine, 40(4), 373–383. https://doi.org/10.1016/j.compbiomed.2009.12.006 Ragab, D. A., Sharkas, M., Marshall, S., & Ren, J. (2019). Breast cancer detection using deep convolutional neural networks and support vector machines. PeerJ, 7, e6201. https://doi. org/10.7717/peerj.6201 Ramadan, S. Z. (2020, March 12). Methods used in computer-aided diagnosis for breast cancer detection using mammograms: A review. Review Article. Journal of Healthcare Engineering. https://doi.org/10.1155/2020/9162464 Setiawan, A. S., Elysia, E., Wesley, J., & Purnama, Y. (2015). Mammogram classification using Law’s texture energy measure and neural networks. Procedia Computer Science, 59, 92–97. https://doi.org/10.1016/j.procs.2015.07.341 Siu, A. L., & on behalf of the US Preventive Services Task Force. (2016). Screening for breast cancer: U.S. Preventive Services Task Force Recommendation Statement. Annals of Internal Medicine, 164(4), 279. https://doi.org/10.7326/M15-2886 Suckling, J., Parker, J., Dance, D., Astley, S., Hutt, I., Boggis, C., Ricketts, I., Stamatakis, E., Cerneaz, N., Kok, S., Taylor, P., Betal, D., & Savage, J. (2015). Mammographic Image Analysis Society (MIAS) database v1.21 [Data set]. https://www.repository.cam.ac.uk/ handle/1810/250394

64

Artificial Intelligence and Internet of Things

Tomar, A, & Nagpal, A. (2016). Comparing accuracy of K-nearest-neighbor and supportvector-machines for age estimation. International Journal of Engineering Trends and Technology, 38(6). DOI: 10.14445/22315381/IJETT-V38P260. Tsochatzidis, L., Costaridou, L., & Pratikakis, I. (2019). Deep learning for breast cancer diagnosis from mammograms—A comparative study. Journal of Imaging, 5(3), 37. https:// doi.org/10.3390/jimaging5030037

3

Breast Cancer Detection and Diagnostic with Convolutional Neural Networks Muhammad Kashif, Amjad Rehman, Tariq Sadad, Zahid Mehmood

CONTENTS 3.1 Introduction.....................................................................................................66 3.2 Breast Cancer Abnormalities...........................................................................66 3.2.1 Masses/lumps...................................................................................... 67 3.2.2 Microcalcifications.............................................................................. 67 3.2.3 Architectural Distortions (AD)............................................................ 67 3.3 Diagnosis......................................................................................................... 67 3.4 Treatments....................................................................................................... 70 3.5 Prevention Strategies....................................................................................... 70 3.6 Preprocessing Techniques............................................................................... 71 3.6.1 Image Enhancement............................................................................ 71 3.6.2 Segmentation....................................................................................... 71 3.6.3 Feature Extraction............................................................................... 71 3.6.4 Classification........................................................................................ 72 3.7 Datasets............................................................................................................ 72 3.7.1 Database for Screening Mammography (DDSM) Mammogram Dataset.......................................................................... 72 3.7.2 CBIS-DDSM Dataset........................................................................... 72 3.7.3 FNAC Database................................................................................... 72 3.7.4 Bioimaging Challenge 2015 Breast Histology Dataset....................... 72 3.7.5 Mammograms-MIAS Dataset............................................................. 72 3.7.6 DMR-IR Dataset.................................................................................. 73 3.7.7 WDBC Dataset.................................................................................... 73 3.7.8 In Breast Cancer Dataset..................................................................... 73 3.8 CNN Techniques............................................................................................. 74 3.9 Analysis and Findings..................................................................................... 74 3.10 Conclusions and Future Challenges................................................................. 77 References................................................................................................................. 77 65

66

Artificial Intelligence and Internet of Things

3.1 INTRODUCTION The study and classification of medical imaging have a significant role to play to identify anomalies of various body species such as blood cancer (Abbas et al., 2018a,b, 2019a,b; Rehman et al., 2018a,b,c), lung cancer (Khan et al., 2019a,b,d; Saba et al., 2019a,c) and brain tumours (Khan et al., 2019,c,e). Organ abnormality also leads to rapid tumour formation, the world’s leading cause of death (Fahad et al., 2018; Rahim et al., 2017a,b; Saba et al., 2012, 2018a,b; Yousaf et al., 2019a,b; Ullah et al., 2019). Generally, breast cancer is detected amongst women; ~2.1 million infected cases are reported yearly. In 2018, the mortality rate of ~627,000 was estimated (Afza et al., 2019; Mughal et al., 2017). Two billion new breast-cancer-infected individuals were indicated globally in 2018 (Roslidar et al., 2020). Breast cancer is caused due to the irregularity and irrepressibly spreading of breast cell tissues that create big tissue lump resulting in cancer (Mughal et al., 2018a). It happens in breast glandular epithelium due to incorrect cell development (Lu et al., 2018). There are four kinds of breast tissues: normal, benign (denotes slight variation), in situ carcinoma (internal breast infection, can be treated if detected in initial stages) and invasive carcinoma (cancer spreads to external organs) (Mughal et al., 2018b). Extensive clinical tests reported that cancer-infected persons could be treated more effectively and have less mortality when detected and diagnosed at initial stages (Abbas et al., 2015, 2018a,b; Mashood Nasir et al., 2020; Yousaf et al., 2019a; Zou et al., 2019). Newly developed medical imaging techniques, such as magnetic resonance imaging (MRI), ultrasound imaging, mammography, computed tomography (CT), positron emission tomography (PET), thermography and more, are applied successfully in early breast cancer diagnosis (Al-Ameen et al., 2015; Sadad et al., 2018; Ullah et al., 2019; Yousaf et al., 2019b). Several scientists have developed various techniques for early-stage breast cancer detection to enhance performance of infection classification (Amin et al., 2018, 2019a,b; Mughal et al., 2018b). Commonly practitioners segment abnormal regions manually at each slice of MR-imaging modalities (Iqbal et al., 2019; Nazir et al., 2019; Norouzi et al., 2014). The early and accurate detection of any type of disease is a keystone in the cure of patients, increasing the survival possibilities. This chapter is further categorized as Section 3.2, which presents breast cancer abnormalities. Section 3.3 provides diagnosis, Section 3.4 analysed treatments; prevention strategies are presented in Section 3.5. Section 3.6 discusses the preprocessing technique; datasets description is provided in Section 3.7. Section 3.8 demonstrates CNN techniques and Section 3.9 presents analysis and findings. Finally, Section 3.10 provides a conclusion and future challenges.

3.2  BREAST CANCER ABNORMALITIES Breast cancer is caused by a hasty and unbalanced partition of cells, resulting in a lump, which spreads to external organs by lymph nodes. The disease mainly infects glandular, ducts or further breast (Osman et al., 2020; Sadad et al., 2018; Saba, 2017; Saba et al., 2012, 2019a,b,c). A biopsy is an effective procedure for cancer confirmation than the detection of breast irregularities by self-analysis, doctors and other methods. Several methods are commonly used for primary breast cancer diagnosis,

Breast Cancer Detection and Diagnostic Analysis

67

FIGURE 3.1  Breast cancer abnormalities with (a) malignant masses, (b) architectural distortions and (c) calcification

such as MRI, ultrasound, mammography and more (Saba et al., 2018a,b; ZuluagaGomez et al., 2019). Breast abnormalities can be categorized into three main types.

3.2.1 Masses/lumps The lump, swelling, knob or hump localization of the breast varies from the breast’s normal tissues called mass. It can be cancerous or non-cancerous by classifying based on size, shape and density (Rehman et al., 2018a,b,c; Saba, 2020; Saba et al., 2019a,b,c, 2020a,b).

3.2.2 Microcalcifications This abnormality can be seen as white dots appearing on mammograms; this spread of calcium bonds in clusters or mammary glands is called microcalcifications (Nazir et al., 2019; Perveen et al., 2020; Quershi et al., 2020).

3.2.3  Architectural Distortions (AD) This abnormality has not been related to masses that can be observable or noticeable. In contrast, irregularity of breast cell tissue parts can be found in circular or scattered patterns due to surgery (Norouzi et al., 2014; Rad et al., 2013, 2016; Rahim et al., 2017a,b; Ramzan et al., 2020a,b). Figure 3.1 shows breast cancer abnormalities.

3.3 DIAGNOSIS National Cancer Institute (NCI) predicted that the breast cancer infection proportion will be 50% by 2030 in the United States. World Cancer Report reported a diagnosis rate of 22.9% and high for breast cancer and the mortality rate of 13.7% globally due to cancer. Breast cancer arises due to a change in close cell tissues, creating a big lump that produces a tumour. It is cancerous or non-cancerous once extended in the breast or other organs by lymph and blood. Cancer might originate in mammary gland lobules due to fats in cell tissues or genetic alterations. Figure 3.2 exhibits different types of breast cancer.

68

Artificial Intelligence and Internet of Things

FIGURE 3.2  (a) Breast structure, (b) and (c) breast having tumour and (d) breast cancer stages

Breast cancer’s (BC) symptoms include size and shape variations, possibly unproblematic solid lump growth and skin variations like colour, bumps, lumpiness and nipple variations. Breast cancer is also diagnosed in patients having no symptoms (Mittal et al., 2020). Convolutional neural network (CNN)-based techniques ResNet101, DenseNet, MobileNetV2 and ShuffleNetV2 were evaluated to detect breast cancer, tested by ImageNet dataset (Saba 2019). The DMR (Database for Mastology Research) dataset containing breast cancer and normal thermal images was used for training. The results show that the DenseNet achieves accurate performance, while ResNet101 and MobileNetV2 (99.6%) and ShuffleNetV2 (98%) have slightly lesser performance (Roslidar et al., 2019). A CNN-based computer-aided design (CAD) system was proposed for breast cancer diagnosis by evaluating an open-source DMR with infrared image (DMR-IR) dataset consisting of 57 patients’ thermal images. The results reported better performance in terms of accuracy, sensitivity, precision and F1 scores of 92%, 91%, 94% and 92%, respectively. Several factors limit the research: low images data prevent better performance (Ejaz et al., 2018, 2020; Fahad et al., 2018; Zuluaga-Gomez et al., 2019). An ensemble boosting and radial-based function neural network (EBLRBFNN) technique was proposed to diagnose and stage breast cancer assessment by evaluating UCI (University of California, Irvine) variation breast cancer datasets. The accuracies achieved for different UCI repository BCP (Breast Cancer Prognostic), BCD (Breast Cancer Dataset), WBC (Breast Cancer Original) and WBCD (Wisconsin Breast Cancer Dataset) datasets were 97.7%, 98.4%, 97.4% and 97.0%, respectively (Osman et al., 2020). DNN-RFE (deep neural network and recursive feature elimination) technique was proposed for breast cancer diagnosing. The experiments were performed on WBCD with an accuracy of 98.62%, and 2% benign was misclassified as malignant and 1% malignant was misclassified as benign. The

Breast Cancer Detection and Diagnostic Analysis

69

study is limited due to DNN training time and time consumption in low GPU (graphics processing units) systems (Abdel-Zaher et al., 2016 ). Computer-aided detection (CADe) system based on 3D CNN was developed to detect and diagnose breast cancer by evaluating ABUS (Automated Breast Ultrasound) dataset due to doctors’ timing cost. The sliding window technique is utilized for dataset categorization into volumes of interests (VOIs). The unbalanced data, high false positive (FP) and false negative (FN) rates were resolved through “focal loss and ensemble learning” technique due to cancer images’ similarity. At last, candidate aggregation technique is used for concatenating data. Extensive experiments performed by exploiting 81 and 165 cases resulted in 57.1% FP reduction at 95.3% sensitivity. This is limited due to the high FP rate in the case of 100% sensitivity, and by taking original images, it might be reduced. This technique is incapable of detecting cancer in the real world (Iqbal et al., 2017, 2018, 2019; Lung et al., 2014; Majid et al., 2020; Moon et al., 2020). CNN-based computer-aided diagnosis system was developed for mass lesions of breast optical tomographic images classification (Javed et al., 2019a,b, 2020a,b; Khan et al., 2019e, 2020e; Rehman et al., 2020a,b). The data contains 63 optical tomographic images of dense breasts women and 1260 2D greyscale images of DOT breast dataset (not open source but accessible on request). Experiments reported accuracy, sensitivity, specificity and area under the ROC curve (AUC) of 90.2%, 95%, 80% and 0.94%, respectively. The 93.3% accuracy, 88% sensitivity, 96% specificity and 0.95% AUC were reported by augmentation technique evaluation of the enhanced dataset. The research is limited due to small data that affect performance. Instead of 3D, 2D images were used, and they have less information than 3D (Xu et al., 2019). The CNN-based mask regions’ method was proposed to predict breast cancer by evaluating a dataset of 307 patients’ ultrasound images resulting in 75% precision and 85% accuracy. The lack of a patient’s data and method enhancement prevents performance (Chiao et al., 2019). Figure 3.3 shows different modality images. A model based on multilayer CNN was implemented, and experiments were performed

FIGURE 3.3  Breast cancer screening through different modalities (a) mammogram clustered microcalcifications, (b) elastography ultrasound image, (c) breast MRI image and (d) magnified histopathology WSI image

70

Artificial Intelligence and Internet of Things

TABLE 3.1 List of Treatments against Breast Cancer Author Motallebnezhad et al. (2016)

Cancer stage I, II

Masoud et al. (2017) Collignon et al. (2016)

Aubele et al. (2017) Martel et al. (2017)

II I/II

II

Treatments AVE 1642, dalotuzumab (MK-0646) Alisertib with taxol Doxorubicin plus cyclophosphamide Anti-PDL1 (MEDi4736) and nab-paclitaxel associated Tamoxifen Trastuzumab + pertuzumab

on enhanced MRI images. The dataset contains 200 breast cancer MRI images collected from Turkey Hospital, Istanbul. The experiment presents a 98.33% accuracy performance, 100% sensitivity, 96.9% specificity, 96.55% precision and 0.0167 loss. The analytical analysis shows that 3.4% benign image was misclassified as malignant (Yurttakal et al., 2019).

3.4 TREATMENTS Breast cancer mostly infects women globally. The risk factors of breast include smoking, alcohol consumption, fatness, inactiveness, genetic history and preceding treatments such as chemotherapy and radiotherapy (Khan et al., 2017; Majid et al., 2020; Marie-Sainte et al., 2019a,b; Martel et al., 2017). Novel antiviral medications such as abemaciclib (CDK4/6 inhibitor), tyrosine phosphatase-1 (SHP-1) and SRC homology 2 (SH2), conjugate ADC (antibody drug conjugates) and fatty acid synthase (FASN) were revealed for treatment effectiveness and medicine resistance. The pertuzumab, T-DM1 and lapatinib development showed greater effectiveness than trastuzumab’s severity and resistance (Liaqat et al., 2020; Mittal et al., 2020). Deep investigation endures further and enhanced treatment identification. Representative treatments comprise taxanes, cyclophosphamide, anthracyclines, FGFR, PARP (poly ADP-ribose polymerase), FGFR and EGFR (epidermal growth factor receptor) (Collignon et al., 2016). Accepted paclitaxel and docetaxel treatment are the main taxanes. Unilateral-mastectomy-treated women were exposed more to death than bilateral mastectomy and BRCA1/2 alteration treatment (Godet and Gilkes, 2017). mTOR inhibitors, CDK4/6, anti-HER2 therapies and chemotherapeutic agents are the possible cardiotoxicity agents (Martel et al., 2017). Table 3.1 presents different treatments and drugs for breast cancer diagnosis.

3.5  PREVENTION STRATEGIES Early breast cancer detection and diagnosis are the preventive strategies for mortality reduction and prediction enhancements (Khan et al., 2017, 2018, 2019c,d, 2020e). Breast cancer prevention strategies such as avoiding extra ionizing radiation, weight

Breast Cancer Detection and Diagnostic Analysis

71

sustaining, evading tobacco, reducing alcohol consumption, breastfeeding and exercise improve threat reduction in initial prevention. The main reason for mortality and the occurrence of cancer is tobacco consumption. Raloxifene and tamoxifen are two FDA-approved selective oestrogen receptor modulators (SERMs) for breast cancer prevention in women (Khan et al., 2020d). Breast cancer threat reduction related to reasonable preservation of exercise, weight and alcohol consumption is suggested by the American Cancer Society. According to World Cancer Research Fund (WCRF) and American Institute for Cancer Research (AICR) reports, prevention strategies include keeping sufficient body weight, activeness, (ED) high-energy-density food reduction, vegetables eating and limiting meat, salt and alcoholic beverages (Javed et al., 2019a,b, 2020a,b).

3.6  PREPROCESSING TECHNIQUES Breast cancer image diagnosis as cancerous/non-cancerous comprises numerous steps such as image enhancement, segmentation, feature extraction and classification (Iftikhar et al., 2017; Sadad et al., 2018).

3.6.1  Image Enhancement The procedure for contrast improvement and noise decreasing of different breast cancer images helps doctors in abnormality detection (Husham et al., 2016; Hussain et al., 2020; Jamal et al., 2017; Ragab et al., 2019). The image enhancement techniques are mainly applied for tags, labels, patient names or supplementary annoying data elimination and contrast augmentations (Dabass et al., 2019). Various image enhancement methods such as homomorphic filter for noise reduction and illumination improvements of breast images—Laplacian of Gaussian, unsharp mask, high-boost filters and more techniques are currently used (Rao, 2019).

3.6.2 Segmentation It is the process of image division into portions with characteristics and simplification for analysis. It comes to be monotonous due to noise, contrast variation and image blur effect (Adeel et al., 2020; Dabass et al., 2019; Khan et al., 2020a,b,c). Many image segmentation techniques such as partial differential equation (PDE), fuzzy theory, edge, artificial neural network (ANN), fuzzy C-means clustering (FCM), Otsu global thresholding, Gaussian kernel FCM, Gaussian mixture model, threshold and region-based segmentation are used (Amin et al., 2018, 2019a,b; Ragab et al., 2019).

3.6.3 Feature Extraction The feature extraction is the main step for disease diagnosis from images. It is the process of extracting local, texture and statistical features from various types of images through some familiar techniques such as angular second moment (ASM), fractal dimension (FD), entropy, scale invariant feature transform (SIFT), local binary pattern (LBP), centre-symmetric LBP (CS-LBP), gray-level co-occurrence matrix (GLCM) features and more (Rao, 2019).

72

Artificial Intelligence and Internet of Things

3.6.4 Classification The process of images categorization is based on labels. Numerous classification techniques were developed and evaluated in research for medical imaging of various types of images. Machine learning (ML) and artificial intelligence (AI) techniques were developed and deeply exploited for breast cancer image classifications such as ensemble learning and several CNN-based models (AlexNet, ResNet and more).

3.7 DATASETS This section provides different datasets evaluated during breast cancer detection and diagnosing with its brief description and access links.

3.7.1 Database for Screening Mammography (DDSM) Mammogram Dataset This dataset contains 43 volumes of 2620 cases, including normal, benign and malignant mammograms in 50 mm and greyscale of 12 and 16 bits.

3.7.2 CBIS-DDSM Dataset Enhanced dataset according to the region of interest (ROI) and segmented data based on digital database for screening mammography (DDSM) called Curated Breast Imaging Subset (CBIS-DDSM) of DDSM consists of 891 mass cases and 753 microcalcification cases of the breast (Ragab et al., 2019).

3.7.3 FNAC Database The fine needle aspiration cytology (FNAC) dataset consists of images, including cell samples collected and processed at Ayursundra Healthcare Pvt. Ltd, Guwahati, India. The dataset contains 212 FNAC images, including 113 malignant and 99 benign of 20 infected peoples. The images are collected through high definition (HD) microscope (Leica ICC50) with a 5-megapixel camera and 400 resolution, and colour depth is 24 bit with reports and labels (Saikia et al., 2019).

3.7.4 Bioimaging Challenge 2015 Breast Histology Dataset The dataset contains two types of files: training and testing files, including 249 and 20 labelled H&E images of normal, benign, in situ carcinoma and invasive carcinoma classes, respectively, and an enhanced test file of 16 images with 2040 × 1536 resolution and 200 × magnified rate (Araújo et al., 2017).

3.7.5 Mammograms-MIAS Dataset Mammographic Image Analysis Society (MIAS) dataset consists of 322 mammograms, including 189 normal and 133 abnormal breast images. The dataset contains

Breast Cancer Detection and Diagnostic Analysis

73

seven class data of abnormalities such as “architectural distortion, asymmetry, ­calcification, spiculated masses, circumscribed masses and varied images” (Charan et al., 2018).

3.7.6 DMR-IR Dataset The DMR-IR dataset consists of 57 infected patients, including normal and infected (19 and 37, respectively) thermal images, ~21–80. The images were recorded at −40 to 500°C through FLIR (forward-looking infrared) thermal camera “model SC620” with 640 × 480 resolution. The dataset has files of the heat map and a matrix point of 640 × 480 (Zuluaga-Gomez et al., 2019).

3.7.7 WDBC Dataset The Wisconsin Data Set for Diagnostic Breast Cancer dataset contains 569 infected patients, including 212 malignant and 357 benign, and several features were collected by ML library at Irvine University of California.

3.7.8  In Breast Cancer Dataset The dataset contains 410 mammogram images of 115 infected cases, including 90 cases of both breasts and mastectomy patients and 25 cases with several breast abnormalities. The dataset is provided on request in XML format. Different benchmark datasets are presented in Table 3.2.

TABLE 3.2 Datasets and Their Access Links Dataset DDSM mammogram dataset CBIS-DDSM dataset FNAC database Bioimaging Challenge 2015 Breast Histology Dataset Mammograms-MIAS dataset DMR-IR dataset WDBC dataset In breast cancer dataset

Access Links http://marathon.csee.usf.edu/Mammography/ Database.html https://wiki.cancerimagingarchive.net/display/ Public/CBIS-DDSM https://1drv. ms/u/s!Al-T6d-_ENf6axsEbvhbEc2gUFs https://rdm.inesctec.pt/dataset/nis-2017-003 https://www.kaggle.com/kmader/ mias-mammography http://visual.ic.uff.br/dmi https://www.kaggle.com/uciml/ breast-cancer-wisconsin-data http://medicalresearch.inescporto.pt/breastresearch/ index.php/Get_INbreast_Database

74

Artificial Intelligence and Internet of Things

3.8  CNN TECHNIQUES Breast cancer patient’s image diagnosis is mostly complex for physicians due to lack of doctors and mistakes during diagnosis processes (Ejaz et al., 2020; Iqbal et al., 2018). Several studies aimed to create such a model that can automatically diagnose the medical images. CNNs resulted in better performance and image classifiers than other networks and models. It is the feature extraction procedure to form feature vectors through input data scale reduction (Amin et al., 2019c,d). A deep CNN is generally classified into basic input, convolution, fully connected (FC) layers and output layer (softmax). Convolution and FC layers are used for appearance and pixel-based features, respectively. Further layers such as normalization and pooling layers are used during the process. CNN is used for features such as automatic detection and prediction and effectiveness, which are reported for breast cancer screening. CNN is the leading model for medical image recognition and classification. Numerous models are based on adaptable CNN, such as AlexNet (Krizhevsky et al., 2012), VGG (Visual Geometry Group) (Simonyan et al., 2014), ResNet, DenseNet (Huang et al., 2017), Inception (Szegedy et al., 2015), and more for breast ultrasound (BUS), MRI, chest X-rays and more image types (Ausawalaithong et al., 2018). Standard CNN architecture is presented in Figure 3.4.

3.9  ANALYSIS AND FINDINGS Several CNN-based systems were evaluated for the analysis of breast cancer. DNN was proposed by Charan et al. (2018) to detect breast cancer by evaluating the MammogramsMIAS dataset. Stochastic gradient descent momentum (SGDM) is applied during training, followed by CNN. The average accuracy of 65% was reported. A novel CAD system was developed for breast mass cancer classification by evaluating multiple mammographic datasets such as DDSM and CBIS-DDSM. Images are segmented through ROI, threshold and region-based methods. Features are extracted through DCNN (deep convolutional neural network), and fine-tuned AlexNet (DCNN based) is used for classification. Extensive experiments of DDSM dataset reported 80.5%, 77.4%, 84.2%, 86%, 81.5% and 88% of accuracy, sensitivity, specificity, precision, F1 score and AUC, respectively. CBIS-DDSM dataset in the case of DCNN + SVM (Support Vector Machine) achieved the performance accuracy of 73.6%, 94% AUC, 86.2% sensitivity, 87.7% specificity, 88% precision and 87.1% F1 score (Ragab et al., 2019).

FIGURE 3.4  CNN architecture

Breast Cancer Detection and Diagnostic Analysis

75

A CAD system was proposed using unsupervised extreme learning machine (US-ELM) clustering and different fused features. Classifying breast mass through ELM classifier resulted accurately and efficiently to detect and classify breast mass (Wang et al., 2019). mixture-ensemble-based CNNs (ME-CNN) has been proposed for breast cancer classification by considering dynamic contrast-enhanced MRI (DCEMRI) breast images. The results efficiently show 96.39% accuracy, 97.73% sensitivity and 94.87% specificity (Rasti et al., 2017). A deep learning (DL) algorithm based on a neural network was developed to diagnose breast cancer by evaluating 11 features from the WBCD dataset, which resulted in 99.67% accuracy. Preprocessing techniques were applied, such as Label Encoder, Normalizer and Standard Scaler (Khuriwal et al., 2018). CNN-based algorithm was developed, followed by enhancing Bayes a­ lgorithm to extract and classify multiple features as normal or infected by evaluating 140 patients’ breast thermal images dataset. The result presents 98.95% accuracy. This study is limited due to blurriness images that restrict the features and affect the performance (Ekici et al., 2020). A CNN-based technique is proposed to classify hematoxylin and eosin (H&E)-stained breast biopsy images (containing normal, benign, in situ carcinoma and invasive carcinoma). The features are extracted both locally and globally through CNN and also evaluated through SVM. Accuracies of 77.8% and 83.3% were achieved for four classes and carcinoma/non-carcinoma, respectively, and sensitivity was 95.6% (Araújo et al., 2017). Fine-tuned transfer learning based on comparison of DCNNs was proposed for cell sample diagnosis by evaluating 2120 images. Multiple CNN models such as ResNet-50, GoogLeNet-V3, VGG16 and VGG19 are also implemented and tested. Cell samples were collected through FNAC through a microscope, and it is used mainly for breast cancer diagnosis. The results achieved 96.25% accuracy for enhanced GoogLeNet-V3 by crucial FNAC (Saikia et al., 2019). Transfer learning such as InceptionV3, ResNet50 and Xception, CNN3 models and traditional ML techniques distinguish cancerous/non-cancerous by BUS images. Deep-featurebased classification techniques were also developed to extract transferred learning features with 89.44% and 0.93% of accuracy and AUC, respectively. The biopsyproven benchmarking dataset consists of 2058 BUS images, including 688 malignant and 1370 benign. Extensive results present that transfer learning models perform better than ML and CNN3 technique. The InceptionV3 attained 85.13% and 0.91% accuracy and AUC, respectively. The study is limited due to low data, and the drawback of the InceptionV3 model is memory consumption (Xiao et al., 2018). DL typically needs big data for network training, and a better technique for small data is transfer learning in terms of medicinal data. At the same time, it can be complex when overfitting happens. The CNN-based CAD system proposed mammograms’ classification by different breast cancer datasets and comparatively evaluated several CNN-based techniques. Extensive results reported accuracy and AUC of 97.35% and 0.98%, respectively, in the DDSM dataset, accuracy and AUC of 95.50% and 0.97%, respectively, in the INbreast database and accuracy and AUC of 96.67% and 0.96%, respectively, in the case of BCDR (breast cancer digital repository) dataset. After some preprocessing from all datasets and evaluated classified-based CNN, the large dataset is created after some preprocessing resulted in 98.94% accuracy and more effective breast cancer prediction (Chougrad et al., 2018). A faster R-CNN CAD system was implemented for cancer detection and classification from breast

76

Artificial Intelligence and Internet of Things

mammograms. The extensive results reported 0.95% and 0.85% AUC for INbreast and Digital Mammography DREAM Challenge data, respectively, with a low FP rate (0.3) per image in the case of INbreast database. The technique is limited due to less pixel-wise label data to detect cancer and big data assessed for model classification (Ribli et al., 2018). Radiologists prefer CAD-based techniques that help them in the precise diagnosis procedures, but the issue remains in breast mammograms’ prediction and classification. You Only Look Once (YOLO) CAD technique based on CNN (ROI-based) was developed. The technique mainly consists of preprocessing, feature extraction, detection and mammogram classification by exploiting multi-convolutional deep layers, confidence models and FC neural network (FC-NN). The statistical analysis reported 96.33% and 85.52% accuracy for detecting and predicting breast cancer, respectively. The analysis shows that 22% of malignant cases were misclassified as benign and 6.8% benign misclassified as malignant. The system accomplished detection and classification at one time (Al-Masni et al., 2017). Table 3.3 reports the different performance measures.

TABLE 3.3 Critical Performance Measures Based on CNN Models

Author Techniques Roslidar ResNet101 and et al. (2019) MobileNetV2 ShuffleNetV2 ZuluagaCNN-based Gomez CAD system et al. (2019) Osman et al. EBL-RBFNN (2020) Abdel-Zaher DNN-RFE et al. (2016) Moon et al. 3D CNN-based (2020) CADe Xu et al. CNN (2019)

database DMR database

F1 Accuracy Sensitivity Specificity Precision Score (%) (%) (%) (%) (%) 99.6 – – – – 98 – – – –

DMR-IR dataset

92

91

–

94

92

WBCD dataset

97

–

–

–

–

WBCD dataset

98.62

–

–

–

–

ABUS dataset

–

95.3

–

–

–

93.3

88

96

–

–

85

–

–

75

–

98.33

100

96.9

96.55

–

DOT breast dataset and optical tomographic images Chiao et al. CNN-based 307 ultrasound (2019) mask regions’ patients’ method images Yurttakal Multilayer 200 MRI et al. (2019) CNN images

Breast Cancer Detection and Diagnostic Analysis

77

3.10  CONCLUSIONS AND FUTURE CHALLENGES Breast cancer exhibits several symptoms such as pain, size and shape variations, nipple variations and more. The genetic adaptation of somatic cells’ abnormality is indicated as cancer. BC’s mortality rate growth is predicted due to age, increasing population, infection awareness, health and family history, radiation disclosure and more. This research aims to analyse breast cancer based on the CNN technique by first providing a comprehensive overview of breast cancer, followed by abnormalities, diagnosis, treatments and prevention strategies, image processing and CNN techniques. The crucial analysis was reported according to published evidence. The future challenges include developing the technique for big data variations’ classification, while the performance remains unchanged. The analysis of techniques based on CAD systems reported limitations such as the fewer dataset size and comparison that should be improved. The CAD models, such as chest CT, mammograms and more, can aid the physicians during the accurate diagnosis of BC due to lack of eye visions, tiredness and interruption. Doctors and oncologists aimed at pragmatic practice variations for translational research to develop new drugs to achieve better breast cancer performance. Its types are overexpressing HER2, basal-like, luminal A and luminal B. The future challenges for breast cancer include accumulating toxicity, tumour tissue biopsy, liquid biopsy, large data analysis and translational biomarkers.

REFERENCES Abbas, N., Mohamad, D., Abdullah, A. H., Saba, T., Al-Rodhaan, M., & Al-Dhelaan, A. (2015). Nuclei segmentation of leukocytes in blood smear digital images. Pakistan Journal of Pharmaceutical Sciences, 28(5), pp. 1801–1806. Abbas, N., Saba, T., Mehmood, Z., Rehman, A., Islam, N., & Ahmed, K. T. (2019a). An automated nuclei segmentation of leukocytes from microscopic digital images. Pakistan Journal of Pharmaceutical Sciences, 32(5), pp. 2123–2138. Abbas, N., Saba, T., Mohamad, D., Rehman, A., Almazyad, A. S., & Al-Ghamdi, J. S. (2018a). Machine aided malaria parasitemia detection in Giemsa-stained thin blood smears. Neural Computing and Applications, 29(3), pp. 803–818. DOI:10.1007/ s00521-016-2474-6. Abbas, A., Saba, T., Rehman, A., Mehmood, Z., Javaid, N., Tahir, M., Khan, N. U., Ahmed, K. T., & Shah, R. (2019b). Plasmodium species aware based quantification of malaria, parasitemia in light microscopy thin blood smear. Microscopy Research and Technique, 82(7), pp. 1198–1214. DOI:10.1002/jemt.23269. Abbas, N., Saba, T., Rehman, A., Mehmood, Z., Kolivand, H., Uddin, M., & Anjum, A. (2018b). Plasmodium life cycle stage classification-based quantification of malaria parasitaemia in thin blood smears. Microscopy Research and Technique. https://doi. org/10.1002/jemt.23170 Abdel-Zaher, A. M., & Eldeib, A. M. (2016). Breast cancer classification using deep belief networks. Expert Systems with Applications, 46, pp. 139–144. Adeel, A., Khan, M. A., Akram, T., Sharif, A., Yasmin, M., Saba, T., & Javed, K. (2020). Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Systems. https://doi.org/10.1111/exsy.12569

78

Artificial Intelligence and Internet of Things

Afza, F., Khan, M. A., Sharif, M., & Rehman, A. (2019). Microscopic skin laceration segmentation and classification: a framework of statistical normal distribution and optimal feature selection. Microscopy Research and Technique, 82(9), pp. 1471–1488. Al-Ameen, Z., Sulong, G., Rehman, A., Al-Dhelaan, A., Saba, T., & Al-Rodhaan, M. (2015). An innovative technique for contrast enhancement of computed tomography images using normalized gamma-corrected contrast-limited adaptive histogram equalization. EURASIP Journal on Advances in Signal Processing, 32. DOI: 10.1186/ s13634-015-0214-1. Al-Masni, M. A., Al-Antari, M. A., Park, J. M., Gi, G., Kim, T. Y., Rivera, P., Valarezo, E., Han, S. M., & Kim, T. S. (2017, July). Detection and classification of the breast abnormalities in digital mammograms via regional convolutional neural network. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 1230–1233). IEEE. DOI:10.1109/EMBC.2017.8037053. Amin, J., Sharif, M., Raza, M., Saba, T., & Anjum, M. A. (2019a). Brain tumor detection using statistical and machine learning method. Computer Methods and Programs in Biomedicine, 177, pp. 69–79. Amin, J., Sharif, M., Raza, M., Saba, T., & Rehman, A. (2019b). Brain tumor classification: feature fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. DOI:10.1109/ICCISci.2019.8716449. Amin, J., Sharif, M., Raza, M., Saba, T., Sial, R., & Shad, S. A. (2019c). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 32, pp. 1–9. Amin, J., Sharif, M., Rehman, A., Raza, M., & Mufti, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy Research and Technique, 81(9), pp. 990–996. Amin, J., Sharif, M., Yasmin, M., Saba, T., & Raza, M. (2019d). Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications, 79(15), pp. 10955–10973. https://doi. org/10.1007/s11042-019-7324-y Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A., & Campilho, A. (2017). Classification of breast cancer histology images using convolutional neural networks. PLOS ONE, 12(6), p. e0177544. Aubele, M., Schmitt, M., Napieralski, R., Paepke, S., Ettl, J., Absmaier, M., … & Kiechle, M. (2017). The predictive value of PITX2 DNA methylation for high-risk breast cancer therapy: current guidelines, medical needs, and challenges. Disease Markers, 2017. DOI:10.1155/2017/4934608. Ausawalaithong, W., Thirach, A., Marukatat, S., & Wilaiprasitporn, T. (2018, November). Automatic lung cancer prediction from chest X-ray images using the deep learning approach. In 2018 11th Biomedical Engineering International Conference (BMEiCON) (pp. 1–5). IEEE. DOI:10.1109/BMEiCON.2018.8609997. Charan, S., Khan, M. J., & Khurshid, K. (2018, March). Breast cancer detection in mammograms using convolutional neural network. In 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1–5). IEEE. DOI:10.1109/ICOMET.2018.8346384. Chiao, J. Y., Chen, K. Y., Liao, K. Y. K., Hsieh, P. H., Zhang, G., & Huang, T. C. (2019). Detection and classification the breast tumors using mask R-CNN on sonograms. Medicine, 98(19), p. e15200. Chougrad, H., Zouaki, H., & Alheyane, O. (2018). Deep convolutional neural networks for breast cancer screening. Computer Methods and Programs in Biomedicine, 157, pp. 19–30. Collignon, J., Lousberg, L., Schroeder, H., & Jerusalem, G. (2016). Triple-negative breast cancer: treatment challenges and solutions. Breast Cancer: Targets and Therapy, 8, p. 93.

Breast Cancer Detection and Diagnostic Analysis

79

Dabass, J., Arora, S., Vig, R., & Hanmandlu, M. (2019). Segmentation techniques for breast cancer imaging modalities – a review. In 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (pp. 658–663). IEEE. DOI:10.1109/CONFLUENCE.2019.8776937. Ejaz, K., Rahim, M. S. M., Rehman, A., Chaudhry, H., Saba, T., & Ejaz, A. (2018). Segmentation method for pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), pp. 394–401. Ejaz, K., Rahim, M. S. M., Bajwa, U. I., Chaudhry, H., Rehman, A., & Ejaz, F. (2020). Hybrid segmentation method with confidence region detection for tumor identification. IEEE Access. DOI:10.1109/ACCESS.2020.3016627. Ekici, S. & Jawzal, H. (2020). Breast cancer diagnosis using thermography and convolutional neural networks. Medical Hypotheses, 137, p. 109542. Fahad, H. M, Khan, M. U. G., Saba, T., Rehman, A., & Iqbal, S. (2018). Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microscopy Research and Technique, 81(5), pp. 449–457. DOI:10.1002/jemt.22998. Godet, I. & Gilkes, D. M. (2017). BRCA1 and BRCA2 mutations and treatment strategies for breast cancer. Integrative Cancer Science and Therapeutics, 4(1). DOI:10.15761/ ICST.1000228. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4700–4708). DOI:10.1109/CVPR.2017.243. Husham, A., Alkawaz, M. H., Saba, T., Rehman, A., & Alghamdi, J. S. (2016). Automated nuclei segmentation of malignant using level sets. Microscopy Research and Technique, 79(10), pp. 993–997. DOI:10.1002/jemt.22733. Hussain, N., Khan, M. A., Sharif, M., Khan, S. A., Albesher, A. A., Saba, T., & Armaghan, A. (2020). A deep neural network and classical features-based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications, https:// doi. org/10.1007/s11042-020-08852-3. Iftikhar, S., Fatima, K., Rehman, A., Almazyad, A. S., & Saba, T. (2017). An evolution-based hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research, 28(8), pp. 3451–3455. Iqbal, S., Ghani, M. U., Saba, T., & Rehman, A. (2018). Brain tumor segmentation in multispectral MRI using convolutional neural networks (CNN). Microscopy Research and Technique, 81(4), pp. 419–427. DOI:10.1002/jemt.22994. Iqbal, S., Khan, M. U. G., Saba, T., Mehmood, Z., Javaid, N., Rehman, A., & Abbasi, R. (2019). Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microscopy Research and Technique. https://doi.org/10.1002/ jemt.23281 Iqbal, S., Khan, M. U. G., Saba, T., & Rehman, A. (2017). Computer assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, 8(1), pp. 5–28. DOI:10.1007/s13534-017-0050-3. Jamal, A., Hazim Alkawaz, M., Rehman, A., & Saba, T. (2017). Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17), pp. 799–811. https://doi.org/10.1002/jemt. Javed, R., Rahim, M. S. M., & Saba, T. (2019a). An improved framework by mapping salient features for skin lesion detection and classification using the optimized hybrid features. International Journal of Advanced Trends in Computer Science and Engineering, 8(1), pp. 95–101. Javed, R., Rahim, M. S. M., Saba, T., & Rashid, M. (2019b). Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), pp. 1–10.

80

Artificial Intelligence and Internet of Things

Javed, R., Rahim, M. S. M., Saba, T., & Rehman, A. (2020a). A comparative study of features selection for skin lesion detection from dermoscopic images. Network Modeling Analysis in Health Informatics and Bioinformatics, 9(1), p. 4. Javed, R., Saba, T., Shafry, M., & Rahim, M. (2020b). An intelligent saliency segmentation technique and classification of low contrast skin lesion dermoscopic images based on histogram decision. In 2019 12th International Conference on Developments in eSystems Engineering (DeSE) (pp. 164–169). DOI:10.1109/DeSE.2019.00039. Khan, M. A., Akram, T., Sharif, M., Javed, K., Raza, M., & Saba, T. (2020a). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, pp. 1–30. DOI:10.1007/s11042-020-08726-8. Khan, M. A., Akram, T., Sharif, M., & Saba, T. (2020c). Fruits diseases classification: exploiting a hierarchical framework for deep features fusion and selection. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-09244-3 Khan, M. A., Akram, T., Sharif, M., Saba, T., Javed, K., Lali, I. U., Tanik, U. J., & Rehman, A. (2019d). Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion. Microscopy Research and Technique, 82(5), pp. 741–763. http://doi.org/10.1002/jemt.23220. Khan, M. A., Ashraf, I., Alhaisoni, M., Damaševičius, R., Scherer, R., Rehman, A., & Bukhari, S. A. C. (2020d). Multimodal brain tumor classification using deep learning and robust feature selection: a machine learning application for radiologists. Diagnostics, 10, p. 565. Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., & Rehman, A. (2019a). Multi-model deep neural network-based features extraction and optimal selection approach for skin lesion classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–7). IEEE. DOI:10.1109/ICCISci.2019.8716400. Khan, M. A., Lali, I. U., Rehman, A., Ishaq, M., Sharif, M., Saba, T., Zahoor, S., & Akram, T. (2019c). Brain tumor detection and classification: a framework of marker based watershed algorithm and multilevel priority features selection. Microscopy Research and Technique, 82(6), pp. 909–922. https://doi.org/10.1002/jemt.23238 Khan, M. A., Sharif, M., Akram, T., Raza, M., Saba, T., & Rehman, A. (2020b). Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing, 87, p. 105986. Khan, M, A., Sharif, M., Javed, M, Y., Akram, T., Yasmin, M., Saba, T. (2018). License number plate recognition system using entropy-based features selection approach with SVM, IET Image Processing 12 (2), 200–209. Khan, M. A., Sharif, M. I., Raza, M., Anjum, A., Saba, T., & Shad, S. A. (2019b). Skin lesion segmentation and classification: A unified framework of deep neural network features fusion and selection. Expert Systems, p. e12497. https://doi.org/10.1111/exsy.12497 Khan, M. W., Sharif, M., Yasmin, M., & Saba, T. (2017). CDR based glaucoma detection using fundus images: a review. International Journal of Applied Pattern Recognition, 4(3), pp. 261–306. Khan, M. Z., Jabeen, S., Khan, M. U. G., Saba, T., Rehmat, A., Rehman, A., & Tariq, U. (2020e). A realistic image generation of face from text description using the fully trained generative adversarial networks. IEEE Access, 9, pp. 1250–1260. Khan, S. A., Nazir, M., Khan, M. A., Saba, T., Javed, K., Rehman, A., … & Awais, M. (2019e). Lungs nodule detection framework from computed tomography images using support vector machine. Microscopy Research and Technique, 82(8), pp. 1256–1266. Khuriwal, N. & Mishra, N. (2018, October). Breast cancer diagnosis using deep learning algorithm. In 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN) (pp. 98–103). IEEE. DOI:10.1109/ ICACCCN.2018.8748777.

Breast Cancer Detection and Diagnostic Analysis

81

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097–1105). https://doi.org/10.1145/3065386 Liaqat, A., Khan, M. A., Sharif, M., Mittal, M., Saba, T., Manic, K. S., & Al Attar, F. N. H. (2020). Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Current Medical Imaging, 16(10), pp. 1229–1242. Lu, Y., Li, J. Y., Su, Y. T., & Liu, A. A. (2018, December). A review of breast cancer detection in medical images. In 2018 IEEE Visual Communications and Image Processing (VCIP) (pp. 1–4). IEEE. DOI:10.1109/VCIP.2018.8698732. Lung, J. W. J., Salam, M. S. H, Rehman, A., Rahim, M. S. M., & Saba, T. (2014). Fuzzy phoneme classification using multi-speaker vocal tract length normalization. IETE Technical Review, vol. 31 (2), pp. 128–136. DOI: 10.1080/02564602.2014.892669. Majid, A., Khan, M. A., Yasmin, M., Rehman, A., Yousafzai, A., & Tariq, U. (2020). Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microscopy Research and Technique, 83(5), pp. 562–576. Marie-Sainte, S. L., Aburahmah, L., Almohaini, R., & Saba, T. (2019a). Current techniques for diabetes prediction: review and case study. Applied Sciences, 9(21), p. 4604. Marie-Sainte, S. L., Saba, T., Alsaleh, D., Alotaibi, A., & Bin, M. (2019b). An improved strategy for predicting diagnosis, survivability, and recurrence of breast cancer. Journal of Computational and Theoretical Nanoscience, 16(9), pp. 3705–3711. Martel, S., Maurer, C., Lambertini, M., Pondé, N., & De Azambuja, E. (2017). Breast cancer treatment-induced cardiotoxicity. Expert Opinion on Drug Safety, 16(9), pp. 1021–1038. Mashood Nasir, I., Attique Khan, M., Alhaisoni, M., Saba, T., Rehman, A., & Iqbal, T. (2020). A hybrid deep learning architecture for the classification of superhero fashion products: an application for medical-tech classification. Computer Modeling in Engineering & Sciences, 124(3), pp. 1017–1033. Masoud, V. & Pagès, G. (2017). Targeted therapies in breast cancer: new challenges to fight against resistance. World Journal of Clinical Oncology, 8(2), p. 120. Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A., & Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), p. 1068. Moon, W. K., Huang, Y. S., Hsu, C. H., Chien, T. Y. C., Chang, J. M., Lee, S. H., Huang, C. S., & Chang, R. F. (2020). Computer-aided tumor detection in automated breast ultrasound using a 3-D convolutional neural network. Computer Methods and Programs in Biomedicine, 190, p. 105360. Motallebnezhad, M., Aghebati-Maleki, L., Jadidi-Niaragh, F., Nickho, H., Samadi-Kafil, H., Shamsasenjan, K., & Yousefi, M. (2016). The insulin-like growth factor-I receptor (IGF-IR) in breast cancer: biology and treatment strategies. Tumor Biology, 37(9), pp. 11711–11721. Mughal, B., Muhammad, N., Sharif, M., Rehman, A., & Saba, T. (2018a). Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC Cancer, 18(1), p. 778. Mughal, B., Muhammad, N., Sharif, M., Saba, T., & Rehman, A. (2017). Extraction of breast border and removal of pectoral muscle in wavelet domain. Biomedical Research, 28(11), pp. 5041–5043. Mughal, B., Sharif, M., Muhammad, N., & Saba, T. (2018b). A novel classification scheme to decline the mortality rate among women due to breast tumor. Microscopy Research and Technique, 81(2), pp. 171–180. Nazir, M., Khan, M. A., Saba, T., & Rehman, A. (2019). Brain tumor detection from MRI images using multi-level wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). DOI:10.1109/ICCISci.2019.8716413.

82

Artificial Intelligence and Internet of Things

Norouzi, A., Rahim, M. S. M, Altameem, A., Saba, T., Rada, A. E., Rehman, A., & Uddin, M. (2014). Medical image segmentation methods, algorithms, and applications. IETE Technical Review, 31(3), pp. 199–213. DOI: 10.1080/02564602.2014.906861. Osman, A. H. & Aljahdali, H. M. A. (2020). An effective of ensemble boosting learning method for breast cancer virtual screening using neural network model. IEEE Access, 8, pp. 39165–39174. Perveen, S., Shahbaz, M., Saba, T., Keshavjee, K., Rehman, A., & Guergachi, A. (2020). Handling irregularly sampled longitudinal data and prognostic modeling of diabetes using machine learning technique. IEEE Access, 8, pp. 21875–21885. Qureshi, I., Khan, M. A., Sharif, M., Saba, T., & Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images. International Journal of Intelligent Systems Technologies and Applications, 19(1), pp. 1–16. DOI:10.1504/IJISTA.2020.105172. Rad, A. E., Rahim, M. S. M, Rehman, A., Altameem, A., & Saba, T. (2013). Evaluation of current dental radiographs segmentation approaches in computer-aided applications. IETE Technical Review, 30(3), pp. 210–222. Rad, A. E., Rahim, M. S. M, Rehman, A., & Saba, T. (2016). Digital dental X-ray database for caries screening. 3D Research, 7(2), pp. 1–5. DOI: 10.1007/s13319-016-0096-5. Ragab, D. A., Sharkas, M., Marshall, S., & Ren, J. (2019). Breast cancer detection using deep convolutional neural networks and support vector machines. PeerJ, 7, p. e6201. Rahim, M. S. M., Norouzi, A., Rehman, A., & Saba, T. (2017a). 3D bones segmentation based on CT images visualization. Biomedical Research, 28(8), pp. 3641–3644. Rahim, M. S. M., Rehman, A., Kurniawan, F., & Saba, T. (2017b). Ear biometrics for human classification based on region features mining. Biomedical Research, 28(10), pp. 4660–4664. Ramzan, F., Khan, M. U. G., Iqbal, S., Saba, T., & Rehman, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, pp. 103697–103709. Ramzan, F., Khan, M. U. G., Rehmat, A., Iqbal, S., Saba, T., Rehman, A., & Mehmood, Z. (2020a). A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), p. 37. Rao, C. S. (2019). Exploration and evaluation of efficient preprocessing and segmentation technique for breast cancer diagnosis based on mammograms. International Journal of Research in Pharmaceutical Sciences, 10(3), pp. 2071–2081. Rasti, R., Teshnehlab, M., & Phung, S. L. (2017). Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recognition, 72, pp. 381–390. Rehman, A., Abbas, N., Saba, T., Mahmood, T., & Kolivand, H. (2018c). Rouleaux red blood cells splitting in microscopic thin blood smear images via local maxima, circles drawing, and mapping with original RBCs. Microscopic Research and Technique, 81(7), pp. 737–744. DOI:10.1002/jemt.23030. Rehman, A., Abbas, N., Saba, T., Mehmood, Z., Mahmood, T., & Ahmed, K. T. (2018b). Microscopic malaria parasitemia diagnosis and grading on benchmark datasets. Microscopic Research and Technique, 81(9), pp. 1042–1058. DOI:10.1002/jemt. 23071. Rehman, A., Abbas, N., Saba, T., Rahman, S. I. U., Mehmood, Z., & Kolivand, K. (2018a). Classification of acute lymphoblastic leukemia using deep learning. Microscopy Research & Technique, 81(11), pp. 1310–1317. DOI: 10.1002/jemt.23139. Rehman, A., Khan, M. A., Mehmood, Z., Saba, T., Sardaraz, M., & Rashid, M. (2020a). Microscopic melanoma detection and classification: a framework of pixel-based fusion and multilevel features reduction. Microscopy Research and Technique, 83(4), pp. 410–423. DOI: 10.1002/ jemt.23429.

Breast Cancer Detection and Diagnostic Analysis

83

Rehman, A., Khan, M. A., Saba, T., Mehmood, Z., Tariq, U., & Ayesha, N. (2020b). Microscopic brain tumor detection and classification using 3D CNN and feature selection architecture. Microscopy Research and Technique. DOI:10.1002/jemt.23597. Ribli, D., Horváth, A., Unger, Z., Pollner, P., & Csabai, I. (2018). Detecting and classifying lesions in mammograms with deep learning. Scientific Reports, 8(1), pp. 1–7. Roslidar, R., Rahman, A., Muharar, R., Syahputra, M. R., Arnia, F., Syukri, M., Pradhan, B., & Munadi, K. (2020). A review on recent progress in thermal imaging and deep learning approaches for breast cancer detection. IEEE Access, 8, pp. 116176–116194. Roslidar, R., Saddami, K., Arnia, F., Syukri, M., & Munadi, K. (2019, August). A study of fine-tuning CNN models based on thermal imaging for breast cancer classification. In 2019 IEEE International Conference on Cybernetics and Computational Intelligence (CyberneticsCom) (pp. 77–81). IEEE. DOI:10.1109/ CYBERNETICSCOM.2019.8875661. Saba, T. (2017). Halal food identification with neural assisted enhanced RFID antenna. Biomedical Research, 28(18), pp. 7760–7762. Saba, T. (2019). Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique, 82(9), pp. 1601–1609. Saba, T. (2020). Recent advancement in cancer detection using machine learning: systematic survey of decades, comparisons and challenges. Journal of Infection and Public Health, 13(9), pp. 1274–1289. Saba, T., Al-Zahrani, S., & Rehman, A. (2012). Expert system for offline clinical guidelines and treatment. Life Science Journal, 9(4), pp. 2639–2658. Saba, T., Bokhari, S. T. F., Sharif, M., Yasmin, M., & Raza, M. (2018b). Fundus image classification methods for the detection of glaucoma: a review. Microscopy Research and Technique, 81(10), pp. 1105–1121. DOI: 10.1002/jemt.23094. Saba, T., Haseeb, K., Ahmed, I., & Rehman, A. (2020a). Secure and energy-efficient framework using Internet of Medical Things for e-healthcare. Journal of Infection and Public Health. DOI:10.1016/j.jiph.2020.06.027. Saba, T., Khan, M. A., & Rehman, A., & Marie-Sainte, S. L. (2019a). Region extraction and classification of skin cancer: a heterogeneous framework of deep CNN features fusion and reduction. Journal of Medical Systems, 43, p. 289. https://doi.org/10.1007/ s10916-019-1413-3 Saba, T., Khan, S. U., Islam, N., Abbas, N., Rehman, A., Javaid, N., & Anjum, A. (2019c). Cloud based decision support system for the detection and classification of malignant cells in breast cancer using breast cytology images. Microscopy Research and Technique, 82(6), pp. 775–785. Saba, T., Mohamed, A. S., El-Affendi, M., Amin, J., & Sharif, M. (2020b). Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, pp. 221–230. Saba, T., Rehman, A., Mehmood, Z., Kolivand, H., & Sharif, M. (2018a). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), pp. 704–715. DOI: 10.2174/15734056136661709121 64546. Saba, T., Sameh, A., Khan, F., Shad, S. A., & Sharif, M. (2019b). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43(12), p. 332. Sadad, T., Munir, A., Saba, T., & Hussain, A. (2018). Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. Journal of Computational Science, 29, pp. 34–45. Saikia, A. R., Bora, K., Mahanta, L. B., & Das, A. K. (2019). Comparative assessment of CNN architectures for classification of breast FNAC images. Tissue and Cell, 57, pp. 8–14.

84

Artificial Intelligence and Internet of Things

Simonyan, K. & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–9). DOI:10.1109/ CVPR.2015.7298594. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z., & Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique, 82(4), pp. 361–372. Wang, Z., Li, M., Wang, H., Jiang, H., Yao, Y., Zhang, H., & Xin, J. (2019). Breast cancer detection using extreme learning machine based on feature fusion with CNN deep features. IEEE Access, 7, pp. 105146–105158. Xiao, T., Liu, L., Li, K., Qin, W., Yu, S., & Li, Z. (2018). Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. BioMed Research International, 2018, pp. 1–9. Xu, Q., Wang, X., & Jiang, H., (2019). Convolutional neural network for breast cancer diagnosis using diffuse optical tomography. Visual Computing for Industry, Biomedicine, and Art, 2(1), p. 1. Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A. M., Alharbey, R., & Rashid, M. (2019a). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: a survey. BioMed Research International, 2019, pp. 1–26. Yousaf, K., Mehmood, Z., Awan, I. A., Saba, T., Alharbey, R., Qadah, T., & Alrige, M. A. (2019b). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, pp. 1–23. DOI:10.1007/s10729-019-09486-0. Yurttakal, A. H., Erbay, H., İkizceli, T., & Karaçavuş, S. (2019). Detection of breast cancer via deep convolution neural networks using MRI images. Multimedia Tools and Applications, pp. 1–19. DOI:10.1007/s11042-019-7479-6. Zou, L., Yu, S., Meng, T., Zhang, Z., Liang, X., & Xie, Y. (2019). A technical review of convolutional neural network-based mammographic breast cancer diagnosis. Computational and Mathematical Methods in Medicine, 2019. https://doi.org/10.1155/2019/6509357 Zuluaga-Gomez, J., Masry, Z. A., Benaggoune, K., Meraghni, S., & Zerhouni, N. (2019). A CNN-based methodology for breast cancer diagnosis using thermal images. arXiv preprint arXiv:1910.13757.

4

Automated Medical Image Analysis in Digital Mammography Mohsen Karimi, Majid Harouni, Shadi Rafieipour

CONTENTS 4.1 Introduction..................................................................................................... 85 4.2 Medical Image Analysis Methods in Different Organs.................................. 88 4.2.1 Tumor and Calcium Microparticle in the Chest.................................. 88 4.2.2 Anatomy of the Breast......................................................................... 89 4.2.3 Breast Cancer....................................................................................... 89 4.2.4 Mammogram Images...........................................................................90 4.2.5 Microparticles in Mammogram Images..............................................90 4.2.6 Diagnosis of Tumors and Calcium Microparticles..............................92 4.2.7 Retina and Retinal Blood Vessels........................................................96 4.2.8 Retinal Anatomy................................................................................ 101 4.2.9 Background of Research on Retinal Blood Vessels........................... 102 4.3 Evaluation...................................................................................................... 108 4.3.1 Image Database................................................................................. 109 4.4 Conclusion..................................................................................................... 110 References............................................................................................................... 111

4.1 INTRODUCTION Cancer is a type of disease that causes the growth of cells in a part of the body abnormally and excessively. These produced cells gather to form a mass. Cancers are classified in two types, benign and malignant; cancerous cells are fixed in benign type but in the malignant type, these cells are transferred to other parts of the body and provide the possibility of cancerous cells growth (Sadhukhan et al., 2020). Breast cancer is one of the most common types of cancer in women. The rate of dangerousness of a tumor in the breast is determined by considering criteria such as the size of the tumor, the extent of its spread and penetration into neighbor organs, and the rate of its progression in the chest. Calcium microparticles are formed before a tumor is produced in the breast, and it is one of the early warning signs of breast cancer (Mughal et al., 2017, 2018a,b). On the other hand, calcium microparticles are usually seen in mammographic images. Microparticles usually increase with age in women. These microparticles are usually benign and cannot endanger one’s health, but some microparticles are malignant and can be dangerous for a person’s health. Malignant 85

86

Artificial Intelligence and Internet of Things

microparticles can create serious problems for the patient over time, in such a way that they have the malignant breast tumors. In other word, these malignant microparticles can evolve into breast tumors by growing. Nearly 200,000 women and 1700 men are diagnosed with breast cancer each year, with more than 40,000 women and 450 men dying. Breast cancer is actually an abnormal growth of breast cells that creates a mass in the breast. Age is one of the most important factors in determining breast cancer. In addition to age, some other factors are race, geographical location, and family history (Rehman et al., 2018, 2019, 2020). Also, they are effective in the risk of breast cancer. One in eight women in the United States and one in six women in Europe have breast cancer. Despite the increasing incidence of this disease, statistics show a decrease in the death rate due to that and its reason is the introduction of new detection methods by the image processing and new treatment methods. One way to detect breast cancer is mammography test. In recent years, new methods based on machine learning and image processing are presented that in the detection of cancerous tumors and calcium microparticles in the breast based on image processing have reduced the need for manpower and significantly eliminated human error. More importantly, it has led to a reduction in medical costs, which has played an important role in the medical profession (Hernández et al., 2016). A mammogram can help in early diagnosis of cancer before a lump is felt in the breast. However, it is challenging for a radiologist to examine mammograms images and diagnose benign and malignant tumors. Many researchers believe that automatic analysis of mammographic images increases the rate of early diagnosis. One of the methods of automatic analysis of mammographic to diagnose cancer is the use of image-processing techniques and machine learning that so far, different methods with high detection rates in this field, have been presented and some of them have been introduced in Priyanka and Kulkarni (2016) and Aličković and Subasi (2017). However, there are challenges that can be addressed. • Gray surface charges in different parts of the image are slight. This makes it difficult to segment areas containing the tumor only through the gray surface. • The tumor is not always obvious. • The presence of high-frequency components and different levels of noise in the mammographic image make 10%–15% of the pixels. So far, different researches have been done in this field that each of them has strengths and weaknesses. Detection of breast cancer in digital mammography images is performed in four stages. These four stages include preprocessing, segmentation, feature extraction, and classification. The output of this process helps the radiologists to achieve more accurate results about the detection of breast cancer (Priyanka and Kulkarni, 2016). Because of the little changes in the gray surface in different parts of mammographic image, it is difficult to segment different areas of breast including tumor; so, in the preprocessing section, noise and additional information are removed and methods such as wavelet transform, multi-resolution analysis, area growth techniques, intermediate filter, adaptive intermediate filter, and Wiener filter (Ramani et al., 2013) are used usually to enhance and increase the image contrast

Automated Medical Image Analysis

87

to provide higher quality zoning in later steps. In the segmentation stage, the image for selection of the desired objects is divided into several areas without overlapping (Harouni et al., 2014; Rehman et al., 2020). This stage is of particular importance because the suspicious area that may contain a mass is removed from the main tissue (Mohammadi Dashti and Harouni, 2018). Suspicious areas are usually brighter than the surrounding areas. Although these borders are completely random, their regular shape and monotonous density are recognizable compared to other tissues. Because the features of masses differ from one image to another, the segmentation phase is difficult and complex (Habibi and Harouni, 2018; Raman et al., 2011). Middle filter, texture analysis, and nearest neighbor algorithm are methods that used in this phase. Retina is one of the important members of the visual system of the human body. It is a thin-cell layer that is located on the back of eyeball and its main task is to convert light into nerve signals. User cooperation is essential for retinal identification that the user should remove their glasses and put their eyes near the scanner and stare at a particular point for 75 seconds in order to extract the retinal patter (Khan et al., 2019, 2021; Waheed et al., 2016). Identification of a retinal pattern is performed by projecting nonvisible infrared beams in the eye. Light beams follow a standard route. Because of that, retinal blood vessels adsorb light easier than the surrounding tissue, and the amount of the reflection is different in during retina scan. The structure of these vessels is known as a necessary sign for diagnosis of ophthalmology and cardiovascular diseases (Moghaddam et al., 2019). Diseases such as glaucoma (blood sugar) and diabetic retinopathy are identified by retina. The properties of retina vessels such as length, width, maze, branch pattern, and angles affect the results of diagnosis of diseases. However, the manual division of the retina vessels requires high skill that needs to rapid analysis of retinal vessels (Sekou et al., 2019). On the other hand, the pattern of blood vessels in each eye is unique to each person, and even this pattern is different in identical twins, although it may change due to some diseases such as glaucoma, diabetes, and autoimmune deficiency syndrome. It is one of the basic steps in the use of retinal blood vessels in the processes of detecting the zoning of blood vessels in the retina. The shape and structure of blood vessels in retinal images lay an important role in detecting the disease or identity identification. It is usually used to diagnose type 1 diabetes that leads to blindness and also it is used in identity identification. The traditional and primary methods of edge recognition cannot segment the vessels in the retina with high accuracy. These methods include matching filter (MF) (Marín et al., 2010), segmentation based on the rigid methods (Staal et al., 2004), edge identification methods based on image geometry (Marín et al., 2010), huff conversion (Hossein-Nejad and Nasri, 2018), and wavelet transform (Leandro et al., 2001). Because most of traditional methods of segmenting blood vessels are based on MF, it is necessary to improve this filter. The classic method of MF has advantages like simplicity and efficiency. In this method, the cross-section of the vessel is modeled as a Gaussian function. So, a set of Gaussian filters will be identified and discovered (Nur and Tjandrasa, 2018). The main disadvantages of the filter methods are strong response to vessels and non-vessels, so that the effective non-blood vessels are detected after filtering. The main purpose of this work is a comprehensive review of zoning methods in difference medical images. The innovation of this research can be represented as follows. Introduction and survey of

88

Artificial Intelligence and Internet of Things

the different organ’s anatomy of body in order to use it in image processing are also given. The presentation of a comprehensive overview of all medical image analysis methods in all body organs including the brain, lungs, liver, chest, and retina are available in Amin et al. (2019), Ejaz et al. (2018, 2020), Iqbal et al. (2017, 2018, 2019). • A review of common imaging techniques for retinal blood vessel analysis. • A comprehensive review of research on the diagnosis of breast tumors on mammographic images. • An overview of evaluation criteria in analyzing medical images. • A comprehensive review of the performed researches in the detection and segmentation of calcium particles in mammographic images. • Introducing the different kinds of databases in the analyzing of medical images in the chest and retina. In the following, this research work is divided as follows. In Section 4.2, the common tools in medical image analysis will be presented. In Section 4.3, the anatomies of each breast and also the retina are completely introduced, and the new methods in medical image analysis are also presented. In this section, common databases are introduced. In Section 4.4, the conclusion of the chapter will be presented.

4.2 MEDICAL IMAGE ANALYSIS METHODS IN DIFFERENT ORGANS Medical image analysis methods based on image types and organ types may be different with each other in their efficiency (Fahad et al., 2018; Javed et al., 2019, 2020). For example, however, the morphological algorithms are used as a tool in nonregulatory methods (Harouni et al., 2010), and the variety of uses of this method may be different in each organ. Or they used in the regulatory methods of support vector machine (SVM) classifiers, but the type of their uses may be different based on the image type (Saba et al., 2018; Tahir et al., 2019). Brain tumors in magnetic resonance images, pulmonary tumors in CT scan images, liver tumors in CT scan images, tumors and calcium microparticles in mammography images, and blood vessels in the retina could be detected (Saba et al., 2020b). At the end this chapter, well-known databases have been introduced in each category along with evaluation criteria.

4.2.1 Tumor and Calcium Microparticle in the Chest Calcium decrease can be one of the symptoms of the breast cancer in women. These microparticles are usually seen in mammography images in the form of bright and they are often safe, and in the other word, they are benign type, but in some cases, it is the malignant ones that can lead to breast cancer if is not treated. Breast cancer is one of the most common types of cancer among women. The probability of breast cancer increases with rising age, as 80% of cases occur after age of 50. At first breast cancer is a small gland and becomes a large cancerous gland during several years and engages the surrounding tissues. Sometimes early diagnosis of small glands accompanied with the microparticles of calcium improves the chance of success in

Automated Medical Image Analysis

89

surgery significantly. Because of that, there is no symptom in the early stages, so it is essential to evaluate that by the early diagnosis methods of breast cancer. Early diagnosis of this disease can significantly decrease the rate of mortality. If the small objects can be diagnosed by the analysis of medical images in the breast tissue, early diagnosis of this disease can be helpful. If these objects are small, they can be easily seen by a radiologist, but if this automatic diagnosis is done by the method based on machine vision and learning ability, the accuracy of diagnosis will be increased. So, at first in this chapter of the research, the breast anatomy is introduced. Then, the breast cancer as well as calcium microparticles will be presented. Then, the algorithm used in research based on proposed method will be introduced.

4.2.2  Anatomy of the Breast Basics first, the breasts have a dark part called the areola, and there is a prominent tip in the center of areola called the nipple. There is a network of milk-dedicated ducts next to adipose tissue under the nipples that are immature until puberty and are supported by a connective tissue called stroma, and when girls reach puberty, the female hormones secretion rate (estrogen and progesterone) increases and breasts grow and evolve. Stroma is multiplied during the puberty and the milk ducts evolve, and a large network forms in the chest and also in the inner part of the breast, and lobules grow like a bud. The lobules are small sacs that produce milk after child birth. The milk that is produced by millions of lobules directs to the ducts and finally to the nipple (Mitchell et al., 2019). Each breast consists of 15–20 parts that surround the nipple in the form of a wheel. Each of these blades is called a lobe. Each lobe has smaller section called lobules that lead to a small mammary gland the produce milk during breast feeding. The network of lobules eventually connects to ducts and gathers at the nipple and directs milk out during lactation period. The breast cancer often forms in the ducts, then in the lobules and in the third degree in other tissues (Mangel et al., 2019). The nipple is located in the center of the breast and the colored part surrounding it is called areola. The color of areola is different in the women and is also varying due to hormonal changes like pregnancy and menstruation. The nipple is protuberant in some women and slightly sunken in others. The areola secretes a fatty substance that softens the nipple during breastfeeding. Each breast has a network of blood vessels (arteries and veins) that are responsible for delivering nutrient and oxygen and excreting feces.

4.2.3 Breast Cancer Breast cancer is one of the most important and common diseases in women, and because of the importance of the disease, it is necessary for women to know the basic information about this field even if they do not have this disease. It should be noted that most of breast masses are not cancerous and their treatment does not always lead to the removal of the breast, and there is a chance for real improvement with new treatment methods. Breast cancer is the most common cause of deaths in women between the ages of 35 and 55. The breast tissue is in vicinity to the muscles around the breast (Couch et al., 2019). Understanding the lymphatic system of chest

90

Artificial Intelligence and Internet of Things

is important in terms of diagnosis and treatment as breast tumors can spread through the lymphatic system and reach the whole of body, and in fact, the lymphatic system is a way for lymph to communicate with blood vessels. Blood supply to the inner and central parts of breast is performed by arterial branches. Also, the breast has a nervous system.

4.2.4 Mammogram Images Mammography is a simple X-ray image of the breast and a tool for early detection of untouchable breast cancers. Mammography can detect breast cancers 10 years before it becomes touchable. One of the most useful ways to detect breast cancers and tumors is to use mammograms (Li et al., 2019). The use of mammogram images to identify tumors has the following advantages: • • • •

This technique is able to detect intangible lesions (smaller than 1 cm). It takes about 20 minutes. In this technique, two pictures are prepared from each breast. In 5%–10% of the cases, there are false-negative results (too much or little breast tissue density). • Reduces breast cancer deaths by up to 30%. • Diagnose cancer before it can be touched, so: • Less toxic effects of treatments • Treatments are more effective • Keep beauty • Less involvement probability of armpit lymph nodes. Mammography is a type of medical imaging based on X-ray with low dose that its resulting image shows the inner tissues of the breast. The quality of mammographic images may not be good because of the lack of adjustment of the image in suitable defocus, malfunction of the camera sensor or low light, for example the contrast of the images is low or it has nonuniform illumination. Therefore, in order to provide a proper processing and analysis, image improvement must first be considered. Because of that, the quality of the images is not very good, so the use of these images will not always be able to diagnose cancer. For this reason, processing methods are necessary for the use of these images (Lestari and Sumarlinda, 2019).

4.2.5 Microparticles in Mammogram Images The particles are usually seen on mammographic images. As the age increases, the density and number of these microparticles increase. Many of these calcium fine particles do not endanger person’s health, and in other words, they are benign, but some of these fine particles can create dangerous patterns, and therefore, they create the malignant tissues and densities. It is important to distinguish between these microparticles or to identify malignant and benign particles. It can be said that more than 55% of breast cancers start from these stiff particles. The particles are usually produced by the manifestation of ductal carcinoma in site. This calcium is the main factor of Distant

Automated Medical Image Analysis

91

Cell Signalling Intercellular Communication (DSIC) and also is in the immaterial part of breast carcinoma cancer. Benign calcium can be commonly seen, and they should be monitored; however, they are not dangerous. While suspected malignant calcium particles are small, they must be considered with high accuracy. Detection and separation of calcium microparticle into benign and malignant group are determined by the chemical compound in each benign microparticle which is usually composed of calcium oxide, but malignant microparticles are formed from calcium phosphate. It should be noted that separation of these two masses is not possible without surgery, and pathology and biopsy tests are needed to identify and distinguish them. The use of digital mammography images and the identification and separation of these two are possible with the help of methods based on machine learning and image processing. Several researches have been performed on microparticles separation since 2008, and their results have shown the effect of imageprocessing methods (Vigeland et al., 2008). The ultrasound imaging method cannot detect fine particles well, and by this imaging method (Khan et al., 2019, 2020; Nazir et al., 2019), only different large tissues in the breast are identified as large tissues or nodules or cysts, and even if they are identified, their separation is not possible (Radiology and D’Orsi, 2013). Morphology descriptors and, of course, their distribution can be used to separate these tissues, but it seems that the combination of these two can be more effective. These descriptors are identified as the formers of calcium in the breast, and in proportion to their being malignant distribution, these descriptors are as important as morphological descriptors. Figure 4.1 shows the type

FIGURE 4.1  Dense breast tissue: (a) fatty, (b) scattered fibroglandular, (c) heterogeneously dense, (d) extremely dense

92

Artificial Intelligence and Internet of Things

of microparticles distribution in the image. In the following, these distributions are described (Hernández et al., 2016).

4.2.6 Diagnosis of Tumors and Calcium Microparticles The use of artificial neural network and K-means algorithm for breast cancer diagnosis has been presented. This system consists of two stages. In the clustering section, the input data are first improved by combining the colonial competition algorithm and the K-means algorithms, which performs the clustering. Then, Euclidean distance of each pattern of determined cluster is calculated. The classification section determines the membership of each pattern by using of the calculated distance. In this section, several neural networks such as Multilayer Perceptron (MLP) neural network, probabilistic neural networks and radial-base neural networks are used (Kalteh et al., 2013). Mohamed et al. proposed the method of histogram uniformity and morphological operators in the preprocessing section. Also, Otsu’s thresholding method was proposed in the segmentation of the desired areas in mammographic images. For the extraction of tissue properties from image, the gray surface event matrix method was used. Finally, the multilayer perceptron neural network, K-nearest neighbors, and support vector machine were used in the classification section. However, in this research, the function of classifiers in the diagnosis of breast cancer has been compared, but they do not have a high degree of accuracy in diagnosis malignancy with mass benignity, and the accuracy rate is approximately 70% (Mohamed et al., 2014). The Otsu’s technique is used to segment the chest muscle in the mammogram image, and the arc edge detection and the straight-line approximation technique are then used to remove neck muscle. In the next step after the matrix, the gray event is used to extract the property, and finally, the support vector classifier is trained to classify cancerous and noncancerous tissues. The proposed method is performed on the mini-MIAS database (Wang et al., 2009). Aličković and Subasi used genetic algorithm to extract features in identifying cancerous tumors. The proposed method uses various data mining techniques to identify the mass. In this method, after extracting the property by genetic algorithm, different data mining methods are used to detect and segment the mass. Because of that, the genetic algorithm inherently searches the problem space to local. In the proposed methods, the features based on image texture such as dispersion wavelet and MLP neural network have been used for classification (Aličković and Subasi,2017). Wang et al. (2009) suggested a method for the automatic detection of breast cancer in mammographic images by using the supported vector method. Preprocessing on image has been performed by using of middle filter and improving the image by filtering. Support vector machine methods with several kernels have been suggested for classification. In this chapter, considering the data structures in the training set, a large structured margin machine called the structure support vector machine, the class problem structure can be formulated as a quadratic conical programming problem, so it will be solvable more efficiently. A new improved segmentation is performed in order to identify the type of area in the breast including tumor and its type more accurately. Experimental results show that the structure support vector machine generally has a better performance in detection compared to the standard support vector

Automated Medical Image Analysis

93

machine (Waheed et al., 2016). Olfati et al. presented a system for diagnosing the breast cancer, in which, first, all main components analyzed to reduce the dimensions of appropriate attributes, and then classified by SVM classifier. The main drawback of this classifier is its genetic approach for learning SVM that searches the problem space in global search not in local search (Olfati et al., 2014). Gayathri et al. used different data mining methods to diagnose breast cancer. Their proposed system consists of two stages. In the first stage, in order to eliminate additional properties, to reduce computational complexities and to develop the mining data process, the genetic algorithm has been used to extract the important data and features. In the second stage, several data mining methods such as logical regression, decision trees, stochastic forests, Bayesian network, multilayer perceptron network, radial-base function network, support vector machine, and rotating forest have been used to detect the mass. Two features that are mainly used in the diagnosis of breast cancer are shape-based features and gene-data-based features. The used database set is WBCD (Gayathri et al., 2013). Otoom et al. have provided experiments in order to compare the performance of several classifiers such as support vector machine, multilayer perceptron network, radial function network, Bayesian, J48 decision tree, and random forest decision tree. Two groups of features include the spectral and textural features and geometric features (Harouni et al., 2010, 2012a). The results of their experiments have shown that the use features based on shape and geometry lead to a higher detection rate. The used database set is WBCD MGE (Otoom et al., 2015). Rejani and Selvi suggested a new method for separating cancerous tumors. In this method, tumor isolation is performed in several steps: (a) increasing the quality of mammographic images such as using a filter and discrete wavelet transform; (b) isolation of the area suspected having a tumor; (c) property extraction from the separated area; and (d) the use of the support vector machine classification method. This method has been performed on 75 mammography images from the mini-MIAS dataset that has a sensitivity of 88.75%. Although the obtained results are very good, it seems that the support vector machine classifier has not been able to perform the classification well (Rejani and Selvi, 2009). Naresh and Kumari have used a morphology operator to eliminate noise and increase the input image contrast in the preprocessing phase. In the suggested method, the complete local binary pattern operator CLBP has been used in order to extract the texture properties from the image that considers the sign size and the center of the gray surface values (Harouni et al., 2012b). Finally, the classification of the mass into two types, benign and malignant, has been done by the support vector classifier (Saba, 2020). In the proposed method, the complementary local binary pattern operator has been used to obtain more accurate results for diagnosis of breast cancer. The results of comparing the suggested methods with other methods have not reported in this chapter, and the results evaluation method is based on cross validation (Naresh and Kumari, 2015). The breast cancer diagnosis system provided by Hiremath and Prasannakumar consists of three main stages: in the first stage, in order to reduce the processing time, the image changes from red, green, and blue space to a gray surface image, and the image is then standardized in size, and the its extra information is deleted like noise. After that, the morphology operator is used to segment the image in order to select areas suspected of having tumor. To reduce the effects of light on the image, Gaussian

94

Artificial Intelligence and Internet of Things

filters and the differences of Gaussian filters in images segmentation are applied; in the second stage, the properties are extracted by the systematic center method of local binary patterns; in the third stage, the extracted properties are classified into two types of normal and abnormal images by classification of the support vector machine. The database set used is mini-MIAS (Hiremath and Prasannakumar, 2015). In summary, in the proposed method, the use of Center Symmetric Local Binary Pattern (CSLBP) for feature extraction and SVM for classification, the use of Gaussian filter and Difference of Gaussians (DOG) filter in order to reduce brightness changes in the image have been presented. In this chapter, the proposed method is not compared with different kernels and other classification methods. In the suggested method, the features based on image texture such as wavelet and MLP neural network are used for classification, and the method of evaluating the results is based on cross validation (Biswas et al., 2016). This work used the extraction process of desired area to delete the artifacts and also used the two-dimensional middle filter to remove noise, and they improved images by using the histogram adjustment algorithm correspondent with limited contrast. In the next step, a gray event is used to extract the properties, and finally, they used K-nearest neighbors, support vector machine, and neural network for classification that have reached 95% accuracy in all three classifications. The proposed method is performed on the mini-MIAS dataset. Antony and Ravi have proposed a two-step system for diagnosis breast cancer. In first step, the noise of image is deleted by using the Gabor filter, and it is normalized by the histogram method. In second stage, the Gaussian filter is applied to the images. The filtered images are converted to binary images, and properties such as stiffness, centrifugal, convex area, direction, environment, small axis length, and large axis length are extracted. These properties are extracted and sent to the K-means clustering algorithm (Antony and Ravi, 2015). Valvano et al. segmented the calcium microparticles, which is known as first step in cancer process. The suggested method is taught on 196 images and tested on 52 images. Mammogram images are segmented by using deep learning methods. Preprocessing is done by wavelet filters. The accuracy of segmentation in the proposed method is about 83%. This method is simulated on the HD5F database (Valvano et al., 2017). Sangeetha and Murthy used a combination of effective methods for the medical images segmentation to identify calcium in the breast. Although the proposed method follows the complete pattern of the classification, there is one major difference that is the use of a clustering in the segmentation section. After clustering, statistical properties are extracted from each cluster. Finally, these obtained feature vectors for classification are classified by support vector machine classifier. The only innovation of the suggested method is the use of clustering in the segmentation stage and feature extraction from these areas (Sangeetha and Murthy, 2017). Velikova et al. used Bayesian neural network to segment calcium microparticles. The proposed method creates a logical relation between the low-level properties and, of course, the high-level properties, for example the relationship between calcium particles and large volumes in the image. This relationship occurs with the probability of distribution in Bayesian networks. The established logical connection is used for training the neural network, and finally, the particles are identified (Velikova et al., 2015). Chen et al. presented a new method following the microparticle in breast tumors in an article. At first, each calcium

Automated Medical Image Analysis

95

microparticle is obtained by using the topology fuzzy clustering algorithms in the proposed method. Then a graph is obtained from all the obtained microparticles, and a general graph is extracted from all the multiscale topologies with necessary properties. The multiscale topological property vector is created from calcium particle graphs set, and finally, they are classified by the K-nearest neighbors classifier. The proposed method is simulated on the MIAS database. The accuracy of the proposed method is that region of interest (ROI) for selection of calcium microparticles is selected manually (Chen et al., 2012b). In the study, by Hailey et al., a combination of K-means algorithm and fuzzy K-means clustering method has been used in order to diagnose breast cancer. In the preprocessing phase, a middle filter and a discrete wavelet transform have been used in order to delete the noise and high-frequency components in the image. After that, the breast cancer has been detected by the extraction of tissue property and sending them to K-means algorithm and fuzzy C-means clustering method (Hailey and Marshall, 1995). In 2012, Chen et al. proposed a combination of Particle Swarm Optimization (PSO) and SVM for the diagnosis of cancerous tumors. In the proposed method, the properties based on texture image such as scattering wavelet and also neural network have been used, and then the relevant properties will be selected by PSO. In order to have a better balance in the local and general search into PSO algorithm, the TVAC method has been used in proposed method. Also, the optimization of SVM parameters such as the used windows in this classifier has been performed in order to increase the accuracy in classification. In this chapter, the comparison of proposed method accuracy with other methods has not been performed, and this method is a combination of SVM and base PSO algorithm (Chen et al., 2012a). In 2015, Gc et al. proposed geometric and morphological properties for the detection of the cancerous tumors. In the presented method, the SVM classifier is used to classify the image. In the proposed method, the image-based features have been used to improve the performance of the classifier by extracting geometric properties and obtaining the morphological unique features of tumor and also the features based on image texture. In this method, although the computational cost is low and it is not very complicated, the tumor boundaries are determined with low accuracy, and also the tumor type is not separated (Gc et al., 2015). Kekre et al. have suggested the separating by using the gradual vector method in order to diagnose cancer on mammography images in their article. The prepared method uses LBG algorithm to separate mammographic images. In this method, a code book of 128 pieces is first produced for mammographic image. Feature vectors extracted from images in eight clusters are clustered using the LGB algorithm. In this algorithm, the center of the first cluster will be the first vector. By considering a threshold rate for the error and finishing the algorithm, at the first step, eight clusters were obtained based on the minimum distance between two vectors (code). In next step, the first cluster is considered as the first center in second step, then the three clusters are clustered in the four clusters. This process continues until two clusters are reached in the image. The two obtained clusters will be healthy clusters and image tumor. Finally, the diffusion algorithm is used to separate the tumor area. Although this method is simple and efficient, it seems that the exact boundaries of the tumor are not recognizable (Kekre et al., 2009). Urooj et al. have used quasiZernic properties to detect breast cancer in a study. In this method, the ROI is first

96

Artificial Intelligence and Internet of Things

extracted from the image manually. Then, the textural properties of the image are extracted by using quasi-Zernic descriptor and in the final step. The feature vectors extracted from the image are divided into three classes of health, benign and malignant by using the support vector machine classifier, but this research has two main weaknesses: first, the area is selected manually and the method is not fully automated, and second, the exact boundaries of tumor area are not achieved (Urooj et al., 2018). Abaspur et al. in a study used the coordinate logic filters that are a group of nonlinear digital filters that are constructed by logic operators. These operators are used for AND, OR, and XOR filters and their combination. They were able to separate the mass from the other parts of the image completely, and their quality and brightness increased to determine the location and size of the mass in a mammogram clearly and with high accuracy. The presented method is very effective in reducing human errors in detecting mass on images, and several images were received from the MIAS mammography database that was analyzed by the proposed model; the obtained result is very acceptable, and its speed and accuracy are higher than the presented models in authentic articles (Abaspur et al., 2009). Table 4.1 shows the comparison of this method. The suggested method is completely described along with its advantages and disadvantages.

4.2.7 Retina and Retinal Blood Vessels Retina is one of the important members of the visual system in the human body. Retina is a thin-cell layer that is located on the back of eye ball, and its task is to change the light into neural signals. User cooperation is necessary for retina identification as the users should take off their glasses and put the eyes near the scanner and stare at a particular point for 15 seconds and then the retina pattern to be extracted (Khan et al., 2019 ; Waheed et al., 2016). The retinal pattern is obtained by projecting nonvisible infrared beams in the eye. The light beams follow a standard route. Because of that, retinal blood vessels absorb light easier than the surrounding tissue, and the amount of reflection is different during scanning retina. The structure of these vessels is very complex (Moghaddam et al., 2019). Also, the structure of retina vessels is known as essential for diagnosis of ophthalmology and cardiovascular disease. Diseases such as glaucoma (blood sugar) and diabetic retinopathy are diseases that are identified by studying the retina. The attributes of retina vessels such as length, width, maze, branch pattern, and angles affect the results of disease diagnosis. However, the manual division of the retina vessels needs high skill that creates a lot of demand for fast analysis of retinal vessels images (Sekou et al., 2019). On the other hand, the blood vessels pattern of each eye is unique for each person, and even this pattern is different in identical twins. While it may be changed due to some disease such as glaucoma, diabetes, and self-immune deficiency syndrome, the retinal structure remains fixed during the life. The function of biometric systems based on retina has high accuracy, because this structure is not fake, and retinas of dead people rot quickly and identification system cannot be deceived by them. Therefore, this property in persons can be used in identifying by the biometrics. It can be seen as a great biometric. The main purpose of this research is to present a new method in order to detect the retinal blood vessels. Therefore, this chapter will present the necessary preparations to detect and identify blood vessels (Saba et al., 2020a).

97

Automated Medical Image Analysis

TABLE 4.1 The Comparison of Research Background Ref. Kalteh et al. (2013)

Mohamed et al. (2014)

Wang et al. (2009)

Aličković and Subasi (2017)

Proposed Method Use of artificial neural network and K-means algorithm to diagnose breast cancer. This system consists of two stages: clustering and classification Histogram uniformity method and morphological operators in preprocessing section, Otsu’s thresholding method in segmentation stage, extraction of texture properties from image

Advantages High accuracy in identifying tumor boundaries, automation of the algorithm Low computational cost, low computational overhead as well as algorithm automation

The proposed method includes three main steps:

High computational accuracy

• Chest division • Neck muscle removal • Classification of breast muscle in mammogram image into cancerous and noncancerous Use of genetic algorithm to extract features in the identification of cancerous tumors, classification with MLP neural network

Waheed et al. (2016)

Automatic detection of breast cancer in mammographic images by support vector machine method

Olfati et al. (2014)

Using the principal component analysis method to reduce the dimensions of the features, the genetic algorithm to select the appropriate features and the support vector machine for classification are used

Algorithm automation, suitable computational cost

Second-degree conical programming formulation, better identification of the tumor area Appropriate accuracy of identifying tumor boundaries in breast images

Disadvantages Algorithm complexity, high computational overhead In diagnosing malignancy with mass benignity, they do not have a high degree of accuracy, the degree of accuracy is approximately 70% Algorithm complexity, high computational overhead

Genetic algorithm inherently searches the problem space in general, has performed poorly locally in the problem space search High computational overhead, complexity of support vector machine classification optimization algorithm The disadvantage of this method is that the genetic algorithm inherently searches the problem space in general and is weak in local search (Continued)

98

Artificial Intelligence and Internet of Things

TABLE 4.1 (Continued) The Comparison of Research Background Ref. Gayathri et al. (2013)

Otoom et al. (2015)

Rejani and Selvi (2009)

Naresh and Kumari (2015)

Hiremath and Prasannakumar (2015)

Proposed Method The proposed system consists of two stages, in the first stage, in order to eliminate additional features, it reduces computational complexity and speeds up the data mining process Experiments have been performed to compare the performance of different classifiers such as support vector machine, multilayer perceptron network, radial function network, Bayesian, J48 decision tree, and random forest decision tree Separation method in steps 1. Increasing the quality of mammographic images such as using a filter and discrete wavelet transform 2. Isolation of the area suspected of having a tumor 3. Extracting a feature from the isolated area 4. Using the support vector car classification method Using the morphology operator to remove the complementary local binary pattern operator CLBP in order to extract the texture properties from the image, which indicates the classification of the mass into two types, benign and malignant, by the support vector classifier Use CSLBP center symmetric local binary patterns for feature extraction and SVM for classification, use Gaussian and DOG filters to reduce brightness changes in the image

Advantages Improved detection accuracy, increased performance efficiency

Disadvantages The complexity of the algorithm, the use of the genetic algorithm to search locally

The results of their experiments have shown that the use of features based on shape and geometry leads to a higher detection rate. Comparison of methods Computational complexity, semiautomation of the algorithm

Failure to provide a new method

Results evaluation method based on cross validation

The results of comparing the proposed method with other methods are not reported in this article

Results evaluation method based on cross validation

The comparison of the proposed methods has not been done by different kernels of the proposed method with other classification methods

Although the results obtained are very good, it seems that the support vector machine classifier has not been able to perform the classification well

99

Automated Medical Image Analysis

TABLE 4.1 (Continued) The Comparison of Research Background Ref. Biswas et al. (2016)

Proposed Method Removal of artifacts from the desired region extraction process, gray event co-matrix for feature extraction, finally from K-nearest neighbors, support vector machine, and neural network for classification

Antony and Ravi (2015)

Two-stage system: in the first stage, noise removal with Gabor filter. In the second step, application of Gaussian filter to the image, extraction of stiffness, centrifugal, convex area, direction, circumference, small axis length, and large axis length and K-means clustering Use of geometric and morphological features in the identification of cancerous tumors

Zhu et al. (2017)

Valvano et al. (2017)

Use of convolutional neural networks in segmentation of calcium particles in mammographic images

Sangeetha and Murthy (2017)

They used a combination of effective medical imaging segmentation methods to identify fine calcium fragments in the breast

Velikova et al. (2015)

The use of Bayesian neural networks to segment calcium microparticles is made from a logical connection, to train the Bayesian neural network, and finally the microparticles are identified

Advantages Appropriate accuracy of identifying tumors and microparticles, the use and evaluation of different classifiers Algorithm automation, the obtained good results

Disadvantages The proposed method lacks particular innovation in the identification of microparticles as well as tumors The complexity of the algorithm

The low computational cost and the suitable computational complexity

In the proposed method, although the computational cost is very low and not very complicated, the tumor boundaries are determined with low accuracy, and the tumor type is not separated Proper accuracy in This study only identifying identifies microparticles microparticles and does not detect cancer from it The proposed method Algorithm follows the complete complexity, low pattern of the accuracy classification process, the automation of the algorithm Algorithm The complexity automation, good of the algorithm accuracy

(Continued)

100

Artificial Intelligence and Internet of Things

TABLE 4.1 (Continued) The Comparison of Research Background Ref.

Proposed Method

Chen et al. (2012b)

The use of fuzzy topology The accuracy of the clustering to identify each proposed method calcium particulate is the use of is 95% K-nearest neighbors in the classification

Hailey and Marshall (1995)

Chen et al. (2012a)

Kekre et al. (2009)

Urooj et al. (2018)

Abaspur et al. (2009)

Advantages

Disadvantages

The disadvantage of the proposed method is that the initial region of interest for selecting calcium particles is manually selected Using a combination of K-means New hybrid method, High complexity, algorithm and fuzzy C-means good accuracy high computational clustering method to diagnose cost breast cancer The combination of PSO and Proper accuracy, In this paper, the SVM has been used to detect automation of the accuracy of the cancerous tumors to better algorithm proposed method is balance local and global search not compared with in the PSO algorithm other proposed methods with a combination of SVM and the basic PSO algorithm Using the gradient vector The method is very The proposed method of LBG algorithm in simple and method is not able the proposed method efficient to detect the exact boundaries of the tumor Use of Zernik-like features to Proper accuracy, The use of this study detect breast cancer unchangeable has two major torques in cancer weaknesses. First, detection the area is selected manually and the method is not fully automated, and second, the exact boundaries of the tumor area are not obtained Use coordinate logic filters to The result is very Computational reduce human error acceptable, and its complexity, high speed and accuracy computational are higher than the overhead models presented in reputable articles

Automated Medical Image Analysis

101

In fact, the human retina is a physical characteristic that is examined in this research. Low variations in the vessel pattern during life, high security, and more stability are features that exist in the retina images. These properties convert the retina into a powerful tool for identifying people (Sabaghi et al., 2012). There were different main stages in retinal detection systems. This identification includes identify recognition or diagnosis of the disease. Usually, in many researches, the preprocessing stage and removing noise have been performed in order to improve quality and image upgrade. One of the basic steps in the use of retinal blood vessels in the identification processes of the blood vessels zoning in retina. The shape and structure of blood vessels in retina images play an important role in disease diagnosis or identity recognition. Usually it is used in diagnosis of type 1 diabetes that leads to blindness and also it is useful in identity recognition. The traditional and basic methods of edge identification cannot segment the vessels in the retina with high accuracy. These methods are the MF (Marín et al., 2010), segmentation based on rigid methods (Staal et al., 2004), edge identification methods based on image geometry (Marín et al., 2010), Huff conversion (Hossein-Nejad and Nasri, 2018) and wavelet conversion (Leandro et al., 2001). Because many of the traditional methods of segmentation of retinal blood vessels are based on the MF, it is essential to improve this filter. The classical method of MF has advantages such as simplicity and effectiveness. In this method, the cross-section of the vessel is modeled in the form of a Gaussian function. Therefore, a set of Gaussian filters will be identified and discovered (Nur and Tjandrasa, 2018). The main weak point of the MF method is that MF has a strong response to vessel, and non-blood vessels are identified after filtering. One of the most important problems during imaging from the retina is the natural movements of the human head and eyes. Therefore, finding the solution to solve this problem has been the most important concern that has been worked on in the past. Furthermore, the extracted images from the retina usually have low contrast and in terms of lighting are nonuniform and have two light and dark areas in retinal image that is a serious obstacle against features extraction from these images (Willoughby et al., 2010). Detection of ineffective blood vessels will reduce the accuracy of identifying the desired algorithms and classifiers.

4.2.8 Retinal Anatomy The retina is the innermost layer of the eye and contains light-receiving cells and neurons. This very thin layer (about 0.5 mm diameter) covers 75% of the eyeball area. The retina forms the light-sensitive layer and gives the brain the ability to see by converting the electromagnetic current of light into a nervous massage and transferring it through optic nerve to occipital lobe. Retinal light-receiving cells are of two types: cone cells that enable the brain to see the colors in bright light (Ramzan et al., 2020a,b). The rate of light-sensitive substances in the dendrite of these cells is less than cylindrical cells. This deficiency is one of the causes of cone sensitivity to light. Cylindrical cells have larger dendrite than cone cells. Cylindrical cells make it possible to see in low light. Cylindrical cells are simulated in low light, and it seems that they are highly sensitive substance in cylindrical cells in comparison

102

Artificial Intelligence and Internet of Things

FIGURE 4.2  Internal structure of the eye: retina position

to cone cells is due to their high sensitivity. Each of these two types of receivers is specialized for a specific purpose. The bars are made for night vision. These cells act in low light and cause colorless sensations. Cones are more suitable for daylight. These cells respond to intense and cause a feeling of color. Cone cells are about 6 million in number and help us to see several colors and bar (rod) receivers that are approximately 125 million, help us to see surrounding environment. This is a pattern of blood vessels in the retina. Figure 4.2 shows the position of the retina. It is seen that the cornea is located in front of the eye, and the retina is located at the end of the eye. Because of that, the retina is located inside the eye (Figure 4.3), and it is not in the front of environment outside the eye; it is considered as a method of stable identification identity or disease diagnosis (Heriot, 2019). Figure 4.4 shows the close-up of the blood vessels pattern inside the eye. Red lines show the blood vessels and the yellow part indicates the location of optical disk (a place that optic nerve connects to retina and information is sent from the eye to the brain at this location). A circle that you see is a place where the device is scanned in order to extract feature.

4.2.9 Background of Research on Retinal Blood Vessels Retinal images analysis is one of the basic steps in the development and progression identification systems and also the medical systems in identifying disease such as retinopathy. The pattern of retinal blood vessels and its appearance features such

Automated Medical Image Analysis

103

FIGURE 4.3  Posterior view of the ciliary body

as length and width of the vessels, branches, and the angle between the branches play an important role in the diagnosis and treatment of eye diseases and also cardiovascular diseases and in the identification by retinal biometric. The processing of initial images of retina has low accuracy and speed, so the extraction of blood pattern from the initial images was proposed. Automatic extraction of retinal blood vessels pattern has become particularly important. Many methods were proposed for automatic extraction of blood vessels pattern that the latest methods will be examined in the following.

FIGURE 4.4  Blood vessels in the eye anatomy (left), an original sample blood vessel inside the eye (right)

104

Artificial Intelligence and Internet of Things

Nazari and Pourghassem presented a study that considers the rotation and deviation of the human identity recognition algorithm based on a new definition from the geometric shape of the retinal raw images features by using a hierarchical matching structure (Nazari and Pourghassem, 2017). In this algorithm, the retina images are surrounded by areas that covered with blood vessels, and this surrounded region is called SR. A complete set of area-based and border-based features is defined on SR. New properties such as SR corner angle, central distance, and weight corner angle were defined in the border features by defining SR corner points that can determine boundary changes rate and SR geometry. A query has been used to match SR with the recognition SR in the database and the extracted features in a hierarchical structure are implemented from the simpler features among the more complex features for filtering the registered SR in database in order to reduce the research space. Finally, candidate SR matching with query SR estimates the identification or rejection of the query image with proposed decision scenario, identification is performed when at least two SRs from the query are same with two SRs from one person in the database. The proposed algorithm was evaluated in STAR and DRIVE databases in six different experiments and was obtained with 700% accuracy. In this study, the computational complexity of the research was reduced and detection function was improved. A retinal detection system without vessels was proposed based on retinal detection system and dimensions were evaluated by using different database and their results were obtained using different performance criteria (Waheed et al., 2016). These methods depend on the number of people and validation modes, processing time, and EER rate and accuracy. To extract the features, they take two images and compare them at similar times. The brightness of I (x,y) is a measure of brightness average for two candidate images x,y. Contrast function c (x,y) is a standard measure of the deviation of x,y images. Then, they combine brightness and contrast to obtain the size of the structure. To match the performance of the used experimental optimization for generating one similar score between two candidate images, they used 34 items for the experiment and obtained an identification rate of 92.5. Dehghani et al presented an algorithm including feature extraction, phase correction, and feature matching (Dehghani et al., 2011). First, they extracted the points of the arteries corners by applying the HARRIS algorithm on the retina images as a feature and then they applied a phase correction and estimated the angle of the image rotation by applying the wavelet transformation, they defined a model function based on the angle difference between each corner point of an image with other corner points that extracted from the same image in the polar coordinate system, and these model functions were compared to match the query image and they matched the query image with its corresponding image in the database by placing a threshold. The proposed algorithm was tested on a database consisting of 40 images from the DRIVE database and 40 images from the STAR database that each of them was rotated six times randomly and achieved 100% performance with an average time of 3.5 seconds per image. Moreover, a new method has been suggested to identify individuals base on the matching of retinal fundus images. The purpose of image capture involves a multiscale-dependent recording that followed by multiscale elastic recording. The main advantage of this particular method is the two-stage image recording

Automated Medical Image Analysis

105

that is able to be considered for both hard and non-hard shape changes with attention to the inherent retinal texture. Then, a decision identification criterion is defined based on a normalized function to decide whether the pair of images belongs to one person or not? The method was tested on database of 21,721 real pairs generated from a collection of 946 retinal funds images that have been taken from 339 different persons including patients with various eye diseases and healthy persons. Its performance evaluation shows that its false rejection rate (FRR) is very low at zero false acceptance rate (FAR) and is equal to 0.084 and the equal error rate (EER) is low and equal to 0.053. In addition, performance testing by only dependent multiscale recording and regardless of multiscale elastic record clearly indicates the advantage of the proposed method. In Lajevardi et al. (2013), first, they used the adapted filters in the frequency space and the morphological operators in order to extract vessels network. After that, they obtained the structural features such as branch points and transition points from the extracted vessels skeleton. To remove the points that were extracted incorrectly, the path of the vessels connected to them was examined, and if this path was short, those points were removed. Then the retina graph was defined by using the feature vectors as g = (v,e,m,v). In this graph, the graph vertices or the sequence of V were considered of vectors containing the extracted transition points and branch points, and the sequence of E or graph edges were consisted of a vector containing each pair of vertices that connected to a vessel in the skeleton image of the vessels. In the defined retina graph, m represented the labeling function of the vertices of the graph, so that each vertex of the maintained graph was mapped to its Cartesian coordinates in the retina image with coordinates (q1, q2). A biometric graph matching (BGM) algorithm is used for matching that is resistant to displacement and to some extent nonlinear perturbation and small rotations. They reduced the error due to erroneously extracted vertices by defining a cost function in the fault tolerance algorithm. In Zhu et al. (2017), a monitoring method based on ELM classifier (extreme learning mechanism) was used for separating of retinal vessels. First, a set of 39-D diagnostic feature, morphological properties, phase adaptation, Hisin and vector fields divergence for each pixel were removed from the fundus image, then they created a matrix for the pixel of the training set based on feature vector and manual markup and used it as ELM classifier input. The output of the classifier is the separation of the binary retinal vessels. Finally, they performed the optimization process to clear the region of less than 30 pixels that is separated from the retinal vessels. The experimental results on digital retinal images from DRIVE database showed that this method is faster than arteries separating methods. At the same time, the mean accuracy and sensitivity were 0.9607 and 0.7140, respectively. A regulatory method is presented based on a fully symmetric and pretrained network, through transfer learning (Jiang et al., 2018). This proposed method simplifies the problem of usual division of retinal vessels from image segmentation with full size to the vessel area identifying element and merging results. In the meantime, the additional unsupervised image past-processing was performed for the proposed method that determines the final results. Extensive experiments were performed in DRIVE, STAR, CHASE-DB1, and HRF database, and the accuracy of the mutual databases in these four databases improved that present the strength of the proposed method. These

106

Artificial Intelligence and Internet of Things

successful results help not only in the area of auto retinal vessels segmentation but also in supporting the effectiveness of learning transfer when the deep learning method is applied to medical images. In Barkhoda et al. (2011), after extracting the vessels, the binary images of vessels were zoned by using two angular and circular segmentation methods. Then, by counting the pixels of vessel in each part and each ring, they formed two feature vectors. To match, the Manhattan distance between two feature vectors was calculated for the query image and the image examined in order to increase the performance of the system, by defining a fuzzy integration system, the obtained distances related to both feature vector are combined to achieve 99.75% performance. In this research, it has been tried to eliminate the sensitivity of feature vectors to rotation by using circular zoning and applying February transform on feature vectors. Also, an attempt has been made to reduce the effect of local movement in the images by considering smaller areas, but because of that the center of these divisions is fixed, this algorithm does not have enough resistance against the natural movement of the eye. The mentioned algorithm was tested on a database consisting of DRIVE database image that each of which has been rotated 11 times and achieved to 99.75% performance. Also, in Bevilacqua et al. (2008), after extracting the vessels, they extracted structural features including transition points and branching as features by windowing on the image of the extracted vessels. By examining the image in detail and selecting square windows that overlap, they reduce the chances of losing the desired points located at border points between the windows in the feature extraction process. Feature vectors extracted from the query image were compared with the images examined in the database by calculating the distance of the extracted points that are connected by the Huff transform with the corresponding points in the examined image and forming an accumulation matrix. Finally, the identification was performed by introducing a similarity criterion and by using the Robert centered window and where the maximum value due to the windowing is the furthest from the minimum value. This algorithm was tested on a database containing 12 images of 10 people and achieved 100% performance. Furthermore, an extraction process was performed by using the extracted blood vessels images from retina (Xu et al., 2006). First, they used the green image of the retina and found the structure curve of blood vessels, and then, they obtained feature vector using same feature points (such as vessel intersections) and directions for each image. In this feature matching method, affine conversion parameters that give the most similarity between input images and existing images have been used. This algorithm has a relatively good interest and low fault rate (FAR) and its biggest disadvantage is the high computational cost. This algorithm was performed on a database containing 200 images and the number of incorrect recognitions of image in it is zero and the number of incorrect rejected images (FAR) is 38 images. An algorithm has been improved computationally in Ortega algorithm in order to upgrade the mentioned algorithm (Ortega et al., 2009). In this method, blood vessels are first removed and then the location of the optical disc is determined by using a circular Huff conversion. To eliminate the rotation of the images, the reference point specification and triangular similarity have been used to calculate the rotation compensator conversion parameters. In following, it examines the points in a circle with r radius around the optical disc and

Automated Medical Image Analysis

107

considers the starting point as the optical disc and describes the feature vector by the information of the blood vessel network. Some of the used features in vessels network are the starting point of the vessels, the end area of the vessels and the branching points of the vessels. The method used in this algorithm requires extensive costly and time-consuming calculations. The proposed method in Köse and İki (2011) counts the number of similar vessels in a processing line between two images. Because of that, the scale change is independent, and the identified image is examined with parallel lines around the main processing line to consider multiple scales and in order not to be sensitive to rotation; these lines around the main lines are considered as cross-to-vertex lines with different angles and each line is examined. The corresponding number of vessels that are along the processing line in the main image and the input image is counted. This algorithm has two measurements for similarity that one of them counts the matching vessels and calculates the rate of similarity for the sample image and the original image, and the maximum match value is calculated for all sampled lines for identification. This method was performed on about 400 retina images and 80 images from the STAR database and had a 95% efficiency. The extracted vessels were examined by using the nested segmentation method and the box counting algorithm. The principles of this algorithm are based on applying nested hypercube on the examined information and counting the occupied pieces in each network cell. In this way, the pixel of the vessel in each block was examined by applying nested segmentation and dividing the image into k block counter and the gray surface of extracted points. At each stage, only points were considered that had a certain distance from each other and the feature vectors were obtained from the main difference gray surfaces in each case (Harouni et al., 2012c). The mentioned algorithm is applied on a database containing 40 images from DRIVE database that they achieved 98.33% performance accuracy with an average time of 3.1 seconds per image. A fast and accurate method has been proposed for segmenting retinal vessels that is effective in diagnosis many eye diseases (Farzin et al., 2008). The retina image that contains different low contrasts weakness the performance of the segmentation process. In order to remove noise, the independent component analysis (ICA) is widely used and includes two structures, ICA2 and ICA1. This study evaluated both ICA structures on retinal funds color images and selected each one that is presented the upgraded values of contrast. For retinal funds images, the ICA2 structure has shown better performance than ICA1, as regards it is more effective in low contrast values. The impact of the proposed segmentation model on the DRIVE and STARE databases was detected. In the case of DRIVE database, the sensitivity was 72%–75% and it was 3% higher than in other studies, and the classification accuracy was estimated about 96%. Optical discs, masses and the spots in the retina (Soomro et al., 2018) were identified simultaneously, first, the morphological operators such as open and close and erosion operators were applied to the retina image by a 3 × 3 square kernel that in the image created by this method, the background pixels have been separated. The rate of light in the retinal green images is uniformed; therefore, the rate of contrast masses and clear glands is increased. In the next step, by relying on digital curvelet conversion (DCUT), the bright spots of the image are kept on the larger coefficients of the curvelet field, thus

108

Artificial Intelligence and Internet of Things

any function and change on the curvelet coefficient cause changes in the bright spots of the image, which contain the optical disc and the masses. The researchers reached this conclusion by examining the considered feature and evaluating the clustering results that more than 50% of the points are correctly identified. During a few steps, the identified boundary is corrected and it gets closer to the real boundary. For example, by optimizing the used K-means clustering algorithm, the accuracy rate of the proposed algorithm is increased. Finally, by implementing the algorithm used in this chapter on 100 experimental images, the optical disk of 94% of the images is correctly zoned. An effective method for extracting blood vessels from retinal color images has been proposed (Shahbeig and Pourghassem, 2013). Applying the brightness equalizer function in this chapter adjusts the brightness of images significantly. Because of the high ability of multiscale conversion of curvelet in introducing image edges in different scales and directions, in this chapter, the edges and the contrast and quality of retinal images are enhanced using local and adaptive correction, of curvelet coefficients upgraded by the introduced modification function, and these images are prepared for the extraction stage of blood vessels. Because the scattering of blood vessels in retinal images in different directions, morphological operators with weighted adaptive construction elements have been used to extract blood vessels. Morphological operators based on geodetic transformations are a good choice for removing surplusage that are smaller in size than image capillaries. Finally, all remaining surplusage in image are removed by analyzing the images interconnected components and applying adaptive filters locally on these components. The suggested algorithm in this chapter has been evaluated by the images in the DRIVE datable. The implementation results show that the proposed algorithm has achieved a high accuracy of 97.11% in the DRIVE database. The implementation results show the high ability and accuracy of the proposed algorithm in extracting blood vessels from retinal images.

4.3 EVALUATION In the process of segmentation of retinal blood vessels, output and results are evaluated for pixels. Each pixel will have two modes; either it is within the vein or it is not within the vein and is considered the background component (Harouni and Baghmaleki, 2020). In this case, four different modes will be created. Two correct classification modes and two incorrect classification modes are as follows: TP: The pixels in the main image and the GT image are identified as veins (pixel is not vessel). TN: The pixels in the main image and the GT image are not recognized as veins (pixels are not veins). FN: In this case, the corresponding pixel in the GT image is a vessel, but it is not detected in the image for vessel segmentation. FP: In this case, the vessel is detected in the fragmented image but it is not vessel in the GT image.

109

Automated Medical Image Analysis

TPR: A correct positive description indicates the percentage of pixels that have been correctly identified as a vessel (Equation (4.1)).

TPR =

FP (4.1) Total Number Of  Vessel Pixels

FPR: A false-positive description (see Equation (4.2)) shows the percentage of pixels that have been incorrectly identified as a vein.

FPR =

FP (4.2) total number of  non-vessel pixels

ACC: The accuracy of the pixels is the total number ratios that are correctly classified (the sum of true-positive TP and false-positive TN) to the total number of pixels in image as shown in Equation (4.3) ACC =



TP + TN (4.3) TP + TN + FP + FN

SN: The competency indicates the ability of the proposed method to detect vascular pixels. SP: The ability of the proposed method to detect non-vein pixels as defined by Equation (4.4), which is also defined as 1 − FPR. SP =



TP (4.4) TP + FN

PPV: The ability of the proposed method to detect non-vein pixels is also defined as 1 − FPR ROC: The ROC curve is the ratio of correctly classification vascular pixels or TPR based on FPR or the classified nonvascular pixels are incorrect. If the curve is near to the upper left, the obtained results are better. AVC: The higher this value, the better the segmentation result.

4.3.1  Image Database The DRIVE database with 40 TIFF and colored images is one of the most common databases for retinal blood vessel detection. Images are divided into two groups of training and testing. There are 40 images in this database that called ground truth for evaluation and the obtained results will be compared with these images. Figure 4.5 shows two images from the DRIVE database with the corresponding ground truth image.

110

Artificial Intelligence and Internet of Things

FIGURE 4.5  Different images database of blood vessels

4.4 CONCLUSION The main purpose of this study was a comprehensive review of segmentation methods in medical images. In this study, medical images of magnetic resonance imaging. CT scan, mammography and also colored images were examined for segmentation of brain, lung, liver, and breast tumors and calcium granules as well as retinal vessels. Segmentation methods were classified into two group: regulatory and nonregulatory methods and in each member, based on the type of proposed researches image was examined. Evaluation criteria and the well-known and useful databases that are used in the segmentation of these images were introduced.

Automated Medical Image Analysis

111

REFERENCES Uncategorized References ABASPUR, K. I., GHAYUMIZADEH, H. & HADDADNIA, J. 2009. A Novel Smart System for Mass Detection in Mammographic Images. Iranian Quarterly Journal of Breast Disease, 2, 46–54. ALIČKOVIĆ, E. & SUBASI, A. 2017. Breast cancer diagnosis using GA feature selection and Rotation Forest. Neural Computing and Applications, 28, 753–763. AMIN, J., SHARIF, M., RAZA, M., SABA, T. & REHMAN, A. 2019b. Brain Tumor Classification: Feature Fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. ANTONY, S. J. S. & RAVI, S. 2015. A new approach to determine the classification of mammographic image using K-means clustering algorithm. International Journal of Advancements in Research & Technology, 4 (2). BARKHODA, W., AKHLAQIAN, F., AMIRI, M. D. & NOUROOZZADEH, M. S. 2011. Retina identification based on the pattern of blood vessels using fuzzy logic. EURASIP Journal on Advances in Signal Processing, 2011, 113. BEVILACQUA, V., CARIELLO, L., COLUMBO, D., DALENO, D., FABIANO, M. D., GIANNINI, M., MASTRONARDI, G. & CASTELLANO, M. Retinal fundus biometric analysis for personal identifications. International Conference on Intelligent Computing, 2008. Springer, 1229–1237. https://doi.org/10.1007/978-3-540-85984-0_147 BISWAS, R., NATH, A. & ROY, S. Mammogram classification using gray-level co-occurrence matrix for diagnosis of breast cancer. 2016 International Conference on MicroElectronics and Telecommunication Engineering (ICMETE), 2016. IEEE, 161–166. Doi: 10.1109/ICMETE.2016.85. CHEN, H.-L., YANG, B., WANG, G., WANG, S.-J., LIU, J. & LIU, D.-Y. 2012a. Support vector machine based diagnostic system for breast cancer using swarm intelligence. Journal of Medical Systems, 36, 2505–2519. CHEN, Z., OLIVER, A., DENTON, E., BOGGIS, C. & ZWIGGELAAR, R. 2012b. Classification of microcalcification clusters using topological structure features. Medical Image Understanding and Analysis, 37–42. COUCH, F., HU, C., HART, S., GNANAOLIVU, R., LILYQUIST, J., LEE, K., GAO, C., ECKLOFF, B., SAMARA, R. & KLEBBA, J. 2019. Abstract GS2-01: Age-related breast cancer risk estimates for the general population based on sequencing of cancer predisposition genes in 19,228 breast cancer patients and 20,211 matched unaffected controls from US based cohorts in the CARRIERS study. AACR. Cancer Research. Doi: 10.1158/1538-7445.SABCS18-GS2-01. DEHGHANI, A., MOGHADDAM, H. A. & MOIN, M.-S. Retinal identification based on rotation invariant moments. 2011 5th International Conference on Bioinformatics and Biomedical Engineering, 2011. IEEE, 1–4. Doi: 10.1109/icbbe.2011.5780312. EJAZ, K., RAHIM, M. S. M., BAJWA, U. I., CHAUDHRY, H., REHMAN, A. & EJAZ, F. 2021. Hybrid Segmentation Method With Confidence Region Detection For Tumor Identification, IEEE Access, vol. 9, 35256–35278, 10.1109/ACCESS.2020.3016627. EJAZ, K., RAHIM, M. S. M., REHMAN, D. A. & EJAZ, E. F. 2018. An Image-Based Multimedia Database and Efficient Detection though Features. VFAST Transactions on Software Engineering, 14(1), 6–15. FAHAD, H. M., KHAN, M. U. G., SABA, T., REHMAN, A. & IQBAL, S. 2018. Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microsc Res Tech. 81(5):449–457. doi: 10.1002/jemt.22998. FARZIN, H., ABRISHAMI-MOGHADDAM, H. & MOIN, M.-S. 2008. A novel retinal identification system. EURASIP Journal on Advances in Signal Processing, 2008, 280635.

112

Artificial Intelligence and Internet of Things

GAYATHRI, B., SUMATHI, C. & SANTHANAM, T. 2013. Breast cancer diagnosis using machine learning algorithms—a survey. International Journal of Distributed and Parallel Systems, 4, 105. GC, S., KASAUDHAN, R., HEO, T. K. & CHOI, H. D. Variability measurement for breast cancer classification of mammographic masses. Proceedings of the 2015 Conference on Research in Adaptive and Convergent Systems, 2015. ACM, 177–182. https://doi. org/10.1145/2811411.2811505 HABIBI, N. & HAROUNI, M. 2018. Estimation of re-hospitalization risk of diabetic patients based on radial base function (RBF) neural network method combined with colonial competition optimization algorithm. Majlesi Journal of Electrical Engineering, 12, 109–116. HAILEY, D. & MARSHALL, D. 1995. The place of magnetic resonance imaging in health care. Health Policy, 31, 43–52. HAROUNI, M. & BAGHMALEKI, H. Y. 2018. Color Image Segmentation Metrics. Encyclopedia of Image Processing, 95. HAROUNI, M., MOHAMAD, D., RAHIM, M. S. M. & HALAWANI, S. M. 2012a. Finding critical points of handwritten Persian/Arabic character. International Journal of Machine Learning and Computing, 2, 573. HAROUNI, M., MOHAMAD, D., RAHIM, M. S. M., HALAWANI, S. M. & AFZALI, M. 2012b. Handwritten Arabic character recognition based on minimal geometric features. International Journal of Machine Learning and Computing, 2, 578. HAROUNI, M., MOHAMAD, D. & RASOULI, A. Deductive method for recognition of online handwritten Persian/Arabic characters. 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE), 2010. IEEE, 791–795. Doi: 10.1109/ ICCAE.2010.5451869. HAROUNI, M., RAHIM, M., AL-RODHAAN, M., SABA, T., REHMAN, A. & AL-DHELAAN, A. 2014. Online Persian/Arabic script classification without contextual information. The Imaging Science Journal, 62, 437–448. HAROUNI, M., RAHIM, M., MOHAMAD, D., REHMAN, A. & SABA, T. 2012c. Online cursive Persian/Arabic character recognition by detecting critical points. International Journal of Academic Research, 4, 209–214. HERIOT, W. J. 2019. Method and Device for Treating Retinal Detachment. Google Patents. HERNÁNDEZ, P. L. A., ESTRADA, T. T., PIZARRO, A. L., CISTERNAS, M. L. D. & TAPIA, C. S. 2016. Breast calcifications: description and classification according to BI-RADS 5th Edition. Revista Chilena de Radiología, 22, 80–91. HIREMATH, B. & PRASANNAKUMAR, S. 2015. Automated evaluation of breast cancer detection using SVM classifier. International Journal of Computer Science Engineering and Information Technology Research (IJCSEITR), 5, 11–20. HOSSEIN-NEJAD, Z. & NASRI, M. 2018. A-RANSAC: adaptive random sample consensus method in multimodal retinal image registration. Biomedical Signal Processing and Control, 45, 325–338. IQBAL, S., KHAN, M. U. G., SABA, T. & REHMAN, A. 2017. Computer assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, vol.8(1), pp.5–28, doi. 10.1007/s13534-017-0050-3. IQBAL, S., GHANI, M. U., SABA, T. & REHMAN, A. 2018. Brain tumor segmentation in multi-spectral MRI using convolutional neural networks (CNN). Microsc Res Tech. 81(4):419–427. doi: 10.1002/jemt.22994. IQBAL, S., KHAN, M. U. G., SABA, T., MEHMOOD, Z., JAVAID, N., REHMAN, A. & ABBASI, R. (2019) Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation, Microscopy Research and Technique, 82(8)1302–1315, doi.org/10.1002/jemt.23281. JAVED, R., RAHIM, M. S. M., SABA, T. & RASHID, M. 2019. Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), 1–10.

Automated Medical Image Analysis

113

JAVED, R., RAHIM, M. S. M., SABA, T. & REHMAN, A. 2020. A comparative study of features selection for skin lesion detection from dermoscopic images, Network Modeling Analysis in Health Informatics and Bioinformatics 9 (1), 4. JIANG, Z., ZHANG, H., WANG, Y. & KO, S.-B. 2018. Retinal blood vessel segmentation using fully convolutional network with transfer learning. Computerized Medical Imaging and Graphics, 68, 1–15. KALTEH, A., ZARBAKHSH, P., JIRABADI, M. & ADDEH, J. 2013. A research about breast cancer detection using different neural networks and K-MICA algorithm. Journal of Cancer Research and Therapeutics, 9, 456. KEKRE, H., SARODE, T. K. & GHARGE, S. M. 2009. Tumor detection in mammography images using vector quantization technique. International Journal of Intelligent Information Technology Application, 2, 237–242. KHAN, A. R., KHAN, S., HAROUNI, M., ABBASI, R., IQBAL, S., & MEHMOOD, Z. 2021. Brain tumor segmentation using K-means clustering and deep learning with synthetic data augmentation for classification. Microscopy Research and Technique. 1–11. https://doi.org/10.1002/jemt.23694 KHAN, M. A., KHAN, T. M., NAQVI, S. S. & KHAN, M. A. 2019. GGM classifier with multi-scale line detectors for retinal vessel segmentation. Signal, Image and Video Processing, 1–9. https://doi.org/10.1007/s11760-019-01515-3 KHAN, M.A., SHARIF, M., AKRAM, T., RAZA, M., SABA, T. & REHMAN, A. 2020. Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing, 87, 105986. KÖSE, C. & IKI, C. 2011. A personal identification system using retinal vasculature in retinal fundus images. Expert Systems with Applications, 38, 13670–13681. LAJEVARDI, S. M., ARAKALA, A., DAVIS, S. A. & HORADAM, K. J. 2013. Retina verification system based on biometric graph matching. IEEE Transactions on Image Processing, 22, 3625–3635. LEANDRO, J. J., CESAR, J. & JELINEK, H. F. Blood vessels segmentation in retina: preliminary assessment of the mathematical morphology and of the wavelet transform techniques. Proceedings XIV Brazilian Symposium on Computer Graphics and Image Processing, 2001. IEEE, 84–90. Doi: 10.1109/SIBGRAPI.2001.963041. LESTARI, W. & SUMARLINDA, S. 2019. Application of mathematical morphology algorithm for image enhancement of breast cancer detection. Proceeding of ICOHETECH, 1, 187–189. LI, H., MENDEL, K. R., LAN, L., SHETH, D. & GIGER, M. L. 2019. Digital mammography in breast cancer: additive value of radiomics of breast parenchyma. Radiology, 291, 15–20. MANGEL, L., MIMOUNI, F. B., MANDEL, D., MORDECHAEV, N. & MAROM, R. 2019. Breastfeeding difficulties, breastfeeding duration, maternal body mass index, and breast anatomy: are they related? Breastfeeding Medicine, 14, 342–346. MARÍN, D., AQUINO, A., GEGÚNDEZ-ARIAS, M. E. & BRAVO, J. M. 2010. A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features. IEEE Transactions on Medical Imaging, 30, 146–158. MITCHELL, K. B., JOHNSON, H. M., EGLASH, A. & MEDICINE, A. O. B. 2019. ABM clinical protocol# 30: breast masses, breast complaints, and diagnostic breast imaging in the lactating woman. Breastfeeding Medicine, 14, 208–214. MOGHADDAM, A. E., AKBARIZADEH, G. & KAABI, H. 2019. Automatic detection and segmentation of blood vessels and pulmonary nodules based on a line tracking method and generalized linear regression model. Signal, Image and Video Processing, 13, 457–464. MOHAMED, H., MABROUK, M. S. & SHARAWY, A. 2014. Computer aided detection system for micro calcifications in digital mammograms. Computer Methods and Programs in Biomedicine, 116, 226–235. MOHAMMADI DASHTI, M. & HAROUNI, M. 2018. Smile and laugh expressions detection based on local minimum key points. Signal and Data Processing, 15, 69–88.

114

Artificial Intelligence and Internet of Things

MUGHAL, B., MUHAMMAD, N., SHARIF, M., REHMAN, A. & SABA, T. 2018a. Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC cancer, 18(1), 1–14. MUGHAL, B., MUHAMMAD, N., SHARIF, M., SABA, T. & REHMAN, A. 2017 Extraction of breast border and removal of pectoral muscle in wavelet domain. Biomedical Research, vol.28(11), pp. 5041–5043. MUGHAL, B., SHARIF, M., MUHAMMAD, N. & SABA, T. 2018b. A novel classification scheme to decline the mortality rate among women due to breast tumor. Microscopy research and technique, 81(2), 171–180. NARESH, S. & KUMARI, S. V. 2015. Breast cancer detection using local binary patterns. International Journal of Computer Applications, 123, 6–9. NAZARI, P. & POURGHASSEM, H. 2017. A novel retina-based human identification algorithm based on geometrical shape features using a hierarchical matching structure. Computer Methods and Programs in Biomedicine, 141, 43–58. NAZIR, M., KHAN, M. A., SABA, T. & REHMAN, A. (2019). Brain Tumor Detection from MRI images using Multi-level Wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). NUR, N. & TJANDRASA, H. 2018. Exudate segmentation in retinal images of diabetic retinopathy using saliency method based on region. Journal of Physics: Conference Series, 1108, 012110. OLFATI, E., ZARABADIPOUR, H. & SHOOREHDELI, M. A. Feature subset selection and parameters optimization for support vector machine in breast cancer diagnosis. 2014 Iranian Conference on Intelligent Systems (ICIS), 2014. IEEE, 1–6. Doi: 10.1109/ IranianCIS.2014.6802601. ORTEGA, M., PENEDO, M. G., ROUCO, J., BARREIRA, N. & CARREIRA, M. J. 2009. Retinal verification using a feature points-based biometric pattern. EURASIP Journal on Advances in Signal Processing, 2009, 2. OTOOM, A. F., ABDALLAH, E. E. & HAMMAD, M. 2015. Breast cancer classification: comparative performance analysis of image shape-based features and microarray gene expression data. International Journal of Bio-Science & Bio-Technology, 7, 37–46. PRIYANKA, B. & KULKARNI, D. 2016. Digital mammography: a review on detection of breast cancer. International Journal of Advanced Research in Computer and Communication Engineering, 5, 386–390. RADIOLOGY, A. C. O. & D’ORSI, C. J. 2013. ACR BI-RADS Atlas: Breast Imaging Reporting and Data System; Mammography, Ultrasound, Magnetic Resonance Imaging, Follow-up and Outcome Monitoring, Data Dictionary, ACR, American College of Radiology, Virginia, United States. RAMAN, V., SUMARI, P., THEN, H. & AL-OMARI, S. A. K. 2011. Review on mammogram mass detection by machine learning techniques. International Journal of Computer and Electrical Engineering, 3, 873. RAMANI, R., VANITHA, N. S. & VALARMATHY, S. 2013. The pre-processing techniques for breast cancer detection in mammography images. International Journal of Image, Graphics and Signal Processing, 5, 47. RAMZAN, F., KHAN, M. U. G., IQBAL, S., SABA, T. & REHMAN, A. 2020a. Volumetric Segmentation of Brain Regions from MRI Scans Using 3D Convolutional Neural Networks. IEEE Access, 8, 103697–103709. RAMZAN, F., KHAN, M. U. G., REHMAT, A., IQBAL, S., SABA, T., REHMAN, A. & MEHMOOD, Z. 2020b. A Deep Learning Approach for Automated Diagnosis and Multi-Class Classification of Alzheimer’s Disease Stages Using Resting-State fMRI and Residual Neural Networks. Journal of Medical Systems, 44(2), 37.

Automated Medical Image Analysis

115

REHMAN, A., HAROUNI, M. & SABA, T. 2020. Cursive multilingual characters recognition based on hard geometric features. International Journal of Computational Vision and Robotics, 10, 213–222. REHMAN, A., KHAN, M. A., MEHMOOD, Z., SABA, T., SARDARAZ, M. & RASHID, M. 2019. Microscopic melanoma detection and classification: A framework of pixelbased fusion and multilevel features reduction. Microsc Res Tech, 1–14. https://doi. org/10.1002/jemt.23429 REHMAN, A., SABA, T., MAHMOOD, T., MEHMOOD, Z., SHAH, M. & ANJUM, A. 2019. Data hiding technique in steganography for information security using number theory. Journal of information science, 45(6), 767–778. REJANI, Y. & SELVI, S. T. 2009. Early detection of breast cancer using SVM classifier technique. arXiv preprint arXiv:0912.2314. SABA, T., BOKHARI, S. T. F., SHARIF, M., YASMIN, M. & RAZA, M. 2018. Fundus image classification methods for the detection of glaucoma: A review. Microscopy research and technique, 81(10), 1105–1121. SABA, T., MOHAMED, A. S., EL-AFFENDI, M., AMIN, J. & SHARIF, M. 2020a. Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, 221–230. SABA, T., HASEEB, K., AHMED, I. & REHMAN, A. 2020b. Secure and energy-efficient framework using Internet of Medical Things for e-healthcare. Journal of Infection and Public Health, 13(10), 1567–1575. SABA, T. 2020. Recent advancement in cancer detection using machine learning: Systematic survey of decades, comparisons and challenges. Journal of Infection and Public Health, 13(9), 1274–1289. SABAGHI, M., HADIANAMREI, S. R., FATTAHI, M., KOUCHAKI, M. R. & ZAHEDI, A. 2012. Retinal identification system based on the combination of Fourier and wavelet transform. Journal of Signal and Information Processing, 3, 35. SADHUKHAN, S., UPADHYAY, N. & CHAKRABORTY, P. 2020. Breast cancer diagnosis using image processing and machine learning. Emerging Technology in Modelling and Graphics. Springer, Singapore, 2020. 113-127. SANGEETHA, R. & MURTHY, K. S. A novel approach for detection of breast cancer at an early stage by identification of breast asymmetry and microcalcification cancer cells using digital image processing techniques. 2017 2nd International Conference for Convergence in Technology (I2CT), 2017. IEEE, 593–596. Doi: 10.1109/ I2CT.2017.8226198. SEKOU, T. B., HIDANE, M., OLIVIER, J. & CARDOT, H. 2019. From patch to image segmentation using fully convolutional networks-application to retinal images. arXiv preprint arXiv:1904.03892. SHAHBEIG, S. & POURGHASSEM, H. 2013. Blood vessels extraction in retinal image using new generation curvelet transform and adaptive weighted morphology operators. Computational Intelligence in Electrical Engineering (Intelligent Systems in Electrical Engineering), 3, 63–76. SOOMRO, T. A., KHAN, T. M., KHAN, M. A., GAO, J., PAUL, M. & ZHENG, L. 2018. Impact of ICA-based image enhancement technique on retinal blood vessels segmentation. IEEE Access, 6, 3524–3538. STAAL, J., ABRÀMOFF, M. D., NIEMEIJER, M., VIERGEVER, M. A. & VAN GINNEKEN, B. 2004. Ridge-based vessel segmentation in color images of the retina. IEEE Transactions on Medical Imaging, 23, 501–509. TAHIR, B., IQBAL, S., KHAN. M. U. G., SABA, T., MEHMOOD, Z., ANJUM, A. & MAHMOOD, T. 2019. Feature enhancement framework for brain tumor segmentation and classification. Microscopy research and Technique, vol. 82(6), 803–811.

116

Artificial Intelligence and Internet of Things

UROOJ, S., SINGH, S. P. & ANSARI, A. 2018. Computer-aided detection of breast cancer using pseudo Zernike moment as texture descriptors. Sensors and Image Processing. Springer, Singapore, 2018. 85–92. VALVANO, G., DELLA LATTA, D., MARTINI, N., SANTINI, G., GORI, A., IACCONI, C., RIPOLI, A., LANDINI, L. & CHIAPPINO, D. 2017. Evaluation of a deep convolutional neural network method for the segmentation of breast microcalcifications in Mammography Imaging. EMBEC & NBC 2017. Springer, Singapore. VELIKOVA, M., DUTRA, I. & BURNSIDE, E. S. 2015. Automated diagnosis of breast cancer on medical images. Foundations of Biomedical Knowledge Representation. Springer, Cham. VIGELAND, E., KLAASEN, H., KLINGEN, T. A., HOFVIND, S. & SKAANE, P. 2008. Full-field digital mammography compared to screen film mammography in the prevalent round of a population-based screening programme: the Vestfold County Study. European Radiology, 18, 183–191. WAHEED, Z., AKRAM, M. U., WAHEED, A., KHAN, M. A., SHAUKAT, A. & ISHAQ, M. 2016. Person identification using vascular and non-vascular retinal features. Computers & Electrical Engineering, 53, 359–371. WANG, D., SHI, L. & HENG, P. A. 2009. Automatic detection of breast cancers in mammograms using structured support vector machines. Neurocomputing, 72, 3296–3302. WILLOUGHBY, C. E., PONZIN, D., FERRARI, S., LOBO, A., LANDAU, K. & OMIDI, Y. 2010. Anatomy and physiology of the human eye: effects of mucopolysaccharidoses disease on structure and function—a review. Clinical & Experimental Ophthalmology, 38, 2–11. XU, Z., GUO, X., HU, X., CHEN, X. & WANG, Z. The identification and recognition based on point for blood vessel of ocular fundus. International Conference on Biometrics, 2006. Springer, 770–776. https://doi.org/10.1007/11608288_103 ZHU, C., ZOU, B., ZHAO, R., CUI, J., DUAN, X., CHEN, Z. & LIANG, Y. 2017. Retinal vessel segmentation in colour fundus images using Extreme Learning Machine. Computerized Medical Imaging and Graphics, 55, 68–77.

5

Precise Segmentation Techniques in Various Medical Images Majid Harouni, Mohsen Karimi, Shadi Rafieipour

CONTENTS 5.1 Introduction................................................................................................... 118 5.2 Common Medical Image Processing Techniques......................................... 120 5.2.1 Unsupervised Image Segmentation................................................... 120 5.2.1.1 Morphological Operators Algorithm.................................. 120 5.2.1.2 Dilation Operator................................................................ 121 5.2.1.3 Erosion Operator................................................................. 121 5.2.1.4 Active-Contour-Based Segmentation.................................. 121 5.2.1.5 Mathematical Description of Active Contour Model......... 122 5.2.1.6 Watershed Algorithm.......................................................... 124 5.2.1.7 Region-Based Segmentation Methods................................ 127 5.2.1.8 Histogram Thresholding Method....................................... 127 5.2.1.9 Otsu’s Algorithm................................................................. 128 5.2.1.10 Multilevel Thresholding Algorithm.................................... 128 5.2.2 Supervised Segmentation Algorithms............................................... 130 5.2.2.1 Superpixel Algorithm......................................................... 131 5.2.2.2 Implementing the Superpixel Segmentation Algorithms... 131 5.2.2.3 Clustering Method.............................................................. 132 5.2.2.4 Artificial Neural Network (ANN)...................................... 136 5.3 Segmentation Methods for CT Scan and MRI Images.................................. 142 5.3.1 Brain Tumors and MRI...................................................................... 142 5.3.1.1 Overall Brain Anatomy....................................................... 142 5.3.1.2 Types of Brain Tumors........................................................ 143 5.3.1.3 Brain Tumors Grades.......................................................... 144 5.3.1.4 Evaluation Criteria.............................................................. 148 5.3.1.5 Database.............................................................................. 148 5.3.2 Lung Cancer...................................................................................... 148 5.3.2.1 Lung Anatomy.................................................................... 149 5.3.2.2 Lung Cancer........................................................................ 150 5.3.2.3 A Review of Lung Segmentation........................................ 152 5.3.2.4 Lung Database.................................................................... 153

117

118

Artificial Intelligence and Internet of Things

5.3.3. Liver Cancer...................................................................................... 155 5.3.3.1 Liver Anatomy.................................................................... 155 5.3.3.2 Review of Literature on Liver Tumor Segmentation.......... 156 5.3.3.3 Liver Database.................................................................... 159 5.4 Conclusion..................................................................................................... 159 References............................................................................................................... 159

5.1 INTRODUCTION Nowadays, the impact of erroneous lifestyle and eating unhealthy and nonorganic food, in addition to increased pressure and stress due to daily activity at full-tension places, has resulted in increased number of patients with refractory diseases. Tumors and cancers comprise a large part of these diseases. Providentially, owing to advances in medical science in modern communities, many of these diseases are curable in case of rapid and accurate diagnosis. Per contra, human error is inevitable and always causes harms for human being. These harms have been evident in old professions and medical cases. Some cases were not compensable and resulted in serious damages as far as it may even cost human life. Therefore, science practitioners have always perused the agendum of eliminating human error. The issue has been taken into account at health-care industry for many years, the great results of which are present. Fortunately, expert systems have been recently introduced to the field of medicine to minimizing human errors using artificial intelligence (Adapa et al., 2020; Rehman et al., 2019; Wiharto and Suryani, 2020; Xie et al., 2019; Zheng et al., 2020). The expert systems are not affected by fatigue and drowsiness and sentimentality do not affect their decision-making. Medical devices are among the strategic equipment all over the world, which are rapidly growing. These devices are partitioned into two class of diagnostic and therapeutic. Many diagnostic devices are image-based. These images utilize X-ray passing through patients’ body and magnetic resonance (MR). Recognizing the proper type of imaging plays a crucial role in disease diagnosis. Generally, there are various techniques of medical imaging, each of which can be utilized for diagnosis of one or more diseases. Magnetic resonance imaging (MRI) is one of the most popular noninvasive methods for tumor diagnosis. The review of literature shows the efficacious fusion of MRI with other types of images, including positron emission tomography (PET) and CT, to detect where tumors are located. An advantage of MRIs is its safety for pregnant women. The resonance of radio waves during MRI exam does not affect fetus and so it is not harmful to the unborn baby. Moreover, soft tissues such as brain, brain tumors, eyes, and heart can be easily monitored (Amin et al., 2018; Chuang et al., 2012; Saba et al., 2020; Soltaninejad et al., 2017). The main weakness of MRI technique is its high sensitivity to subject motion. In the case of subject move during data acquisition, the recorded image is highly affected. However, this weakness is covered through image fusion. CT scan is a medical imaging technique with proved efficiency in detection and diagnosis of various diseases. Both MRI and CT scan images are commonly used for disease diagnosis (Li et al., 2020, Xie et al., 2019). CT scan images are successfully used for three-dimensional (3D) imaging such as 3D simulation of tumors. CT scan images have benefit as short scan time with high resolution. CT scan imaging is most commonly used for the detection of lung tumors (Saba, 2019).

Precise Segmentation Techniques

119

Image processing mostly refers to digital processing of images. However, there are optical and analog image processing algorithms, which have less importance due to less usage. Most of the image processing algorithms use an image as a two- or moredimensional signal. It seems that standard signal processing algorithms and techniques are practically useful. Image processing aims to measure and recognize the relevant pattern of the subject under processing (Bansal and Singh, 2017; Chen et al., 2019; Hossain et al., 2019; Khan, 2018; Rehman et al., 2019; Xie et al., 2019). At early diagnosis of tumors or tumor vasculature, image processing helps medicines as a decision-making instrument. Early diagnosis of tumors through image screening or isolation of images with cancerous tumors of certain types from the images without tumor has the most significant effect in reducing mortality due to a certain type of cancer. In other words, images form a significant part of standard protocols of cancer diagnosis and can provide functional, metabolic, structural, and morphological data about a certain cancerous tumor, and be helpful along with other clinical diagnostic instruments. There are various imaging systems as CT scan, MRI X-ray, radiography, nuclear medicine and ultrasound, of which, the present research focuses on MRI, CT scan, mammographic, and color images. Segmentation of medical images means to assign label and group pixels in a way that region of interest (ROI) are semantically meaningful. During segmentation, image is split into chunks, each of which has similar features and characteristics, including similar brightness distribution. Precise recognition of tissue boundary, more specifically differentiation of tumor tissues from the healthy ones, is of significance in medical imaging. Various segmentation techniques have been presented in recent years (Adapa et al., 2020; Badawy et al., 2017; Bilic et al., 2019; Chen et al., 2019; Harouni et al., 2012b; Havaei et al., 2017; Li et al., 2020; Padlia and Sharma, 2019; Rehman et al., 2019; Wiharto and Suryani, 2020). These techniques and algorithms can be categorized into supervised and unsupervised segmentation. Supervised segmentation methods mostly use machine learning algorithms, where a number of visual features of each pixel are extracted (Harouni et al., 2012c). Then a classifier such as support vector machine (SVM) or other classifier K-nearest neighbors can be used for classification. Extracted features of each pixel are used in classifiers. Each pixel is classified into one of the ROIs, for instance, tumor or non-tumor, nodules or non-nodules, and vessel or non-vessel (Hua et al., 2015; Orobinskyi et al., 2019). In unsupervised methods, however, image segmentation is done through image processing methods such as morphological operators, thresholding, and online tracking. This chapter provides a comprehensive survey of image segmentation methods. The present study is significant in terms of: • Presenting and reviewing brain, liver, and lung anatomy for the purpose of image processing • A comprehensive review of all segmentation techniques used for brain, lung, and liver images • A comprehensive review of literature on diagnosis of brain tumors at MRI images • A comprehensive review of literature on diagnosis of lung and liver tumors at CT scan imaging • A review of evaluation criteria at medical image segmentation • Introducing prominent databases of medical image processing for lung, brain, and liver

120

Artificial Intelligence and Internet of Things

The remainder of the chapter is structured as follows. Section 5.2 presents the most frequently used medical imaging devices and also introduces the anatomy and the most recent image segmentation techniques of each organ. Section 5.3 tries to present the intended and most common databases. Section 5.4 is conclusion.

5.2  COMMON MEDICAL IMAGE PROCESSING TECHNIQUES Segmentation means labeling and classification of pixels in an image (Habibi and Harouni, 2018; Harouni et al., 2014; Rehman et al., 2020). Division an image into different meaningful regions called image segmentation. Usually, the meaningful regions have the same brightness. In medical imaging, segmentation is of high significance for checking tissues, and precise boundary and edge detection. Generally, segmentation methods can be divided based on function and procedure. Hitherto, various medical imaging segmentation techniques and methods have been proposed for the detection of cancerous tumors and nodules at CT scan or MRI images, retinal blood vessel segmentation, liver detection at CT scan images, and detection of fine-grained tumors at mammogram images. These methods aimed to improve the segmentation accuracy in face of common challenges of medical imagining. This section classifies them into supervised segmentation and unsupervised segmentation methods (Harouni, 2013). It tries to provide general and comprehensive description of each method.

5.2.1 Unsupervised Image Segmentation Unsupervised segmentation methods work by recognizing the patterns of various areas for classification of image pixels (Harouni et al., 2010, 2014; Rehman et al., 2020). The labeled data or training data are not directly used for segmentation (Wiharto and Suryani, 2020). These methods are reviewed in the following sections. 5.2.1.1  Morphological Operators Algorithm Morphological operators are among the most important stages of image processing and efficient algorithms for image processing. Morphology processes images based on shapes. Output image pixel values are compared to the corresponding input image pixel value and its neighbors. Specific kernel shapes and size in morphological operators would create the specific results. Using kernel element, create an output image in the same size as input. There are two most basic morphological operations named dilation and erosion. In dilation process, pixels add to the intended image objects’ boundaries, while pixels are removed in erosion. The number of added or removed pixels from the objects in an image depends on the size and shape of the structuring element. A basic part of dilation and erosion operators is a structuring element. A flat or two-dimensional (2D) structuring element is a matrix that is much smaller than the original imager. The center pixel of the structuring element is called origin. It identifies the pixel in the image being processed. Morphological operators are implemented and ran on binary value image (Rubini et al., 2018; Sharma and Meghrajani,

Precise Segmentation Techniques

121

2014). Following, two operators (dilation and erosion) implemented in this study are described. 5.2.1.2  Dilation Operator This operation is defined as Equation (5.1):

U     Ax A  ⊕   B   =  x  ∈ B

(5.1)

where B is the structuring element, and A is the input image. It means that for each x point of B, there is x ∈ B transition on image A. Accordingly,

A  ⊕   B   = U {( x ⋅ y ) + (u ⋅ v ) : ( x ⋅ y )   ∈   A ⋅  (u ⋅ v )   ∈   B )} (5.2)

In other words, the expansion of A by B means that if B slides through A, and it is not empty at each intersection of B with the ROI on A, the central pixel of B will have a value of 1. 5.2.1.3  Erosion Operator As the name implies, points 1 on the image are eroded or it erodes any 1 value. For each pixel, the structuring element positions on the pixel. Now if B structuring element slides through A, a pixel in the original image will be considered 1 only if all the pixels under the structuring element is 1, otherwise, it is 0 and eroded, as shown in Equation (5.3):

A  B = w  |  Bw   ⊆   A (5.3)

The combination of dilation and erosion operator creates two other efficient operators called opening and closing. Opening operator removes the bright foreground pixels on binary images, improves the image, and makes it smoother. The combination of dilation and erosion derives opening operator as Equation (5.4):

A   B   =  A   B   ⊕   B (5.4)

Moreover, operating the closing operator on the image removes the small holes. The combination of dilation and erosion derives closing operator as shown by Equation (5.5) (see Figures 5.1 and 5.2):

A  B = A  ⊕   B    B (5.5)

5.2.1.4  Active-Contour-Based Segmentation Finding the objects’ boundaries in image is called edge detection. Edge detection is a fundamental and useful tool in various image processing applications. It seems that edges are highly determinative at segmentation algorithms and more specifically when a region is going to be presented or removed. Contours of objects are connected curves that can be detected using and edge detector. Active contour algorithm, also called snakes, is one of the most important edge detection methods.

122

Artificial Intelligence and Internet of Things

FIGURE 5.1  An example of mathematical morphology operations of a sample binary image

Active contour algorithm is a very strong tool at edge detection. Accordingly, the active contour or curvature for the regions of target object is automatically defined for segmentation. Then, the contour evolves with an energy function to match the target object boundaries. As stated, active contour model is also called snake model due to its specific shape and moves. Kass et al. (1988) first introduced the active contour model. 5.2.1.5  Mathematical Description of Active Contour Model As the most commonly used model in image segmentation, active contour model, first introduced by Kass et al. (1988), is a parametric curve at image plane, as shown by Equation (5.6):

s(u) = I ( x (u) ⋅ y(u)),        u = [0.1] (5.6)

123

Precise Segmentation Techniques

FIGURE 5.2  The results of morphology operations on an original UWFFA image

Energy function transformed the curve based on the approximate desired object favorite features, which is shown in Equation (5.7): 1

E =  ∫ Esnake   ( S (u )) du (5.7)



0

Which has internal and image energy as Equation (5.8):

1

E = ∫ Eint ( S (u ))  +Eimg ( S (u )) du (5.8) 0

Where internal energy depends on the internal features of the contour such as elasticity rate and curvature, as shown by Equation (5.9): 2

2



Eint =

α  δ β δ2 S (u ) du + S (u ) du (5.9) 2 δu 2 δu 2

The first part of the internal energy makes the contour to act as a spring and determine the elastic curve. The second part determines the internal energy curve of the rate of resistance against bending. In the abovementioned equation, the coefficients α , β are weighted parameters that control sensitivity rate of counter against stretching and bending. Energy of image, direct the contour curves to the favorite features and distinct image such as edges, lines, and corners. This energy is estimated as edge detection at early formulation and is calculated as (5.10) and/or (5.11): 2



Eimg = Eedge =  − p ∇Is ( s ) (5.10)



Eimg = Eedge =  − p ∇(Gδ (s)* I (( s )) (5.11)

2

124

Artificial Intelligence and Internet of Things

Equation (5.11) is used to minimize noise effect, and where parameter P is a huge amount of image energy, ∇ indicates the gradient operator, and Gδ* I is the Gaussian convolution of the image with a standard deviation δ. Hence, the total energy of the active contour is defined as Equation (5.12):

E   = 

α 2

∫

|

δ β S (u ) |2 du  +  δu 2

∫

|

δ2 S (u ) |2 du  +  δu2

∫ E

edge

( S (u)) du (5.12)

If there is a visual prominent feature (strong edges), energy function correctly guides the contour curve toward the target. However, in the absence of strong edges, it becomes difficult for the curve of contour to find the target object. To overcome this problem, the new energy called the color pressure energy has been used and is replaced by edge energy in Equation (5.13). This energy is a function of statistical features of the model and condenses or expands the contour toward the target object by producing a compressive force. The color pressure energy can be defined as Equation (5.13): ⊥



 δs  E pressure   =  ρ   G ( I ( s ))   (5.13)  δu 

where ρ is a parameter determining the magnitude of the pressure energy and is loaded by the user. G is a function defined in accordance as Equation (5.14):

 +1      if      I   ( s ) ≥ T G ( I ( s ))   =     −1       otherwise      

(5.14)

where T is the intensity threshold of the image. According to Kass et al. (1988), G function can be defined as Equation (5.15):



  G ( I ( s )) =  +1   −1

if   

(I (S ) − µ ) ≤ k σ otherwise

(5.15)

where μ is the mean and σ is the standard deviation of the target object pixel value, available as the previous data or calculated through the image. k is the constant, determined by the user. This method works well, when the target object and foreground are simple. However, it faces some difficulties for target object or foreground of color and texture complications as shown in Figure 5.3. 5.2.1.6  Watershed Algorithm As stated in Ng et al. (2006), watershed algorithm is a simple and fast method, which can be parallelized, and always produces a complete partition of the image in separated regions even if the contrast is poor. There are several definitions for the watershed and various algorithms to calculate it. Generally, image has a 3D coordination,

Precise Segmentation Techniques

125

FIGURE 5.3  Segmentation result of active contour model

including two spatial or Cartesian coordinates, and the third dimension is the gray matter (GM) surface. 5.2.1.6.1  Regions of a Local Minimum Regions that if a simulated drop of water falls onto them will flow in the direction of the same minimum or the regions from where water flows toward two or more minimum regions. For a certain local minimum, the set of regions observing these two conditions is called watershed of the region. The main purpose of the watershed-based algorithms is to find watershed lines or crest lines. There are two main approaches to finding these watersheds: Rain falling approach: In this approach, local minima are found all over the image and every local minimum is labeled. Then, it is assumed that a drop of water falls onto any non-labeled (higher level or peak) region toward the lower level neighboring regions (valleys) with the smallest value to reach a labeled region. The raindrops that fall over a point will flow along the path of the steepest descent until reaching a distinct minimum. Flooding approach: It is assumed that there are holes pierced in every local minima, and then the entire relief is flooded with the same speed due to the penetration of water. The algorithm makes a barrier to prevent the water from two or more local minimum to be merged. At the end of the process, each local minimum is surrounded by catchment basins. The boundaries corresponding to the barriers are the ones extracted by watershed segmentation algorithms (Ng et al., 2006). 5.2.1.6.2  Watershed Algorithm Incorporating Gradient One of the common uses of watershed algorithm is to segment objects with approximately uniform (homogenous) foreground region. Detected regions with fewer changes at grayscale have less gradient value. Thus, watershed segmentation is used for image gradient. The gradient value is used at preprocessing of grayscale image prior to using basin transformation for segmentation. The image gradient value is high for pixels at edges while it is low at any other region. In the simple watershed, a marker is placed at the local minimum of the image gradient. This leads to oversegmentation and so the created regions are merged. If M1 ,…, M n are sets where g( x , y) are the coordinates of the points at local maxima of the image. If set  C ( M i ) has the coordinates of the points at local minimum basin M i which connects a component and max and min indicate the maximum and

126

Artificial Intelligence and Internet of Things

minimum values of g( x , y), respectively, then, T [ n ] is a set of coordinates (s, t ) where g ( s, t ) < N , so (Equation (5.16)) (Gauch, 1999):

T [ n ] = {(s, t ) |  g(s, t ) < N} (5.16)

Geographically, T [ n ] is a set of coordinates at points g( x , y) located below g( x , y) = n plane. This topography is flooded by water surface rising at next steps from n = min + 1 to n = max+ 1. At each n stage of immersion, the algorithm has to know the number of points beneath the water surface. Conceptually, if coordinates at T [ n ] below g( x , y) = n, it is marked with black and other coordinates are marked as white, then for every n increase of immersion, looking down the ( x , y ) coordinate plane, there is a binary image, where the black points correspond to the points of the function below g( x , y) = n plane. Let us assume Cn ( M i ) is the set of coordinates at the basin of local minimum M i , immersed at every n stage, Cn ( M i ) can be seen as a binary image presented by Equation (5.17).

Cn ( M i ) = C ( M i ) ∩  T [ n ] (5.17)

In other word, Cn ( M i ) = 1 at ( x , y ) coordination. If ( x , y ) ∈ C  ( M i ) AND   ( x , y ) ∈ T [ n ], otherwise Cn ( M i ) = 0. At immersion n, AND operator is used to separate a part of binary image at T [ n ], which is related to local minimum M i . Let C [ n ] be the collection of catchment basins immersed at stage n, as described in Equation (5.18). C [ max+ 1] is the collection of all catchment basins calculated using Equation (5.19):

c[n] =

R

∪C ( M ) n

(5.18)

i

i =1



C [ max+ 1] =

R

∪C ( M ) i

(5.19)

i =1

Cn ( M i ) and T [ n ] are not replaced while running the algorithm and the members of these set are added or remain same by n increase. Therefore, C [ n − 1] is a subset of C [ n ]. According to Equations (5.18) and (5.19), C [ n ] is a subset of T [ n ], then C [ n − 1] is a subset of T [ n ] . Thus, every connected component of C [ n − 1] exactly locates at a connected component of T [ n ] . Line detection algorithm is initialized with C [ min + 1] = T [ min + 1]. Then, it continues recursively and calculates C [ n ] from C [ n − 1]. The procedure of obtaining C [ n ] from C [ n − 1] has been described next. If Q is a set of connected components at T [ n ] , there exist three possibilities for each connected component of q ∈ Q [ n ], q ∩ C [ n − 1] (1) is null or empty, (2) has a connected component of C [ n − 1], and (3) has a connected component of C [ n − 1] (Gauch, 1999). Obtaining C [ n ] from C [ n − 1] depends on which one of the following conditions is met. The first condition is met when a new minimum is detected, then connected component q is located at C [ n − 1] to obtain C [ n ]. The second condition is met when q is in the basin of a local minimum, then q is located at C [ n − 1] to obtain C [ n ]. The third

Precise Segmentation Techniques

127

condition is met when all or part of the protrusion of the second wall or several basins are observed, more immersion leads to the merging of water surface at the basins. Thus, dam (or dams, in case of more than two basins) must be built at q to prevent overflow in basins. For this purpose, q ∩ C [ n − 1] dilates with a structuring element of 1s with a size of 3 in 3 and limiting the dilation to q . This algorithm can be enhanced just by using n values corresponding to the intensity value of g ( x , y ) . 5.2.1.7  Region-Based Segmentation Methods These methods extract the similar regions based on a predetermined criterion, which can be similar brightness, similar texture, homogeneity, or sharpness at the image. Region growing method is one of them (Wu, 2019). 5.2.1.7.1  Region Growing Method Region growing method is one of the common techniques well applied for medical image segmentation. In this method, connected regions of a vessel are extracted based on a predefined criterion, including brightness, edge, or other data. First, a small point on the image is selected as the seed by the operators based on the predefined criterion. The operator criterion for selecting a seed can be growing the seed at a certain region of the image until that region reaches the edge. The pixels of the seed point at a certain context are extracted. The main limitation of the method is the need for an operator to locate the seed point, precisely. Thus, every region to be dilated needs to grow a seed point. This algorithm is easy and fast. However, sensitivity to noise, development of holes or discontinuity at the image, and the need for an operator to locate the seed point are among the disadvantages of these algorithms (Khan et al., 2020, 2021). 5.2.1.8  Histogram Thresholding Method In this method, segmentation is done using brightness histogram or encoding color information. It is assumed that an image has different brightness levels illustrated as peak and valley at brightness histogram, where image boundaries are separated at the valleys. Moreover, at initial stages of image processing, one can use multiple thresholds for segmentation. In this case, the number of regions under segmentation has a value of greater than 1 that of threshold point’s value. This method can segment healthy and tumor tissues at two distinct classes. In the case of proper resolution, this method can recognize the subject (target). This method only uses color (brightness) information of the image histogram. Therefore, it does not include local information (Hijazi et al., 2010). 5.2.1.8.1  Multilevel Thresholding and Calculating 2D Thresholding Coefficient In thresholding and multilevel thresholding segmentation of an image, thresholding parameter determines that a pixel is a value specified whether a pixel of the image under processing is located at a specific region or not. There are various solutions for thresholding, some of which are described in the following sections (Singh et al., 2015). 5.2.1.8.1.1  Balanced Histogram Thresholding  The balanced histogram analysis method, which is also called balanced histogram thresholding in some papers, weights the image histogram, checks which of the two sides is heavier (right or left side), and then removes weight from the heavier side until it becomes the lighter. It repeats the

128

Artificial Intelligence and Internet of Things

FIGURE 5.4  Image segmentation results using Otsu’s algorithm

same operation until the edges of the weighing scale meet. In other words, the point that the edges met is defined as the threshold point (Akram and Khan, 2013). 5.2.1.8.1.2  A Constant Value for Threshold  This value is mostly calculated experimentally or through trial and error, to be considered for computations (Akram and Khan, 2013). 5.2.1.9  Otsu’s Algorithm Otsu’s algorithm is a simple and popular thresholding method for image segmentation as illustrated in Figure 5.4, by the clustering or dividing the image histogram into two classes. The algorithm does clustering and defines as a weighted sum of variances of each cluster to obtain a threshold value for the image (Rubini et al., 2018). 5.2.1.10  Multilevel Thresholding Algorithm In this method, more than two threshold values are used due to the need to determine three regions to be segmented. At one-dimensional thresholding, the algorithm defines a boundary between two regions. However, multilevel thresholding algorithm defines a boundary between region one and two (binary regions) and then a boundary between region two and three. So, a threshold value is calculated for each boundary. Each of these threshold values can be computed using previous algorithms. 5.2.1.10.1  Edge Detection An edge in an image can be defined as a single pixel with local discontinuity in intensity where an abrupt change of color intensity, texture, and lightness value occurs. Physical changes in color and brightness appear as edge on image. In fact, edge determines the boundary of any object at the image (Xiaoming et al., 2019). Edge is one of the most important characteristics of any image. Figure 5.5 shows the light change at the edge. Human vision can easily detect the edge. In fact, the edges make the object on an image to be easily detected (Harouni et al., 2012a). Human vision detects edges as changes in the color (hue), color intensity difference and object distance difference (at real 3D scene) (Halder et al., 2019). An edge has pixels with

Precise Segmentation Techniques

129

FIGURE 5.5  Different kinds of edge shapes in an image

intense changes of image intensity function. Edges (or some areas of edges) are collections of connected pixels. Edge detection methods are the local image processing methods designed for the detection of edge pixels (Mohammadi Dashti and Harouni, 2018). A line is a part of an edge where the background intensity at both sides is different from the line pixel intensity. Similarly, a separate point can be seen as a line, the width and length of which has a value of 1 pixel. Line refers to narrow structures with a thickness of 1 pixel. Sever local changes in lightness intensity are detected by derivative. First- and second-order derivatives are proper for this purpose.

130

Artificial Intelligence and Internet of Things

5.2.1.10.2  Edge Detection Models Edges can be modeled based on intensity profile as shown in Figure 5.5 and described in the following. • Step edge involves a transition between two intensity levels ideally occurring over distance of 1 pixel. This type of edge occurs at computer-generated images used for use in areas such as solid modeling and animations. • Ramp edges are noisy and blurred. Slope of the ramp is inversely proportional to the degree of burring in the edge. • Roof edges are models of lines through a region, with base or width of the roof being determined by thickness and sharpness of line (Medina et al., 2017). • Edge detection refers to the process of identifying and locating sharp discontinuities in an image. The discontinuities are abrupt changes in pixel intensity that characterize boundaries of objects in a scene. • Variables involved in the selection of an edge detection operator include: • Edge orientation: The geometry of the operator determines a characteristic direction in which it is most sensitive to edges. Operators can be optimized to look for horizontal, vertical, or diagonal edges. • Noise environment: Edge detection is difficult in noisy images, since both the noise and the edges contain high-frequency content (Marr and Hildreth, 1980). 5.2.1.10.3  Edge Structures Not all edges involve a step change in intensity. Newer detection techniques actually characterize the nature of the transition for each edge in order to distinguish, for example, edges associated with a face. Edge detection methods are grouped into two structures. 5.2.1.10.3.1 Gradient The gradient method detects the edges by looking for the maximum and minimum in the first derivative of the image. 5.2.1.10.3.2 Laplacian The Laplacian method searches for zero crossings in the second derivative of the image to find edges. Clearly, the derivative shows a maximum located at the center of the edge in the original signal. This method of locating an edge is known as the “gradient filter” family of edge detection filters. If the value of the gradient exceeds some threshold, a pixel location is declared an edge location. As mentioned earlier, edges will have higher pixel intensity values than those surrounding them. Therefore, if a threshold is set, one can compare the gradient value to the threshold value and detect an edge whenever the threshold is exceeded. When the first derivative is at a maximum, the second derivative is zero. As a result, another alternative to finding the location of an edge is to locate the zeroes in the second derivative, which is known as the Laplacian (Marr and Hildreth, 1980).

5.2.2 Supervised Segmentation Algorithms Supervised learning methods evaluate segmentation algorithms by comparing the resulting segmented image against a manually segmented reference image, which

Precise Segmentation Techniques

131

is often referred to as a gold standard or ground truth. An expert segments a gold standard to ground truth. In supervised methods, the imaging should be based on the ground truth. The results of supervised-based methods tend to be more accurate than other methods due to the use of training data (Adapa et al., 2020). 5.2.2.1  Superpixel Algorithm The basic idea of superpixel algorithm is to reduce the number of required sample at image processing (Achanta et al., 2012). Every superpixel of a dependent homogenous region has a semantic meaning in the image composed of a set of pixels. This new set can be separately processed. Semantic regions are formed by superpixels. These regions are used in various processes such as image segmentation, contour obscure, object location, object tracking, and 2D and 3D detections. The advantages of superpixel algorithm are as follows: • Connectivity: Each superpixel is a simple single connected area with the regular shape and every pixel of an image belongs only to a superpixel region. • Compactness: The regions without any feature, or the regions with very low frequency, and the superpixel regions have regular shape and size. • Feature preservation: Superpixels maintain all the boundaries and edges of an image. In other words, no feature is removed following the superpixelbased segmentation process on the image. • Sensitive to the concepts: Density of superpixel is adaptive to the local image content. That is, if an image has high local content, then the number and density of superpixels are high. • Efficiency: Superpixels algorithms are fast to compute and memory efficient. Running the algorithm is cost-effective in terms of processing and so it is an ideal method. It has also good performance for the segmentation of high-quality images. • Ease of use: Users can simply simulate the algorithm and run it on the image. The user can change the number of pixels and their smoothness by changing the values. 5.2.2.2  Implementing the Superpixel Segmentation Algorithms There are two methods of simulation, running and computing the superpixel algorithms, (1) graph-based methods, and (2) clustering-based methods. In graph-based methods, image is considered as a graph, the peaks of which are pixels. In graph-based algorithms, a cost function is defined on a graph and minimized to generate superpixels. For this purpose, superpixel lattice or graph cut and graph cut optimization method is used. Although graph-based algorithms are efficient, they have only theoretical use and their simulation is very difficult. Clustering-based superpixel segmentation algorithms divide the image pixels into clusters. Then an iterative process optimizes the clusters so that the algorithm converges or reaches the predefined threshold. Some common clustering-based methods are simple linear iterative clustering (SLIC), structure-sensitive superpixel (SSS), and manifold SLIC. In the following, SLIC algorithm is described.

132

Artificial Intelligence and Internet of Things

SLIC iteratively applies K-means clustering and the combined five-dimensional color and coordinate space of LAB are defined by L, a, b. SLIC enjoys simplicity in application and simulation, and good performance for many projects in practice. It should be noted that in clustering methods, the number of clusters determined by the user is mapped in the image. More comprehensive descriptions about SLIC and other cluster-based superpixel segmentation methods have been provided. Since the present study used SLIC, the following section deals with this algorithm. 5.2.2.2.1 SLIC SLIC iteratively applies K-means clustering n the combined five-dimensional color and coordinate space of LAB defined by L, a, b (Achanta et al., 2012). Let us I be the input image with P pixels and n number of superpixel, then the size of every superpixel is obtained through Equation (5.20). P (5.20) n

d=



In the assignment step of the algorithm, each pixel is associated with the nearest cluster center, the search region of which overlaps its location. Then, an update stage adjusts the cluster centers. Therefore, SLIC acts similar to a clustering algorithm at a five-dimensional feature space and constrained local space. Thus, a method should be suggested for calculating the distance between each pixel to the cluster center. In order to observe consistency in distance calculation, given that the elements of the space feature are not of the same type, the distance is defined as Equation (5.21) for color and pixel coordinate features (Achanta et al., 2012):

dc =

(l j − li ) + (a j − ai ) + (b j − bi ) 2

ds =

2

(rj − ri ) + (c j − ci ) 2

2

2

(5.21)

(5.22)

where dc is the color distance and ds is the spatial distance. There is a need for a unit criterion for distance at SLIC. To normalize the obtained distances and combining them as a certain criterion, we use the maximum value for color and spatial distance as Nc and Nc, respectively. Then, the distance value is calculated in Equation (5.23): 2



2

d  d  D =  c  +  s  (5.23)  Nc   Ns 

5.2.2.3  Clustering Method These methods do segmentation given the brightness feature of the image and so they are brightness dependent. Some of these methods include thresholding, K-means clustering, and fuzzy clustering. Cluster refers to a collection of points or nodes and pixels on an image. Clustering allows mapping on multiple servers (cluster points). In general, “clustering group’s data instances into some subsets in which similar instances are grouped together”. Clustering algorithms are similar to classifying algorithms in many ways. However,

Precise Segmentation Techniques

133

Clustering is often called an unsupervised learning task as no class values denoting an a priori grouping of the data instances are given, which is the case in supervised learning. The clustering algorithms iterate due to a lack of learning data (Manjaramkar and Manesh, 2018). These methods are self-learning through the existed data. Clustering has two main parameters, intercluster distance and intra-cluster distance. The intra-cluster distance between the learning data should be reduced as much as possible and clustering should be done in a way that the intercluster distance is increased while the intra-cluster is reduced to its minimum value. In this case, the most optimal clustering is done. Some instances of common clustering algorithms are K-means, fuzzy C-means (FCM) algorithm that is extended form of K-means, and expectation–maximization (EM) algorithm, which is similar to FCM (Bansal and Singh, 2017). 5.2.2.3.1  K-Means Clustering Algorithm K-means is one of the clustering algorithms, which stores centroids to define clusters. A point is considered to be in a particular cluster if it is closer to that cluster’s centroid than any other centroid. K-means finds the best centroids by alternating between assigning data points to clusters based on the current centroids and choosing centroids (points that are the center of a cluster) based on the current assignment of data points to clusters. It iterates over steps until the sum of the squared errors in each group cannot be decreased any more. The following function is defined as the objective function (Jain, 2010). • In K-means, first k member (k number of clusters) is randomly selected from among n members as the center of cluster. • Then n-k members are assigned to the nearest cluster. • Following the assignment of all the members, the centroids are calculated again, and the members are assigned to the new centroids until the centroids are fixed. The best clustering is the one that maximizes the set of similarities intra-cluster and minimizes the similarity intercluster. To select the best cluster, first, a suggested range for the number of clusters is determined based on expert evaluation and previous studies. Then, the ρ(k) value is calculated for each k, and the k with maximum ρ(k) is selected as the optimal value of the clusters. Accordingly, the number of clusters can be selected in a way that there are maximum intercluster distance and intra-cluster similarity. The quality of clustering with K-means is defined by Equations (5.24)–(5.29). Type of selecting the cluster center, distance function for clustering, and calculation complexity are considered in these equations (Khan, 2018; Saffarzadeh et al., 2014).

O = {c n | n = 1,…, k } (5.24)



O n = {Ci | i = 1,…,|| T c − O ||} (5.25)



ρ (k ) =

 η + ηm   1 k  ∑  min  n  (5.26) k n=1   δnm 

134

Artificial Intelligence and Internet of Things



ηn =

1 ∑ Sim (ci ⋅ c n ) (5.27) || O n || ci On



ηm =

1 ∑ Sim (c j ⋅ c m ) (5.28) || O m || ci Om



δnm = Sim (c m ⋅ c n ) (5.29)

5.2.2.3.2  Fuzzy C-Means Clustering (FCM) FCM is a fuzzy clustering algorithm applied for a wide range of analysis. The clusters are grouped based on distance between the data points and centroids. FCM is a clustering-based method used to divide the one group data into two or more clusters. This method is mostly used to detect the process and there is no rapid transition in rating the members. FCM is a technique for clustering data, where a set of data is divided into n groups and each data point is subdivided to into certain points (Rashid, 2013). Image segmentation is one of the most important processes of image analysis. Now, one of the main issues in this regard is to design a proper algorithm for clustering various images. Image segmentation means dividing an image to the regions with similar features such as color, hue, and texture. There are four main image segmentation techniques, thresholding method, edge-detection-based techniques, region-based techniques, and clustering-based techniques (Badawy et al., 2017). Clustering-based techniques are the techniques, which segment the image into clusters having pixels with similar characteristics (Figures 5.6 and 5.7). Similar patterns are set in one cluster. There are two types of clustering: hard clustering and soft or fuzzy clustering. In hard clustering, each pixel can belong to exactly one cluster, the result of which is a wavy image. Poor resolution, poor contrast, noise, and high interference, nonhomogeneous brightness result are drawbacks of hard clustering method. Fuzzy theory works on the concept of partial membership. Fuzzy-set-based clustering methods are used as a part of soft clustering method. Given the fuzzy clustering, FCM algorithm is a common clustering method at image segmentation since it has advantages for detecting blurred points and can reserve more information compared to the hard clustering. FCM works well on noise-free images. This method is highly sensitive to noise and other imaging artifacts (Kumar et al., 2019). The FCM algorithm was first suggested in Cannon et al. (1986). This algorithm is iterative clustering method that segments the image by minimizing a criterion function, which is the total square weight error as described in Equation (5.30):

N

c

Jm = ∑ ∑ uijm d 2 ( xi , v j ) (5.30) i =1 j =1

where X = { x1 , x 2 ,…, x N } and X ⊆ R m are the input data at a vertical space of m dimension, N number of pixels, c number of clusters, uij degree of membership for pixel xi in cluster vi , m is the weighted square on every member of the membership matrix, vi is the centroid of cluster j, and d 2 ( xi , v j ) intra-cluster distance measure point. Clustering is done as the following:

135

Precise Segmentation Techniques

FIGURE 5.6  Compering the performance of different clustering algorithms

1. Assigning value to parameters c, m, and ε 2. Initial assignment of value to membership matric U(0) 3. Set the loop counter = 0 4. Cluster centers are calculated by Equation (5.31) using membership matric:



(b)

vj =

( )  x ∑ (u ( ) )

∑iN=1 uij(b) N i =1

b ij

m

m

i

(5.31)

FIGURE 5.7  The segmentation results of different clustering algorithms

136

Artificial Intelligence and Internet of Things

5. Membership matric value is subject to Equation (5.32): u (jib+1) =



1 ∑

c k =1

(d ji / d ki )

2/ m −1

(5.32)

}

If max {U b −  U (b+1) , then iteration ends, otherwise if b = b + 1, then it iterates step 4. 5.2.2.4  Artificial Neural Network (ANN) Artificial neural network (ANN) is one of the most successful decision-making systems, which can be used for doing complicated tasks at different applications, including prediction or forecasting, pattern detection, optimization, detection, and classification. A neural network (NN) is a network of connected processing elements inspired by biological NNs. That is, the ANN is an attempt to simulate the behavior of biological NN s to make machines acting like human brain (Amin et al., 2019a,b,c,d). The readers who are interested in artificial intelligence should start with understanding the NNs. The main feature of NNs is that such systems learn to perform tasks by considering examples, generally without being programmed. That is, most of the complicated applications in the past are applicable now. 5.2.2.4.1  Neural Network Architecture Perceptrons are a very popular NN architecture. The terms were projected by Rosenblatt for describing various types of NNs. The main idea was to practically simulate the performance of neurons in an algorithm instead of making a physical model of neurons as shown in Figure 5.8. NNs are mathematical parallel and distributed information processing systems that are inspired and derived from biological learning systems (Ejaz et al., 2019, 2020). As given in Figure 5.8, the output signal (O) is obtained by Equation (5.33):

 n  O = f (net ) = f  w j x j  (5.33)    i =1 

∑

FIGURE 5.8  Mathematical model of an artificial neuron

137

Precise Segmentation Techniques

where wj is the weight vector and f (net) is the activation or transfer function. The net value is calculated by Equation (5.34):

net = w t x = w1 x1 + w2 x 2 +  + wn x n

(5.34)

where t is the transpose of a matric, and in its simplest form, the value of O output is obtained as Equation (5.35):



  1 if W t x ≥ θ O = f (net ) =   0 otherwise

(5.35)

where θ means threshold level. Generally, the NN architecture is divided into two groups: feed-forward and backpropagation. In feed-forward architecture, data are processed in one way from input layer to the output layer (forward) and thus connections between the nodes do not form a cycle. Backpropagation has feedback cycles and neurons at each layer and feeds information from previous and next layers (Samala et al., 2016). 5.2.2.4.2  Deep Neural Network Convolution NN (CNN) is one of the most important deep learning methods, where multiple layers are learned using a robust method. This method is very reliable and popular in various image processing applications. Figure 5.9 shows the general architecture of a CNN with classification layer (Ejaz et al., 2018a,b; Zheng et al., 2020). One of the well-known deep medical image segmentation method that is developed is named U-net, which is derived from CNN as shown in Figure 5.10 (Ronneberger et al., 2015).

FIGURE 5.9  The layout of CNN architecture

138

Artificial Intelligence and Internet of Things

FIGURE 5.10  The basic U-net architecture

A CNN network is generally composed of three main layers, convolution, pooling, and fully connected layers. The layers perform different tasks. As illustrated in Figure 5.9, there are two steps of learning at any CNN: first feedforward and then backpropagation. At first step, the network is fed with an input image, through multiplying the input and parameters of each neuron and finally performing convolution operation at each layer. Then, the output is calculated. In order to set the parameters or learn the network, the output is used for calculating the network error. The network output is compared with correct response through a loss function. At next step, backpropagation begins based on the calculated error. At this step, gradient of each parameter is calculated based on chain rule and all the parameters are updated based on the error effect. Following the parameters’ update, the next feed-forward starts. Network learning ends following proper iteration of these steps (Samala et al., 2016; Xie et al., 2019; Zheng et al., 2020). 5.2.2.4.2.1  Convolution Layers  In this layer as shown in Figure 5.11, a CNN layer uses various kernels for convolving the input image and also feature map, to produce different feature maps. Convolution operation has three advantages: • Weight sharing mechanism at each feature maps reduced the number of parameters • Learns the local connectivity or connectivity of neighboring pixels • Results in consistency in terms of spatial location of the object Due to the advantages of convolution operations, some research papers have used it for translating fully connected layers to increase the learning process speed. 5.2.2.4.2.2  Pooling Layers  A pooling layer usually precedes a convolution layer. It can be used to reduce the feature maps or dimension of data and network

Precise Segmentation Techniques

139

FIGURE 5.11  Layer convolution operation

parameters. Same as convolution layers, pooling layers resist translation because of taking neighboring pixels into their computation. Pooling may compute max and average as two common values. Figure 5.12 illustrates a max pooling process. A max pooling filter of size 2 produces a feature map of size 8 and an input of size 4. Pooling layers are the only layers of convolution network, which have been mostly studied. There are three popular methods in this regard, each of which follows a different objective. 5.2.2.4.3  CNN-Support Vector Machine (SVM) CNN-SVM is one of the best image classifiers for identity recognition and has been used in many researches (Bansal et al., 2012; Tapia et al., 2014; Thomas et al., 2007). This section reviews the performance of this classifier. Assume we have a set of data

FIGURE 5.12  Max pooling layer operation

140

Artificial Intelligence and Internet of Things

FIGURE 5.13  Hyperplane with max margin along with separating boundary for two different classes. The data on the boundary are called support vector

points {( x1 , c1 ),( x 2 , c2 ),…,( x n , cn )}, which needs to be divided into two classification ci = {−1,1}, and every xi is p-dimensional vector of real numbers. Classification methods try to make a hyperplane (e.g. a linear equation) to separate the data. CNN-SVM, which a linear classification method, discovers the best hyperplane to separate two classes with the maximum margin. For a better understanding, Figure 5.13 shows a set of data of two classes, for separation of which CNN-SVM selects the best hyperplane. This section describes the formation of separating hyperplane on an example in detail. First, a convex is considered around the points of each class. In Figure 5.13, the convex is mapped around of −1 class and +1 class points, in which, p is the line showing the nearest distance between two convenes; h is the separating hyperplane, the line cut p in half and perpendicular to p; b is the y-intercept of the separating hyperplane with maximal margin, if b is disregarded, the answer is only those hyperplanes intercepting y-vector. The perpendicular distance of hyperplane to y-vectors is obtained by dividing the absolute value (modulus) of parameter b to length w. The main idea is to select the best separator, which is the one with the maximal margin from the neighboring points of both two classes. This response has the longest boundary with the related points of two classes and can be bounded into one by the hyperplanes passing at least one point of two classes. These vectors are called support vectors. The mathematical formula of two parallel hyperplane forming the separating boundary is Equation (5.36):

 w ⋅ x   −  b   = 1   w ⋅ x   −  b   =  −1

(5.36)

If the training data is linearly separable, we can select two parallel hyperplanes that separate the two classes of data, so that the distance between them is as large as possible. Geometrically, the distance between these two hyperplanes is 2 / w , so to maximize the distance between the planes we want to minimize w . We also have

141

Precise Segmentation Techniques

to prevent data points from falling into the margin; therefore, we add the following constraint for each i as shown in Equation (5.37):

 wi ⋅  x   −  b   = 1   wi ⋅ x   −  b   =  −1

for first class data for second class data

(5.37)

This constraint can be shown as (5.38):

ci ( w ⋅  xi   −  b ) ≥ 1

1 ≤  i   ≤  n

(5.38)

5.2.2.4.4  K the Nearest Neighbor Algorithm (KNN) The K-nearest neighbor (KNN) algorithm is one of the best classifiers in pattern recognition, more specifically identity recognition (Zhang and Guan, 2012). In this classification method, k is a user-defined constant, and an unlabeled vector (a query or test point) is classified by assigning the label which is most frequent among the k training samples nearest to that query point. Various methods of computing neighboring are used to obtain the nearest neighbors of a sample, which are described in the following sections. 5.2.2.4.4.1  Euclidean Distance  This distance metric of neighbor points (Equation (5.39)) is one of the most important distance metrics in various science, so that MATLAB assumed as its distance metric.

d st2 = ( x s − xt ) + ( ys − yt ) (5.39) 2

2

where s is the initial point and t is the target point. 5.2.2.4.4.2  City Block Distance  The block is sum of the absolute differences between coordinates of a pair of objects, calculated as Equation (5.40):

n

d st = ∑ x sj − ytj (5.40) j =1

where s is the initial point and t is the target point. 5.2.2.4.4.3  Cosine Distance (Cosine Similarity)  It is defined to equal the cosine of the angle between two nonzero vectors and is less than 1 for any angle in the interval, which is calculated as Equation (5.41):

 d st = 1 −  

  (5.41) ( xs xs′ )( yt yt′ )  x s yt′

where also s is the initial point and t is the target point.

142

Artificial Intelligence and Internet of Things

5.2.2.4.4.4  Distance Correlation  Distance correlation is a measure of dependence between two paired random vectors of arbitrary, not necessarily equal, dimension, as calculated via Equation (5.42):

d st = 1 −

( xs − xs ) ( yt − yt ) ′

′

( xs − xs )( xs − xs ) ( yt − yt )( yt − yt )

(5.42)

where s and t are the same as the previous equation. 5.2.2.4.4.5  Hamming Distance  Hamming distance is a metric for comparing two binary data strings, obtained as Equation (5.43):

d st = ( # ( x sj ≠ ytj ) / n ) (5.43)

so that s is the initial point and t is the target point.

5.3  SEGMENTATION METHODS FOR CT SCAN AND MRI IMAGES Image segmentation methods may vary based on the type of the image and organ in terms of usage. For instance, although morphological algorithms are used as unsupervised learning algorithms, there are different varieties of this method, or SVMs are used as supervised learning algorithms, but they may differ based on image type. The following section studies the MRI images of brain tumors, CT scan images of lung tumors, and CT scan image of liver and liver tumors (Saba et al., 2019). The researcher has tried to introduce the well-known databases of each organ. Finally, the evaluation criterion is introduced.

5.3.1 Brain Tumors and MRI Brain as the control center of the body is arguably the most important organ in the human body. Any damage to any part of brain can lead to irreparable injuries and sometimes fatal. Brain tumors are among the most common brain injuries with a significant rate of mortality. Accordingly, timely and precise detection of brain tumors can contribute to the treatment and reduced art of mortality. Using machine learning techniques and algorithms can be effective in detection and impeding the brain tumor development using MRI. Following section reviews the overall brain anatomy and brain tumors. Then, MRI technique is described followed by machine learning algorithms and techniques used for brain tumor segmentation. 5.3.1.1  Overall Brain Anatomy Brain is an important part of central nervous system (CNS), acting as the control center of human body. Brain is composed of three main parts, the cerebrum, cerebellum, and brainstem, each of which has different parts and various functions. Here, only the surface of the cerebrum or cortex and its specific layers are presented. As

Precise Segmentation Techniques

143

FIGURE 5.14  Brain cortex (a) input image, (b) gray matter, (c) white matter, (d) substantia nigra (black matter) (Chuang et al., 2012)

shown in Figure 5.14, cortex contains GM, white matter (WM) and substantia nigra (black matter). The CNS contains the following parts: GM consists of neurons, the main task of which is to transfer electrical signals. It makes up 30% of an adult human brain (Chuang et al., 2012). WM refers to areas of the CNS mainly made up of myelinated axons, also called tracts. It makes up 60% of adult human brain. Cerebrospinal fluid (CSF) is also known as black brain due to its dark appearance in MRI images. It makes up 10% of an adult human brain. 5.3.1.2  Types of Brain Tumors Usually, cells die when they get too old or damaged. Then, new cells take their place. Sometimes, genetic changes interfere with this orderly process. Cells start to grow uncontrollably. These cells may form a mass called a tumor. A tumor can be cancerous or benign. A cancerous tumor is malignant, meaning it can grow and spread to other parts of the body. A benign tumor means the tumor can grow but will not spread. Benign brain tumors are noncancerous and are removable. They rarely grow again and have distinct boundaries. BB-tumors rarely invade the adjacent tissues and do not develop in other body parts. However, BB-tumors can cause serious health problems due to pressure on critical part of brain (Black, 1991). Malignant brain tumors (MB-tumors), which are also called brain cancer, contain cancerous cells and are generally more serious and life-threatening. They possibly grow rapidly and invade the adjacent brain tissues. In some cases, these tissues feed the blood and adjacent tissues for rapid growth. Cancerous cells may spread out of the MB-tumors and invade other parts of brain. However, they rarely spread to other parts of the body.

144

Artificial Intelligence and Internet of Things

5.3.1.3  Brain Tumors Grades Cancer and histology specialists classify the brain tumors based on tissue or grade, as described next: • Grade I. This kind of tumor is benign. Cells in a grade I tumor look a lot like normal brain cells and grow slowly. • Grade II. This kind of tumor is malignant. They look less like normal brain cells compared to Grade I. • Grade III. This kind of tumor is malignant. The cells of tumor are different from normal cells. They grow abnormally and they are so-called anaplastic astrocytoma. • Grade IV. This kind of tumor is the most malignant. Grade IV tumor cells are very abnormal. They grow and spread quickly into areas of the brain. Brooks et al. (1978) found that low-grade brain tumors (Grades I and II) look normal under microscope and grow slower than high-grade tumors (Grades III and IV). A low-grade tumor can transform into a high-grade tumor gradually. The transformation is more common among adults compared to children. A study on brain tumors using PET/MRI images developed in Buchbender et al. (2012). In this study, a developed possibilistic neuro-FCM algorithm (PNFCM) is presented to segment the tumor, WM, GM and skull based on imaging features at MRI of brain. To review, Nanthagopal and Sukanesh (2013) used wavelet statistical features and wavelet co-occurrence texture feature for segmentation and classification of benign and malignant tumor slices in brain computed tomography images. Following the extraction of features, they used probabilistic neural network (PNN) classifiers with the selected features to reduce algorithm dimension. The limitation of this method is that there is need for new training set whenever there is a change in the slice dataset, resulting in higher stability of SVM Gaussian classifiers and this method can be applied to only brain CT scan images. In Kalbkhani et al. (2013), first, the proposed method calculated two-level 2D discrete wavelet transform (2D DWT) of an input image. After feature vector normalization, principal component analysis (PCA) and linear discriminant analysis (LDA) are used to extract the proper features and remove the redundancy from the primary feature vector. Finally, the extracted features are applied to the KNN and SVM classifiers separately to determine the normal image or disease type. Experimental results indicate that the proposed algorithm achieves high classification rate and outperforms recently introduced methods while it needs a smaller number of features for classification. The proposed method in Sumitra and Saxena (2013) is to define a technique to enhance the classification of the MR human brain images based on feature extraction and independent component analysis (ICA) (SC-ICA). First, the MR images were segmented into various clusters. The comparative analysis with ICA based on SVM and other common classifiers were used for stability and efficacy of SC-ICA. The main limitation of the technique was feature extraction cost, which is high given the cluster. On the other hand, in Jayachandran and Dhanasekaran (2013), a hybrid algorithm is proposed for detecting the brain tumor in MR images using statistical features and fuzzy support vector machine (FSVM) classifier. In the first stage, anisotropic filter is applied for noise reduction

Precise Segmentation Techniques

145

and to make the image suitable for extracting features. In the second stage, it obtains the texture features related to MRI images. In the third stage, the features of MR images have been reduced using principles component analysis to the most essential features. At the last stage, the supervisor-classifier-based FSVM has been used to classify subjects as normal and abnormal brain MR images. The main limitation of the method was using principles component analysis reduced the low dimensions of texture feature. An approach is reported in Sharma and Meghrajani (2014) to investigate the brain tumor extraction from MRI images using mathematical morphological reconstruction and developed an SVM for detecting nonuniform intensity regions of the brain tumor. The significance of the study was the automatic combination of tumor area and then filtration of the base contour of the tumor. They used fuzzy patterns and functions for detection and filtration of tumor and its growing areas. Similarly, Bauer et al. (2013) worked on a brain tumor extracted from MRI images using mathematical reconstruction of the image and developed an SVM for computing the tumor area in cancerous process. The research in Kasturi et al. (2017) is to propose an algorithm combined with detection criterion for brain tumor segmentation. The research succeeded in computing the segmentation of 2D and 3D brain tumor MRI images with difference in brightness level of tumor and non-tumor area. Shenbagarajan et al. (2016) worked on brain tumor detection at MRI images using a region-based segmentation algorithm for classification of cancerous areas of brain. They used livewire G-wire algorithm based on extended multidimensional formula. The main feature of their method was the exact segmentation of brain tumor areas at noisy images. A method in Damodharan and Raghavan (2015) presented an effective brain tumor detection technique based on NN and a previously designed brain tissue segmentation. The proposed technique involved three steps for skull stripping, binarization via thresholding, morphological operators, and region-based binary mask extraction. The algorithms were used for preprocessing of the brain images, segmentation of pathological tissues (tumor), normal tissues, WM, GM, and CSF with fluid, extraction of the relevant features from each segmented tissues and classification of the tumor images with NN. Moreover, the experimental results and analysis are evaluated by means of quality rate (QR) with normal and the abnormal MRI images. The performance of the proposed technique has been validated and compared with the standard evaluation metrics such as sensitivity, specificity, and accuracy values for NN, KNN classification, and Bayesian classification techniques (Harouni and Baghmaleki, 2020). The obtained results depict that the proposed method with a detection rate of 83% had the best result. An algorithm for brain tumor segmentation is presented in Chandra and Rao (2016). The algorithm focused on the interest of soft thresholding DWT for enhancement and genetic algorithms (GAs) for image segmentation. The developed method achieved SNR value from 20 to 44 and segmentation accuracy from 82% to 97% of detected tumor pixels based on ground truth. Accordingly, a constant value is considered for the genes of the GA. Then, the suspicious regions with tumor detected by algorithm are matched based on what learned at learning phase. Next, if tumor areas are out of ROI, the number of genes is updated and then the algorithm is iterated. The iteration continues until the optimal number of regions of segmentation is achieved. The proposed method was tested on the images collected by the researchers and had the highest detection rate of 97.94%. In

146

Artificial Intelligence and Internet of Things

recent, A CNN architecture is presented in Havaei et al. (2017), a fully automatic brain tumor segmentation method, based on deep NNs (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. In the proposed architecture, the input images are linear and enter the convolution filters as two inputs, where one includes big input and the other small input. At the second step, max pooling was performed on images filtered with big filters to resize them as the match the size of the second input images. At pre-final stage, the filtered images of the second input are max out so that the final output layer enters the fully connected NN. The highest recorded sensitivity rate was 82% and the highest value of Dice parameter was 88%. Instead, a fully automated method for detection and segmentation of the abnormal tissue associated with brain tumor (tumor core and edema) from fluid-attenuated inversion recovery (FLAIR) MRI is proposed in Soltaninejad et al. (2017). The method evaluated on two datasets of 19 MRI FLAIR images of patients with gliomas of Grades II–IV, and BRATS 2012 dataset of 30 FLAIR images with 10 low-grade and 20 high-grade gliomas. Finally, the corresponding evaluation results had 88.09% precision. After that, a scheme for the detection and segmentation of the brain tumor from T1-weighted and FLAIR brain images is reported in Padlia and Sharma (2019). Fractional Sobel filter is used to prevent the effect of noise and enhance texture of the brain image. Histogram asymmetry method is applied to detect the hemisphere, which contained tumor. To segment the tumor region from the tumor hemisphere, the statistical features of a defined window are calculated and classified using SVM. Simulations are performed on the images, taken from the BRATS-2013 dataset, and performance parameters such as accuracy, sensitivity, and specificity for different values of α are computed. The research reported the highest detection rate of 98%. Chen et al. (2019) proposed a dual-force convolutional NN (CNN) for accurate brain tumor segmentation at MR images. They used the novel dual-force training scheme to promote the quality of multilevel features learnt from deep models. It can be applied to many exiting architectures, including DeepMedic NN. Finally, they proposed a novel multilayer perceptron-based post-processing approach to refine the prediction results of deep models. And then in 2020, an algorithm is presented in Vijh et al. (2020) to propose a tumor segmentation algorithm using OTSU embedded adaptive particle swarm optimization method and CNN. The analysis was performed in proposed work to provide automation in brain tumor segmentation. The adaptive particle swarm optimization along with OTSU was contributed to determine the optimal threshold value. Anisotropic diffusion filtering was applied on brain MRI to remove the noise and improve the image quality. The extracted features provided as data for training the CNN and performing classification. The proposed research achieved higher accuracy of 98%. Although the proposed model used OTUS to remove noise at MRI images, the deep-learning-based methods require big image database, which results in increased computation complexity and high load on computing systems. Moreover, in Gao et al. (2019), a DNN coupled with imaging strategy using nanoparticle is reported to propose a new tumor segmentation method. The deep network used higher level imaging using previous imaging algorithms and learning transformation. This method is highly useful in tumor detection. The proposed model was simulated at of GoogLeNet and also AlexNet DNNs. It has high precision but significant

147

Precise Segmentation Techniques

TABLE 5.1 Review of Related Literature Ref. Buchbender et al. (2012)

Method The possibilistic neuro-fuzzy C-means algorithm (PNFCM)

Limitations Low detection accuracy

Nanthagopal and Sukanesh (2013)

Combined two stable algorithm of WCT and WST DWT, PCA, SVM, KNN

Lack of stability for different dataset

Kalbkhani et al. (2013) Sumitra and Saxena (2013)

Single-channel independent component analysis (SC-ICA) Jayachandran and Texture features and fuzzy Dhanasekaran (2013) support vector machine (FSVM) classifier Bauer et al. (2013) Support vector machine (SVM) classifier Sharma and Meghrajani Active contour SVM and (2014) fuzzy algorithm

Advantages Addressing the main challenge of white matter usually mistaken with tumor High detection accuracy

Low detection accuracy High feature extraction cost

Simple implementation

Low detection accuracy due to increased dimension Analysis on only one dataset Weak tumor segmentation (low accuracy) Weak accuracy

Simple implementation

Shenbagarajan et al. (2016)

Livewire G-wire structural algorithm

Damodharan and Raghavan (2015)

Neural network and morphologic functions

Low accuracy

Chandra and Rao (2016)

Genetic algorithm

Havaei et al. (2017)

CNN

High computing load to reach ideal genetic algorithm mood Low accuracy

Vijh et al. (2020)

CNN

High computation load

High detection accuracy

Predicting tumor growth process High accuracy in tumor detection at images (two-class) Constant results against noise challenge Examining the main challenge of white and gray matter High accuracy

Predicting tumor growth pattern High accuracy at tumor region

computation complexities (Mughal et al., 2018; Ramzan et al., 2020a,b). Table 5.1 shows the literature of previous work on medical image segmentation methods. The detection of brain tumors at MRI images has always faced various challenges. Shape and random location of tumors resulted in low contracts, for which various methods have been proposed. This study proposed a simple and efficient method at three steps, preprocessing, segmentation and representation. At this section, various experimental models are evaluated using qualitative and quantitative methods and the results will be compared.

148

Artificial Intelligence and Internet of Things

FIGURE 5.15  Parameter and algorithm test accuracy

5.3.1.4  Evaluation Criteria Two measures of sensitivity and Dice similarity coefficients (Equations (5.44) and (5.45)) were often used in order to measure the efficacy of the proposed segmentation method and to compare it with the related efficient systems (Harouni and Baghmaleki, 2020). The evaluation parameters are defined as TP: number of true-positive pixels detected as tumor boundary, TN: number of true negative pixels detected as tumor boundary, FP: number of false-positive pixels detected as tumor boundary, and FN: number of false-negative pixels detected as tumor boundary (see Figure 5.15).

Sensitivity = Dice =

TP (5.44) TP + FN

2 × TP (5.45) (2 × TP) + FN + FP

5.3.1.5 Database The present studies used well-known BRATS2018 database (Menze et al., 2014). This database includes 250 LGG patients, where tumor is at hidden development stage and may cause paralysis in individuals. There are also 500 HGG patients, where tumor is resulted due to changing to LGG. Tumors usually begin to develop from white brain matter and gradually develop into the brain. These areas usually have random and abnormal boundary and different sizes. In HGG, the healthy tissue and brain tumor tissue are completely separated. In other words, disease is high developed so that tumor tissues are easily detected. In addition, tumor and normal tissue can hardly separable. There are five types of images, TIMR, POST GADOLINIUM, FIAIR, T2, T1 at this database, of which only FIAIR images (21 images) are used for testing proposed algorithm.

5.3.2 Lung Cancer Caner, as the uncontrollable growth of cells spreading to other organs, is a significant public health concern at global level. At present, cancer is the second leading cause of death in the United States following heart disease. However, a comprehensive

Precise Segmentation Techniques

149

investigation reported more than 6000 cases of death due to cancer. Accordingly, lung cancer is responsible as the most common cause of death due to cancer among men and women, so that more than 1.4% and 27% of all deaths are due to lung cancer (Feng et al., 2019; Hua et al., 2015). Lung cancer is a highly life-threatening disease, the main feature of which is uncontrollable growth of cells into lung tissues. In case of nontreatment, the abnormal growth of cells can lead to metastases into other organs and adjacent tissues, followed by death in many cases. Image-processingbased technique is a very good solution with good expense and low mistake in the detection of disease at initial stage. This section presents the lung anatomy and lung cancer. The efficient techniques, including CNN and morphology operators, used for tumor detection along with active contour algorithm are presented in this section. 5.3.2.1  Lung Anatomy The lungs are a pair of spongy, air-filled organs located on either side of the chest (thorax) divided into three parts called lobes. Left lung has two lobes and is smaller since it shares space in the chest with the heart. Figure 5.16 shows the lung anatomy. Air passes through trachea into the lungs. Trachea is divided into bronchi which further divide into alveolar ducts that give rise to the alveolar sacs that contain the alveoli. The alveoli absorb the air oxygen and eliminate carbon dioxide. The main function of lungs is to absorb oxygen the air oxygen and eliminate carbon dioxide. A thin layer called pleura wrap around the lings and facilitates the proper function

FIGURE 5.16  The lung anatomy details (Ye et al., 2019)

150

Artificial Intelligence and Internet of Things

of lungs during expansion and contraction. Lungs work in connection with heart to circulate oxygen all through the body. When heart circulates blood through cardiovascular circle, the oxygen-poor blood is pumped into lungs while returning to heart. The pulmonary artery carries blood from the right side of the heart to the lungs to pick up a fresh supply of oxygen. The aorta is the main artery that carries oxygenrich blood from the left side of the heart to the body. Coronary arteries are the other important arteries attached to the heart. They carry oxygen-rich blood from the aorta to the heart muscle, which must have its own blood supply to function. The alveolar membrane makes the air inside the respiratory system liquid. The oxygen released all over epithelium of alveoli sacs is released into the capillaries. Carbon dioxide also releases into the alveolar sacs from the blood inside the capillaries. Now oxygen-rich blood returns to the heart through pulmonary capillaries. Carbon dioxide is evacuated through lungs by aspiration (Gao et al., 2019). Air reaches lungs through breathing process. Diaphragm plays a key role in breathing. Diaphragm is a muscle, which separates chest and abdominal cavity. It has a domical shape at rest, which limits the space at the chest. When diaphragm is contracted, it moves down to the abdominal cavity that causes the expansion of chest. Accordingly, air pressure in lungs is reduced and so the air is absorbed through aspiratory airways; this is called aspiration. During exhalation, the diaphragm is relaxed which decreases the volume of the lung cavity pushing air out. Inspiration is an automatic function of neural system. It is controlled by a brain area called medulla oblongata. Neurons of this area sent signals to diaphragm and chest muscles to regulate the contractions for aspiration. 5.3.2.2  Lung Cancer All cancers began from the cells as the basic units making up the human body. Understanding the concept of cancer is necessary for awareness about how normal cells become cancerous. Body is made up of many cells. The cells usually grow and divide into different cells to protect body health and function. However, sometimes, the cells excessively grow and divide. These cells may form a mass called a tumor, which can be malignant or benign (Xie et al., 2019). The benign tumors are not cancerous and they are mostly removed and do not recur. The benign tumor cells do not develop into other parts of the body. It is essential to know that they are rarely life-threatening. The malignant tumors are cancerous. The tumors are abnormal cells, which are uncontrollably divided. They invade to and damage the adjacent tissue. The cancerous cells can develop into blood circulation system or lymphatic system (network of tissues and organs transporting lymph, a fluid containing infection-fighting white blood cells, throughout the body), from the malignant tumor. This is known as metastasis, through which cancer is developed from initial tumor and forms new tumors (secondary) at other organs. Figure 5.17 shows the cancerous tumor at left lung (Orobinskyi et al., 2019). Lung cancer is categorized histologically. This categorization is crucial for disease management and predicting the consequences. Lung cancer is categorized into two main histological groups: • Non-small-cell lung carcinoma (NSCLC) and • Small-cell lung carcinoma (SCLC)

Precise Segmentation Techniques

151

FIGURE 5.17  Tumor at left lung

This categorization is done based on microscopic analysis of cells. These cells have different types of development and metastases and so different treatments (Hossain et al., 2019). 5.3.2.2.1.  Non-Small-Cell Lung Carcinoma (NSCLC) Most cancers that start in the lung, known as primary lung cancers, are carcinomas (malignancy due to carcinoma). Lung carcinoma is categorized based on the size and shape of the malignant cells observed by microscopic analysis by a histopathology expert. Two main types are SCLC and NSCLC. NSCLC is the most common lung cancer with slow metastasis. NSCLC has three most common types, based on cancerous cells: • Squamous cell carcinoma (also squamous cells carcinoma, which is known as epidermoid carcinomas) • Large cell carcinoma • Adenocarcinoma 5.3.2.2.2  Small-Cell Lung Carcinoma (SCLC) It is a type of highly malignant cancer, most commonly arising within the lung. The SCLC includes neural secretion sacs containing endocrine hormones. Smallcell carcinoma of the lung usually presents in the central airways and infiltrates the submucosa leading to narrowing of bronchial airways. It is highly metastasis and is sometimes called “oat cell carcinoma” due to the flat cell shape and scanty cytoplasm. It has less prevalence compared to NSCLC. They grow quickly and spread to other nearby organs (Feng et al., 2019). 5.3.2.2.3  Lung Nodules The lung nodule is a small round mass that enlarges intrapulmonary and can be noted by the physician in a chest X-ray graph or CT scan images. These nodules being smaller than a green pea, as big as a golf ball, or even bigger are detected while the patient is being examined for other reasons (e.g. chest X-ray for pneumonia). Nodules can also be categorized as tumors. A lung nodule can be benign (noncancerous) or

152

Artificial Intelligence and Internet of Things

malignant (cancerous or possible cancerous). Some patients are subject to malignant lung nodules. Those who smoke or smoked before and those over 40 and other individuals suffering from various cancers are in danger of lung cancer. 5.3.2.3  A Review of Lung Segmentation Tumor detection is based on medical imaging techniques that are lung region extraction in MRI or CT scan images, lung region segmentation, differentiation between normal and tumor areas, and segmentation of lung tumor region. Various studies have been conducted in this regard, including lung nodule segmentation using ensemble learning is proposed in Farahani et al. (2015), in which the proposed computer-aided classification method used computed tomography (CT) images of the lung based on ensemble of three classifiers, MLP, KNN, and SVM. It had a detection accuracy of approximately 94%. First, the CT scan images are preprocessed and then the nodule features are extracted using DWT algorithm. Finally, the features are classified using combined classifiers. CT images were scanned by using GAs and morphological technique is presented in Jaffar et al. (2009). The GA has been used for automated segmentation of lungs. The SVM has been used to classify extracted ROIs that contain nodule. The proposed system is capable to perform fully automatic segmentation and nodule detection from CT scan lungs images. 97% of the ROIs contained lung nodules. In Namin et al. (2010), a research was reported to propose a computer-aided diagnosis (CAD) system for automated detection and classification of pulmonary nodules in 3D computerized tomography (CT) images. First, they segment lung parenchyma from the CT data using a thresholding method. Afterward, the research applied Gaussian filters for noise reduction and nodule enhancement. The features such as sphericity, mean and variance of the gray level, elongation, and border variation of potential nodules were extracted to classify detected nodules to malignant and benign groups. Fuzzy KNN is employed to classify potential nodules as non-nodule or nodule with different degrees of malignancy. The proposed method achieved sensitivity of 87.5% for nodule with a 3-mm dimension. Tan et al. (2011) used a CAD system for the detection of lung nodules in computed tomography images with four main functions: preprocessing, caner nodule detection, feature selection, and classification. The performance of the novel feature-selective classifier based on GAs and ANNs is compared with that of two other established classifiers, SVMs and fixed-topology NNs. The proposed method achieved sensitivity of 90% for nodule with a 3-mm dimension. A set of basic image processing techniques such as erosion, median filter, dilation, outlining, and lung border extraction are applied to the CT scan images in order to detect the lung region in Sharma and Jindal (2011). Then they used thresholding by OTSU algorithm and ROIs. Finally, they differentiated between cancerous and noncancerous regions using feature extraction and ANN. This system recognized lung nodules as small as 3 mm and 90% sensitivity. Following the preprocessing and noise elimination from CT scan images in Sivakumar and Chandrasekar (2013), they used weighted fuzzy clustering for segmentation and SVM for classification to differentiate cancerous and noncancerous nodules. They reached the accuracy, specificity, and sensitivity of 71.43%, 75%, and 70.83% for linear kernel; 66.07%, 52.94%, and 71.79% for polynomial kernel; and 80.36%, 76.47%, and 82.05% for radial base function (RBF) kernel, respectively. The detection of cancerous nodules from noncancerous one is presented in Keshani et al. (2013). The method first used active segmentation contour and then SVM classifier. The system could detect the cancerous nodules with

Precise Segmentation Techniques

153

an accuracy of 89%. Likewise, Kuruvilla and Gunavathi (2014) used a computer-aided classification method in computed tomography (CT) images of lungs developed using ANN. The entire lung was segmented from the CT images and the parameters were calculated from the segmented image. The statistical parameters like mean, standard deviation, skewness, kurtosis, fifth central moment, and sixth central moment are used for classification. Compared to feed-forward networks, backpropagation network gives better classification and skewness with the maximum classification accuracy. Two new training functions were proposed in this chapter. The results showed that the proposed training function 1 gives an accuracy of 93.3%, specificity of 100%, sensitivity of 91.4%, and a mean square error of 0.998. The proposed training function 2 gives a classification accuracy of 93.3% and minimum mean square error of 0.0942. A detection method of cancerous tumor at three stages, preprocessing, threshold segmentation, color segmentation, and feature extraction morphology (including average intensity, area, circumference, and eccentricity) is presented in Gajdhane and Deshpande (2014). SVM classifier is used to find tumors of various sizes. The research work in Orozco et al. (2015) used wavelet feature descriptor and SVM to detect lung tumor and then to extract the nodules areas. As a result, 11 features are selected and combined in pairs as inputs to the SVM, which is used to distinguish CT images containing cancerous nodules from those not containing nodules. The total preciseness obtained was 82%; the sensitivity was 90.90%, whereas the specificity was 73.91%. A method is proposed to use active contour model and C-mean fuzzy clustering to detect nodules in Nithila and Kumar (2016). The study used binary and Gaussian filters to simulate lung tissue and also segmented images using active contour model and classified nodules using C-means fuzzy clustering. This method could detect nodules with 3- to 30-mm size. Moreover, the method reduced the error rate significantly. The proposed method in Vorontsov et al. (2017) presented a hyperpixel algorithm for segmentation of ROIs and then used CNN for lung cancer detection. The method proposed the networks as end-to-end. First, network was used for learning and initial presentation of lung volume and then nodule area. CNN was used for computational evaluation. The Dice parameter was used and resulted in accuracy of 0.65 which is high compared to other methods. Finally, a method in Ye et al. (2019) proposed a new computer-based fuzzy algorithm for CT scan image processing for nodule detection. This method includes several steps. First, the ROIs on CT scan image are segmented using a fuzzy thresholding algorithm. Then natural and shape features were used for feature extraction. Finally, SVM was used to reduce positive error rate. The accuracy of 90.2% and positive error of 8 resulted. Various tumors and nodules at clinical application were very well recognized. The main limitation of the method was lack of detection at turbidity of glass floor and high positive error. Two measures of sensitivity and Dice similarity coefficients (Equations (5.44) and (5.45)) are often used in the related works. The methods proposed in tumor segmentation tried to solve the challenges, each of which had some limitations and disadvantages, as shown in Table 5.2. 5.3.2.4  Lung Database Long image database consortium (LIDC) is one of the most commonly used image databases for lung tumor detection (Armato III et al., 2011). The database includes 1018 CT scan images of 1010 patients with DICOM format. Figure 5.18 presents a sample of the database.

154

Artificial Intelligence and Internet of Things

TABLE 5.2 A Review of Literature on Tumor Segmentation Ref. Jaffar et al. (2009)

Method Genetic algorithm and MLP, KNN, and SVM

limitation High computation cost, supervised algorithm

Namin et al. (2010)

A computer-based detection system for early detection of lung tumor and classification with fuzzy KNN Classification with genetic algorithm and artificial neural network

Low efficiency, low computation cost

Tan et al. (2011)

High computation cost, complexity of the algorithm

Sharma and Using image enhancement Jindal (2011) techniques, using thresholding by OTSU algorithm and thresholding mechanism Sivakumar and Using weighted fuzzy clustering Chandrasekar for segmentation and SVM for (2013) classification of caner nodules

Lengthy processing process

Keshani et al. (2013)

Semi-supervised method

Gajdhane and Deshpande (2014)

Orozco et al. (2015) Nithila and Kumar (2016)

Vorontsov et al. (2017) Ye et al. (2019)

Using multiple steps for detection of cancerous nodules from normal cells and using active contour Detection of tumor at three steps of preprocessing, threshold segmentation and color and morphology extraction, including average intensity, area, circumference, and eccentricity Extracting nodule region using wavelet transformation and SVM Using active contour model and C-mean fuzzy clustering to detect nodules. They used binary and Gaussian filters to simulate lung tissue Using hyperpixel algorithm and then CNN for cancer segmentation Using fuzzy algorithm for nodules detection and SVM classifier

Low efficacy, supervised algorithm

Output only shows the difference in tumor size. It is a supervised method

Advantages Automatic segmentation and detection of lung nodules at CT scan images High sensitivity of 0/88% and using various features Fixed sensitivity of 87.5% topology for nodules bigger than 3 mm Detection of lung nodules with 3 mm diameter and 90% sensitivity the accuracy, specificity, and sensitivity of 71.43%, 75%, and 70.83% for linear kernel Good performance for detection of cancerous nodules from normal cells Using various techniques for detection and computation of tumor size

Complexity and high processing cost

Automatic segmentation and nodule detection

Complexity of the proposed algorithm

Significant reduction in errors, unsupervised algorithm

Complexity, notable Dice parameter

Applicable for lung tumors and nodules simultaneously High detection, fast 90.2% accuracy and computation for various positive error of 8 image condition

Precise Segmentation Techniques

155

FIGURE 5.18  Sample images from LIDC database

5.3.3. Liver Cancer The liver is an organ only found in vertebrates which detoxifies various metabolites, synthesizes proteins, and produces biochemicals necessary for digestion and growth. Viral hepatitis, excessive alcohol consumption, and obesity are the major factors causing liver injury. If liver disease is detected at early stage, there is chance to preserve and protect liver from serious problems. Accordingly, most of the studies tried to develop CAD system to detect injured or normal liver. Liver images have a grained appearance and are similar to other abdominal tissues such as kidney, lung, and heart, which results in difficult detection. This section describes liver anatomy along with methods for the detection of liver boundaries. Then, there will be a review of literature. 5.3.3.1  Liver Anatomy The liver is a wedge-shaped organ located at the right upper quadrant of the abdomen in front of stomach and behind the ribs. A human liver normally weighs approximately 1.5 kg. It is divided into four parts, left, right, caudate, and quadrate lobes. Each of which is wrapped by a thin layer that develops into the lobes and divides then into lobules, and the entire body blood passes liver every 2 minutes (Bismuth, 1982). The liver is a reddish-brown due to high blood content. The liver blood is supported by two sources: • The hepatic portal vein delivers around 75% of the liver’s blood supply and carries venous blood drained from the spleen, gastrointestinal tract, and its associated organs. • The hepatic arteries supply arterial blood to the liver, accounting for the remaining quarter of its blood flow. These blood vessels subdivide into small capillaries known as liver sinusoids, which then lead to lobules. Blood flows through the liver sinusoids and empties into the central vein of each lobule. The central veins coalesce into hepatic veins, which leave the liver and drain into the inferior vena cava. The liver mainly consists of hepatocytes, and the remainder includes phagocytic Kupffer cells and hepatic stellate (or fat-storing) cells. Hepatocytes have hexagonal structure. The hepatic portal blood is

156

Artificial Intelligence and Internet of Things

carried to the sinus from region 1 and passing through the sinusoids, it flows to the last hepatic portal vein. Finally, hepatic arteries supply arterial blood to the liver. The Kupffer cells are attached to sinusoidal endothelial cells. The primary function of the Kupffer cell is to remove foreign debris and particles that have come from the portal system when passing through the liver (Bilic et al., 2019). 5.3.3.2  Review of Literature on Liver Tumor Segmentation A level set technique for the automatic segmentation of the liver in abdominal CT scans is presented in Pan and Dawant (2001). The method proposed a novel speed function that is designed to stop the propagating front at organ boundaries with weak edges and incorporate an initial knowledge on the liver relative position and other structures. 3D PAN algorithm initiated segmentation with initial quantification of the curve by setting a small circle on the liver region. An operator is needed for early quantification of a circle for each region. An algorithm learns similarity parameter for ROIs is proposed in Pohle and Toennies (2002). Moreover, the similarity parameter was estimated for the sample ROIs. These regions were alternatively selected by a random assignment from the pixel position and similarity parameters were regularly updated. The main advantage of the method was automatic pixel position detection and segmentation. A statistical shape model constructed built from 20 manually segmented individual for liver in CT datasets is built in Lamecker et al. (2002). They proposed a new geometric method based on minimizing the distortion of the mapping between two surfaces. In this approach, an operator defines feature points through analysis of surfaces into patches. The patch boundaries are built by determining some points on the surface and then the shortest distance is computed. The mean of 2D shapes, pure translation for gravity center balance of shapes, and rigid transformation by MLS were computed. The PCA for analysis of the changes on a set of learning data to the corresponding set of surfaces. The study of an unsupervised liver segmentation algorithm with three steps is reported in Lim et al. (2005). In the preprocessing, they simplified the input CT image by estimating the liver position using a prior knowledge about the location of the liver and by performing multilevel threshold on the estimated liver position. The proposed scheme utilized the multiscale morphological filter recursively for labeling the region and clustering them to detect the search range in deformable contouring. Most of the liver contours were positioned within the search range. In order to perform an accurate segmentation, they produced the gradient-label map, which represents the gradient magnitude in the search range. The proposed algorithm performed deformable contouring on the gradient-label map by using regular patterns of the liver boundary. Further, a gradient vector flow snake model (GGVF-snake) based on canny algorithm for semiautomatic delineation of the liver contours on CT images is presented in Gui et al. (2007). Firstly, the CT images were enhanced and denoised by a method based on histogram equalization and anisotropic diffusion filtering; then, a manually delineated boundary using hermite-spline interpolation was chosen as the rough segmentation result; finally, an improved generalized gradient vector flow snake model (GGVF-snake) based on canny algorithm was adopted for refinement of the rough segmentation. Experiment results showed that the proposed method can precisely extract the liver region. In Freiman et al. (2008), using abdominal computed tomography angiography (CTA) images, novel semiautomatic liver volume estimation

Precise Segmentation Techniques

157

and segmentation and also its validation are presented. The hybrid algorithm used a multiresolution iterative scheme. It started from a single-user-defined pixel seed inside the liver and repeatedly applied smoothed Bayesian classification to identify the liver and other organs, followed by adaptive morphological operations and active contours refinement. They evaluated the algorithm with two retrospective studies on 56 validated CTA images. The first study compared it to ground truth manual segmentation and semiautomatic and automatic commercial methods. The second study used the public dataset SLIVER07 and its comparison methodology. The algorithm required minimal interaction by a nonexpert user and was accurate, efficient, and robust to initial seed selection. Moreover, an algorithm based on quadtree analysis to detect the soft tissue regions using EM is reported in Heimann et al. (2009). They used classification and regression C&RT method to initially determine region of liver. Finally, they used thresholding-based method to detect the ROI. This semiautomated algorithm is based on 2D developing regions with knowledge based on constraints. Following that, an approach for automatic segmentation of liver and tumor from CT images mainly used for CAD of liver is proposed in Kumar et al. (2011). The method uses regiongrowing, facilitated by pre- and post-processing functions for automatic segmentation of liver and alternative FCM (AFCM) algorithm for tumor segmentation. The effectiveness of the algorithm was evaluated by comparing automatic segmentation results to the manual segmentation results. Quantitative comparison shows a close correlation between the automatic and manual as well as high spatial overlap between the ROIs generated by the two methods. A method in Chi et al. (2011) presented a novel vessel context-based voting for automatic liver vasculature segmentation in CT images for full vessel segmentation and recognition of multiple vasculatures based on classification through multiple feature point voting. Recognizing abdominal liver cancer in CT images based on novel multi-instance learning (MIL) algorithm. MIL method is based on instance optimization (IO) and SVM with parameters optimized by a combination algorithm of particle swarm optimization and local optimization (CPSO-SVM) is presented in Jiang et al. (2013). Introducing MIL to liver cancer recognition can solve the problem of multiple ROI classifications. The proposed method consisted of two main steps: (1) obtaining the key instances through IO by texture features and a classification threshold in classification of instances with CPSO-SVM and (2) predicting unknown samples with the key instances and the classification threshold. The results showed that the proposed method recognizes liver cancer images in two kinds of cancer CT images and improves the recognition accuracy significantly. An research study that focused on a new automatic liver tumor segmentation in CT scans using fully convolutional networks (FCN) and nonnegative matrix factorization (NMF) is reported in Zheng et al. (2017). The study used low-resolution CT scan liver images. Energy was used for the main boundary detection. First, liver region is extracted using CNN and then nonnegative boundary segmentation matric was improved. However, the algorithm suffers computational complexity. Kasturi et al. (2017) proposed a method to locate and detect the cancerous cells effectively from the 2D and 3D CT scan images by reducing the detection error made by the physicians’ naked eye. Early stage of cancer is diagnosed by using image enhancement and Sobel edge detection. However, the results are not significant. The computational time was more than 1 minute and further research in medical images segmentation, recognition is still desired. Table 5.3 shows related literature.

158

Artificial Intelligence and Internet of Things

TABLE 5.3 Review of the Related Literature on Live Cancer Ref. Method Pan and Dawant Automatic 3D segmentation of the liver using level (2001) set, stopping the propagating front at organ boundaries with weak edges using speed function, and incorporate an initial knowledge on the liver relative position and other structures Pohle and Proposed a three-dimensional automatic Toennies segmentation algorithm. This algorithm learns (2002) similarity parameter for ROIs. Moreover, the similarity parameter estimated for the sample regions of interest. The regions were alternatively selected by a random assignment from the pixel position and similarity parameters were regularly updated Lamecker et al. Proposing a novel geometric approach based on (2002) minimizing the distortion of the mapping between two surfaces. Principal component analysis (PCA) for analysis of the changes on a set of learning data to the corresponding set of surfaces was used Lim et al. Three steps unsupervised liver segmentation (2005) algorithm, preprocessing, simplified the input CT image using estimating the liver position using a-prior knowledge about the location of the liver. Performing estimated liver position multilevel threshold. Multiscale morphological filter recursively with region labeling and clustering to detect the search range for deformable contouring Gui et al. (2007) Gradient vector flow snake model (GGVF-snake) based on canny algorithm for semiautomatic delineation of the liver contours on CT images Freiman et al. New algorithm for nearly automatic liver (2008) segmentation and volume estimation from abdominal computed tomography angiography (CTA) images and its validation at three steps Heimann et al. (2009)

Kumar et al. (2011)

Limitations and Advantages Semiautomatic, need for an operator for initial quantification, a circle for each segmentation region Automatic detection of seed points is the main advantage. Low accuracy is the main limitation

Semiautomated, time-consuming

Limitation using a-priori information on the relative position of the liver Advantage: high accuracy and speed of segmentation

Limitation: semiautomatic Advantage: simple implementation Limitation: defining seed by operator, semiautomatic, repeated algorithm, need for samples with proper distribution An algorithm based on quadtree analysis to detect theSemiautomated algorithm is soft tissue regions using expectation minimization. based on 2D developing regions They used classification and regression C&RT with knowledge based on method to initially determine region of liver. constraints Finally, they used thresholding-based method to detect the region of interest Region-growing, automatic segmentation of liver Semiautomated algorithm is based on 2D developing regions and based on prior information

Precise Segmentation Techniques

159

TABLE 5.3 (Continued) Review of the Related Literature on Live Cancer Ref. Chi et al. (2011)

Jiang et al. (2013)

Zheng et al. (2017)

Kasturi et al. (2017)

Method A novel vessel context-based voting (CBV) for automatic liver vasculature segmentation in CT images for full vessel segmentation and recognition of multiple vasculatures based on classification through multiple feature point voting Multi-instance learning (MIL) method to recognize liver cancer with abdominal CT images using abdominal CT scan

Limitations and Advantages Limitation: need for high contrast images

Ignoring the tumor region segmentation, low demand for precise liver region segmentation Automatic method to segment liver tumor in Using CT scan with low contrast abdomen images from CT scans by using fully and resolution. Advantages are convolutional networks (FCN) and nonnegative short computing time but the matrix factorization (NMF). liver region extracted parameters suffer from using CNN and then nonnegative boundary computation complexity segmentation matric was improved Locating and detecting the cancerous cells from the Advantages: works on both 2D 2D and 3D CT scan images effectively using and 3D images. It is very reducing the detection error, and using image simple but the results are not enhancement and Sobel edge detection significant. The computational time was more than 1 minute

5.3.3.3  Liver Database MICCAI-SLiver07 is one of the reliable databases, available at http://www.sliver07. org/ (Li et al., 2020). The database includes 30 CT scan images of 512 in 512. There are 46–502 slides for each image. The images have a thickness of 0.87.

5.4 CONCLUSION The present study aimed to present a comprehensive review of the segmentation methods for medical images. Accordingly, MRI, CT scan, mammography, and also color images for the segmentation of brain, liver and lung tumors were reviewed. The segmentation methods were divided into supervised and unsupervised categories. Some images of each category were investigated. Evaluation parameters and popular database for the segmentation of these images were also introduced.

REFERENCES ACHANTA, R., SHAJI, A., SMITH, K., LUCCHI, A., FUA, P. & SÜSSTRUNK, S. (2012). SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 2274–2282. ADAPA, D., JOSEPH RAJ, A. N., ALISETTI, S. N., ZHUANG, Z. & NAIK, G. (2020). A supervised blood vessel segmentation technique for digital fundus images using Zernike Moment based features. PLOS ONE, 15, e0229831.

160

Artificial Intelligence and Internet of Things

AKRAM, M. U. & KHAN, S. A. (2013). Multilayered thresholding-based blood vessel segmentation for screening of diabetic retinopathy. Engineering with Computers, 29, 165–173. AMIN, J., SHARIF, M., REHMAN, A., RAZA, M. & MUFTI, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy Research and Technique, 81(9), 990–996. AMIN, J., SHARIF, M., RAZA, M., SABA, T. & ANJUM, M. A. (2019a). Brain tumor detection using statistical and machine learning method. Computer Methods and Programs in Biomedicine, 177, 69–79. AMIN, J., SHARIF, M., RAZA, M., SABA, T. & REHMAN, A. (2019b). Brain tumor classification: feature fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. Doi: 10.1109/ICCISci.2019.8716449. AMIN, J., SHARIF, M., RAZA, M., SABA, T., SIAL, R. & SHAD, S. A. (2019c). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 1–9. https://doi.org/10.1007/s00521-019-04650-7 AMIN, J., SHARIF, M., YASMIN, M., SABA, T., & RAZA, M. (2019d) Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications, 79(15), 10955–10973. https:// doi.org/10.1007/s11042-019-7324-y ARMATO III, S. G., MCLENNAN, G., BIDAUT, L., MCNITT-GRAY, M. F., MEYER, C. R., REEVES, A. P., ZHAO, B., ABERLE, D. R., HENSCHKE, C. I. & HOFFMAN, E. A. (2011). The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical Physics, 38, 915–931. BADAWY, S. M., HEFNAWY, A. A., ZIDAN, H. E. & GADALLAH, M. T. (2017). Breast cancer detection with mammogram segmentation: a qualitative study. IJACSA International Journal of Advanced Computer Science and Applications, 8. Doi: 10.14569/IJACSA.2017.081016. BANSAL, A., AGARWAL, R. & SHARMA, R. SVM based gender classification using iris images. In Computational Intelligence and Communication Networks (CICN), 2012 Fourth International Conference on, 2012. IEEE, 425–429. Doi: 10.1109/ CICN.2012.192. BANSAL, M. & SINGH, N. Retinal vessel segmentation techniques: a review. In Proceedings of the World Congress on Engineering and Computer Science, San Francisco, CA, October 25–27, 2017. BAUER, S., WIEST, R., NOLTE, L.-P. & REYES, M. (2013). A survey of MRI-based medical image analysis for brain tumor studies. Physics in Medicine and Biology, 58, R97. BILIC, P., CHRIST, P. F., VORONTSOV, E., CHLEBUS, G., CHEN, H., DOU, Q., FU, C.-W., HAN, X., HENG, P.-A. & HESSER, J. (2019). The Liver Tumor Segmentation Benchmark (LiTS). arXiv preprint arXiv:1901.04056. BISMUTH, H. (1982). Surgical anatomy and anatomical surgery of the liver. World Journal of Surgery, 6, 3–9. BLACK, P. M. (1991). Brain tumors. New England Journal of Medicine, 324, 1555–1564. BROOKS, W., MARKESBERY, W., GUPTA, G. & ROSZMAN, T. (1978). Relationship of lymphocyte invasion and survival of brain tumor patients. Annals of Neurology, 4, 219–224. BUCHBENDER, C., HEUSNER, T. A., LAUENSTEIN, T. C., BOCKISCH, A. & ANTOCH, G. (2012). Oncologic PET/MRI, part 1: tumors of the brain, head and neck, chest, abdomen, and pelvis. Journal of Nuclear Medicine, 53, 928–938. CANNON, R. L., DAVE, J. V. & BEZDEK, J. C. (1986). Efficient implementation of the fuzzy C-means clustering algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8, 248–255.

Precise Segmentation Techniques

161

CHANDRA, G. R. & RAO, K. R. H. (2016). Tumor detection in brain using genetic algorithm. Procedia Computer Science, 79, 449–457. CHEN, S., DING, C. & LIU, M. (2019). Dual-force convolutional neural networks for accurate brain tumor segmentation. Pattern Recognition, 88, 90–100. CHI, Y., LIU, J., VENKATESH, S. K., HUANG, S., ZHOU, J., TIAN, Q. & NOWINSKI, W. L. (2011). Segmentation of liver vasculature from contrast enhanced CT images using context-based voting. IEEE Transactions on Biomedical Engineering, 58, 2144–2153. CHUANG, C.-C., LEE, Y.-T., CHEN, C.-M., HSIEH, Y.-S., LIU, T.-C. & SUN, C.-W. (2012). Patient-oriented simulation based on Monte Carlo algorithm by using MRI data. Biomedical Engineering Online, 11, 21. DAMODHARAN, S. & RAGHAVAN, D. (2015). Combining tissue segmentation and neural network for brain tumor detection. International Arab Journal of Information Technology (IAJIT), 12, 42–52. EJAZ, K., RAHIM, M. S. M., REHMAN, A., CHAUDHRY, H., SABA, T. & EJAZ, A. (2018a). Segmentation method for pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), 394–401. EJAZ, K., RAHIM, D. M. S. M., REHMAN, D. A. & EJAZ, E. F. (2018b). An Image-Based Multimedia Database and Efficient Detection though Features. VFAST Transactions on Software Engineering, 14(1), 6–15. EJAZ, K., RAHIM, M. S. M., BAJWA, U. I., RANA, N. & REHMAN, A. (2019). An unsupervised learning with feature approach for brain tumor segmentation using magnetic resonance imaging. In Proceedings of the 2019 9th International Conference on Bioscience, Biochemistry and Bioinformatics (pp. 1–7). https://doi.org/10.1145/3314367.3314384 EJAZ, K, RAHIM, M.S.M., BAJWA, U.I., CHAUDHRY, H., REHMAN, A. & EJAZ, F (2020). Hybrid segmentation method with confidence region detection for tumor identification. IEEE Access. Doi: 10.1109/ACCESS.2020.3016627. FARAHANI, F. V., AHMADI, A. & ZARANDI, M. F. Lung nodule diagnosis from CT images based on ensemble learning. In Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), 2015 IEEE Conference on, 2015. IEEE, 1–7. Doi: 10.1109/CIBCB.2015.7300281. FENG, L., DELACOSTE, J., SMITH, D., WEISSBROT, J., FLAGG, E., MOORE, W. H., GIRVIN, F., RAAD, R., BHATTACHARJI, P. & STOFFEL, D. (2019). Simultaneous evaluation of lung anatomy and ventilation using 4D respiratory-motion-resolved ultrashort echo time sparse MRI. Journal of Magnetic Resonance Imaging, 49, 411–422. FREIMAN, M., ELIASSAF, O., TAIEB, Y., JOSKOWICZ, L., AZRAQ, Y. & SOSNA, J. (2008). An iterative Bayesian approach for nearly automatic liver segmentation: algorithm and validation. International Journal of Computer Assisted Radiology and Surgery, 3, 439. GAJDHANE, A. V. & DESHPANDE, L. (2014). Detection of lung cancer stages on CT scan images by using various image processing techniques. IOSR Journal of Computer Engineering (IOSR-JCE), 16, 28–35. GAO, J., JIANG, Q., ZHOU, B. & CHEN, D. (2019). Convolutional neural networks for computer-aided detection or diagnosis in medical image analysis: an overview. Mathematical Biosciences and Engineering, 16, 6536. GAUCH, J. M. (1999). Image segmentation and analysis via multiscale gradient watershed hierarchies. IEEE Transactions on Image Processing, 8, 69–79. GUI, T., HUANG, L.-L. & SHIMIZU, A. Liver segmentation for CT images using an improved GGVF-snake. In SICE, 2007 Annual Conference, 2007. IEEE, 676–681. Doi: 10.1109/SICE.2007.4421068. HABIBI, N. & HAROUNI, M. (2018). Estimation of re-hospitalization risk of diabetic patients based on radial base function (RBF) neural network method combined with colonial competition optimization algorithm. Majlesi Journal of Electrical Engineering, 12, 109–116.

162

Artificial Intelligence and Internet of Things

HALDER, A., BHATTACHARYA, P. & KUNDU, A. (2019). Edge detection method using Richardson’s extrapolation formula. Soft Computing in Data Analytics. Singapore: Springer. HAROUNI, M. (2013). A Smoothing Embedded Segmentation Algorithm in Recognition of Online Handwritten Persian Character. Johor Bahru: Universiti Teknologi Malaysia. HAROUNI, M. & BAGHMALEKI, H. Y. 2018. Color Image Segmentation Metrics. Encyclopedia of Image Processing, 95. HAROUNI, M., MOHAMAD, D., RAHIM, M. S. M. & HALAWANI, S. M. (2012a). Finding critical points of handwritten Persian/Arabic character. International Journal of Machine Learning and Computing, 2, 573. HAROUNI, M., MOHAMAD, D., RAHIM, M. S. M., HALAWANI, S. M. & AFZALI, M. (2012b). Handwritten Arabic character recognition based on minimal geometric features. International Journal of Machine Learning and Computing, 2, 578. HAROUNI, M., RAHIM, M., MOHAMAD, D., REHMAN, A. & SABA, T. (2012c). Online cursive Persian/Arabic character recognition by detecting critical points. International Journal of Academic Research, 4, 208–213. HAROUNI, M., MOHAMAD, D. & RASOULI, A. Deductive method for recognition of online handwritten Persian/Arabic characters. In 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE), 2010. IEEE, 791–795. Doi: 10.1109/ICCAE.2010.5451869. HAROUNI, M., RAHIM, M., AL-RODHAAN, M., SABA, T., REHMAN, A. & AL-DHELAAN, A. (2014). Online Persian/Arabic script classification without contextual information. The Imaging Science Journal, 62, 437–448. HAVAEI, M., DAVY, A., WARDE-FARLEY, D., BIARD, A., COURVILLE, A., BENGIO, Y., PAL, C., JODOIN, P.-M. & LAROCHELLE, H. (2017). Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35, 18–31. HEIMANN, T., VAN GINNEKEN, B., STYNER, M. A., ARZHAEVA, Y., AURICH, V., BAUER, C., BECK, A., BECKER, C., BEICHEL, R. & BEKES, G. (2009). Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Transactions on Medical Imaging, 28, 1251–1265. HIJAZI, M. H. A., COENEN, F. & ZHENG, Y. Retinal image classification using a histogram based approach. In The 2010 International Joint Conference on Neural Networks (IJCNN), 2010. IEEE, 1–7. Doi: 10.1109/IJCNN.2010.5596320. HOSSAIN, S., NAJEEB, S., SHAHRIYAR, A., ABDULLAH, Z. R. & HAQUE, M. A. A pipeline for lung tumor detection and segmentation from CT scans using dilated convolutional neural networks. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019. IEEE, 1348–1352. Doi: 10.1109/ICASSP.2019.8683802. HUA, K.-L., HSU, C.-H., HIDAYATI, S. C., CHENG, W.-H. & CHEN, Y.-J. (2015). Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and Therapy, 8., 2015–2022. JAFFAR, M. A., HUSSAIN, A., JABEEN, F., NAZIR, M. & MIRZA, A. M. (2009). GA-SVM based lungs nodule detection and classification. Signal Processing, Image Processing and Pattern Recognition. Berlin, Heidelberg: Springer. JAIN, A. K. (2010). Data clustering: 50 years beyond k-means. Pattern Recognition Letters, 31, 651–666. JAYACHANDRAN, A. & DHANASEKARAN, R. (2013). Brain tumor detection and classification of MR images using texture features and fuzzy SVM classifier. Research Journal of Applied Sciences, Engineering and Technology, 6, 2264–2269. JIANG, H., ZHENG, R., YI, D. & ZHAO, D. (2013). A novel multiinstance learning approach for liver cancer recognition on abdominal CT images based on CPSO-SVM and IO. Computational and Mathematical Methods in Medicine, 2013. https://doi. org/10.1155/2013/434969

Precise Segmentation Techniques

163

KALBKHANI, H., SHAYESTEH, M. G. & ZALI-VARGAHAN, B. (2013). Robust algorithm for brain magnetic resonance image (MRI) classification based on GARCH variances series. Biomedical Signal Processing and Control, 8, 909–919. KASS, M., WITKIN, A. & TERZOPOULOS, D. (1988). Snakes: active contour models. International Journal of Computer Vision, 1, 321–331. KASTURI, M. M., SUTHA, P. & JAYANTHI, V. Efficient edge detection method for diagnosis of 2D and 3D lung and liver images. In Inventive Systems and Control (ICISC), 2017 International Conference on, 2017. IEEE, 1–5. Doi: 10.1109/ICISC.2017.8068661. KESHANI, M., AZIMIFAR, Z., TAJERIPOUR, F. & BOOSTANI, R. (2013). Lung nodule segmentation and recognition using SVM classifier and active contour modeling: a complete intelligent system. Computers in Biology and Medicine, 43, 287–300. KHAN, A. R., KHAN, S., HAROUNI, M., ABBASI, R., IQBAL, S., & MEHMOOD, Z. 2021. Brain tumor segmentation using K-means clustering and deep learning with synthetic data augmentation for classification.  Microscopy Research and Technique. 1–11. https://doi.org/10.1002/jemt.23694 KHAN, F. (2018). Automated segmentation of CT liver images: a review. Journal of Communications Technology, Electronics and Computer Science, 19, 5–9. KHAN, T. M., MEHMOOD, M., NAQVI, S. S. & BUTT, M. F. U. (2020). A region growing and local adaptive thresholding-based optic disc detection. PLOS ONE, 15, e0227566. KUMAR, M. A., KUMAR, M. & SHESHADRI, H. (2019). Computer aided detection of clustered microcalcification: a survey. Current Medical Imaging Reviews, 15, 132–149. KUMAR, S., MONI, R. & RAJEESH, J. (2011). Automatic segmentation of liver and tumor for CAD of liver. Journal of Advances in Information Technology, 2, 63–70. KURUVILLA, J. & GUNAVATHI, K. (2014). Lung cancer classification using neural networks for CT images. Computer Methods and Programs in Biomedicine, 113, 202–209. LAMECKER, H., LANGE, T. & SEEBASS, M. A statistical shape model for the liver. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2002. Springer, 421–427. https://doi.org/10.1007/3-540-45787-9_53 LI, Y., ZHAO, Y.-Q., ZHANG, F., LIAO, M., YU, L.-L., CHEN, B.-F., & WANG, Y.-J. (2020). Liver segmentation from abdominal CT volumes based on level set and sparse shape composition. Computer Methods and Programs in Biomedicine, 195, 105533. LIM, S.-J., JEONG, Y.-Y. & HO, Y.-S. Segmentation of the liver using the deformable contour method on CT images. In Pacific-Rim Conference on Multimedia, 2005. Springer, 570–581. https://doi.org/10.1007/11581772_50 MANJARAMKAR, A. & MANESH, K. (2018). Connected component clustering based hemorrhage detection in color fundus images. International Journal of Intelligent Engineering and Systems, 11, 143–151. MARR, D. & HILDRETH, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London. Series B. Biological Sciences, 207, 187–217. MEDINA, R., LLAMAS, J., GÓMEZ-GARCÍA-BERMEJO, J., ZALAMA, E. & SEGARRA, M. (2017). Crack detection in concrete tunnels using a Gabor filter invariant to rotation. Sensors, 17, 1670. MENZE, B. H., JAKAB, A., BAUER, S., KALPATHY-CRAMER, J., FARAHANI, K., KIRBY, J., BURREN, Y., PORZ, N., SLOTBOOM, J. & WIEST, R. (2014). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34, 1993–2024. MOHAMMADI DASHTI, M. & HAROUNI, M. (2018). Smile and laugh expressions detection based on local minimum key points. Signal and Data Processing, 15, 69–88. MUGHAL, B., MUHAMMAD, N., SHARIF, M., REHMAN, A., & SABA, T. (2018). Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC Cancer, 18(1), 1–14. NAMIN, S. T., MOGHADDAM, H. A., JAFARI, R., ESMAEIL-ZADEH, M. & GITY, M. Automated detection and classification of pulmonary nodules in 3D thoracic CT

164

Artificial Intelligence and Internet of Things

images. 2010 IEEE International Conference on Systems Man and Cybernetics (SMC), 2010. IEEE, 3774–3779. Doi: 10.1109/ICSMC.2010.5641820. NANTHAGOPAL, A. P. & SUKANESH, R. (2013). Wavelet statistical texture featuresbased segmentation and classification of brain computed tomography images. IET Image Processing, 7, 25–32. NG, H., ONG, S., FOONG, K., GOH, P. & NOWINSKI, W. Medical image segmentation using k-means clustering and improved watershed algorithm. In 2006 IEEE Southwest Symposium on Image Analysis and Interpretation, 2006. IEEE, 61–65. Doi: 10.1109/ SSIAI.2006.1633722. NITHILA, E. E. & KUMAR, S. (2016). Segmentation of lung nodule in CT data using active contour model and Fuzzy C-mean clustering. Alexandria Engineering Journal, 55, 2583–2588. OROBINSKYI, P., PETRENKO, D. & LYASHENKO, V. Novel approach to computer-aided detection of lung nodules of difficult location with use of multifactorial models and deep neural networks. In 2019 IEEE 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM), 2019. IEEE, 1–5. Doi: 10.1109/CADSM.2019.8779340. OROZCO, H. M., VILLEGAS, O. O. V., SÁNCHEZ, V. G. C., DOMÍNGUEZ, H. D. J. O. & ALFARO, M. D. J. N. (2015). Automated system for lung nodules classification based on wavelet feature descriptor and support vector machine. Biomedical Engineering Online, 14, 9. PADLIA, M. & SHARMA, J. (2019). Fractional Sobel filter based brain tumor detection and segmentation using statistical features and SVM. Nanoelectronics, Circuits and Communication Systems. Singapore: Springer. PAN, S. & DAWANT, B. M. (2001). Automatic 3D segmentation of the liver from abdominal CT images: a level-set approach. Medical Imaging 2001: Image Processing. International Society for Optics and Photonics, 128–139. Doi: 10.1117/12.431019. POHLE, R. & TOENNIES, K. D. (2002). Three-level evaluation process for segmentation methods in medical imaging. In Medical Imaging 2002: Image Processing. International Society for Optics and Photonics, 287–298. Doi: 10.1117/12.467169. RASHID, S. (2013). Computerized exudate detection in fundus images using statistical feature based fuzzy c-mean clustering. International Journal of Computing and Digital Systems, 2. http://dx.doi.org/10.12785/IJCDS/020305. REHMAN, A., HAROUNI, M. & SABA, T. (2020). Cursive multilingual characters recognition based on hard geometric features. International Journal of Computational Vision and Robotics, 10, 213–222. REHMAN, Z. U., NAQVI, S. S., KHAN, T. M., KHAN, M. A. & BASHIR, T. (2019). Fully automated multi-parametric brain tumour segmentation using superpixel based classification. Expert Systems with Applications, 118, 598–613. RAMZAN, F., KHAN, M. U. G., REHMAT, A., IQBAL, S., SABA, T., REHMAN, A. & MEHMOOD, Z. (2020a). A deep learning approach for automated diagnosis and multiclass classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), 37. RAMZAN, F., KHAN, M. U. G., IQBAL, S., SABA, T. & REHMAN, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, 103697–103709. RONNEBERGER, O., FISCHER, P. & BROX, T. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015. Cham: Springer, 234–241. RUBINI, S. S., KUNTHAVAI, A., SACHIN, M. & VENKATESH, S. D. (2018). Morphological contour based blood vessel segmentation in retinal images using Otsu thresholding. International Journal of Applied Evolutionary Computation (IJAEC), 9, 48–63.

Precise Segmentation Techniques

165

SABA, T. (2019). Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique, 82(9), 1601–1609. SABA, T., SAMEH, A., KHAN, F., SHAD, S.A., SHARIF, M. (2019). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43 (12), 332. SABA, T., MOHAMED, A. S., EL-AFFENDI, M., AMIN, J., & SHARIF, M. (2020). Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, 221–230. SAFFARZADEH, V. M., OSAREH, A. & SHADGAR, B. (2014). Vessel segmentation in retinal images using multi-scale line operator and k-means clustering. Journal of Medical Signals and Sensors, 4, 122. SAMALA, R. K., CHAN, H. P., HADJIISKI, L., HELVIE, M. A., WEI, J. & CHA, K. (2016). Mass detection in digital breast tomosynthesis: deep convolutional neural network with transfer learning from mammography. Medical Physics, 43, 6654–6666. SHARMA, D. & JINDAL, G. Identifying lung cancer using image processing techniques. In International Conference on Computational Techniques and Artificial Intelligence (ICCTAI), 2011. 115–120. Pattaya, Thailand. SHARMA, Y. & MEGHRAJANI, Y. K. Brain tumor extraction from MRI image using mathematical morphological reconstruction. In 2014 2nd International Conference on Emerging Technology Trends in Electronics, Communication and Networking (ET2ECN), 2014. IEEE, 1–4. Doi: 10.1109/ET2ECN.2014.7044982. SHENBAGARAJAN, A., RAMALINGAM, V., BALASUBRAMANIAN, C. & PALANIVEL, S. (2016). Tumor diagnosis in MRI brain image using ACM segmentation and ANN-LM classification techniques. Indian Journal of Science and Technology, 9. Doi: 10.17485/ijst/2016/v9i1/78766. SINGH, N. P., KUMAR, R. & SRIVASTAVA, R. Local entropy thresholding based fast retinal vessels segmentation by modifying matched filter. In International Conference on Computing, Communication & Automation, 2015. IEEE, 1166–1170. Doi: 10.1109/ CCAA.2015.7148552. SIVAKUMAR, S. & CHANDRASEKAR, C. (2013). Lung nodule detection using fuzzy clustering and support vector machines. International Journal of Engineering and Technology, 5, 179–185. SOLTANINEJAD, M., YANG, G., LAMBROU, T., ALLINSON, N., JONES, T. L., BARRICK, T. R., HOWE, F. A. & YE, X. (2017). Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI. International Journal of Computer Assisted Radiology and Surgery, 12, 183–203. SUMITRA, N. & SAXENA, R. K. (2013). Brain tumor classification using back propagation neural network. International Journal of Image, Graphics and Signal Processing, 5, 45. TAN, M., DEKLERCK, R., JANSEN, B., BISTER, M. & CORNELIS, J. (2011). A novel computer-aided lung nodule detection system for CT images. Medical Physics, 38, 5630–5645. TAPIA, J. E., PEREZ, C. A. & BOWYER, K. W. Gender classification from iris images using fusion of uniform local binary patterns. In European Conference on Computer Vision, 2014. Springer, 751–763. https://doi.org/10.1007/978-3-319-16181-5_57 THOMAS, V., CHAWLA, N. V., BOWYER, K. W. & FLYNN, P. J. Learning to predict gender from iris images. In First IEEE International Conference on Biometrics: Theory, Applications, and Systems, 2007. BTAS 2007, 2007. IEEE, 1–5. Doi: 10.1109/ BTAS.2007.4401911. VIJH, S., SHARMA, S. & GAURAV, P. (2020). Brain tumor segmentation using OTSU embedded adaptive particle swarm optimization method and convolutional neural network. Data Visualization and Knowledge Engineering. Cham: Springer.

166

Artificial Intelligence and Internet of Things

VORONTSOV, E., CHARTRAND, G., TANG, A., PAL, C. & KADOURY, S. (2017). Liver lesion segmentation informed by joint liver segmentation. arXiv preprint arXiv:1707.07734. WIHARTO, W. & SURYANI, E. (2020). The comparison of clustering algorithms k-means and fuzzy c-means for segmentation retinal blood vessels. Acta Informatica Medica, 28, 42. WU, Q. (2019). Segmentation-Based Retinal Image Analysis. Karlskrona, Sweden: Blekinge Institute of Technology. XIAOMING, L., KE, X., PENG, Z. & JIANNAN, C. (2019). Edge detection of retinal OCT image based on complex shearlet transform. IET Image Processing, 13, 1686–1693. XIE, H., YANG, D., SUN, N., CHEN, Z. & ZHANG, Y. (2019). Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognition, 85, 109–119. YE, Z., CHEN, Q., ZHANG, Y., ZOU, J. & ZHENG, Y. (2019). Identification of vortex structures in flow field images based on convolutional neural network and dynamic mode decomposition identification of vortex structures in flow field images based on convolutional neural network and dynamic mode decomposition. Traitement Du Signal, 36, 501–506. ZHANG, H. & GUAN, X. Iris recognition based on grouping KNN and rectangle conversion. In 2012 IEEE 3rd International Conference on Software Engineering and Service Science (ICSESS), 2012. IEEE, 131–134. Doi: 10.1109/ICSESS.2012.6269422. ZHENG, S., FANG, B., LI, L., GAO, M., WANG, Y. & PENG, K. (2017). Automatic liver lesion segmentation in CT combining fully convolutional networks and non-negative matrix factorization. Imaging for Patient-Customized Simulations and Systems for Point-of-Care Ultrasound. Cham: Springer. ZHENG, S., CORNELISSEN, L. J., CUI, X., JING, X., VELDHUIS, R. N., OUDKERK, M. & VAN OOIJEN, P. (2020). Efficient convolutional neural networks for multi-planar lung nodule detection: improvement on small nodule identification. arXiv preprint arXiv:2001.04537.

6

Lung Cancer Detection and Diagnosis with Deep Learning Models Evaluation Tanzila Saba, Muhammad Kashif, Hind Alaskar, Erum Afzal

CONTENTS 6.1 Introduction................................................................................................... 167 6.2 Diagnosis....................................................................................................... 168 6.3 Treatments..................................................................................................... 171 6.4 Prevention...................................................................................................... 173 6.5 Datasets.......................................................................................................... 175 6.5.1 Kaggle Data Science Bowl (KDSB) 2017 Challenge Dataset........... 175 6.5.2 Lung Nodule Analysis 2016 (LUNA16) Challenge Dataset.............. 175 6.5.3 Japanese Society of Radiological Technology (JSRT) Dataset......... 175 6.5.4 Lung Image Database Consortium and Image Database Resource Initiative (LIDC/IDRI)...................................................... 176 6.5.5 ChestX-Ray14 Dataset....................................................................... 176 6.5.6 Lung1 Dataset.................................................................................... 176 6.6 Deep Learning Techniques............................................................................ 176 6.7 Analysis and Findings................................................................................... 177 6.8 Conclusions and Future Challenges............................................................... 178 References�������������������������������������������������������������������������������������������������������������� 179

6.1 INTRODUCTION Lung cancer is identified as the second most fatal amongst different kinds of cancers existed globally. The prostate and bosom malignancy is normal in males and females, respectively. The results show the survival rate reduction due to the most severe lung malignancy. The deaths of 150,000 were reported in the 225,000 registered lung cancer patients in the United States with the $12 billion budget spend only on lung cancer (Saba, 2019). In 2018, almost 1.8 million (18.4%) overall lung cancer deaths were reported in around 9.6 million cancer deaths. Lung cancer has two main types, non-small-cell lung cancer (NSCLC) and small cell lung cancer 167

168

Artificial Intelligence and Internet of Things

(SCLC), approximately 85% and 15% cases, respectively. Since 1985, lung cancer has remained a severe disease globally based on prevalence and survival rate (Saba et al., 2019b). The deep learning (DL) strategies are recently put in place by experts in machine learning (ML) and provide a good reliable solution for all computer vision applications (Khan et al., 2019a), such as skin cancer (Javed et al., 2019a,b; Khan et al., 2019b; Saba et al., 2019a), agriculture (Khan et al., 2020a,c), breast cancer (Mughal et al., 2017, 2018a,b; Saba et al., 2019c), blood cancer (Abbas et al., 2015, 2018a,b, 2019a,b) and brain tumour (Amin et al., 2018, 2019a,b,c,d; Ejaz et al., 2018, 2020; Rehman et al., 2020). A basic profound learning model typically consists of various layers, such as convolution that translates the input image into different patches, ReLu activation mechanism, the feature reduction pooling layer and the fully connected (FC) layer. In the convolutional neural network (CNN), which extracts the strongly correlated and depth-based features of a particular problem such as different classifications of tumours, the FC layer is most relevant. Many medical imaging methods such as computed tomography (CT), chest X-ray, magnetic resonance imaging (MRI) and positron emission tomography (PET) are used to detect various kinds of cancers. The CT images are profound in finding the tumour growth as well as low noise and better than other methods (Afza et al., 2019; Hussain et al., 2020; Khan et al., 2019a,b,c; Liaqat et al., 2020; Saba, 2020; Saba et al., 2020a). Currently, CT screening in the detection of lung cancer is a successful method due to comprehensive nodules size and localities data. In 1980, the efficacy and mortality rate improvement, computer-aided diagnosis (CAD) techniques, were developed and helped physicians examine images. Several ML techniques such as support vector machine (SVM), K-nearest neighbors (KNN), Naive Bayes (NB), decision trees, random forest, linear regression and others have shown deep health effectiveness (Husham et al., 2016; Hussain et al., 2020; Iftikhar et al., 2017; Iqbal et al., 2017, 2018; Jamal et al., 2017). In this chapter, the efficiency of DL techniques has been discussed for lung cancer detection and diagnosis. The cause of lung cancer is the unnecessary growing cell in the lung by developing tissues and blood vessels’ outer surfaces. The typical symptoms contain cough, breath squatness, pain in the chest and weight loss. Doctors examine the CT scans for cell growth to distinguish the infection (Javed et al., 2019a,b, 2020a,b). Various researches reported the effectiveness of low-dose helical CT by identifying early-stage lung cancers and tumours better than chest radiography (National Lung Screening Trial Research Team, 2011).

6.2 DIAGNOSIS In the last several decades, cancer diagnosis and treatment have been one of the major challenges confronted by humankind (Al-Ameen et al., 2015; Amin et al., 2019c,d; Ejaz et al., 2018, 2020; Fahad et al., 2018). Lung cancer is caused due to the unusual growth of normal cells in the lung(s) convert into cancer through blood flow with oxygen. Four stages are determined by cancer locality and further spread. The appearance of a pulmonary nodule in the lung points out (not all) lung cancer of ~30-mm in diameter at the initial stage. The analyses of tiny nodules are a challenge

Lung Cancer Detection and Diagnosis

169

for experts (Khan et al., 2020a,b). Chemotherapy is used to treat complex lung cancer, where 50% of patients are identified at the final stages (Fahad et al., 2018; Husham et al., 2016; Shah et al., 2020). CAD systems were developed for precise cancer diagnosis and treatment due to high diagnosis costs and fewer experts. DL techniques are accomplished with various speech, object recognition and natural language processing (NLP) complications. DL, especially CNNs, has distinctive possessions of three-dimensional (3D) invariance and multi-feature evaluation (Hussain et al., 2020; Khan et al., 2020c,d; Rao et al., 2016). 3D U-Net combined with Capsule Networks (CapNets) investigated to collect low- and high-level (edges and semantic) nodule information. The results reported in 0.84%, 92.9% and 84.5% of area under the ROC curve (AUC), sensitivity and accuracy, respectively, but low specificity of 70% due to the excessiveness of benign samples. This research is limited due to a lack of sample size and not taking nodule less than 3-cm diameter that changes the results. The solution for these issues would be to take a large sample size and categorize pulmonary nodules, DL parameters and lesion size, appearance and geometric features (Lung et al., 2014; Yang et al., 2020). Massion et al. (2020) executed lung cancer prediction CNN (LCP-CNN) model by utilizing two datasets from Vanderbilt University Medical Center (VUMC) and Oxford University Hospitals National Health Service Foundation Trust (OUH) to detect indeterminate pulmonary nodules (IPNs). DL networks (3D CNN) were executed on seven different datasets—three radiotherapy datasets (Harvard, Radboud and Maestro containing 317, 147 and 307 NSCLC stage I–IIIb patients, respectively), three surgery datasets (Moffitt, MUMC and M-SPORE containing 200, 90 and 101 NSCLC stage I–IIIb patients, respectively) and a stability assessment dataset (RIDER containing 32 patients with NSCLC) to predict survival rate of two years according to CT images with some limitation due to the data, tests and sensitivity variations of medical and imaginary acquirement factors (Hosny et al., 2018; Iftikhar et al., 2017; Iqbal et al., 2017, 2018, 2019). Figure 6.1 shows the activation map in CT images. The F-fluorodeoxyglucose/ positron emission tomography (FDG-PET) data (3936 PET slices) using deep neural networks (DNNs) was examined to predict lung cancer. The clinical outcomes show that the AUCs for PET100% (standard dose images), and PET10% (reduced dose) and PET3.3% reconstruction were 0.989, 0.983 and 0.970. Neural networks (NNs) achieved 95.9% and 91.5% sensitivity and 98.1% and 94.2% specificity at ultralowdose PET3.3% and standard dose, respectively. However, 43, 48 for PET3.3%, 14, 23 and 14, 23 false positive and false negative were reported, respectively (Schwyzer et al., 2018). Benign and the malignant nodules appeared similar at initial stages. Therefore, two deep 3D-modified mixed-link networks (CMixNet) and faster R-CNN on efficiently learned features from CMixNet to detect nodules were investigated for accurate malignancy diagnosis. Gradient boosting machine (GBM) used for classification executed on LUNA16 and LIDC-IDRI datasets resulted in 94% sensitivity and 91% specificity (Nasrullah et al., 2019). 3D DL ProNet and radionics com_radNet methods were developed for lung adenocarcinomas (ADCs), squamous cell carcinomas (SCCs) and SCLCs classification by Guo et al. (2020). The evaluation of both ProNet and com_radNet model using CT images data of 920 patients in terms of F1 scores

170

Artificial Intelligence and Internet of Things

FIGURE 6.1  (a) Malignant and (b) benign lung CT images

for ADC, SCC and SCLC abnormalities is 72.4%, 90.0% and 83.7%, respectively, and of 73.2%, 83.1%, 75.4%, 85.1%, respectively. The accuracy of and area under the receiver operating characteristic curve were reported 71.6% and 74.7%, respectively, for ProNet model. The accuracy of and area under the receiver operating characteristic curve were reported 0.840 and 0.789, respectively, for com_radNet model. DNN-based whole-slide training method and unified memory (UM) CNNs were proposed for the classification of lung cancer types through whole-slide images (WSIs). The analysis conducted on a dataset containing 9662 hematoxylin and eosin (H&E) damaged samples composed of 2843 cancer or non-cancer tissue resulted in 0.924 and 0.950 AUC SCC ADC by distinct test set, respectively. Researchers explained that the class activation map (CAM) technique exposes the risky areas associated with tumour regions. UM is used to control GPU memory by influencing the short-term data to host memory due to the effectiveness variations (Chen et al., 2020; Jamal et al., 2017; Javed et al., 2019a, 2020a). A multi-stream multiscale technique based on convolutional networks (ConvNet) was evaluated on Italian MILD screening trial (containing 943 patients and 1352 nodules) and Danish DLCST screening trial (containing 468 patients and 639 nodules) by Ciompi et al. (2017). Two CNN-based Lung-RADS and PanCan proposed nodules detection and rating risk in terms of size and additional features from the identified nodules (Majid et al., 2020; Marie-Sainte et al., 2019a,b; Mittal et al., 2020). Techniques were evaluated on 1030 CT cases with 5010 regions of interesting (ROIs) comprising two sets thin group (less or equals 2 mm) and thick group (greater or equals 5 mm). CT images resulted in an accuracy of 92.4% and 93.4% for a thin group and thick group, respectively. Lung-RADS screening might decrease the rate of false positive for lung cancer than the state of the art (Saba, 2019). Figure 6.2 shows CT images of different stage cancers.

Lung Cancer Detection and Diagnosis

171

FIGURE 6.2  LC-type PET images: (a) squamous cell carcinoma, (b) adenocarcinoma and (c) SCLC

A cascaded DL technique was proposed by Shrey et al. (2020) for segmentation and classification of malignant or benign lung nodules. They employed CT images for experiments and attained 97.96% and 95.67% accuracy and AUC, respectively. The real-world experiments performed in five different Chinese clinics by Pulmonary Nodules Artificial Intelligence Diagnostic System (PNAIDS) evaluation through 534 persons containing 611 lung CT images nodule size of 5- to 30-mm diameter. The outcome shows 76.5%, 63.0% and 75.3% for AUC, sensitivity and specificity, respectively. According to variances with Mayo Clinic’s model, AI results recommended 11 members were negative and positive, according to experts. Most were diagnosed malignant due to CACs (genetically abnormal cells) found in their blood. The study is limited due to fewer data; it might improve the performance if tested on more data (Xu et al., 2020). Two different techniques were proposed based on state-of-the-art DL (Khan et al., 2019e; Ramzan et al., 2020a,b; Adeel et al., 2020; Rad et al., 2013, 2016), modified AlexNet (MAN) with SVM for classification of lung CT and chest X-ray images (pneumonia and normal) and serial fusion and principal component analysis (PCA) added for classification accuracy enhancement. The proposed technique was evaluated on the LIDC-IDRI dataset with 97.27% accuracy, but 3.7% of normal patient images were misclassified as pneumonia (Bhandary et al., 2020). Modified SegNet and CNNs were proposed by Agnes et al. (2020) for lung segmentation out of chest CT images and for normalizing batch (BN) on the LUNA16 dataset. The results reported 94.8% and 0.89 ± 0.23 of sensitivity (nodules classified into cancer and non-cancer) and average DICE coefficient, respectively, after the PCA validation. The critical analysis indicated a high misclassification rate of true nodules as false nodules on CNN without BN and lowered with CNN + BN.

6.3 TREATMENTS SCLC treatments are incompetent and endured for the last 30 years without proper cures (Norouzi et al., 2014; Perveen et al., 2020; Qureshi et al., 2020; Rahim et al., 2017a,b). The concatenation of low-dose CT by multidisciplinary complex treatment techniques comprises minimally invasive surgery evaluating 13,491 patients’ data two years prior from Zhejiang Chinese province for lung cancer diagnosis. The use of LDCT in primary diagnosing cancer from smokers and non-smokers

172

Artificial Intelligence and Internet of Things

TABLE 6.1 List of Treatments against Lung Cancer Author Ott et al. (2017) Antonia et al. (2016) Reck et al. (2016) Brahmer et al. (2012) Van Meerbeeck et al. (2020)

Cancer Stage Ib I,II III I III

Treatments Pembrolizumab Nivolumab and ipilimumab variation Ipilimumab and platinum-etoposide vs. platinum-etoposide αPD-L1 antibody Atezolizumab and platinum doublet vs. platinum doublet

achieves better performance (Zhu et al., 2020). Table 6.1 provides a list of evaluated treatments. Pneumonectomy treatment is significant in lung cancer investigated by the evaluation of Dutch Lung Cancer Audit for Surgery (DLCA-S) data selected 8446 patients from 51 different hospitals in the Netherlands. Multivariable logistic regression is used for pneumonectomy recognition and hospital-wise proportion prediction through observed/expected ratio (O/E ratio). The results reported 0.80 AUC for multivariable logistic regression (Beck et al., 2020). Multiple techniques such as the logistic regression model, Kaplan-Meier method and Cox proportional-hazard model were utilized for treatment, survival and mortality assessment, respectively, of primary lung cancers. Among 583 early cancer patients, 71.9% and 46.7% were treated through curative intent and surgery. Existing smokers were possibly treated through radiotherapy and chemotherapy, not through surgery—the survival rate of 87.8% for two years and 70% for two years with surgery treatment. Survival and treatment compared among Māori and non-Māori and with stereotactic ablative body radiotherapy (SABR) reported unaffected results (Lawrenson et al., 2020). A male lung cancer adult patient in China reported through reverse transcription polymerase chain reaction (RT-PCR) coronavirus infection was cured by treating lopinavir/ritonavir (Kaletra). The analysis reported possibilities of preserving treatment continuousness in healthier patients aiming for crucial examination in the future (Iqbal et al., 2019; Khan et al., 2017). The pooled epidermal growth factor receptor (EGFR) tyrosine kinase inhibitor by cytotoxic chemotherapy in complex NSCLC reported extreme treatment effectiveness (Hosomi et al., 2020). NSCLC’s serious infections of stage 4 were treated through platinum-based doublet chemotherapy. Adaptable immunotherapy mechanisms reported better performance for the final complex NSCLC (Shah et al., 2020). The ramucirumab and docetaxel (R&D) efficiency assessment was analysed as the third phase when platinum-based chemotherapy and immune-checkpoint-inhibitor (ICI) treatments were unsuccessful as the first and second phases, respectively, for NSCLC final stages. The effectiveness is reported based on progression-free survival (PFS), objective response rate (ORR) and overall survival (OS) measures and aimed to be examined as the second phase after failing as third-phase treatment (Brueckl et al., 2020). The analysis of atezolizumab antiviral against PD-L1 accomplished improvements for NSCLC. Major enhancements were reported in terms of

Lung Cancer Detection and Diagnosis

173

PFS and OS in IMpower research for NSCLC subtypes. The atezolizumab is used as a key drug by adding carboplatin and nab-paclitaxel and carboplatin and etoposide for complex non-squamous NSCLC and SCLC, respectively (Manzo et al., 2020). The investigation suggested by European Society of Medical Oncology (ESMO) for survival treatment shows the effective usage of irinotecan and cisplatin, especially in Asians, not by adding PCI or radiotherapy for extensive-stage ED-SCLC. Survival for ED-SCLC and limited-SCLC (LDSCLC) cancer stages achieves an enhancement in both the situations and is low in the case of carboplatin and etoposide, but this treatment is not studied well due to fewer data. The momentous differences are established by the use of cisplatin and carboplatin in their long-lasting survival by origin. The study is limited due to considering data without groups. It is important for using several factors. The outcomes represented survival, not the real world, but only observed research was involved (Mashood Nasir et al., 2020; Mughal et al., 2017, 2018a,b; Nazir et al., 2019; Yousaf et al., 2019a,b). Due to the variations during NSCLC final stages (IIIAN2), treatment disagreements were found. Neoadjuvant chemotherapy (NA-C) treatment results were compared by sequential or concurrent chemoradiotherapy (CRT) treatment for 53 infected patients. The PSF and OS reported effectiveness among NA-C, surgery and CRT techniques. The NA-C immunotherapy could alter treatment culture in the upcoming years (Van Meerbeeck et al., 2020). Biomarker such as blood and breath represents effectiveness examination through bronchoscopy, surgery (video-aided thoracoscopic), radiation, (adjuvant) immunotherapy treatments and amalgamation of some multi-analyte CancerSEEK tests (ctDNA and protein (old serum cancer) markers) and low offensive substitutions—however, still under examination (McLellan et al., 2020). Nivolumab treatment for advanced NSCLC therapy is undergone by examining blood samples. The outcomes recommendation might assess anti-PD-1 treatment’s prediction by lymphocyte division’s dissemination and programmed cell death protein-1 (PD-1) on T cells’ appearance (Ottonello et al., 2020).

6.4 PREVENTION The prevention presents the reports based on the published data. The United States Preventive Services Task Force (USPSTF) suggested LDCT lung screening annually for smokers amongst 55–80 years old who smoke a minimum of 30 packs per year or quit within 15 years (Saba et al., 2012; Sadad et al., 2018; Ullah et al., 2019). Lung patients with allograft dysfunction and immunosuppression infections increased cancer risk than normal patients and possibly appeared after lung transplant (Saba, 2017). HIV patients are on the risk of lung cancer due to smoking, lung disease, immune system deficiency, pneumonia or tenderness infection risk factors (Marcus et al., 2017). Several studies indicate the risk of lung cancer based on diet and alcohol that eating deep-fried or red meat in high amounts might result in nitrosamines creation through the cooking process, whole or soaked fat and drinking alcohol and coffee (Malhotra et al., 2016). Krstić (2017) reported the radon is the second and most important risk factor in the United States and Canada in lung cancer. Yearly prediction of case reports in America shows that 13% radon, 9% insufficient diet, 3% ETS, 8% PM2.5, 2% arsenic and 2% asbestos (overall 37%) result in lung cancer (Saba et al., 2019b).

174

Artificial Intelligence and Internet of Things

Currently, the number of smokers is around 1.1 billion globally; if the rate remains alike, it will rise to 1.9 billion by 2025. Several lung cancer cases are diagnosed in non-smokers due to ecological experience, air contamination, alterations and single-nucleotide polymorphisms (Akhtar et al., 2017; Saba et al., 2018a,b). A DNN model is developed by Chen et al. (2020), including environmental, disease history, behavioural and demographic risk factors by open-source Behavioral Risk Factor Surveillance System (BRFSS) of 50 US states, 235,673 individuals. The results show better performance with (0.927–0.962) accuracy, and the area under the curve is 0.913–0.931. The study has limitations in taking open-source data rather than individuals genomic and dietetic and region-based factors. Bernatsky et al. (2018) evaluated lung cancer risk factors in 14 systemic lupus erythematosus (SLE) cohorts by considering mean accustomed HR followed by assessing demographics, medications, infection history, and smoking information through SLE Disease Activity Index (SLEDAI) adjusted model scores. The major risk factor of adaptable lung cancer identified might be smoking than other medications in SLE. The analysis of 15 research studies contained lung cancer patients (4732) and controls (4337) to evaluate the effectiveness of lung cancer through GC genes and vitamin D receptor (VDR). The Cdx2 and Bsm1 reduced, whereas Taq1 polymorphism increased lung cancer (Duan et al., 2020). Akhtar et al. (2017) reported the association of different factors with lung cancer without strong indications such as smoking marijuana, inappropriate diet, drinking alcohol, Epstein-Barr, HIV and human papillomavirus (HPV) infections. The study by Yu et al. (2019) investigated the combination of food with alcohol and smoking using unconditional logistic regression for analysis of 1902 infected patients of age ranges 24–90 years from the two Chinese hospitals and 2026 controls of age ranges 23–87 years of normal patients’ data from Jan 2006 to Dec 2013. The result shows 2.954- and 6.774-fold increase in risk by comparing with those who eat fired food and smoked food less or more than three times/week, respectively. The mutual comparison indicates 2.108-fold increase in lung cancer risk as non-smokers and smoking drinkers had 0 and 1 dietary risk scores, respectively. Women have a greater lung cancer risk due to hormone replacement therapy (HRT) (Stapelfeld et al., 2020). The analysis of 161,808 women from Women’s Health Initiative (WHI) data in the United States was performed for taking extra vitamins B6, B12 and folic acid. HR and CI were evaluated through Cox regression techniques for vitamin B consumption and lung cancer relation risk. Vitamin B6 consumption greater or equal to 50 mg/day resulted in 16% in the risk of lung cancer reduction but not expressive in smoking, vitamin B12 and folic acid (Brasky et al., 2020). A CXR-LC model based on a CNN was proposed to identify lung cancer through an electronic medical record (EMR), NLST and Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. The results showed performance of CXR-LC in terms of AUC and sensitivity which is better than CMS eligibility in the case of PLCO (0.755 vs. 0.634) and (74.9% vs. 63.8%), respectively, and reported that the smokers have a high risk of lung cancer. Critical and decision curve analysis reported the lung cancer occurrence reduction of 30.7%. The research is limited due to the data taken from PLCO and NLST, mostly of one generic and older smoker. Big data could be evaluated. Radiographs of symptoms

Lung Cancer Detection and Diagnosis

175

TABLE 6.2 Datasets and Their Access Links Dataset Kaggle Data Science Bowl KDSB (2017) LUNA16 JSRT LIDC/IDRI ChestX-ray14 Lung1

Access Links https://www.kaggle.com/c/data-science-bowl-2017/data https://luna16.grand-challenge.org/ http://db.jsrt.or.jp/eng.php https://wiki.cancerimagingarchive.net/display/Public/ LIDC-IDRI https://academictorrents.com/details/557481faacd824c83 fbf57dcf7b6da9383b3235a https://wiki.cancerimagingarchive.net/display/Public/ NSCLC-Radiomics

were not included, and low-resolution (224 pixels) chest radiographs were used, resulting in performance degradation (Lu et al., 2020). A segmentation-free survival analysis system is based on DL and survival analysis techniques (locality-constrained linear coding (LLC), based bag of words (BoW), encoding algorithm, Cox proportional hazards model and biomarker interpretation module). The experiments were evaluated through The Cancer Genome Atlas (TCGA) dataset on the Kaplan-Meier estimate and concordance index (c-index) log-rank test (p-value) (Cui et al., 2020).

6.5 DATASETS This section presents benchmark datasets for lung cancer diagnosis. Table 6.2 presents different datasets and their access links.

6.5.1 K aggle Data Science Bowl (KDSB) 2017 Challenge Dataset The dataset contains 2101 patients’ data with ids and 0 and 1 labels denoted noncancerous and cancerous 3D CT images that have a resolution of 512 × 512 pixels in DICOM set-up (Sori et al., 2019).

6.5.2 Lung Nodule Analysis 2016 (LUNA16) Challenge Dataset The LUNA16 dataset involves CT scan data and a nodule label of 888 patients’ data that comprises labelled data for 888 patients having several nodule parameters with 512 × 512-pixel images. The candidate’s CSV file contains 1120/1186 nodules from 551,065 candidates (Sori et al., 2019).

6.5.3  Japanese Society of Radiological Technology (JSRT) Dataset JSRT dataset contains 93 uninfected and 154 lung nodules infected images amongst 247 chest X-ray frontal images having a high resolution of 2048 × 2048 pixels. The nodule size varies 5–40 mm in diameter.

176

Artificial Intelligence and Internet of Things

6.5.4 Lung Image Database Consortium and Image Database Resource Initiative (LIDC/IDRI) The database consists of 1018 patients’ lung nodules CT images in DICOM formats approximately 150–550. It includes malignant, benign, metastatic and unknown images (Rao et al., 2016).

6.5.5 ChestX-Ray14 Dataset The dataset of 30,805 patients having chest X-ray frontal images is currently the largest available database, including several parameters and 14 disease labels such as “Atelectasis, Consolidation, Infiltration, Pneumothorax, Edema, Emphysema, Fibrosis, Effusion, Pneumonia, Pleural_thickening, Cardiomegaly, Nodule, Mass and Hernia”. The PNG resolution is 1024 × 1024 pixels (Ausawalaithong et al., 2018).

6.5.6 Lung1 Dataset The dataset contains 422 NSCLC infected images obtained from TCIA (“the Cancer Imaging Archive”).

6.6  DEEP LEARNING TECHNIQUES CAD will decrease the observational overhead and false-negative rates. It is software that examines a radiographic image to detect specific diseases (Saba et al., 2019a, 2020a). A CAD system helps radiologists detect and diagnose abnormalities earlier and faster (Saba et al., 2018a,b; Ullah et al., 2019). Fortunately, DNNs have recently gained considerable interest in feature computation due to their ability to learn mid- and high-level image representation (Rehman et al., 2021). According to WHO data, lung cancer mortality is 20%. If detected in the initial phases, the LDCT treatment presented significant performance. DL techniques are extremely efficient while evaluated on big data. Numerous methods have been proposed in recent years (Rehman et al., 2018a,b,c; Saba, 2020; Saba et al., 2020b). DL is of mainly two types: supervised and unsupervised learning. Supervised learning models include NN, CNN (ConvNet), deep CNNs (general multiclassification and segmentation models) and recurrent neural networks (RNNs). Unsupervised learning models include auto-encoders (AEs), stacked AEs (SAEs), AE and SAE variations, restricted Boltzmann machines (RBMs), deep belief networks (DBNs), variational AEs and generative adversarial networks (Ramzan et al., 2020a,b). CNN evaluation provides better recognition of medical imaginary and classification process. Several existing research developed the adaptable CNN, that is, DenseNet (Khan et al., 2020a,b,c,d), ResNet (He et al., 2016), Inception (Perveen et al., 2020), AlexNet (Qureshi et al., 2020), VGG (Majid et al., 2020; Nazir et al., 2019) and more, for chest X-rays to detect defects (Mughal et al., 2018a,b).

Lung Cancer Detection and Diagnosis

177

TABLE 6.3 Performance Measures Based on CT Images Author

Techniques

Database

AUC (%) Accuracy Sensitivity Specificity F1 Score (%) (%) (%) (%) Bhandary MAN-SVM LIDC-IDRI _ 97.27 98.09 95.63 97.95 et al. (2020) Chest X-rays _ 96.8 96.97 96.63 96.78 Chae et al. CT-lungNET Chest CT 0.85 _ _ _ _ (2020) images Yang et al. 3D U-Net + LIDC-IDRI 0.84 84.50 92.90 70.00 _ (2020) CapNets integration Pang et al. Adaptable CT images _ _ _ _ _ (2020) VGG16-T Majid et al., 3D CNN LUNA 16 80 _ _ _ 2020 Qin et al. MFSCA0.92 0.72 _ _ _ CT + PET (2020) DenseNet (GMU+ joint optimization)

6.7  ANALYSIS AND FINDINGS This section reports the analysis based on the existing state-of-the-art DL techniques. The early and final stages of cancer are limited only to the lungs and dispersed further regions. Due to the low level of available indications, a lung cancer diagnosis is challenging through existing treatment techniques comprising biopsies and imaging and more fatality enhancement. The analysis through CAD of less than 10-mm diameter nodule detection from chest CT of approximately 200 mm × 400 mm × 400 mm presented that the patient has primary lung cancer (Mughal et al., 2017). The analysis of different studies presented in Table 6.3. CT-lungNET based on DL was proposed to predict malignancy from 208 pulmonary nodules of 173 whole unenhanced chest CT images comprising size 5–20 mm. The results enhanced in terms of AUROC and confidence interval of 0.85 and 95%, respectively. The study has various limitations due to taking small test sets and 2-mm slice thickness resulting in external validation and nodule classification consequences, respectively. However, it is proposed only for the classification not for the detection (Chae et al., 2020). Automatic Cancer Detection and Classification in Whole-slide Lung Histopathology (ACDC@LungHP) developed and evaluated WSI of 200 patients to segment lung cancer tissues pixel-wise. Ten research comparisons were performed based on the single- and multi-model techniques. The multi-model performs better than the single-model method DICE coefficient of 0.79660.0898 and 0.75440.099, respectively. The future challenge will be based on the WSI biopsy method to classify lung cancer subtypes from approximately 4000 slides. The networks are based on fine-grained features (channel and spatial dimensions) and GMU for lung cancer diagnosis. The results reported in terms of area under the

178

Artificial Intelligence and Internet of Things

ROC curve and accuracy of 0.92% and 0.72 %, respectively. The research is limited due to the small dataset; less segmented labelled data was used to use large and labelled data to achieve high-level performance (Qin et al., 2020). A DNN U-Net model with CNN was developed and evaluated on a radiomic dataset containing 435 cancer patients and 3363 benign nodules (NLST CT images ranges from 4 to 30 mm of diameter). Lasso regression was utilized for dataset reduction to 40 features and then combined with every image, respectively. The result of U-Net model with CNN described 86.0% and 88.4% of precision and accuracy, respectively. The performance did not enhance due to a combination of U-Net model with CNN 40 features accomplished 89.0% and 84.3% of precision and accuracy, respectively (Masquelin et al., 2020). An algorithm based on DL was proposed to evaluate existing and preceding CT images for lung cancer risk prediction. The technique involves four steps containing: lung segmentation (Mask-RCNN evaluated on LUNA dataset), cancer ROI detection model (LIDC and NLST dataset were used), full-volume model (3D Inception V1 (CNN) using ImageNet dataset) and cancer risk prediction model; during all the process, TensorFlow platform (Google Inc.) was used. The results reported AUC, false positives and false negatives of 94.4%, 11% and 5%, respectively, using the National Lung Cancer Screening Trial dataset. The study has several consequences: the general radiologist’s rate was higher and lower in the preceding and NLST data. Fewer cancer data was predicted during the early screening of lung cancer (Ardila et al., 2019). 3D CNN-based on a DNN was proposed to detect initial stages from 3D CT images of lung cancer. The thresholding method and Vanilla 3D CNN classifier were used for preprocessing and classification process LUNA 16 dataset resulted in 80% accuracy. The study is limited by considering fewer data and adding extra features (i.e., nodule size, texture and position) for performance enhancement (Majid et al., 2020). Nanoplasmonic sensing technique, DL-based SERS (surface-enhanced Raman spectroscopy), proposed to analyse cell and likeness with human plasma exosomes (nanosized extracellular vesicles in the blood) detect initial stages. The classification result shows 0.912 and 95% of AUC and accuracy, respectively. The technique effectively detected the initial lung cancer with some limitations of clarity and many cancer cells, low patient samples and less memory (Shin et al., 2020).

6.8  CONCLUSIONS AND FUTURE CHALLENGES Lung cancer is frequently identified due to fewer symptoms and knowledge. During the early and final stages, cancer diagnosis and treatment of cancer cell detection are crucial. Experts’ analysis of tiny nodules examination is most difficult and inefficient because human visualization is limited. In 2020, approximately 228,820 older people experience through diagnosis procedures. Early lung cancers diagnose CT images in lung nodules (≤3 diameters) ~5%–10%. This research analysed diagnosis, treatments, prevention and clinical manifestation based on DL techniques followed by a brief description of lung datasets and DL techniques. The future challenges are to develop the models that classify concurrent resolutions and parameters variant big data, although effectiveness sustains. More

Lung Cancer Detection and Diagnosis

179

clinical manifestations and techniques are important for cancer cells and exosome separation due to limitations and variations, particularly for miRNAs. The extra crizotinib medication resistance will be acute during ALK+ NSCLC (anaplastic lymphoma kinase (ALK) gene) treatment. The future immunotherapy techniques will be expected for cancer treatments due to obtaining ICIs conflict for complex cancer patients.

REFERENCES Abbas, N., Mohamad, D., Abdullah, A.H., Saba, T., Al-Rodhaan, M., and Al-Dhelaan, A. (2015). Nuclei segmentation of leukocytes in blood smear digital images. Pakistan Journal of Pharmaceutical Sciences, 28(5), pp. 1801–1806. Abbas, N., Saba, T., Mehmood, Z., Rehman, A., Islam, N., and Ahmed, K.T. (2019a). An automated nuclei segmentation of leukocytes from microscopic digital images. Pakistan Journal of Pharmaceutical Sciences, 32(5), pp. 2123–2138. Abbas, N., Saba, T., Mohamad, D., Rehman, A., Almazyad, A.S., and Al-Ghamdi, J.S. (2018a). Machine aided malaria parasitemia detection in Giemsa-stained thin blood smears. Neural Computing and Applications, 29(3), pp. 803–818, doi:10.1007/ s00521-016-2474-6. Abbas, A., Saba, T., Rehman, A., Mehmood, Z., Javaid, N., Tahir, M., Khan, N.U., Ahmed, K.T., and Shah, R. (2019b) Plasmodium species aware based quantification of malaria, parasitemia in light microscopy thin blood smear. Microscopy Research and Technique, 82(7), pp. 1198–1214. doi: 10.1002/jemt.23269. Abbas, N., Saba, T., Rehman, A., Mehmood, Z., Kolivand, H., Uddin, M., and Anjum, A. (2018b). Plasmodium life cycle stage classification based quantification of malaria parasitaemia in thin blood smears. Microscopy Research and Technique, 82(3), pp. 283–295, doi.org/10.1002/jemt.23170. Adeel, A., Khan, M. A., Akram, T., Sharif, A., Yasmin, M., Saba, T., and Javed, K. (2020). Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Systems. https://doi.org/10.1111/exsy.12569. Afza, F., Khan, M.A. Sharif, M., and Rehman, A. (2019) Microscopic skin laceration segmentation and classification: a framework of statistical normal distribution and optimal feature selection. Microscopy Research and Technique, 82(9), pp. 1471–1488. https:// doi.org/10.1002/jemt.23301 Agnes, S.A. and Anitha, J. (2020). Appraisal of deep-learning techniques on computeraided lung cancer diagnosis with computed tomography screening. Journal of Medical Physics, 45(2), p. 98. Akhtar, N. and Bansal, J.G. (2017). Risk factors of lung cancer in nonsmoker. Current Problems in Cancer, 41(5), pp. 328–339. Al-Ameen, Z., Sulong, G., Rehman, A., Al-Dhelaan, A., Saba, T., and Al-Rodhaan, M. (2015) An innovative technique for contrast enhancement of computed tomography images using normalized gamma-corrected contrast-limited adaptive histogram equalization. EURASIP Journal on Advances in Signal Processing, 32, doi:10.1186/ s13634-015-0214-1. Amin, J., Sharif, M., Raza, M., Saba, T., and Anjum, M.A. (2019a). Brain tumor detection using statistical and machine learning method. Computer Methods and Programs in Biomedicine, 177, pp. 69–79. Amin, J., Sharif, M., Raza, M., Saba, T., and Rehman, A. (2019b). Brain Tumor Classification: Feature Fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. doi:10.1109/ICCISci.2019.8716449.

180

Artificial Intelligence and Internet of Things

Amin, J., Sharif, M., Raza, M., Saba, T., Sial, R., and Shad, S.A. (2019c). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 32(20), pp. 15965–15973. Amin, J., Sharif, M., Rehman, A., Raza, M., and Mufti, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy Research and Technique, 81(9), pp. 990–996. Amin, J., Sharif, M., Yasmin, M., Saba, T., and Raza, M. (2019d). Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications, 79(15), pp. 10955–10973, https://doi. org/10.1007/s11042-019-7324-y Antonia, S.J., López-Martin, J.A., Bendell, J., Ott, P.A., Taylor, M., Eder, J.P., Jäger, D., Pietanza, M.C., Le, D.T., de Braud, F., and Morse, M.A. (2016). Nivolumab alone and nivolumab plus ipilimumab in recurrent small-cell lung cancer (CheckMate 032): a multicentre, open-label, phase 1/2 trial. The Lancet Oncology, 17(7), pp. 883–895. Ardila, D., Kiraly, A.P., Bharadwaj, S., Choi, B., Reicher, J.J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., and Naidich, D.P. (2019). End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine, 25(6), pp. 954–961. Ausawalaithong, W., Thirach, A., Marukatat, S., and Wilaiprasitporn, T. (2018, November). Automatic lung cancer prediction from chest X-ray images using the deep learning approach. In 2018 11th Biomedical Engineering International Conference (BMEiCON) (pp. 1–5). IEEE. Beck, N., van Brakel, T.J., Smit, H.J., van Klaveren, D., Wouters, M.W., and Schreurs, W.H. (2020). Pneumonectomy for lung cancer treatment in the Netherlands: between-hospital variation and outcomes. World Journal of Surgery, 44(1), pp. 285–294. Bernatsky, S., Ramsey-Goldman, R., Petri, M., Urowitz, M.B., Gladman, D.D., Fortin, P.R., Yelin, E.H., Ginzler, E., Hanly, J.G., Peschken, C., and Gordon, C. (2018). Smoking is the most significant modifiable lung cancer risk factor in systemic lupus erythematosus. The Journal of Rheumatology, 45(3), pp. 393–396. Bhandary, A., Prabhu, G.A., Rajinikanth, V., Thanaraj, K.P., Satapathy, S.C., Robbins, D.E., Shasky, C., Zhang, Y.D., Tavares, J.M.R., and Raja, N.S.M. (2020). Deep-learning framework to detect lung abnormality—a study with chest X-ray and lung CT scan images. Pattern Recognition Letters, 129, pp. 271–278. Brahmer, J.R., Tykodi, S.S., Chow, L.Q., Hwu, W.J., Topalian, S.L., Hwu, P., Drake, C.G., Camacho, L.H., Kauh, J., Odunsi, K., and Pitot, H.C. (2012). Safety and activity of antiPD-L1 antibody in patients with advanced cancer. New England Journal of Medicine, 366(26), pp. 2455–2465. Brasky, T.M., Ray, R.M., Navarro, S.L., Schenk, J.M., Newton, A.M., and Neuhouser, M.L. (2020). Supplemental one-carbon metabolism related B vitamins and lung cancer risk in the Women’s Health Initiative. International Journal of Cancer, 147, 1374–1384. Brueckl, W.M., Reck, M., Rittmeyer, A., Kollmeier, J., Wesseler, C., Wiest, G.H., Christopoulos, P., Tufman, A., Hoffknecht, P., Ulm, B., and Reich, F. (2020). Efficacy of docetaxel plus ramucirumab as palliative third-line therapy following second-line immune-checkpoint-inhibitor treatment in patients with non-small-cell lung cancer stage IV. Clinical Medicine Insights: Oncology, 14. doi:10.1177/1179554920951358. Chae, K.J., Jin, G.Y., Ko, S.B., Wang, Y., Zhang, H., Choi, E.J., and Choi, H. (2020). Deep learning for the classification of small (≤2 cm) pulmonary nodules on CT imaging: a Preliminary Study. Academic Radiology, 27(4), pp. 55–63. Chen, S. and Wu, S. (2020). Identifying lung cancer risk factors in the elderly using deep neural networks: quantitative analysis of web-based survey data. Journal of Medical Internet Research, 22(3), p. e17695.

Lung Cancer Detection and Diagnosis

181

Ciompi, F., Chung, K., Van Riel, S.J., Setio, A.A.A., Gerke, P.K., Jacobs, C., Scholten, E.T., Schaefer-Prokop, C., Wille, M.M., Marchiano, A., and Pastorino, U. (2017). Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Scientific Reports, 7, p. 46479. Cui, L., Li, H., Hui, W., Chen, S., Yang, L., Kang, Y., Bo, Q., and Feng, J. (2020). A deep learning-based framework for lung cancer survival analysis with biomarker interpretation. BMC Bioinformatics, 21(1), pp. 1–14. Duan, G.Q., Zheng, X., Li, W.K., Zhang, W., Li, Z., and Tan, W. (2020). The association between VDR and GC polymorphisms and lung cancer risk: a systematic review and meta-analysis. Genetic Testing and Molecular Biomarkers, 24(5), pp. 285–295. Ejaz, K, Rahim, M.S.M., Bajwa, U.I., Chaudhry, H., Rehman, A., and Ejaz, F. (2020). Hybrid segmentation method with confidence region detection for tumor identification. IEEE Access. doi:10.1109/ACCESS.2020.3016627. Ejaz, K., Rahim, M.S.M., Rehman, A., Chaudhry, H., Saba, T., Ejaz, A., and Ej, C.F. (2018). Segmentation method for pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), pp. 394–401. Fahad, H.M, Khan, M.U.G, Saba, T., Rehman, A., and Iqbal, S. (2018). Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microscopy Research and Technique. 81(5), pp. 449–457. doi:10.1002/jemt.22998. Guo, Y., Song, Q., Jiang, M., Guo, Y., Xu, P., Zhang, Y., Fu, C.C., Fang, Q., Zeng, M., and Yao, X. (2020). Histological subtypes classification of lung cancers on CT images using 3D deep learning and radiomics. Academic Radiology. doi:10.1016/j.acra.2020.06.010. He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778). doi:10.1109/CVPR.2016.90. Hosny, A., Parmar, C., Coroller, T.P., Grossmann, P., Zeleznik, R., Kumar, A., Bussink, J., Gillies, R.J., Mak, R.H., and Aerts, H.J. (2018). Deep learning for lung cancer prognostication: a retrospective multi-cohort radiomics study. PLoS Medicine, 15(11), p. e1002711. Hosomi, Y., Morita, S., Sugawara, S., Kato, T., Fukuhara, T., Gemma, A., Takahashi, K., Fujita, Y., Harada, T., Minato, K., and Takamura, K. (2020). Gefitinib alone versus gefitinib plus chemotherapy for non–small-cell lung cancer with mutated epidermal growth factor receptor: NEJ009 study. Journal of Clinical Oncology, 38(2), pp. 115–123. Husham, A., Alkawaz, M. H., Saba, T., Rehman, A., and Alghamdi, J.S. (2016). Automated nuclei segmentation of malignant using level sets. Microscopy Research and Technique, 79(10), pp. 993–997, doi. 10.1002/jemt.22733. Hussain, N., Khan, M. A., Sharif, M., Khan, S. A., Albesher, A. A., Saba, T., and Armaghan, A. (2020). A deep neural network and classical features-based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-08852-3. Iftikhar, S., Fatima, K., Rehman, A., Almazyad, A.S., and Saba, T. (2017). An evolutionbased hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research, 28(8), pp. 3451–3455. Iqbal, S., Ghani, M.U., Saba, T., and Rehman, A. (2018). Brain tumor segmentation in multispectral MRI using convolutional neural networks (CNN). Microscopy Research and Technique, 81(4), pp. 419–427. doi: 10.1002/jemt.22994. Iqbal, S., Khan, M.U.G., Saba, T., Mehmood, Z., Javaid, N., Rehman, A., and Abbasi, R. (2019). Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microscopy Research and Technique, 82(8), 1302–1315, doi. org/10.1002/jemt.23281.

182

Artificial Intelligence and Internet of Things

Iqbal, S., Khan, M.U.G., Saba, T., and Rehman, A. (2017). Computer assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, 8(1), pp. 5–28, doi. 10.1007/s13534-017-0050-3. Jamal, A., Hazim Alkawaz, M., Rehman, A., and Saba, T. (2017). Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17), pp. 799–811. https://doi.org/10.1002/jemt. Javed, R., Rahim, M.S.M., and Saba, T. (2019b). An improved framework by mapping salient features for skin lesion detection and classification using the optimized hybrid features. International Journal of Advanced Trends in Computer Science and Engineering, 8(1), pp. 95–101. Javed, R., Rahim, M. S. M., Saba, T., and Rashid, M. (2019a). Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), pp. 1–10. Javed, R., Rahim, M.S.M., Saba, T., and Rehman, A. (2020a). A comparative study of features selection for skin lesion detection from dermoscopic images. Network Modeling Analysis in Health Informatics and Bioinformatics 9 (1), p. 4. Javed, R., Saba, T., Shafry, M., and Rahim, M. (2020b). An intelligent saliency segmentation technique and classification of low contrast skin lesion dermoscopic images based on histogram decision. In 2019 12th International Conference on Developments in eSystems Engineering (DeSE) (pp. 164–169). doi:10.1109/DeSE.2019.00039. Kaggle Data Science Bowl (KDSB). (2017). https://www.kaggle.com/c/data-science-bowl2017/data Khan, M.A., Akram, T., Sharif, M., Javed, K., Raza, M., and Saba, T. (2020a). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, pp. 1–30. doi:10.1007/s11042-020-08726-8. Khan, M.A., Akram, T., Sharif, M., and Saba, T. (2020d). Fruits diseases classification: exploiting a hierarchical framework for deep features fusion and selection. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-09244-3 Khan, M.A., Akram, T., Sharif, M., Saba, T., Javed, K., Lali, I.U., Tanik, U.J., and Rehman, A. (2019d). Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion. Microscopy Research and Technique, 82(5), pp. 741–763, http://doi.org/10.1002/jemt.23220 Khan, M.A., Ashraf, I., Alhaisoni, M., Damaševičius, R., Scherer, R., Rehman, A., and Bukhari, S.A.C. (2020c). Multimodal brain tumor classification using deep learning and robust feature selection: A machine learning application for radiologists. Diagnostics, 10, p. 565. Khan, M.A., Javed, M.Y., Sharif, M., Saba, T., and Rehman, A. (2019a). Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–7). IEEE. doi:10.1109/ICCISci.2019.8716400. Khan, M.A., Lali, I.U., Rehman, A., Ishaq, M., Sharif, M., Saba, T., Zahoor, S., and Akram, T. (2019c). Brain tumor detection and classification: a framework of marker-based watershed algorithm and multilevel priority features selection. Microscopy Research and Technique, 82(6), pp. 909–922, https://doi.org/10.1002/jemt.23238 Khan, M.A., Sharif, M., Akram, T., Raza, M., Saba, T., and Rehman, A. (2020b). Handcrafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing 87, p. 105986. Khan, M.A., Sharif, M.I., Raza, M., Anjum, A., Saba, T., and Shad, S.A. (2019b). Skin lesion segmentation and classification: a unified framework of deep neural network features fusion and selection. Expert Systems, p. e12497. https://doi.org/10.1111/exsy.12497

Lung Cancer Detection and Diagnosis

183

Khan, M.W., Sharif, M., Yasmin, M., and Saba, T. (2017). CDR based glaucoma detection using fundus images: a review. International Journal of Applied Pattern Recognition, 4(3), pp. 261–306. Khan, S.A., Nazir, M., Khan, M.A., Saba, T., Javed, K., Rehman, A., …, and Awais, M. (2019e). Lungs nodule detection framework from computed tomography images using support vector machine. Microscopy Research and Technique, 82(8), pp. 1256–1266. Krstić, G. (2017). Radon versus other lung cancer risk factors: How accurate are the attribution estimates? Journal of the Air & Waste Management Association, 67(3), pp. 261–266. Lawrenson, R., Lao, C., Brown, L., Moosa, L., Chepulis, L., Keenan, R., Kidd, J., Middleton, K., Conaglen, P., de Groot, C., and Aitken, D. (2020). Management of patients with early stage lung cancer—why do some patients not receive treatment with curative intent? BMC Cancer, 20(1), p. 109. Liaqat, A., Khan, M.A., Sharif, M., Mittal, M., Saba, T., Manic, K.S., and Al Attar, F.N.H. (2020). Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Current Medical Imaging, 16, 1229–1242. Lu, M.T., Raghu, V.K., Mayrhofer, T., Aerts, H.J., and Hoffmann, U. (2020). Deep learning using chest radiographs to identify high-risk smokers for lung cancer screening computed tomography: development and validation of a prediction model. Annals of Internal Medicine, 173, 704–713. Lung, J.W.J., Salam, M.S.H., Rehman, A., Rahim, M.S.M., and Saba, T. (2014). Fuzzy phoneme classification using multi-speaker vocal tract length normalization. IETE Technical Review, 31(2), pp. 128–136. doi. 10.1080/02564602.2014.892669. Lung Nodule Analysis (LUNA). (2016). https://luna16.grand-challenge.org/ Majid, A., Khan, M.A., Yasmin, M., Rehman, A., Yousafzai, A., and Tariq, U. (2020). Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microscopy Research and Technique, 83(5), pp. 562–576. Malhotra, J., Malvezzi, M., Negri, E., La Vecchia, C., and Boffetta, P. (2016). Risk factors for lung cancer worldwide. European Respiratory Journal, 48(3), pp. 889–902. Manzo, A., Carillio, G., Montanino, A., Sforza, V., Palumbo, G., Esposito, G., Costanzo, R., Sandomenico, C., La Manna, C., Martucci, N., and La Rocca, A. (2020). The safety of atezolizumab plus chemotherapy for the treatment of metastatic lung cancer. Expert Opinion on Drug Safety, 19(7), pp.775–783. Marcus, J.L., Leyden, W.A., Chao, C.R., Horberg, M.A., Klein, D.B., Quesenberry Jr, C.P., Towner, W.J., and Silverberg, M.J. (2017). Immunodeficiency, AIDS-related pneumonia, and risk of lung cancer among HIV-infected individuals. AIDS (London, England), 31(7), p. 989. Marie-Sainte, S.L., Aburahmah, L., Almohaini, R., and Saba, T. (2019a). Current techniques for diabetes prediction: review and case study. Applied Sciences, 9(21), p. 4604. Marie-Sainte, S.L., Saba, T., Alsaleh, D., Alotaibi, A., and Bin, M. (2019b). An improved strategy for predicting diagnosis, survivability, and recurrence of breast cancer. Journal of Computational and Theoretical Nanoscience, 16(9), pp. 3705–3711. Mashood Nasir, I., Attique Khan, M., Alhaisoni, M., Saba, T., Rehman, A., and Iqbal, T. (2020). A hybrid deep learning architecture for the classification of superhero fashion products: An application for medical-tech classification. Computer Modeling in Engineering & Sciences, 124(3), pp. 1017–1033. Masquelin, A.H., Whitney, D., Stevenson, C., Spira, A., Bates, J.H.T., San Jose Estepar, R., and Kinsey, C.M. (2020). Radiomics in deep learning-feature augmentation for lung cancer prediction. In D99, Advancing Risk Assessment for Pulmonary Nodules (pp. A7690–A7690). American Thoracic Society.

184

Artificial Intelligence and Internet of Things

Massion, P.P., Antic, S., Ather, S., Arteta, C., Brabec, J., Chen, H., Declerck, J., Dufek, D., Hickes, W., Kadir, T., and Kunst, J. (2020). Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules. American Journal of Respiratory and Critical Care Medicine, 202(2), pp. 241–249. McLellan, R., Marshall, H., Dent, A., Bowman, R.V., Yang, I.A., and Fong, K.M. (2020). Diagnosis and treatment of early lung cancer. Australian Journal of General Practice, 49(8), p. 508. Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A., and Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), p. 1068. Mughal, B., Muhammad, N., Sharif, M., Rehman, A., and Saba, T. (2018a). Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC Cancer, 18(1), p. 778. Mughal, B., Muhammad, N., Sharif, M., Saba, T., and Rehman, A. (2017). Extraction of breast border and removal of pectoral muscle in wavelet domain. Biomedical Research, 28(11), pp. 5041–5043. Mughal, B., Sharif, M., Muhammad, N., and Saba, T. (2018b). A novel classification scheme to decline the mortality rate among women due to breast tumor. Microscopy Research and Technique, 81(2), pp. 171–180. Nasrullah, N., Sang, J., Alam, M.S., Mateen, M., Cai, B., and Hu, H. (2019). Automated lung nodule detection and classification using deep learning combined with multiple strategies. Sensors, 19(17), p. 3722. National Lung Screening Trial Research Team. (2011). Reduced lung-cancer mortality with low-dose computed tomographic screening. New England Journal of Medicine, 365(5), pp. 395–409. Nazir, M., Khan, M. A., Saba, T., and Rehman, A. (2019). Brain tumor detection from MRI images using Multi-level Wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). doi:10.1109/ICCISci.2019.8716413. Norouzi, A., Rahim, M.S.M, Altameem, A., Saba, T., Rada, A.E., Rehman, A., and Uddin, M. (2014). Medical image segmentation methods, algorithms, and applications. IETE Technical Review, 31(3), pp. 199–213. doi:10.1080/02564602.2014.906861. Ott, P.A., Elez, E., Hiret, S., Kim, D.W., Morosky, A., Saraf, S., Piperdi, B., and Mehnert, J.M. (2017). Pembrolizumab in patients with extensive-stage small-cell lung cancer: results from the phase Ib KEYNOTE-028 study. Journal of Clinical Oncology, 35(34), pp. 3823–3829. Ottonello, S., Genova, C., Cossu, I., Fontana, V., Rijavec, E., Rossi, G., Biello, F., Dal Bello, M.G., Tagliamento, M., Alama, A., and Coco, S. (2020). Association between response to nivolumab treatment and peripheral blood lymphocyte subsets in patients with nonsmall cell lung cancer. Frontiers in Immunology, 11, p. 125. Pang, S., Meng, F., Wang, X., Wang, J., Song, T., Wang, X., and Cheng, X. (2020). VGG16-T: a novel deep convolutional neural network with boosting to identify pathological type of lung cancer in early stage by CT images. International Journal of Computational Intelligence Systems, 13(1), pp. 771–780. Perveen, S., Shahbaz, M., Saba, T., Keshavjee, K., Rehman, A., and Guergachi, A. (2020). Handling irregularly sampled longitudinal data and prognostic modeling of diabetes using machine learning technique. IEEE Access, 8, pp. 21875–21885. Qin, R., Wang, Z., Jiang, L., Qiao, K., Hai, J., Chen, J., Xu, J., Shi, D., and Yan, B. (2020). Fine-grained lung cancer classification from PET and CT images based on multidimensional attention mechanism. Complexity, 2020. https://doi.org/10.1155/2020/6153657. Qureshi, I., Khan, M.A., Sharif, M., Saba, T., and Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images. International Journal of Intelligent Systems Technologies and Applications, 19(1), pp. 1–16. doi:10.1504/IJISTA.2020.105172.

Lung Cancer Detection and Diagnosis

185

Rad, A.E., Rahim, M.S.M, Rehman, A., Altameem, A., and Saba, T. (2013) Evaluation of current dental radiographs segmentation approaches in computer-aided applications. IETE Technical Review, 30(3), pp. 210–222. Rad, A.E., Rahim, M.S.M, Rehman, A., and Saba, T. (2016) Digital dental X-ray database for caries screening. 3D Research, 7(2), pp. 1–5, doi:10.1007/s13319-016-0096-5. Rahim, M.S.M., Norouzi, A., Rehman, A., and Saba, T. (2017a). 3D bones segmentation based on CT images visualization. Biomedical Research, 28(8), pp. 3641–3644. Rahim, M.S.M., Rehman, A., Kurniawan, F., and Saba, T. (2017b). Ear biometrics for human classification based on region features mining, Biomedical Research, 28(10), pp. 4660–4664. Ramzan, F., Khan, M.U.G., Iqbal, S., Saba, T., and Rehman, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, pp. 103697–103709. Ramzan, F., Khan, M.U.G., Rehmat, A., Iqbal, S., Saba, T., Rehman, A., and Mehmood, Z. (2020a). A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), p. 37. Rao, P., Pereira, N.A., and Srinivasan, R. (December 2016). Convolutional neural networks for lung cancer screening in computed tomography (CT) scans. In 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I) (pp. 489–493). IEEE. doi: 10.1109/IC3I.2016.7918014. Reck, M., Luft, A., Szczesna, A., Havel, L., Kim, S.W., Akerley, W., Pietanza, M.C., Wu, Y.L., Zielinski, C., Thomas, M., and Felip, E. (2016). Phase III randomized trial of ipilimumab plus etoposide and platinum versus placebo plus etoposide and platinum in extensive-stage small-cell lung cancer. Journal of Clinical Oncology, 34(31), pp. 3740–3748. Rehman, A., Abbas, N., Saba, T., Mahmood, T., and Kolivand, H. (2018c). Rouleaux red blood cells splitting in microscopic thin blood smear images via local maxima, circles drawing, and mapping with original RBCs. Microscopic Research and Technique, 81(7), pp. 737–744. doi: 10.1002/jemt.23030. Rehman, A., Abbas, N., Saba, T., Mehmood, Z., Mahmood, T., and Ahmed, K.T. (2018b). Microscopic malaria parasitemia diagnosis and grading on benchmark datasets. Microscopic Research and Technique, 81(9), pp. 1042–1058. doi:10.1002/jemt.23071. Rehman, A., Abbas, N, Saba, T., Rahman, S.I.U., Mehmood, Z., and Kolivand, K. (2018a). Classification of acute lymphoblastic leukemia using deep learning. Microscopy Research & Technique, 81(11), pp. 1310–1317. doi:10.1002/jemt.23139. Rehman, A., Khan, M.A., Mehmood, Z., Saba, T., Sardaraz, M., and Rashid, M. (2020). Microscopic melanoma detection and classification: a framework of pixel-based fusion and multilevel features reduction. Microscopy Research and Technique, 83(4), pp. 410–423. doi:10.1002/jemt.23429. Rehman, A., Khan, M.A., Saba, T., Mehmood, Z., Tariq, U., and Ayesha, N. (2021). Microscopic brain tumor detection and classification using 3D CNN and feature selection architecture. Microscopy Research and Technique. https://doi.org/10.1002/ jemt.23597 Saba, T. (2017). Halal food identification with neural assisted enhanced RFID antenna. Biomedical Research, 28(18), pp. 7760–7762. Saba, T. (2019). Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique, 2019, pp. 1–9. https://doi.org/10.1002/ jemt.23326 Saba, T. (2020). Recent advancement in cancer detection using machine learning: systematic survey of decades, comparisons and challenges. Journal of Infection and Public Health, 13(9), pp. 1274–1289, https://doi.org/10.1016/j.jiph.2020.06.033

186

Artificial Intelligence and Internet of Things

Saba, T., Al-Zahrani, S., and Rehman, A. (2012). Expert system for offline clinical guidelines and treatment. Life Science Journal, 9(4), pp. 2639–2658. Saba, T., Bokhari, S.T.F., Sharif, M., Yasmin, M., and Raza, M. (2018b). Fundus image classification methods for the detection of glaucoma: a review. Microscopy Research and Technique. doi:10.1002/jemt.23094. Saba, T., Haseeb, K., Ahmed, I., and Rehman, A. (2020a). Secure and energy-efficient framework using Internet of Medical Things for e-healthcare. Journal of Infection and Public Health, 13, 1567–1575. Saba, T., Khan, M.A., and Rehman, A., and Marie-Sainte, S.L. (2019a) Region extraction and classification of skin cancer: a heterogeneous framework of deep CNN features fusion and reduction. Journal of Medical Systems, 43, p. 289. https://doi.org/10.1007/ s10916-019-1413-3 Saba, T., Khan, S.U., Islam, N., Abbas, N., Rehman, A., Javaid, N., and Anjum, A. (2019c). Cloud-based decision support system for the detection and classification of malignant cells in breast cancer using breast cytology images. Microscopy Research and Technique, 82(6), pp. 775–785. Saba, T., Mohamed, A.S., El-Affendi, M., Amin, J., and Sharif, M. (2020b). Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, pp. 221–230. Saba, T., Rehman, A. Mehmood, Z., Kolivand, H., and Sharif, M. (2018a). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), pp. 704–715. doi:10.2174/15734056136661 70912164546. Saba, T., Sameh, A., Khan, F., Shad, S. A., and Sharif, M. (2019b). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43(12), p. 332. Sadad, T., Munir, A., Saba, T., and Hussain, A. (2018). Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. Journal of Computational Science, 29, pp. 34–45. Schwyzer, M., Ferraro, D.A., Muehlematter, U.J., Curioni-Fontecedro, A., Huellner, M.W., Von Schulthess, G.K., Kaufmann, P.A., Burger, I.A., and Messerli, M. (2018). Automated detection of lung cancer at ultralow dose PET/CT by deep neural networks– initial results. Lung Cancer, 126, pp. 170–173. Shah, D.R. and Masters, G.A. (2020). Precision medicine in lung cancer treatment. Surgical Oncology Clinics, 29(1), pp. 15–21. Shin, H., Oh, S., Hong, S., Kang, M., Kang, D., Ji, Y.G., Choi, B.H., Kang, K.W., Jeong, H., Park, Y., and Hong, S. (2020). Early-stage lung cancer diagnosis by deep learning-based spectroscopic analysis of circulating exosomes. ACS Nano, 14(5), pp. 5435–5444. Shrey, S.B., Hakim, L., Kavitha, M., Kim, H.W., and Kurita, T. (February 2020). Transfer learning by cascaded network to identify and classify lung nodules for cancer detection. In International Workshop on Frontiers of Computer Vision (pp. 262–273). Springer, Singapore. Sori, W.J., Feng, J., and Liu, S. (2019). Multi-path convolutional neural network for lung cancer detection. Multidimensional Systems and Signal Processing, 30(4), pp. 1749–1768. Stapelfeld, C., Dammann, C., and Maser, E. (2020). Sex-specificity in lung cancer risk. International Journal of Cancer, 146(9), pp. 2376–2382. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z., and Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique. doi:10.1002/jemt.23178.

Lung Cancer Detection and Diagnosis

187

Van Meerbeeck, J.P., De Pooter, C., Raskin, J., Janssens, A., and Van Schil, P.E. (2020). Local treatment of stage IIIA-N2 nonsmall cell lung cancer: Surgery and/or radiotherapy. Current Opinion in Oncology, 32(1), pp. 54–62. Xu, T., Huang, C., Liu, Y., Gao, J., Chang, H., Yang, R., Jiang, T., Cheng, Z., Yu, W., Zhang, J., and Bai, C. (2020). Artificial intelligence based on deep learning for differential diagnosis between benign and malignant pulmonary nodules: A real-world, multicenter, diagnostic study. Journal of Clinical Oncology. doi:10.1200/jco.2020.38.15_suppl.9037. Yang, K., Liu, J., Tang, W., Zhang, H., Zhang, R., Gu, J., Zhu, R., Xiong, J., Ru, X., and Wu, J. (2020). Identification of benign and malignant pulmonary nodules on chest CT using improved 3D U-net deep learning framework. European Journal of Radiology, 129, p. 109013. Yousaf, K., Mehmood, Z., Awan, I.A., Saba, T., Alharbey, R., Qadah, T., and Alrige, M.A. (2019a). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, 287–309, pp. 1–23. Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A.M., Alharbey, R., and Rashid, M. (2019b). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: A survey. BioMed Research International, 2019, pp. 1–26. Yu, H., Xu, Q., Xiong, W., Liu, Z., Cai, L., and He, F. (2019). Association of pickled food, fired food and smoked food combined with smoking and alcohol drinking with lung cancer: a case-control study. Wei Sheng yan jiu Journal of Hygiene Research, 48(6), pp. 925–931. Zhu, Y.C., Wang, W.X., Xu, C., Lan, G., Chen, H., Li, X., Zhai, Z., Huang, L., Du, K., and Fang, M. (2020). Community-based practice of early lung cancer screening with lowdose spiral computed tomography in Jiaxing city from Zhejiang Province. Journal of Clinical Oncology, doi:10.1200/jco.2020.38.15_suppl.e13557.

7

Image-Based Glioblastoma Segmentation Traditional versus Deep Learning Approaches: Analysis, Comparisons and Future Look Amjad Rehman

CONTENTS 7.1 Introduction................................................................................................... 189 7.2 Tradational Machine Learning and Deep Learning Techniques ................ 191 7.2.1 Tradational Machine Learning Methods........................................... 191 7.2.1.1 Classification....................................................................... 192 7.2.2 Deep Learning (DL) Techniques....................................................... 197 7.2.2.1 2D-CNN Techniques.......................................................... 197 7.2.2.2 3D-CNN Techniques.......................................................... 198 7.3 Conclusions and Future Directions................................................................200 Acknowledgment��������������������������������������������������������������������������������������������������� 201 References�������������������������������������������������������������������������������������������������������������� 201

7.1 INTRODUCTION The brain tumor is the most harmful cancer since it affects the primary nervous system of the humans; it could be categorized into benign and malignant tumors (Amin et al., 2019a,b). The first category is not fatal enough, while the malignant brain tumor is very harmful, spreadable, if not properly treated and removed well in time. Brain tumor detection and segmentation is a complex and sensitive process (Iqbal et al., 2017). Meningioma, glioma and pituitary are the types of brain tumors that commonly arise depending on the area affected. There is a certain degree of 189

190

Artificial Intelligence and Internet of Things

malignancy in each type of these tumors. Glioma is the most extreme type of tumor that develops in the region of glial tissues and spinal cord; meningioma is another type of brain tumor that develops in the membrane area, while the pituitary tumor grows in area of pituitary gland (Amin et al., 2019c; Iqbal et al., 2018; Saba et al., 2020a,b). Normally, the initial examination of brain tumors by oncologists is normally conducted using diagnostic imaging techniques such as magnetic resonance imaging (MRI) and CT scans. These two modalities are commonly used to provide a large amount of brain function knowledge. For a detailed diagnosis by a physician, however, a surgical biopsy of the diagnosed tissue (tumor) is needed if the doctor suspects a brain tumor and needs more information about its form. Commonly neurologists segment abnormal regions manually. The manual segmentation of brain tumors is subjective, costly, consuming more time, error-prone and requiring highly experienced neurologists. The early and accurate detection of any type of disease is a keystone in the cure of patients, which increases the survival possibilities, reduces the risk to patients’ lives and raises their hopes of being cured to 90%. However, early and accurate detection of the tumor is a process that involves the intervention of expert people in all evaluation processes of the patient. This is costly and nearly impossible to be achieved for a huge number of people. The two main grades of brain tumors are low-grade glioma (LGG) and high-grade glioma (HGG). HGG is aggressive, while LGG is less aggressive. The patients suffering from HGG have an average life expectancy of 1 year, while in LGG, this expectancy is 5 years. A brain tumor is treated based on surgery, radiation and chemotherapy. Therefore, computer-aided diagnostic system to identify region of interest (ROI) is extremely needed. The computer aided diagnosis (CAD) is a process in which the first stage of tumor detection could be achieved automatically using machine learning (ML) algorithms (Rahim et al., 2017a,b). The MRI system generates the brain images while ML will detect any different sections or areas in the brain like a tumor. The CAD will then assist the human expert in generating the first report of tumor possibilities. Computer-aided cancer detection techniques could play a significant role in brain cancer detection (Iqbal et al., 2019; Khan et al., 2019a,b; Pereira et al., 2016; Tahir et al., 2019). In a state of the art, several ML methods are used to segment brain tumors that could be classified into two major categories, i.e., traditional ML- and deep learning (DL)-based methods. All traditional ML methods consist of four main stages: preprocessing, feature extraction, feature selection and classifier training and testing (Abbas et al., 2019a,b). The preprocessing step is used for noise removal from an image and enhanced image quality to smoothen further processing. Secondly, handcrafted features are extracted, and feature selection is used for dimensionality reduction to decrease the size of the feature vector. It decreases training time, chances of overfitting and enhances classification accuracy. Finally, classifier is trained based on the selected feature vector set to achieve high accuracy (Mittal et al., 2020; Norouzi et al., 2014; Saba et al., 2020b). Currently for medical image analysis and cancer detection, DL techniques showed a significant improvement on traditional ML techniques (Perveen et al., 2020; Saba, 2020; Saba et al., 2020a, Nazir et al., 2019). Many researchers

Image-Based Glioblastoma Segmentation

191

FIGURE 7.1  Machine learning taxonomy

proposed convolutional neural network (CNN) models for brain tumor segmentation from MRI, and these models are based on 3D or 2D-CNN and attained promising results (Abbas et al., 2015, 2018a,b; Hussain et al., 2020; Liaqat et al., 2020; Qureshi et al., 2020; Ramzan et al., 2020a,b; Rehman et al., 2020a,b; Saba, 2019; Saba et al., 2018a,b, 2020a,b). This chapter presents a detailed comparison of traditional ML to DL methods for brain tumor classification. ML techniques are categorized into traditional ML and DL methods, as shown in Figure 7.1. Each category is further categorized into subcategories based on the methods being used. Several latest traditional and DL state-of-the-art techniques are compared in terms of high accuracy, dice similarity, error rate, sensitivity and specificity as performance measures. The remaining chapter is organized as follows. Section 7.2 details different ML methods elaborated to segment brain tumors, which are classified into traditional ML and DL methods. Section 7.3 presents a conclusion with possible solutions to the challenges.

7.2 TRADATIONAL MACHINE LEARNING AND DEEP LEARNING TECHNIQUES 7.2.1 Tradational Machine Learning Methods In this section, different ML methods were employed to segment brain tumors consist of support vector machine (SVM), K-nearest neighbor (K-NN), random forest (RF), fuzzy C means and K means. These methods are further categorized into two classes: clustering algorithms and classification (Al-Ameen et al., 2015; Fahad et al., 2018; Husham et al., 2016; Iftikhar et al., 2017; Saba, 2017; Saba et al., 2012; Tahir et al., 2019).

192

Artificial Intelligence and Internet of Things

7.2.1.1 Classification Most ML methods perform classification on pixel level to detect ROI using MRI (Khan et al., 2019c,d,e; Ullah et al., 2019). These classification-based methods are discussed in the following section that is used to segment brain tumor. 7.2.1.1.1  SVM-Based Techniques This section presents detailed review of biomedical image analysis to segment ROI. There are numerous existing methods in medical image analysis, such as efficient detection of ROI and accurate classification, which are still challenging and needs improvement (Jamal et al., 2017; Javed et al., 2019a,b, 2020a,b; Khan et al., 2020a,b). There are different traditional segmentation techniques used to segment brain tumors, i.e., manual thresholding, OTSU threshold, region grown, watershed, K-means segmentation, etc. A segmentation technique named Otsu binarization can select the best threshold value to segment brain tumors in MRI. In this method, for the selection of optimum threshold, hit and trial method is used. This technique is easy to implement but not good for the low quality of data. Watershed segmentation technique is used for overlapped objects; it first calculates the gradient magnitude from the input data, and then the calculated gradients are used to segment the ROI. The results are observed. This technique is found best for 2D data and is not suitable for 3D data. Region growing segmentation, this method works with seed point selection, and for best seed point first, it selects a single seed point, is known as starting seed point. It grows based on homogeneity criteria and group neighbor pixels of the same region of an image; otherwise, not group neighbor pixels. The results show that this technique is sensitive to noisy data (Ejaz et al., 2018a,b). Usman and Rajpoot (2017) employed RF for brain tumor segmentation using neighborhood intensity, intensity differences, and wavelet texture features. El Abbadi and Kadhim (2017) used gray-level run length matrices (GLRLM) and GLCM for features extraction and probabilistic neural networks (PNNs) incorporated for brain tumor classification. All the above-mentioned studies for brain tumor classification have one significant disadvantage, i.e., that the classification of brain tumor defends best feature selection and is quite challenging and reduces the capability of generalization. Some other approaches used to classify brain tumors by using different handcrafted and deep features with different classifier, i.e., SVM (Khan et al., 2019c,d), artificial neural network (ANN) (Rehman et al., 2018a,b,c, 2020a) and KNN (Jamal et al., 2017; Khan et al., 2017). Ren et al. (2019) proposed a fuzzy clustering technique to segment brain tumors using MRI. This technique works like K-means clustering-based segmentation. It segments the input image into two voxel classes, i.e., brain voxels and tumor voxels. However, the time complexity is more than K-means, which is the main drawback of their method. Raja et al. (2018) proposed a semi-automated technique that could investigate medical MRI. This method used a hybrid segmentation approach, and Bat algorithm and Tsallis-based thresholding were fused with region growing technique to a segment brain tumor in MRI. Better results are reported such as Jaccard (87.41%), dice (90.36%) and accuracy (97.53%). Ma et al. (2018) proposed a hybrid scheme to combine the active contour model and the RF algorithm for brain tumor

Image-Based Glioblastoma Segmentation

193

segmentation. For feature extraction, a specific modality of RFs, i.e., feature learning kernel, is utilized to extract local and contextual information. The experimental evaluation proves that the proposed technique provides efficient and accurate results compared to other existing techniques (Ejaz et al., 2019). In Nguyen et al. (2019), a multilevel discrete wavelet transform (DWT) is used, in which the input is first decomposed, then the fisher discriminates ratio technique is used to segment brain tumor. Before feature extraction, some morphological operations are applied for removing noise. GLCM feature descriptor is used for feature extraction followed by a PNN to classify the brain tumor into two classes, i.e., benign and malignant. Talo et al. (2019) proposed a deep-transfer-learning-based technique to classify brain tumors into normal and abnormal using MR images. In this technique, ResNet34 is fine-tuned for optimal learning rate with extensive data augmentation and higher classification accuracy. Zhang et al. (2011) addressed the brain tumor’s challenge more sophisticatedly. In their method, multispectral MR-images are fused. The benefit of their research work is decreasing computational cost, reducing segmentation error and increasing inference time. Feature extraction, selection of features and presented extra adaptive training steps were redefined in their work. Three features, i.e., wavelet transformation, texture information and intensities, were calculated in different filters that spread across multispectral images. SVM is employed for training and testing purposes. The brain tumor classification process is demonstrated in Figure 7.2. Rathi and Palani (2012) proposed that brain tumors’ segmentation is a pixel classification task. In their method, three features are extracted: intensity (kurtosis, skewness, median intensity, standard deviation, variance and mean), texture (square variance sum, cluster shade, homogeneity, energy, entropy, correlation and contrast)

FIGURE 7.2  Brain tumor detection using support vector machine (SVM)

194

Artificial Intelligence and Internet of Things

and shape features (shape index, perimeter, area, irregularity and circularity). For the training purpose, the SVM classifier is used to classify each pixel as gray matter, white matter, CSF, non-tumor and tumor. The experiments were performed on 140 MRI of IBSR dataset. The classifier classifies these MR-images into malignant, benign or normal. Reddy et al. (2012) presented a technique in which three feature descriptors, i.e., mean intensity, LBP and HOG feature, were used to extract features. The extracted features were fed to train the SVM classifier. The Gaussian filter used the smoothed the result of SVM to generated a confidence surface, which controlled every pixel’s probability as non-tumor or tumor. Vaishnavee and Amshakala (2015) presented a hybrid technique and employed SOM, SVM to segment brain tumor. In the preprocessing phase, they used histogram equalization. In the next phase, three features are extracted, i.e., variance, intensity mean and number-of-occurrences for classifier training. Following this, SOM clustering method is used to segment and localize abnormal brain clusters into sub-tumors, and texture features of GLCM were extracted. Finally, PCA is applied to reduce dimensionality. Arikan et al. (2016) proposed a semi-automatic hybrid methodology of SVM and interactive selection of a seed-based approach to segment brain tumors. At the preprocessing phase, the anisotropic diffusion kernel was used to remove noise from MR-images. They selected random seeds from preprocessed MR-images for SVM training. Finally, MICCIA BraTS 2015 datasets were used for performance assessment. The proposed work reached 81% of an average dice similarity with ground truth comparison. Mehmood et al. (2019) proposed a 3D-efficient and effective framework to analyze and visualize brain MRI. The BOW is used to compute features from images using SURF feature descriptor, and for classification purposes, the SVM classifier is used. The proposed research work reached an accuracy of 99.0 on the local datasets. Table 7.1 presents an accuracy comparison of current methods for brain tumor segmentation.

TABLE 7.1 Comparisons of Brain Tumor Segmentation Techniques References Zhang et al. (2011) Rathi and Palani (2012) Reddy et al. (2012) Vaishnavee and Amshakala (2015) Arikan et al. (2016)

Techniques PCA-based SVM Multi-kernel SVM SVM and Gaussian filter SVM PSVM

Dataset IBSR SG SG BraTS 2015

Random seed selection B-2015 SVM Mehmood et al. (2019) Bow-Surf-based SVM LRH Das et al. (2019) GLCM, GRLN, HOG, GMCH Tamura and LBP features and six classifiers

Dice – – 69

Recall – – –

Accuracy (%) 97.8 98.87 –

–

90

–

80

–

–

– –

– –

99.0 100

Image-Based Glioblastoma Segmentation

195

FIGURE 7.3  KNN-based classification of brain tumor

7.2.1.1.2  K-Nearest Neighbor (K-NN)-Based Techniques The KNN are supervised ML techniques that are successfully applied to classification and regression problems (Mughal et al., 2018a,b, 2017). Zhang et al. (2011) proposed a hybrid method to classify MR-images consisting of three steps including features extraction, reducing dimensionality and classification. Originally, DWT is used to extract MRI features. In the next step, PCA was used to reduce the feature set’s dimension extracted from MR-images, which makes the feature set more meaningful. Finally, for predication, they used two classifiers, FP-ANN and K-NN, to classify brain tumors into abnormal or normal. Their proposed approach’s classification accuracy is 97% on FP-ANN and 98% on K-NN, respectively. In Figure 7.3, KNN-based classification of brain tumor is illustrated. Wasule and Sonar (2017) presented a technique in which brain tumors are automatically classified into benign or malignant and HGG or LGG. In this work, features are extracted from the images using GLCM method and stored these extracted features in a feature vector. Finally, KNN and SVM classifier fused for classification. To train classifiers, 251 images are taken from the clinical database in which 166 were benign and 85 were malignant and 80 images are taken from BraTS 2012 in which 30 belonged to HGG and 50 were from LGG. The accuracy of the proposed method on SVM using a clinical database is 96% and on K-NN is 86%, respectively, and using the BraTS database, SVM-based classification accuracy reached up to 85%, and K-NN is 72.50%. Table 7.2 presents K-NN based methods for brain tumor segmentation. 7.2.1.1.3  Random Decision Forest-Based Methods Random decision forests are ensembled learning systems for classification. However, their output could be influenced by data characteristics. The following techniques are currently reported to detect ROI using RF.

196

Artificial Intelligence and Internet of Things

TABLE 7.2 Brain Tumor Segmentation Using K-Nearest Neighbor (K-NN) References Zhang et al. (2011) Kalbkhani et al. (2013)

Techniques DWT + PCA + K-NN PCA + LDA + K-NN

Dataset T2-weighted MRI database T2-weighted MRI database

Havaei et al. (2016) Wasule and Sonar (2017)

K-NN-CRF GLCM-features-based K-NN

MICCAIB BraTS 2013 Clinical database BraTS 2012

Accuracy (%) 98 1st scenario, 97.62 2nd scenario, 100 Not reported 86 72.50

Ellwaa et al. (2016) presented an MRI-based approach with iterative RF as the classifier. The sample BraTS 2016 was used for testing purposes of this methodology. Clear knowledge allowed the patient screening criterion to produce a successful outcome. Meier et al. (2016) presented a discriminative model based on RF using high discriminative features. For the standardization of intensities at the preprocessing stage, MedPy was used to match the intensity profile sequence. The RF classifier was trained using voxel-wise features based on intensities values. Reza and Iftekharuddin (2014) presented an enhanced brain tumor segmentation method that extracts texture features on multi-model MR-images. Median and mean were computed to check the effectiveness of the presented work. In their work, they used RF and performed a pixel-level classification. Furthermore, the objects having the smallest area were deleted using a connected components labeling technique after preprocessing. Finally, the tumor’s regions were detected and the holes were filled with the help of connected neighbor intensities. Experiments were conducted on BraTS 2013 and BraTS 2014 datasets. Abbasi and Tajeripour (2017) presented an automatic 3D model to detect and segment brain tumor using RF. In preprocessing steps, bias field correction and histogram matching were used. Following this, ROIs were separated from the FLAIR images. In their approach for feature extraction, they used LBP and HOG descriptors. Soltaninejad et al. (2017) proposed brain tumor grading based on a superpixel technique using an extremely randomized tree (ERT) classifier. In preprocessing, the skull is removed from MR images to extract all other features. They used BraTS 2012 and a self-created datasets consisting of 30 FLAIR images. Their result showed a 7% improvement as compared to SVM. Table 7.3 presents RF methods for brain tumor segmentation. 7.2.1.1.4 Clustering Brain MRI scans also utilize clustering-based methods to identify and find anomalies (Yousaf et al., 2019a,b; Saba et al., 2020a,b). The current research utilizes the brain tumor segmentation clustering method in MR images described later in this chapter. Verma et al. (2009) suggested a system focused on mountain clustering methodology to cluster brain tumors. Their approach was comparable to the popular C-means and K-means clustering strategies focused on cluster entropy. They developed their

197

Image-Based Glioblastoma Segmentation

TABLE 7.3 Brain Tumor Segmentation Using Random Forest References Reza and Iftekharuddin (2014) Ellwaa et al. (2016) Meier et al. (2016) Soltaninejad et al. (2017) Abbasi and Tajeripour (2017)

Technique EF with morph Iterative RF RF ERT RF

Dataset BraTS 2014 BraTS 2016 SG BraTS 2012 BraTS 2013

Complete Core Enhanced DICE 81 66 71 82 72 56 84 66 39 0.88 83.8 76 76

mountain clustering algorithm in science. Their method was efficient and displayed the lowest average entropy (0.000831) against K-means (0.000840) and fuzzy C-means algorithms (0.000839). Singh et al. (2014) proposed a two-stage standard method to classify tumors via the ROI. At the first stage, a fuzzy C-means was added to separate brain MRI into different clusters, and in the second stage, MRIs’ odd areas are separated with assistance of level-set algorithm. Their cascaded strategy, which involved various methods, yielded positive results. Abdel-Maksoud et al. (2015) proposed the fusion of K-means and fuzzy C-means (KIFCM) to segmentation brain tumors. The five phases included their suggested method: preprocessing, clustering, selection, contouring and segmentation. During the extraction and contouring step, the threshold was applied, and threshold output was smoothed with the denoising process. Lastly, the tumor region in the MRI was sculpted with the standard collection. Kaya et al. (2017) found that the PCA-based technique segmented brain tumor using T1-weighted MRI. Five popular PCA-dependent techniques were applied to minimize the dimension of vector function. To test the process of restoration and Euclidean distance errors, the probabilistic PCA has implemented four further strategies. Zeinalkhani et al. (2018) proposed computerized techniques to identify tumor blockages and recognize tumors by utilizing multi-patient MRI ANN algorithms. They removed noise from segmented images using a high pass filter in the preprocessing step. Finally, extracted features were used for MRI analysis to detect brain tumors.

7.2.2 Deep Learning (DL) Techniques DL is part of a larger family of ANNs-based ML approaches. DL models are successfully applied to computer vision, natural language processing, bioinformatics, drug design and medical image recognition problems in which they have achieved excellent results. CNN-based brain tumor segmentation strategies are categories into two main types: 2D-CNN and 3D-CNN (Adeel et al., 2020; Amin et al., 2019a,b,c; Ejaz et al., 2020; Saba et al., 2019a,b,c; Marie-Sainte et al., 2019a,b). 7.2.2.1  2D-CNN Techniques Chang (2016) presented a complete CNN brain tumor segmentation system with hyperlocal local features (HLFC). Five convolution layers of non-linear activation

198

Artificial Intelligence and Internet of Things

FIGURE 7.4  Brain tumor classification using CNN models

functions occurred in the complete CNN. Bilinear interpolation has extracted the final convolution layer data from the initial input signal. The hyperlocal concatenation function reintroduced the original network data through channel-wide concatenation, utilizing a two narrow-scale (3 × 3) convolutionary filter segmentation map after concatenating hyperlocal features and bilinear interplay images. The suggested BRATS 2016 dataset protocol was checked. They were computationally effective, and in less than a second, they were capable of completing the segmentation process on an optimized GPU. The patch-wise interpretation of the CNN model lacks the overall contextual details. Lun and Hsu (2016) proposed a completely automatic CNN system to segment brain tumors using MRI. They implemented MR-images in multimodalities and used a CNN layout focused on global functions. Local regions were classified as a central pixel patch isolated from a 30 × 30 kernel file. Such patches have been used for pattern testing. They also applied a certainly weighting scheme to boost the mismatch mark problem on the failure layer on CNN. The BRATS 2015 dataset validates their process. Their global reweighting approach outperformed the existing patch system of intelligent thinking. A new automatic CNN system for the segmentation of brain tumors utilizing four weighted modalities of the MR (T1, T1c, T2, FLAIR) was introduced (Pereira et al., 2016). Defined grains in scale 3 × 3 were used. The convolutional fixed-size kernel has helped them to create a more detailed CNN with fewer non-linearities and weights. The model was trained and tested on BraTS 2013 and BraTS 2015. The proposed CNN model is shown in Figure 7.4. Dong et al. (2017) employed UNet CNN model to segment brain tumor using BraTS 2015 dataset. Accuracies of 86%, 86% and 65% for complete, core and enhanced regions, respectively, were reported. Fidon et al. (2017) proposed a novel scalable multimodal DL architecture to detect ROI with CNN support. They used BraTS 2013 dataset. The leather board’s achieved dice scores are 0.77, 0.64 and 0.56 for the whole core and enhance, respectively. Table 7.4 presents a comparison of 2D-CNN techniques for brain tumor segmentation. 7.2.2.2  3D-CNN Techniques Brain tumor detection using 3D-CNN is a relatively new concept (Hussain et al., 2020, Rehman et al., 2018a,b,c). Kayalibay et al. (2017) also conducted the segmentation of medical imaging data with 3D-CNN. Their approach merged with 3D-convolutional layers and CNN. The BRATS 2013 and BRATS 2015 datasets were used for experiments. 3D-CNN was also employed by (Yi et al., 2016) for

199

Image-Based Glioblastoma Segmentation

TABLE 7.4 Brain Tumor Segmentation with 2D CNN Model References Pereira et al. (2016) Dong et al. (2017) Fidon et al. (2017) Seetha and Raja (2018)

Khawaldeh et al. (2018) Sajjad et al. (2019)

Özyurt et al. (2019) Amin et al. (2020)

Technique 2D-CNN U-Net Scalable multimodal CNN FCM segmentation Texture and shape feature fed to SVM and DNN AlexNet architecture VGG-19 CNN architecture using data augmentation

Dataset BraTS 2015 BraTS 2013 BraTS 2013 BraTS 2015

Accuracy (%) 76 – – 97.5

TCIA BraTS 2016

NS-CNN feature fed to SVM classifier Threshold method, discrete wavelet transform (DWT)

TCGA-GBM

91.16 Without augmentation 87.38 With 90.67 95.62 97

glioblastoma segmentation. For 3D convolution processing, the convolutional layers in their design have been paired with the discrepancy of Gaussian (DoG) filters. They used a data collection of 274 tumor samples from BRATS 2015 and used dice similitude to calculate their proposed system’s efficiency. Nie et al. (2016) suggested an automated 3D-CNN-dependent brain tumor segmentation technique utilizing T1, functional MR modality and diffusion tensor imaging (DTI). Various preprocess methods have been determined for each MR model, i.e., pressure standardization for T1, tensor modeling for DTI, BOLD fluctuation (blood oxygen dependent) for fMRI. 3D-CNN architecture is employed for tumor segmentation and finally classified using SVM. Table 7.5 presents a comparison of 3D-CNN techniques for brain tumor segmentation.

TABLE 7.5 Brain Tumor Segmentation Using 3D-CNN Architecture References Urban et al. (2014) Yi et al. (2016) Nie et al. (2016) Kayalibay et al. (2017) Yogananda et al.(2019) Rehman et al. (2020b)

Technique CNN with DoG 3D-CNN 3D-CNN with SVM 3D-CNN 3D-Dense-UNets 3D-CNN

Dataset BraTS 2015 BraTS 2015 SG BraTS 2015 BraTS 2019 PH2 and ISBI2016

Accuracy (%) 89 89.9 55 98.20 95.42

200

Artificial Intelligence and Internet of Things

7.3  CONCLUSIONS AND FUTURE DIRECTIONS This chapter has presented an extensive analysis of strategies for ML-based brain tumor segmentation from MRI. The ML approaches have been divided into two distinct categories: traditional and DL strategies. The traditional techniques are classified according to the ML algorithms used, whereas the methods of profound learning are categorized into 2D-CNN and 3D-CNN. However, CNN’s method surpassed the traditional procedure dependent on ML and improved performance on brain tumors segmentation. 3D-CNN-based systems are complicated for biomedical image segmentation and are hard due to high processor demanding. 2D-CNN architectures, on the other hand, are primarily developed to tackle the limited data problem inherent to medical optimization and are computationally less complex. In comparison, 2D-CNN often obtained similar 3D-CNN performance. It is also essential to develop new methodologies and strategies for removing tumors at an early level. For each group, tumor segmentation results are compared on benchmark dataset BraTS. The full details of these methods used within the same group are listed in each list. For example, the comprehensive findings of techniques focused on SVM, KNN, RF, 2D-CNN and 3D-CNN are given in Tables 7.1–7.4, respectively. The tables further outline their testing methods, sample and output assessment matrices used to calculate the techniques’ efficiency. Different methods used specific evaluation parameters for results. Traditional ML approaches rely heavily on customized apps to model domain information. On the other side, DL techniques have the remarkable capabilities to know an even more complicated structure of highly unregulated features that could not be expressed using side-made software. These additional features allow DL-based techniques to surpass traditional ML techniques, especially in viewing and biomedical image differentiation tasks with the CNN. The ML approach commonly used for the segmentation of brain tumors is clustering techniques. Such procedures are semi-automated to create completely clustered tumor areas requiring involvement by neurologists. Even so, these strategies rely heavily on the spatial distribution of MR-images details (i.e., pressure, texture, etc.). These correlations also allow the effects of clustering strategies to deviate. 3D-CNN-based approaches aim to model 3D MR artefacts through the use of 3D kernels and achieve positive performance. Nonetheless, the usage of 3D-CNN in medical image segmentation is unfavorable. Since the lack of labeled data is often a concern in the medical industry, to overcome this problem, 2D-CNN architectures are added that process 2D patches. The approaches focused on 2D-CNN achieved comparable findings to 3D-CNN. Even though several existing methods to segment and classify the brain tumor still have limitations and need improvements. In most of the existing schemes, only tumors are segmented, and for a radiologist or medical expert, it is not enough to diagnose, cure a patient. The radiologist or medical experts need clear and accurate segmented and classified brain tumor support to diagnose and treat a patient properly. Moreover, the lack of medical data has adverse effects to achieve accurate and robust classification outcomes in medical imaging. To report these margins, we suggested an efficient adaptive gray-level threshold technique to segment tumor in brain MR images followed by a DL model with a data augmentation approach to improve tumor classification results.

Image-Based Glioblastoma Segmentation

201

Finally, the output of published classification findings is difficult and often impossible to compare, as many authors use non-public databases for training/testing their approaches. To allow comparability, future researchers should use publicly accessible benchmark datasets only and fully report approaches used for preparation.

ACKNOWLEDGMENT This work was supported by Artificial Intelligence and Data Analytics (AIDA) Lab CCIS, Prince Sultan University, Riyadh, Saudi Arabia. The author is thankful for the support.

REFERENCES Abbas, A., Saba, T., Rehman, A., Mehmood, Z., Javaid, N., Tahir, M., Khan, N. U., Ahmed, K. T., and Shah, R. (2019b). Plasmodium species aware based quantification of malaria, parasitemia in light microscopy thin blood smear. Microscopy Research and Technique, 82(7), 1198–1214. doi:10.1002/jemt.23269. Abbas, N., Mohamad, D., Abdullah, A. H., Saba, T., Al-Rodhaan, M., and Al-Dhelaan, A. (2015). Nuclei segmentation of leukocytes in blood smear digital images. Pakistan Journal of Pharmaceutical Sciences, 28(5), 1801–1806. Abbas, N., Saba, T., Mehmood, Z., Rehman, A., Islam, N., and Ahmed, K. T. (2019a). An automated nuclei segmentation of leukocytes from microscopic digital images. Pakistan Journal of Pharmaceutical Sciences, 32(5), 2123–2138. Abbas, N., Saba, T., Mohamad, D., Rehman, A., Almazyad, A. S., and Al-Ghamdi, J. S. (2018a). Machine aided malaria parasitemia detection in Giemsa-stained thin blood smears. Neural Computing and Applications, 29(3), 803–818. doi:10.1007/s00521-016-2474-6. Abbas, N., Saba, T., Rehman, A., Mehmood, Z., Kolivand, H., Uddin, M., and Anjum, A. (2018b). Plasmodium life cycle stage classification based quantification of malaria parasitaemia in thin blood smears. Microscopy Research and Technique, 82(3), 283–295. https://doi.org/10.1002/jemt.23170. Abbasi, S., and Tajeripour, F. (2017). Detection of brain tumor in 3D MRI images using local binary patterns and histogram orientation gradient. Neurocomputing, 219, 526–535. Abdel-Maksoud, E., Elmogy, M., and Al-Awadi, R. (2015). Brain tumor segmentation based on a hybrid clustering technique. Egyptian Informatics Journal, 16(1), 71–81. Adeel, A., Khan, M. A., Akram, T., Sharif, A., Yasmin, M., Saba, T., and Javed, K. (2020). Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Systems. https://doi.org/10.1111/exsy.12569. Al-Ameen, Z., Sulong, G., Rehman, A., Al-Dhelaan, A., Saba, T., and Al-Rodhaan, M. (2015). An innovative technique for contrast enhancement of computed tomography images using normalized gamma-corrected contrast-limited adaptive histogram equalization. EURASIP Journal on Advances in Signal Processing, 32. doi:10.1186/ s13634-015-0214-1. Amin, J., Sharif, M., Raza, M., Saba, T., and Anjum, M. A. (2019a). Brain tumor detection using statistical and machine learning method. Computer Methods and Programs in Biomedicine, 177, 69–79. Amin, J., Sharif, M., Raza, M., Saba, T., and Rehman, A. (2019b). Brain Tumor Classification: Feature Fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. doi:10.1109/ICCISci.2019.8716449. Amin, J., Sharif, M., Raza, M., Saba, T., Sial, R., and Shad, S. A. (2020). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 32(20), 15965–15973.

202

Artificial Intelligence and Internet of Things

Amin, J., Sharif, M., Yasmin, M., Saba, T., and M Raza (2019c). Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications, 79(15), 10955–10973. https://doi. org/10.1007/s11042-019-7324-y. Arikan, M., Fröhler, B., and Möller, T. (2016). Semi-Automatic Brain Tumor Segmentation Using Support Vector Machines and Interactive Seed Selection. In Proceedings MICCAI-BRATS Workshop (pp. 1–3). Chang, P. D. (2016). Fully Convolutional Neural Networks with Hyperlocal Features for Brain Tumor Segmentation. In Proceedings of MICCAI-BRATS Workshop (pp. 4–9). Das, D., Mahanta, L. B., Ahmed, S., Baishya, B. K., and Haque, I. (2019). Automated classification of childhood brain tumours based on texture feature. Songklanakarin Journal of Science and Technology, 41(5), 1014–1020. Dong, H., Yang, G., Liu, F., Mo, Y., and Guo, Y. (2017). Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks. In Annual Conference on Medical Image Understanding and Analysis (pp. 506–517). Ejaz, K., Rahim, D. M. S. M., Rehman, D. A., and Ejaz, E. F. (2018b). An image-based multimedia database and efficient detection though features. VFAST Transactions on Software Engineering, 14(1), 6–15. Ejaz, K., Rahim, M. S. M., Bajwa, U. I., Chaudhry, H., Rehman, A., and Ejaz, F. (2020). Hybrid segmentation method with confidence region detection For Tumor identification. IEEE Access. doi: 10.1109/ACCESS.2020.3016627. Ejaz, K., Rahim, M. S. M., Bajwa, U. I., Rana, N., and Rehman, A. (2019). An Unsupervised Learning with Feature Approach for Brain Tumor Segmentation Using Magnetic Resonance Imaging. In Proceedings of the 2019 9th International Conference on Bioscience, Biochemistry and Bioinformatics (pp. 1–7). Ejaz, K., Rahim, M. S. M., Rehman, A., Chaudhry, H., Saba, T., and Ejaz, A. (2018a). Segmentation method for pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), 394–401. El Abbadi, N. K., and Kadhim, N. E. (2017). Brain cancer classification based on features and artificial neural network. Brain, 6(1), 123–134. Ellwaa, A., Hussein, A., AlNaggar, E., Zidan, M., Zaki, M., Ismail, M. A., and Ghanem, N. M. (2016). Brain Tumor Segmantation Using Random Forest Trained on Iteratively Selected Patients. In International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. doi: 10.1007/978-3-319-55524-9_13. Fahad, H. M., Khan, M. U. G., Saba, T., Rehman, A., and Iqbal, S. (2018). Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microscopy Research and Technique, 81(5), 449–457. doi:10.1002/jemt.22998. Fidon, L., Li, W., Garcia-Peraza-Herrera, L. C., Ekanayake, J., Kitchen, N., Ourselin, S., and Vercauteren, T. (2017). Scalable Multimodal Convolutional Networks for Brain Tumour Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 285–293). doi: 10.1007/978-3-319-66179-7_33. Havaei, M., Larochelle, H., Poulin, P., and Jodoin, P.-M. (2016). Within-brain classification for brain tumor segmentation. International Journal of Computer Assisted Radiology and Surgery, 11(5), 777–788. Husham, A., Alkawaz, M. H., Saba, T., Rehman, A., and Alghamdi, J. S. (2016). Automated nuclei segmentation of malignant using level sets. Microscopy Research and Technique, 79(10), 993–997. doi:10.1002/jemt.22733. Hussain, N., Khan, M. A., Sharif, M., Khan, S. A., Albesher, A. A., Saba, T., and Armaghan, A. (2020). A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-08852-3.

Image-Based Glioblastoma Segmentation

203

Iftikhar, S., Fatima, K., Rehman, A., Almazyad, A. S., and Saba, T. (2017). An evolution based hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research, 28(8), 3451–3455. Iqbal, S., Ghani, M. U., Saba, T., and Rehman, A. (2018). Brain tumor segmentation in multispectral MRI using convolutional neural networks (CNN). Microscopy Research and Technique. 81(4), 419–427. doi:10.1002/jemt.22994. Iqbal, S., Khan, M. U. G., Saba, T., Mehmood, Z., Javaid, N., Rehman, A., and Abbasi, R. (2019). Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microscopy Research and Technique, 82(8), 1302–1315. https:// doi.org/10.1002/jemt.23281. Iqbal, S., Khan, M. U. G., Saba, T., and Rehman, A. (2017). Computer assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, 8(1), 5–28. doi:10.1007/s13534-017-0050-3. Jamal, A., Hazim Alkawaz, M., Rehman, A., and Saba, T. (2017). Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17), 799–811. https://doi.org/10.1002/jemt. Javed, R., Rahim, M. S. M, and Saba, T. (2019a). An improved framework by mapping salient features for skin lesion detection and classification using the optimized hybrid features. International Journal of Advanced Trends in Computer Science and Engineering, 8(1), 95–101. Javed, R., Rahim, M. S. M., Saba, T., and Rashid, M. (2019b). Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), 1–10. Javed, R., Rahim, M. S. M., Saba, T., and Rehman, A. (2020a). A comparative study of features selection for skin lesion detection from dermoscopic images. Network Modeling Analysis in Health Informatics and Bioinformatics, 9(1), 4. Javed, R., Saba, T., Shafry, M., and Rahim, M. (2020b). An Intelligent Saliency Segmentation Technique and Classification of Low Contrast Skin Lesion Dermoscopic Images Based on Histogram Decision. In 2019 12th International Conference on Developments in eSystems Engineering (DeSE) (pp. 164–169). doi: 10.1109/DeSE.2019.00039. Kalbkhani, H., Shayesteh, M. G., and Zali-Vargahan, B. (2013). Robust algorithm for brain magnetic resonance image (MRI) classification based on GARCH variances series. Biomedical Signal Processing and Control, 8(6), 909–919. Kaya, I. E., Pehlivanlı, A. Ç., Sekizkardeş, E. G., and Ibrikci, T. (2017). PCA based clustering for brain tumor segmentation of T1w MRI images. Computer Methods and Programs in Biomedicine, 140, 19–28. Kayalibay, B., Jensen, G., and van der Smagt, P. (2017). CNN-based segmentation of medical imaging data. arXiv preprint arXiv:1701.03056. Khan, M. A., Ashraf, I., Alhaisoni, M., Damaševičius, R., Scherer, R., Rehman, A., and Bukhari, S. A. C. (2020c). Multimodal brain tumor classification using deep learning and robust feature selection: a machine learning application for radiologists. Diagnostics, 10, 565. Khan, M. A., Akram, T., Sharif, M., Javed, K., Raza, M., and Saba, T. (2020a). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, 1–30. https://doi.org/10.1007/s11042-020-08726-8. Khan, M. A., Akram, T., Sharif, M., and Saba, T. (2020d). Fruits diseases classification: exploiting a hierarchical framework for deep features fusion and selection. Multimedia Tools and Applications https://doi.org/10.1007/s11042-020-09244-3. Khan, M. A., Akram, T., Sharif, M., Saba, T., Javed, K., Lali, I. U., Tanik, U. J., and Rehman, A. (2019d). Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion, Microscopy Research and Technique, 82(6), 741–763. http://doi.org/10.1002/jemt.23220.

204

Artificial Intelligence and Internet of Things

Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., and Rehman, A. (2019a). Multi-Model Deep Neural Network Based Features Extraction and Optimal Selection Approach for Skin Lesion Classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–7). IEEE. doi: 10.1109/ICCISci.2019.8716400. Khan, M. A., Lali, I. U., Rehman, A., Ishaq, M., Sharif, M., Saba, T., Zahoor, S., and Akram, T. (2019c). Brain tumor detection and classification: a framework of marker-based watershed algorithm andmultilevell priority features selection. Microscopy Research and Technique, 82(6), 909–922. https://doi.org/10.1002/jemt.23238. Khan, M. A., Sharif, M., Akram, T., Raza, M., Saba, T., and Rehman, A. (2020b). Handcrafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing, 87, 105986. Khan, M. A., Sharif, M. I., Raza, M., Anjum, A., Saba, T., and Shad, S. A. (2019b). Skin lesion segmentation and classification: a unified framework of deep neural network features fusion and selection. Expert Systems, e12497. https://doi.org/10.1111/exsy.12497. Khan, M. W., Sharif, M., Yasmin, M., and Saba, T. (2017). CDR based glaucoma detection using fundus images: a review. International Journal of Applied Pattern Recognition, 4(3), 261–306. Khan, S. A., Nazir, M., Khan, M. A., Saba, T., Javed, K., Rehman, A., … Awais, M. (2019e). Lungs nodule detection framework from computed tomography images using support vector machine. Microscopy Research and Technique, 82(8), 1256–1266. Khawaldeh, S., Pervaiz, U., Rafiq, A., and Alkhawaldeh, R. S. (2018). Noninvasive grading of glioma tumor using magnetic resonance imaging with convolutional neural networks. Applied Sciences, 8(1), 27. Liaqat, A., Khan, M. A., Sharif, M., Mittal, M., Saba, T., Manic, K. S., and Al Attar, F. N. H. (2020). Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Current Medical Imaging, 16(10), 1229–1242. Lun, T., and Hsu, W. (2016). Brain Tumor Segmentation Using Deep Convolutional Neural Network. In Proceedings of BRATS-MICCAI. https://sites.google.com/site/ braintumorsegmentation/home/brats_2016 Ma, C., Luo, G., and Wang, K. (2018). Concatenated and connected random forests with multiscale patch driven active contour model for automated brain tumor segmentation of MR images. IEEE Transactions on Medical Imaging, 37(8), 1943–1954. Marie-Sainte, S. L., Aburahmah, L., Almohaini, R., and Saba, T. (2019a). Current techniques for diabetes prediction: review and case study. Applied Sciences, 9(21), 4604. Marie-Sainte, S. L., Saba, T., Alsaleh, D., Alotaibi, A., and Bin, M. (2019b). An improved strategy for predicting diagnosis, survivability, and recurrence of Breast Cancer. Journal of Computational and Theoretical Nanoscience, 16(9), 3705–3711. Mehmood, I., Sajjad, M., Muhammad, K., Shah, S. I. A., Sangaiah, A. K., Shoaib, M., and Baik, S. W. (2019). An efficient computerized decision support system for the analysis and 3D visualization of brain tumor. Multimedia Tools and Applications, 78(10), 12723–12748. Meier, R., Knecht, U., Wiest, R., and Reyes, M. (2016). CRF-based brain tumor segmentation: alleviating the shrinking bias. In International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. doi: 10.1007/978-3-319-55524-9_10. Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A., and Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), 1068. Mughal, B., Muhammad, N., Sharif, M., Saba, T., and Rehman, A. (2017). Extraction of breast border and removal of pectoral muscle in wavelet domain. Biomedical Research, 28(11), 5041–5043.

Image-Based Glioblastoma Segmentation

205

Mughal, B., Muhammad, N., Sharif, M., Rehman, A., and Saba, T. (2018a). Removal of pectoral muscle based on topographic map and shape-shifting silhouette. BMC Cancer, 18(1), 778. Mughal, B., Sharif, M., Muhammad, N., and Saba, T. (2018b). A novel classification scheme to decline the mortality rate among women due to breast tumor. Microscopy Research and Technique, 81(2), 171–180. Nazir, M., Khan, M. A., Saba, T., and Rehman, A. (2019). Brain Tumor Detection from MRI images Using Multilevel Wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). doi: 10.1109/ICCISci.2019.8716413. Nguyen, P. T., Shankar, K., Hashim, W., and Maseleno, A. (2019). Brain tumor segmentation and classification using KNN algorithm. International Journal of Engineering and Advanced Technology, 8(6 Special Issue), 706–711. Nie, D., Zhang, H., Adeli, E., Liu, L., and Shen, D. (2016). 3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients. In International Conference on Medical Image Computing and Computer-Assisted Intervention. https:// doi.org/10.1007/978-3-319-46723-8_25. Norouzi, A., Rahim, M. S. M, Altameem, A., Saba, T., Rada, A. E., Rehman, A., and Uddin, M. (2014). Medical image segmentation methods, algorithms, and applications. IETE Technical Review, 31(3), 199–213. doi:10.1080/02564602.2014.906861. Özyurt, F., Sert, E., Avci, E., and Dogantekin, E. (2019). Brain tumor detection based on convolutional neural network with neutrosophic expert maximum fuzzy sure entropy. Measurement, 147, 106830. Pereira, S., Pinto, A., Alves, V., and Silva, C. A. (2016). Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5), 1240–1251. Perveen, S., Shahbaz, M., Saba, T., Keshavjee, K., Rehman, A., and Guergachi, A. (2020). Handling irregularly sampled longitudinal data and prognostic modeling of diabetes using machine learning technique. IEEE Access, 8, 21875–21885. Qureshi, I., Khan, M. A., Sharif, M., Saba, T., and Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images. International Journal of Intelligent Systems Technologies and Applications, 19(1), 1–16. doi: 10.1504/IJISTA.2020.105172. Rahim, M. S. M, Norouzi, A., Rehman, A., and Saba, T. (2017a). 3D bones segmentation based on CT images visualization. Biomedical Research, 28(8), 3641–3644. Rahim, M. S. M, Rehman, A., Kurniawan, F., and Saba, T. (2017b). Ear biometrics for human classification based on region features mining. Biomedical Research, 28 (10), 4660–4664. Raja, N. S. M., Fernandes, S., Dey, N., Satapathy, S. C., and Rajinikanth, V. (2018). Contrast enhanced medical MRI evaluation using Tsallis entropy and region growing segmentation. Journal of Ambient Intelligence and Humanized Computing, 1–12. https://doi. org/10.1007/s12652-018-0854-8. Ramzan, F., Khan, M. U. G., Iqbal, S., Saba, T., and Rehman, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, 103697–103709. Ramzan, F., Khan, M. U. G., Rehmat, A., Iqbal, S., Saba, T., Rehman, A., and Mehmood, Z. (2020a). A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), 37. Rathi, V., and Palani, S. (2012). Brain tumor MRI image classification with feature selection and extraction using linear discriminant analysis. arXiv preprint arXiv:1208.2128. Reddy, K. K., Solmaz, B., Yan, P., Avgeropoulos, N. G., Rippe, D. J., and Shah, M. (2012). Confidence Guided Enhancing Brain Tumor Segmentation in Multi-Parametric MRI. In 9th IEEE International Symposium on Biomedical Imaging (pp. 366–369). doi: 10.1109/ISBI.2012.6235560.

206

Artificial Intelligence and Internet of Things

Rehman, A., Abbas, N., Saba, T., Mahmood, T., and Kolivand, H. (2018c). Rouleaux red blood cells splitting in microscopic thin blood smear images via local maxima, circles drawing, and mapping with original RBCs. Microscopic Research and Technique, 81(7), 737–744. doi:10.1002/jemt.23030. Rehman, A., Abbas, N., Saba, T., Mehmood, Z., Mahmood, T., and Ahmed, K. T. (2018b). Microscopic malaria parasitemia diagnosis and grading on benchmark datasets. Microscopic Research and Technique, 81(9), 1042–1058. doi:10.1002/jemt.23071. Rehman, A., Abbas, N., Saba, T., Rahman, S. I. U., Mehmood, Z., Kolivand, K. (2018a). Classification of acute lymphoblastic leukemia using deep learning. Microscopy Research and Technique, 81(11), 1310–1317. doi:10.1002/jemt.23139. Rehman, A., Khan, M. A., Mehmood, Z., Saba, T., Sardaraz, M., and Rashid, M. (2020a). Microscopic melanoma detection and classification: a framework of pixel based fusion andmultilevell features reduction. Microscopy Research and Technique, 83(4), 410–423. doi:10.1002/jemt.23429. Rehman, A., Khan, M. A., Saba, T., Mehmood, Z., Tariq, U., Ayesha, N. (2020b). Microscopic brain tumor detection and classification using 3D CNN and feature selection architecture. Microscopic Research and Technique. https://doi.org/10.1002/jemt.23597. Ren, T., Wang, H., Feng, H., Xu, C., Liu, G., and Ding, P. (2019). Study on the improved fuzzy clustering algorithm and its application in brain image segmentation. Applied Soft Computing, 81, 105503. Reza, S., and Iftekharuddin, K. (14 September 2014). Improved Brain Tumor Tissue Segmentation Using Texture Features. In Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge) (pp. 27–30). Boston, MA. Saba, T. (2017). Halal food identification with neural assisted enhanced RFID antenna. Biomedical Research, 28(18), 7760–7762. Saba, T. (2019). Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique, 82(9), 1601–1609. doi.org/10.1002/ jemt.23326. Saba, T. (2020). Recent advancement in cancer detection using machine learning: systematic survey of decades, comparisons and challenges, Journal of Infection and Public Health, 13(9), 1274–1289, doi.org/10.1016/j.jiph.2020.06.033. Saba, T., Al-Zahrani, S., and Rehman, A. (2012). Expert system for offline clinical guidelines and treatment. Life Science Journal, 9(4), 2639–2658. Saba, T., Bokhari, S. T. F., Sharif, M., Yasmin, M., and Raza, M. (2018b). Fundus image classification methods for the detection of glaucoma: a review. Microscopy Research and Technique, 81(10), 1105–1121. doi:10.1002/jemt.23094. Saba, T., Haseeb, K., Ahmed, I., and Rehman, A. (2020a). Secure and energy-efficient framework using Internet of medical things for e-healthcare. Journal of Infection and Public Health, 13(10), 1567–1575. Saba, T., Mohamed, A. S., El-Affendi, M., Amin, J., Sharif, M. (2020b). Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, 221–230. Saba, T., Khan, M. A., Rehman, A., Marie-Sainte, S. L. (2019a). Region extraction and classification of skin cancer: a heterogeneous framework of deep CNN features fusion and reduction. Journal of Medical Systems, 43: 289. https://doi.org/10.1007/s10916-019-1413-3. Saba, T., Khan, S. U., Islam, N., Abbas, N., Rehman, A., Javaid, N., and Anjum, A., (2019c). Cloud-based decision support system for the detection and classification of malignant cells in breast cancer using breast cytology images. Microscopy Research and Technique, 82(6), 775–785. Saba, T., Rehman, A. Mehmood, Z., Kolivand, H., and Sharif, M. (2018a). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), 704–715. doi:10.2174/1573405613666170912164546.

Image-Based Glioblastoma Segmentation

207

Saba, T., Sameh, A., Khan, F., Shad, S. A., and Sharif, M. (2019b). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43(12), 332. Sajjad, M., Khan, S., Muhammad, K., Wu, W., Ullah, A., and Baik, S. W. (2019). Multi-grade brain tumor classification using deep CNN with extensive data augmentation. Journal of Computational Science, 30, 174–182. Seetha, J., and Raja, S. S. (2018). Brain tumor classification using convolutional neural networks. Biomedical and Pharmacology Journal, 11(3), 1457. Singh, P., Bhadauria, H., and Singh, A. (2014). Automatic Brain MRI Image Segmentation Using FCM and LSM. In Paper Presented at the Proceedings of 3rd International Conference on Reliability, Infocom Technologies and Optimization. doi: 10.1109/ ICRITO.2014.7014706. Soltaninejad, M., Yang, G., Lambrou, T., Allinson, N., Jones, T. L., Barrick, T. R., … Ye, X. (2017). Automated brain tumour detection and segmentation using superpixelbased extremely randomized trees in FLAIR MRI. International Journal of Computer Assisted Radiology and Surgery, 12(2), 183–203. Tahir, B., Iqbal, S., Khan. M. U. G., Saba, T., Mehmood, Z., Anjum, A., and Mahmood, T. (2019). Feature enhancement framework for brain tumor segmentation and classification. Microscopy Research and Technique, 82(6), 803–811. Talo, M., Baloglu, U. B., Yıldırım, Ö., and Acharya, U. R. (2019). Application of deep transfer learning for automated brain abnormality classification using MR images. Cognitive Systems Research, 54, 176–188. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z., and Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique, 82(4), 361–372. Urban, G., Bendszus, M., Hamprecht, F., and Kleesiek, J. (2014). Multimodal brain tumor segmentation using deep convolutional neural networks. In MICCAI BraTS (Brain Tumor Segmentation) Challenge. Proceedings, Winning Contribution (pp. 31–35). Usman, K., and Rajpoot, K. (2017). Brain tumor classification from multi-modality MRI using wavelets and machine learning. Pattern Analysis and Applications, 20(3), 871–881. Vaishnavee, K., and Amshakala, K. (2015). An Automated MRI Brain Image Segmentation and Tumor Detection Using SOM-Clustering and Proximal Support Vector Machine Classifier. In IEEE International Conference on Engineering and Technology (ICETECH). doi: 10.1109/ICETECH.2015.7275030. Verma, N. K., Gupta, P., Agrawal, P., and Cui, Y. (2009). MRI Brain Image Segmentation for Spotting Tumors Using Improved Mountain Clustering Approach. In IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009). doi: 10.1109/ AIPR.2009.5466301. Wasule, V., and Sonar, P. (2017). Classification of Brain MRI Using SVM and KNN Classifier. In Third International Conference on Sensing, Signal Processing and Security (ICSSS). doi: 10.1109/SSPS.2017.8071594. Yi, D., Zhou, M., Chen, Z., and Gevaert, O. (2016). 3-D convolutional neural networks for glioblastoma segmentation. arXiv preprint arXiv:1611.04534. Yogananda, C. G. B., Nalawade, S. S., Murugesan, G. K., Wagner, B., Pinho, M. C., Fei, B., Madhuranthakam, A. J., and Maldjian, J. A. (2019). Fully Automated Brain Tumor Segmentation and Survival Prediction of Gliomas using Deep Learning and MRI. In International MICCAI Brainlesion Workshop (pp. 99–112). Springer, Cham. Yousaf, K., Mehmood, Z., Awan, I. A., Saba, T., Alharbey, R., Qadah, T., and Alrige, M. A. (2019a). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, 1–23. doi: 10.1007/s10729-019-09486-0.

208

Artificial Intelligence and Internet of Things

Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A. M., Alharbey, R., and Rashid, M. (2019b). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: a survey. BioMed Research International, 2019, 1–26. Zeinalkhani, L., Jamaat, A. A., and Rostami, K. (2018). Diagnosis of brain tumor using combination of K-means clustering and genetic algorithm. Frontiers in Health Informatics, 7, 6. Zhang, N., Ruan, S., Lebonvallet, S., Liao, Q., and Zhu, Y. (2011). Kernel feature selection to fuse multi-spectral MRI images for brain tumor segmentation. Computer Vision and Image Understanding, 115(2), 256–269. Zhang, Y., Dong, Z., Wu, L., and Wang, S. (2011). A hybrid method for MRI brain image classification. Expert Systems with Applications, 38(8), 10049–10053.

8

Artificial Intelligence Techniques for Glaucoma Detection Through Retinal Images State of the Art Ayesha Shoukat and Shahzad Akbar

CONTENTS 8.1 Introduction��������������������������������������������������������������������������������������������������209 8.2 L iterature Review������������������������������������������������������������������������������������������� 213 8.2.1 Glaucoma Detection Through Conventional Machine Learning����� 215 8.2.2 Glaucoma Detection Through Deep Learning��������������������������������� 223 8.3 Discussion����������������������������������������������������������������������������������������������������� 231 8.4 Conclusion���������������������������������������������������������������������������������������������������� 234 Acknowledgment��������������������������������������������������������������������������������������������������� 235 References�������������������������������������������������������������������������������������������������������������� 235

8.1 INTRODUCTION The glaucoma is a group of eye conditions that damage the optic nerve and results in the vision loss. The structural changes in the eye cause the functional changes and detection of structural changes at early stage can help in diagnosis of glaucoma. The structure of the eye (Human Eye Anatomy, 2020) is shown in Figure 8.1 as follows. The aqueous humor is a fluid within the eye. Through some channel, the aqueous humor flows out of the eye. On that channel blockage, the fluid increases inside the eye and causes intraocular pressure (IOP). The extreme fluid pressure in the eye due to obstruction of drainage and the angle between the iris and the cornea gets narrow or closure. The glaucoma damages the retinal ganglion cells (RGC), which leads to the vision loss (Li et al., 2019). The glaucoma disease also occurs due to the expansion of the optic cup (OC) within the optic disk (OD). The increase in the OC with respect to the OD due to the increase in dead optic nerve fibers also increases the cup-to-disk ratio (CDR ) value (Mittapalli and Kande, 2016). The glaucoma is widely accepted as the second leading disease that causes permanent loss of vision 209

210

Artificial Intelligence and Internet of Things

FIGURE 8.1  Anatomy of eye

that is irreversible worldwide (Civit-Masot et al., 2020). The blindness caused by the glaucoma cannot be cured but can be avoided if detected earlier. The glaucoma is classified into many types as open-angle glaucoma (OAG), angular closure glaucoma (ACG), normal tension glaucoma and congenital glaucoma. The major types are OAG and the ACG (Qureshi et al., 2020; Saba et al., 2018). Most of the cases are reported under the OAG as it is commonly known glaucoma and almost 90% of glaucomatous patients are affected by the OAG. The drainage angle remains open in OAG. The ACG occurs due to the blocking of the drainage canal, which suddenly increases the IOP (Civit-Masot et al., 2020; Jamal et al., 2017). There are many risk factors of glaucoma such as increased IOP, increased CDR, the visual field loss and damaged optic nerve head (ONH). The aging is the main risk factor of glaucoma. The majority of the people remain unaware of the disease that leads to the severity of the disease to an advanced stage (Amin et al., 2018). World Health Organization (WHO) reported that about 64 million people in the world are suffering in glaucoma and 76 million people of age 40–80 years will suffer in glaucoma till 2020 (World Health Organization, 2019). The number of glaucoma patients in 2040 will exceed 118 million (World Health Organization, 2019). The increasing number of glaucomatous patients leads to the automatic diagnosis of the glaucoma due to the small number of trained physicians. The physician uses different methods for glaucoma detection such as gonioscopy, ophthalmoscopy, tonometry, perimetry and pachymetry. In tonometry, eye pressure is checked. Normally pressure exceeds than 22 mmHg suspected to glaucoma. In ophthalmoscopy, the physician examines the optic nerve shape and color. The angle between the cornea and the iris is observed through gonioscopy. The perimetry test is conducted to observe the visual field. For measuring the cornea thickness, the pachymetry test is performed. These tests help the experts to detect the glaucoma in clinic. The medical image processing captures the inner structure of the body for the clinical and medical analysis. Through the medical image processing, we can observe the internal body structure that cannot be seen by the human eye. The poor image quality is a hurdle in the analysis and the interpretation of the image for the disease diagnosis. The need of the time is to enhance the quality of the image for the better interpretation of the image such as for feature extraction and recognition.

AI Techniques for Glaucoma Detection

211

FIGURE 8.2  Healthy fundus image

The medical image processing has made possible the diagnosis of the diseases in early stages and through treatment of the disease saved human lives. In image processing, the image is taken as input and converted into the digital form where mathematical operations are performed to improve the quality of the image. The preprocessing of the image helps in better diagnosis of the disease. The fundus images are captured with the fundus camera that is the best tool to diagnose the glaucoma. The fundus camera is easier to operate and inexpensive. Easy availability and clear ocular structure of retinal fundus images help in accurate diagnosis of glaucoma. The retinal fundus images are a 2D representation of the inner structure of the eye that depicts the changes in CDR and ONH. Since the fundus images do not depict the glaucoma signs in early stage so the optical coherence tomography (OCT) is the 3D source for the glaucoma detection in the early stage. Though it is more expensive method to diagnose the glaucoma, it is used in many studies for the glaucoma detection. The following fundus images show the OD and OC in a healthy (Akbar et al., 2017a,b; Akram et al., 2020) and glaucomatous eye (Raja et al., 2020) in Figures 8.2 and 8.3.

FIGURE 8.3  Glaucoma affected fundus image with CDR

212

Artificial Intelligence and Internet of Things

FIGURE 8.4  Normal OCT image

The use of OCT for the evaluation of the structural changes due to glaucoma is widespread. It can detect the abnormality in the eye and its progression with accuracy. The OCT images can detect the glaucoma in the early stages. The OCT is an expensive tool being used for screening of large population and requires skillful and experienced clinicians for image processing. The detailed examination of the eye is possible with the OCT images. The following Figures 8.4 and 8.5 (Normal Retinal OCT Image, 2020) show normal images that depict the structure of the eye. The use of computer-aided diagnosis (CAD) system in glaucoma diagnosis results in fast and accurate results. For manual detection of glaucoma, clinicians require expertise for fast and accurate result, but there is a chance of mistake and it is also a time-consuming task. The CAD can perform this task efficiently and in less time. The feature extraction and the classification are performed in CAD. The CAD using

FIGURE 8.5  Glaucoma affected OCT images layers

AI Techniques for Glaucoma Detection

213

fundus image can detect the abnormality in the eye structure without considering the observer variability that is seen in the clinical diagnosis.

8.2  LITERATURE REVIEW The datasets are publicly available for the glaucoma diagnosis containing the retinal fundus photographs. As the convolution neural network (CNN) architecture require a mountain of data for the training, so huge repository of data is only possible with the data augmentation to train the CNN and get the good results. There are several available datasets such as DRISHTI-Glaucoma Screening (DRISHTI-GS) (Sivaswamy et al., 2014), glaucoma dataset containing both OCT and fundus images (Raja et al., 2020) dataset, retinal image database for optic nerve evaluation (RIMONE) (Fumero et al., 2011), online retinal fundus image database for glaucoma analysis (ORIGA) (Zhang et al., 2010), retinal fundus glaucoma challenge (REFUGE) (Orlando et al., 2020) and G1020 (Bajwa et al., 2020) for the glaucoma detection. Both the OCT and the fundus images are included in most of the datasets. The datasets contain both the glaucoma affected and the healthy images. The DRISHTI-GS dataset contained total 101 images for segmentation of OD and OC along with ground truth images. The training images were 50 and 51 images for the testing purpose. All the images focused on OD with field of view (FOV) 30 degrees and resolution of 2896 × 1944 in PNG format. The fundus region was focused only from the ground truth images by discarding the non-fundus region. The ground truth of each image contains three types of information such as region boundary, segmentation soft map and CDR. The manual segmentation of the OC and disk regions were performed by the experts with the dedicated marking tool. This is the publically available dataset. Raja et al. (2020) presented the dataset that contained 50 fundus and the corresponding OCT images. The data collection parameters were ONH, CDR and retinal layers. The OCT and fundus images were captures through TOPCON’S 3D OCT1000 camera. The outlining of inner limiting membrane (ILM) layer and retinal pigmented epithelium (RPE) layers was performed by the ophthalmologists. The OCT scan was ONH centered and their resolution was 951 × 456. The RIM-ONE dataset contains 169 high-resolution images that were specified for the ONH segmentation. The images were captured through fundus camera Nidek AFC-210. The classification of image was in four categories such as normal 118 images, 12 images for early glaucoma, 14 for moderate, 14 for deep and for ocular hypertension 11 images were specified. This is also publicly available. The ORIGA dataset consists of 650 retinal images. The trained professionals segmented the images and performed annotation on them. The grading information was attached with every image. Through the web interface in online depository, the user can request the images according to the required grading information. The data in the ORIGA database can be used for the image processing algorithms, the detection of the peripapillary atrophy (PPA) and junction of the disk boundary blood vessels. This dataset can be accessed online on request. The G1020 dataset consist of 1020 high-resolution fundus images for the glaucoma detection, CDR calculation, OD and OC segmentation, size of neuroretinal

214

Artificial Intelligence and Internet of Things

TABLE 8.1 Datasets with the Number of Glaucoma and Non-Glaucoma Images Serial No. 1 2 3 4 5 6

Dataset DRISHTI-GS Data on OCT and fundus images RIM-ONE ORIGA G1020 REFUGE

Glaucomatous Images 70 32 51 168 296 120

Healthy 31 18 118 482 724 1080

rim in inferior, superior, nasal and temporal regions and location of bounding box for OD. The images in the dataset were focused only on the fundus region by removing the unrelated region. The size of the images was between the 1944 × 2108 and 2426 × 3007 pixels. The dataset is publicly available. The REFUGE dataset contains 1200 fundus images with ground truth OD and OC segmentation. The fundus camera with 2124 × 2056 resolution and second device Canon-CR-2 with 1634 × 1634 resolution were used to capture the fundus images. The main focus in the images was on posterior pole to assess the ONH and retinal nerve fiber layer (RNFL) defects. The manual annotation of the OD and OC was performed by the seven different experts in glaucoma disease diagnosis. This dataset is also publicly available and considered largest dataset for glaucoma diagnosis. To evaluate the diagnosis performance for glaucoma of different algorithms, different performance metrics are used. The accuracy, receiver operating characteristic (ROC), sensitivity, specificity, dice similarity coefficient (DSC), confusion metrics, area under the curve (AUC) etc. are used to determine the efficiency of the algorithms. The accuracy represents the truly classified images such as healthy or glaucomatous to the total number of images. It can be calculated as: Accuracy =

( TP + TN )

( TP + FP + TN + FN )

The confusion matrix shows the classification results in the table format to visualize the results for performance evaluation. The true positive is predicted positive by the model, and it is true in predicting the image as glaucomatous. Similarly, true negative (TN) refers to truly predicted negative as the predicted image is healthy image and it is true. The false positive is predicted positive by the model, but it is false, and false negative is predicted negative by the model and it is also false. The sensitivity measures the correctly classified positive data points such as correctly classified glaucoma images. It is the proportion of the affected people whom test results are positive and they suffer in glaucoma. The proposed system performance is measured in case of uncertainty with the sensitivity results. The specificity measures the negative data points such as images labeled as the healthy images. It measures the correctly classified negative cases. It is the proportion

AI Techniques for Glaucoma Detection

215

of the healthy people with the negative test results. Both sensitivity and specificity can be measure as:

Sensitivity = Total No.of True Positive / Total No.of True Positive + Total No.of False Negative

Specificity = Total No.of True Negative / Total No.of True Negative + Total No.of False Positive

The ROC curve is represented by plotting the true positive against the FP at some threshold value. The classification results as glaucoma or healthy image are best represented by the AUC. The higher the AUC, the better the classification results are. The DICE is mostly used in medical field for the image segmentation and shows the similarity between two objects. It can be calculated as:

DICE = 2 × True Positive / 2 × True Positive + False Positive + False Negative

8.2.1 Glaucoma Detection Through Conventional Machine Learning The machine learning is a branch of artificial intelligence (AI) that performs the tasks in the same way as the humans. The meaningful pattern is extracted and is used for the training of the algorithms for performing the automated tasks. The trained algorithms find the abnormal part from the given image and point to the areas that need attention. The machine learning has smoothened the way for making more intelligent and the robust systems to accomplish the tasks with accuracy and the reliability. This has made the machine learning the most appealing field in the modern age. The machine learning is widely being used in speech recognition, vehicle automation and product recommendation. The machine learning is successfully being applied in medical imaging for the diagnosis of the diseases. Different machine learning algorithms exist for the classification of retinal diseases according to the features provided to the algorithms (Akbar et al., 2017a,b; 2018a,b; 2019). Different applications of machine learning are being used in heart disease diagnosis, prediction of diabetes, cancer prediction and robotic surgery (Iftikhar et al., 2017; Ullah et al., 2019). The algorithms of machine learning are categorized into three domains such as supervised, unsupervised and semi-supervised (Saba, 2020). The training of the algorithm with the given data label is known as supervised machine learning algorithm such as support vector machine (SVM), Naïve Bayes (NB), decision tree (DT), random forest (RF) and logical regression (LR), whereas the algorithm trained without the given data label is classified as unsupervised. Examples of unsupervised learning algorithms are principal component analysis (PCA) and clustering methods. In semisupervised machine learning algorithm, the model is trained by using both labeled and unlabeled data. As in machine learning, the features are handcrafted. So the more accurate features correctly classify the diseased images. The segmentation on

216

Artificial Intelligence and Internet of Things

FIGURE 8.6  Glaucoma diagnosis using fundus and OCT images

the preprocessed image is applied to focus on the affected region by eliminating the unnecessary part (Saba et al., 2018). The large number of features extracted from the images is reduced to a simplified features space by eliminating the irrelevant features to reduce the memory and the processing time. The most representative features help in the simplifying the model for the glaucoma detection with greater efficiency. The machine learning classifiers are trained on the fabricated features and tested by feeding the retinal images. In the unsupervised models, the model is trained without labeled data. The framework used in supervised machine learning models for the glaucoma classification is shown as follows in Figure 8.6. Different machine learning model has been introduced to diagnose the glaucoma. Acharya et al. (2011) developed a glaucoma detection method using the texture and high-order spectra (HOS) features using the fundus images. The texture and the HOS features proved robust for the classification into two classes. The features were normalized with the t-test technique and five different classifiers such as NB, SVM, RF and sequential minimal optimization (SMO) were used and the best results were shown by the RF algorithm with 91.7% accuracy on 60 fundus images. Mookiah et al. (2012) developed a glaucoma diagnosis method using the discrete wavelet transform (DWT) and higher order spectra (HOS) features using the fundus images. The images were processed with the histogram equalization and radon transform. The DWT and HOS features were classified by the SVM with radial basis function (RBF) kernel and achieved 95% of accuracy, 93.33% of sensitivity and 96.67% of specificity. More the glaucoma risk index (GRI) in combination with DWT and HOS features was also developed to assist the clinicians to classify the glaucoma and the normal images.

AI Techniques for Glaucoma Detection

217

Yousefi et al. (2013) developed a glaucoma diagnostic model using structural and functional features from OCT images. The RNFL thickness measurement and threshold for each standard automated perimetry (SAP) test value were used for the machine learning classifiers training and testing such as Bayesian net, Meta classification, Lazy K Star, Meta ensemble selection, alternating DT (ADT), RF tree, regression tree and simple classification. The best performed classifier was RF with area under receiver operating characteristics curve (AUROC) of 0.88%. Simonthomas et al. (2014) developed a novel glaucoma detection method using the fundus images dataset. The fundus images were preprocessed with gray level co-occurrence matrix (GLCM) technique. The features were extracted by combining the Haralick texture features with GLCM to get the robust image features for the classification as glaucomatous or the healthy images. The K-nearest neighbors (K-NN) classifier was used and achieved the best performance with accuracy of 98% on local dataset of 60 images. Sakthivel and Narayanan (2015) developed an automatic early-stage glaucoma detection method using the fundus images. The dilation and erosion morphology performed to find the region of interest (ROI) and the OD was detected through the ROI. The 2D Gabor filter applied for the localization of the OD. The feature extraction algorithms Daugman and local binary patterns (LBP) were applied for the extraction of the histogram features. Both the magnitude and phase features performed better in the glaucoma diagnosis. The Euclidean distance was used to find the distance between the features to detect glaucoma. The proposed model’s performance was measured using the time consumption, sensitivity, specificity and the AUC parameters. The proposed algorithm took less time and achieved 95.45% sensitivity, specificity and the ROC results by using the 44 fundus images. Acharya et al. (2015) developed a novel glaucoma diagnostic method using 510 fundus images. The Gabor transform was applied for feature extracting. The PCA reduced the dimensionality of the extracted features. Various features ranking approaches such as Wilcoxon test, Bhattacharyya space algorithm, ROC, t-test and entropy were used and the best ranking was done with the t-test approach. The classification was performed with the SVM and achieved the 93.10% accuracy, 89.75% sensitivity value and 96.20% specificity. The GRI with principal components was used to perform classification between two classes. Salam et al. (2015) proposed an algorithm using CDR and hybrid textual and intensive features for the detection of glaucoma. The color features such as color moments and autocorrelograms and texture features LBP were merged. The PCA was used to reduce the dimensions of these features. Thus, SVM was trained and tested on these features to get the classification result. The decision was made by calculating the CDR and the result from the classifier. The proposed algorithm obtained the sensitivity of 1, specificity of 0.88 and the accuracy of 0.92 on a local dataset containing 50 fundus images. Akram et al. (2015) proposed a novel scheme for the early-stage glaucoma diagnosis using the fundus images. The OD detected from the preprocessed image by focusing on the brighter region where blood vessels originate. The ROI was extracted and a feature space formed containing CDR, spatial and spectral features. The multivariate m-mediods approach was used for the classification of the glaucoma and

218

Artificial Intelligence and Internet of Things

non-glaucoma-based images. The model performance was measured and sensitivity, specificity and accuracy were found, respectively, as 85.72, 92.91 and 90.84. Dey and Bandyopadhyay (2016) developed a glaucoma diagnostic machine learning method to perform image processing on fundus images. The preprocessing on image was applied for the noise removal and image enhancement while features were extracted using the PCA technique. The SVM classification approach was used for the glaucoma and the non-glaucoma images. The method achieved the accuracy of 86%, positive predictive accuracy of 81.08%, sensitivity of 100%, specificity of 65% and negative predictive accuracy of 100%. Singh et al. (2016) developed a retinal image processing approach for the glaucoma diagnosis using the fundus images. The wavelet features from the segmented ROI extracted and normalized using the z-score normalization technique. Two methods for the feature reduction PCA and evolutionary attribute selection were used. Various machine learning classifiers including SVM, K-NN, RF, NB and artificial neural network (ANN) were used and the best classification results were obtained from the SVM and the K-NN with accuracy of 94.75%. Kim et al. (2017) developed a model for the glaucoma diagnosis by calculating the thickness of RNFL and the visual field. The best handcrafted features were extracted with t-test features evaluation. Four machine learning algorithms C4, SVM, RF and K-NN were used. A dataset with 100 cases for testing and 399 for training and validation was used. Best performance for the prediction of the glaucoma was achieved with the RF model with accuracy of 0.98, sensitivity of 0.983 and specificity of 0.975 and AUC of 0.979. Maheshwari et al. (2017) proposed automatic glaucoma detection method using the fundus images. The image decomposed through variational mode decomposition (VMD), and features were mined using ReliefF algorithm. The least squares-support vector machines (LS-SVM) performed the classification. The proposed approach achieved the accuracy of 95.19 on three-fold cross validation and 94.74% on ten-fold cross validation. Khalil et al. (2017) proposed a novel approach using hybrid approach of structural and textual features for the glaucoma automatic detection. The hybrid structural features (HSF) and hybrid texture features (HTF) modules examine different textual and intensive features and used SVM for the classification. In case of ambiguity in the decision, a third class was used. The super-pixel method was introduced for the detection of the damaged cup. The 100% accuracy for the calculation of CDR in two different channels was achieved for glaucoma detection. Acharya et al. (2017) proposed a novel CAD system for glaucoma detection. Multiple operations such as filters, normalization, equalization histogram, binning and image plane separation were performed in preprocessing of the image. The textons that depict the natural structure of the image were obtained by applying multiple filters such as Schmid filters, maximum response and Leung-Malik. The required features from texton are obtained through local configuration pattern (LCP). The sequential floating forward search (SFFS) technique was applied to decrease the features and t-test was used for ranking. Six different classifiers such as SVM, K-NN, DT, probabilistic neural network, linear discriminant analysis and quadratic discriminant analysis were used for the classification while measuring the performance

AI Techniques for Glaucoma Detection

219

with sensitivity, accuracy, specificity and positive predictive value. The K-NN performs the best classification as compared to other classifiers with accuracy of 95.7%, sensitivity of 96.2%, specificity of 93.7% and par plana vitrectomy of 98.3% on six characteristic features using 702 images. Sengar et al. (2017) proposed an automatic suspected glaucoma detection method by detecting the existence of hemorrhages in surrounding region of the OD using the fundus images. The adaptive threshold and geometrical features were used to segment the OD and hemorrhage in a particular region automatically. The model first segmented the ROI and then detected the hemorrhage detection for the suspected glaucoma. The model achieved an accuracy of 93.57% for the diagnosis of suspected glaucoma using fundus images. Christopher et al. (2018b) proposed the approach for glaucoma detection and progression using swept source optical coherence tomography (SS-OCT) images. The OCT circumpapillary RNFL circle scans, SAP, (SD)-OCT and FDT tests conducted on patients. The RNFL thickness map extracted from SS-OCT images and input to the unsupervised model logistic regression using the PCA technique to get the structural features. The RNFL PCA accuracy for the diagnosis of glaucoma was measured with AUC of 0.95. Kausu et al. (2018) developed glaucoma detection method using the 86 fundus images. The three fuzzy c-means clustering classes were used to segment the OD and the Otsu’s thresholding for the OC segmentation. The features were extracted using 2D- annual average daily traffic (AADT)-CWT, which were fed to four classifiers: multilayer perceptron (MLP), RF, SVM and AdaBoost. The best results were obtained using MLP with 97.67% accuracy, 98% specificity and 97.1% sensitivity. An et al. (2019) developed an algorithm for OAG detection using OCT and fundus images. The CNN was trained by using five types of input scales. The input to the CNN were the fundus images of optical disc in gray, thickness map of ganglion cell complex (GCC) and deviation map, RNFL thickness map and deviation map. A RF was used to combine and classify the results of the glaucomatous images. The results from each separate CNN model were obtained. The AUC values were obtained as RNFL thickness maps of 0.942, RNFL deviation maps of 0.949, macular GCC thickness maps of 0.944 and macular GCC deviation maps of 0.952. The machine learning classifier RF was used instead of fully connected layer for classification. The combination of five different CNN models improved the AUC to 0.963 by RF. The proposed model diagnosed the glaucomatous images with great accuracy. Sharma et al. (2019) developed an automatic diagnostic model of glaucoma using the high-order statistics (HOS) approach using the fundus images. The center slice bicepstrum was applied on the fundus images and many features were mined. To reduce the features, locality sensitive discriminant analysis (LSDA) technique was applied. The SVM was used for the classification using these features and achieved accuracies of 98.8% and 95% on the local and the public datasets. Mohamed et al. (2019) proposed an automatic glaucoma detection approach using super pixel classification approach. The Image was preprocessed for noise removal and illustration. Then the image pixel was aggregated using the simple linear iterative clustering (SLIC) approach, and features were extracted using the statistical pixel level (SPL) method. The SVM classified each super pixel as OC, OD, background

220

TABLE 8.2 Published Studies for the Diagnosis of Glaucoma Using Machine Learning Serial No. Authors 1 Acharya et al.

Mookiah et al.

2012

3

Yousefi et al.

2013

4

Simonthomas et al. Sakthivel and Narayanan

2014

6

Acharya et al.

2015

7

Salam et al.

2015

8

Akram et al.

2015

The CDR, spatial and spectral features space were classified by Multivariate m-mediods

2015

Dataset 60 images of local dataset

Results Accuracy 91.7%

60 images of local dataset

Accuracy 95% Specificity 96.67% Sensitivity 93.33%

632 images of local dataset 60 images of local dataset 44 images of local dataset

AUROC 0.88%

510 images of local dataset

554 images of local dataset

Accuracy 93.10% Sensitivity 89.75% Specificity 96.20% Sensitivity 1% Specificity 0.88% Accuracy 0.92% Sensitivity 85.72% Specificity 92.91% Accuracy 90.84%

50 images of local dataset

Accuracy 98% 95.45% Sensitivity, Specificity and ROC

Artificial Intelligence and Internet of Things

2

Technique The different features such as texture and HOS normalized with t-test technique and five different classifiers such as NB, SVM, RF and SMO were used and the best results were shown by the Random forest The images were processed with the histogram equalization and radon transform. The DWT and HOS features were classified by the SVM with RBF kernel. The GRI in combination with DWT and HOS features was also developed to assist the clinicians to differentiate between the glaucoma and the normal images The functional and structural features were fed to different classifiers and the best results were obtained with the random forest classifier The fundus images preprocessed with GLCM technique and Haralick texture features extracted from fundus images fed to K-NN classifier The optic disk detected through ROI and localization was performed using Gabor filter. The Daugman and LBP were applied for the extraction of the histogram features. The Euclidean distance was used to find the distance between the features to detect glaucoma The features extracted using the Gabor transform from fundus images and the dimensionality was decreased using PCA. The feature ranking was best performed by t-test approach and classification through SVM The textual and intensive features reduced with PCA fed to SVM

5

Year 2011

Serial No. Authors Year 9 Dey and 2016 Bandyopadhyay

Technique The features accessed through PCA after image preprocessing and fed to SVM

Dataset 100 images of local dataset

10

Singh et al.

2016

11

Kim et al.

2017

The wavelet features extracted from ROI, reduced with PCA and normalized using z-score normalization achieved the best result by K-NN and SVM classifiers The RNFL thickness and visual field used where features evaluated with t-test and fed to different classifiers and the best results were obtained with RF

63 images of local dataset 499 images of local dataset

12

Maheshwari et al.

2017

The VMD used for the image decomposition iteratively and features extracted using ReliefF algorithm fed to LS-SVM

488 images of local dataset

13

Khalil et al.

2017

Two modules HSF and HTF used textual and intensive features that were fed to SVM

14

Acharya et al.

2017

The features from texton obtained through LCP and SFFS technique was applied to decrease the features and t-test for ranking, which then fed to different classifiers and the best result obtained with K-NN classifier

100 images of local dataset 702 images of local dataset

15

Sengar et al.

2017

The adaptive threshold and geometrical features were used to segment OD and hemorrhage in a particular region to detect glaucoma

140 images of local dataset

Results Accuracy 86% PPA 81.08% Sensitivity 100% Specificity 65% NPA 100% Accuracy 94.75% Accuracy 0.98% AUC 0.979% Sensitivity 0.983% Specificity 0.975% Accuracy 95.19% (three-fold) 94.74% (ten-fold) Accuracy 100%

AI Techniques for Glaucoma Detection

TABLE 8.2  (Continued) Published Studies for the Diagnosis of Glaucoma Using Machine Learning

Accuracy 95.7% Sensitivity 96.2% Specificity 93.7% PPV 98.3% Accuracy 93.57%

221

(Continued)

222

TABLE 8.2  (Continued) Published Studies for the Diagnosis of Glaucoma Using Machine Learning Year Technique 2018a,b The RNFL thickness map mined from SS-OCT images fed to the unsupervised logistic regression model using PCA technique to get the structural features 2018 The three fuzzy c-means clustering classes segmented OD and Otsu’s thresholding OC. The features extracted using 2D-AADT and the best results obtained using MLP

Dataset 179 images of local dataset 86 images of local dataset

18

An et al.

2019

357 images of local dataset

19

Sharma et al.

2019

20

Mohamed et al. 2019

21

Gour and Khanna

2019

22

Bisneto et al.

2020

The transfer learning with CNNs on five different types of inputs were used. The data augmentation and dropout were applied and RF classifier was used for the classification instead of the fully connected layer The features extracted through center slice bicepstrum and reduced with LSDA (locality sensitive discriminant analysis) technique. The SVM is used for the classification The image pixel aggregated using the SLIC and features extracted through SPL (statistical pixel level) method. The SVM classified each super pixel and CDR was measured to determine the detection of glaucoma The image preprocessed using CLAHE approach. The GIST and PHOG features were extracted and normalized using z-score. The feature selection was done by PCA and fed to the SVM for classification The GAN algorithm applied for OD segmentation and texture features were extracted through taxonomy indexes. The classification is performed using MLP, RF and SMO

PRIVATEPUBLIC datasets RIM-ONE

Results AUC 0.95% Accuracy 97.67% Sensitivity 97.1% Specificity 98% AUC 0.963%

Accuracy 98.8%

Accuracy 98.6% Sensitivity 92.3%

DRISHTI-GS1 Accuracy 83.40% and HRF AUC 0.88% DRISHTI-GS RIM-ONE v2

Accuracy 100% AUC 1% Sensitivity 100% Specificity 100%

Artificial Intelligence and Internet of Things

Serial No. Authors 16 Christopher et al. 17 Kausu et al.

AI Techniques for Glaucoma Detection

223

region or blood vessel. Then CDR was measured to determine the detection of glaucoma with accuracy of 98.6% and sensitivity of 92.3% on RIM-ONE dataset. Gour and Khanna (2019) developed a glaucoma diagnostic system from the fundus images. In image preprocessing by using contrast limited adaptive histogram equalization (CLAHE) approach, small region of the image was extracted. The global image descriptor (GIST) and pyramid histogram of oriented gradients (PHOG) features were extracted and normalized using z-score. The feature selection was done by the PCA technique and fed to the SVM classifier for the classification of glaucoma and non-glaucoma images. The performance was achieved with AUC of 0.86 and accuracy of 79.20% on DRISHTI-GS and high-resolution fundus (HRF) datasets. Bisneto et al. (2020) developed a generative adversarial network (GAN) using the texture features from the fundus images. The GAN algorithm was applied for OD segmentation on images from the DRISHTI-GS dataset and RIM-ONE datasets. After post-processing of image, texture features were extracted through taxonomy indexes. The classification was performed using MLP, RF and SMO classifiers and achieved 100% accuracy, 100% specificity, 100% sensitivity and AUC.

8.2.2 Glaucoma Detection Through Deep Learning The deep learning is the branch of machine learning that is more advance than machine learning techniques. Deep learning architectures are widely being used for the development of different system for voice recognition, natural language processing and medical image analysis (Mashood Nasir et al., 2020). Various applications are being developed and successfully used across the world for the automatic disease detection and showed the best performance. The successful application of deep learning in medical imaging paved the way for the early-stage diagnosis of the glaucoma diseases. The CNN is widely used deep learning neural network. Unlike machine learning where the features are hand crafted, CNN extracts the features itself. The CNN architecture learns itself during the training. There are different layers in CNN such as convolution layers, activation (rectified linear unit [ReLU]) layer, max pooling layer, fully connected layers (Saba et al., 2020a,b). The function of these layers is described here. The convolution layer applies the filter on the input image and outputs a feature map. The activation layer results in the activation of the output on the given input. The pooling layer reduces or compresses the feature map size. The last layer of the CNN is the fully connected layer (Saba et al., 2019). It performs the classification to label the input image as the glaucomatous or healthy. The framework of deep learning architecture for the glaucoma diagnosis using the fundus images is shown in Figure 8.7. Different architectures of CNN (Fukushima and Miyake, 1982) are developed such as AlexNet (Krizhevsky et al., 2012), ResNet (He et al., 2016), GoogleNet (Szegedy et al., 2015), Inception v2 (Ioffe and Szegedy, 2015) and visual geometry group (VGG) (Simonyan and Zisserman, 2014). These architectures have their own structures in which different number of layers are used, which affects the speed and efficiency of the architecture. This chapter provides an overview of studies that uses the deep learning architectures to diagnose the glaucoma automatically. Gayathri

224

Artificial Intelligence and Internet of Things

FIGURE 8.7  The deep learning CNN architecture

et al. (2014) developed a glaucoma diagnosis method using the energy and the structural features. After applying DWT transformation, it normalized with the z-score that is then fed to the MLP and back propagation (BP) ANN and NB. The best classification results were obtained by the ANN with accuracy of 97.6% on the 30 fundus images. Yadav et al. (2014) proposed a glaucoma detection method using texture features with neural network on fundus images. The texture features such as energy, homogeneity, entropy, correlation, contrast and standard deviation from the color fundus images were extracted using the GLCM technique. The PCA approach was used for the statistical analysis. The adaptive resonance theory (ART) method was used for the classification and got an accuracy of 75% on 20 fundus images dataset. Chen et al. (2015) developed an automatic deep learning model CNN modified from the traditional model for the glaucoma detection using the fundus images. The proposed model adds MLP for the abstraction of data in the receptive field. The contextualizing training approach was used for the learning of features and obtained 83.8% AUC value on ORIGA and 89.8% AUC on Singapore Chinese eye study (SCES) datasets. Asaoka et al. (2016) developed a deep Feed forwarding neural network (FNN) for the glaucoma detection using 159 fundus images. The proposed model used stacked auto-encoder for the distinction of the healthy and glaucomatous images. The FNN was compared with other machine learning SVM, RF and neural network and showed a significant AUC of 92.6%. Sevastopolsky (2017) developed a deep learning modified U-Net architecture for the automatic cup and disk segmentation. Three public datasets such as RIM-ONE v3, DRISHTI-GS and digital retinal images for optic nerve segmentation database (DRIONS-DB) were used. In preprocessing, the CLAHE approach was applied. In the U-Net architecture, the contracting path repeatedly applied on the architecture of the convolution part for the classification. The expensive path merged the information of the layers from the contracting and the expensive path to recognize the pattern in the image. The modification from the typical architecture was that there were less number of filters in the convolution layers, and in case of decreasing resolution, the number of filters was also decreasing. The proposed approach required less parameters and the training time. The architecture performed well with DICE of 0.95 and intersection over union (IOU) of 0.89 for disk segmentation on RIM-ONE-v3 dataset and cup segmentation with DICE of 0.85 and IOU of 0.75 on DRISHTI-GS dataset.

AI Techniques for Glaucoma Detection

225

Abbas (2017) proposed a model for the glaucoma detection named as glaucoma deep using the retinal fundus images. The model first detected the ROI from the high-intensity green plane using red green blue (RGB) image and extracted the deep pixel level features. The optimization of features was done by deep belief network (DBN) and classified using soft-max linear classifier between normal and the glaucoma images. The deep glaucoma achieved the sensitivity of 84.50%, specificity of 98.01%, accuracy of 99% and precision of 84%. Al-Bander et al. (2017) developed a fully automated CNN for the classification of normal and the glaucomatous fundus images. The RIM-ONE dataset was used where images were resized to 227 × 227 without applying any other enhancement. The AlexNet is a pretrained model consisting of 23 layers was developed. The convolution layers, max pooling layers, softmax layer and fully connected layers and output layer were used as feature extractor in the model. The fully connected layer that performs classification was removed for just extracting the features from the model. The model was designed for extracting the features as the high-level features were found in the deeper layers of the CNN. The extracted features were then fed to the SVM for classification. The network obtained sensitivity, specificity and accuracy as 85%, 90.8% and 88.2%, respectively. Raghavendra et al. (2018) proposed a CAD tool for glaucoma classification using the deep learning techniques. A CNN with 18 layers was trained to get the best features for the classification as healthy or glaucomatous using the fundus images. The network architecture was designed as the first layer got 2D or 3D image as input. The next convolution layer performed convolution on the image that created the feature map. The next layer was batch normalization layer that helped in faster learning and improved overall performance. The next was the ReLU that removed the redundant data and only the important features. The next was the max pooling layer that reduced the size of the feature map. This layer reduced the size of output. The fully connected layer specified the classes for the classification. Finally, the soft-max layer transformed the multidimensional data into 0 or 1. It helped to reduce the outliers. The tool got the best accuracy of 98.13 at the 0.0001 learning rate. Christopher et al. (2018a) developed three deep learning models ResNet50, Inception and VGG16 for the glaucoma detection using the retinal fundus images. The origin on the ONH was extracted from the fundus images. The location of the disk center was extracted through the CNN model. The data augmentation was applied and 148,220 images were created for training. The models ResNet, VGG16 and Inception were trained and their performance was compared. The number of layers and the arrangement of each architecture were different from each other. The models were evaluated through native data and the transfer learning technique. The GON images were classified as mild or moderate to severe. The validation performed through ten-fold. The occlusion testing was used for the decision-making. The ResNet50 architecture achieved the highest AUC of 0.91 among all architectures with transfer learning for healthy eyes, the AUC of 0.97 for the mild-to-severe functional loss and 0.98 AUC with the mild loss on 14,822 images. Ahn et al. (2018) developed a deep learning model for the automatic glaucoma detection using fundus photographs. The logistic classification model, CNN and pretrained GoogleNet Inception v3 model were trained and tested on dataset of

226

Artificial Intelligence and Internet of Things

1542 fundus images. The data augmentation was applied to provide a large training data. The accuracy of logistic classification was 82.9%, validation accuracy was 79.9% and testing accuracy was 77.2%. The CNN accuracy were calculated as 92.2%, 88.6%, 87.9% on training, validating testing data and AUROC was as 0.98, 0.95, 0.94 on training, validation and test data, respectively. The pretrained GoogleNet Inception v3 got AUROC and accuracy of 0.99% and 99.7% on training data, 0.95% and 87.7% on validation data and 0.93% and 84.5% on test data. CNN model predicts the early glaucoma efficiently among all. Shibata et al. (2018) developed a deep residual learning model for the glaucoma detection utilizing the fundus images and compared the results with the ophthalmologists. The fundus images were cropped using the Hough transform technique. The data augmentation was also applied by inverting images vertically and horizontally to avoid overfitting. The 1364 glaucomatous and 1768 without glaucoma images were used for the training. The algorithm obtained the high performance with AROC 96.5 as compared to the ophthalmologists that was 72.6% and 91.2%. Al-Bander et al. (2018) developed a deep learning model DenseNet for the automatic detection of glaucoma in early stages. Five datasets such as RIM-ONE, DRIONS-DB, DRISHTI-GS, optic nerve head segmentation dataset (ONHSD) and ORIGA were used. The preprocessing was performed on images considering the green channel only from the fundus images and cropped OD from the ROI. The dense model with fully convolution network (U-shaped architecture) was developed for the classification. The optimization algorithm RMSprop was trained using the ORIGA dataset and tested on the other four datasets. The CDR was calculated to diagnose the glaucoma with AUROC of 0.7776. Gómez-Valverde et al. (2019) developed a CNN model based on various architectures CNN, VGG19, ResNet50, DENet and GoogLeNet using the DRISHTI-GS, RIM-ONE and local datasets in total 2313 fundus images. The performance of all the architectures was measured, and the best performance was achieved by the pretrained VGG19 model as AUC calculated as 0.94, sensitivity as 87.01 and specificity as 89.01. Phan et al. (2019) used three deep CNN for the glaucoma diagnosis using 3312 fundus images. The layers used by the ResNet152, DenseNet201 and VGG19 were the convolution, max pooling and activation and the fully connected layer. The ResNet152 and DenseNet201 further used residual connections or dense connections. The impact of image size on the network ability was determined by using the adaptive average pooling layer before the first FCL. The heat map areas that contribute most for the diagnosis of the glaucoma were also determined with the class activation map. More 465 poor quality images were also used for determining the ability of the system to diagnose the glaucoma. All three DCNN ResNet152, DenseNet201 and VGG19 achieved the AUC of 0.9. Liao et al. (2019) proposed a novel CNN-based scheme that used ResBlock architecture for the diagnosis of glaucoma using ORIGA dataset. The model not only diagnosed the glaucoma but also provided the transparent interpretation based on visual evidences by highlighting the affected area. The model named EAMNet contained three parts such as CNN ResNet architecture extracted the features and aggregation, the multi layers average pooling (M-LAP) linked the semantic detail

AI Techniques for Glaucoma Detection

227

and information of the localization. The evidence activation map (EAP) for the evidence of the affected area was used by the physician for the final decision. The ResNet contained convolution layer and the pooling layer. More the dropout, more the batch normalization layer was used to avoid overfitting. The activation map was used to provide the clinical basis for glaucoma. The proposed scheme diagnosed the glaucoma with great efficiency like AUC of 0.88. Serte and Serener (2019) developed a generalized CNN model for the classification of the glaucoma disease using fundus images of five different datasets. The model was trained on four datasets and one dataset was used for the testing. A preprocessing was applied on the images for better assessment of the disease. For classification, three models such as ResNet50, ResNet152 and GoogleNet were used. All the models used 50, 152 and 21 layers, respectively. All the models were tested on one dataset, whereas trained on all other four datasets. Different performance measures were used to compare the results with the other studies and showed the better results as AUC of 87% better than previous, accuracy of 53% and specificity of the model was 100% better than the previous. Overall, the proposed model was 80% better than the previous. Juneja et al. (2019) developed the G-Net model based on CNN for the detection of glaucoma on DRISHTI-GS dataset. The model used two neural networks (U-Net) for the segmentation of the disk and cup separately in conjunction. The cropped fundus images of size 128 × 128 in red channel were fed to the model. The model contained 31 layers consisting of convolutional, max-pooling, up-sampling and merge layer. The filters were applied of size (3, 3), (1, 1) and the number of filters were 1, 32 and 64 on different layers. The dropout layer was discarded due to the small number of the training data in the dataset. The model labels the pixel as black on segmenting the OD in real image otherwise white. The convolutional layer uses sigmoid function output in the form of 1 or 0 on all the input. The output images were fed to the other model for the segmentation of the cup. The second model was same as the first with a single difference was that the size of filters was increased as (4, 4). The output of this model was segmented cup. These two outputs were used to calculate the CDR for the glaucoma prediction. This algorithm worked using two NN working in combination to attain high accuracy of 95.8% of OD and 93.0% of OC segmentation. Yu et al. (2019) developed a deep learning model using modified version of U-Net architecture for the glaucoma diagnosis using the fundus images. The U-Net used the pretrained ResNet34 for encoder and the classical U-Net architecture as decoding layer. The decoding layer combined the up-sampled feature map from preceding layers parallel to the down-sampling feature map layers. For the segmentation of the disk, the U-Net architecture was applied. This was done to get the center and the diameter of the disk, which estimated the ROI. The resized ROI was used to calculate the cup and the disk segmentation. The segmented disk and the cup were further processed by the morphological preprocessed module. The CDR was measured by the vertical diameter of OC and OD. The proposed model used three datasets RIGA, DRISHTI-GS and RIM-ONE. The model achieved the best performance as compared to the experts who manually predicted. Then the model was tested on the other two datasets without training on them and got good performance with 97.38% of disc DICE value and 88.77% of cup DICE value for DRISHTI-GS test set, 96.10%

228

Artificial Intelligence and Internet of Things

of disc DICE and 84.45% of disc DICE for RIM-ONE database. The model used the pretrained Res-Net34 and U-Net that showed the best performance instead of training the model from scratch. It saved the time enabling fast training of model with less epochs and avoided overfitting. Maetschke et al. (2019) developed a CNN model for the glaucoma detection and compared it with the machine learning algorithm using 1110 OCT images. The total 22 features were extracted and fed to different machine learning classifiers such as NB, RF, SVM, LR, Gradient AdaBoost and Extra Trees. The other feature agnostic technique CNN was also used for the classification and achieved the better results with AUC of 0.97 than obtained from handcrafted machine learning approaches such as logistic regression with AUC of 0.89. Civit-Masot et al. (2020) proposed an ensemble technique in which two independent subsystems were being used for glaucoma diagnosis using the fundus images on RIM-ONE r3 and DRISHTI combined datasets. The first one based on dual CNN (U-Net architecture) for segmentation and the second on transfer-learning-based MobileNet V2 architecture. The preprocessing was performed on the images. Dual CNN U-Net architecture segmented the cup and disc and extracted their features that assist in the diagnosis of glaucoma. The static and dynamic data augmentation was performed for the training and testing. Six layers network with the 64 channels, and 2.5M parameters were used for segmentation. The MobileNet V2 directly classified the images as the glaucomatous or healthy. There was no need for the image preprocessing and the feature extraction in the CNN. In MobileNet V2, upper most layers were discarded, and the classifier was added based on average pooling layer. An average pooling layer, dense layer with 64 nodes, dropout and a final dense layer for classification of the classes has been used. Both architectures diagnosed the glaucoma independently and combined their results for final decision. This approach collected the output from both the subsystems and provided the report to the physician for a final decision. This approach achieved the specificity of 86%, accuracy of 88%, sensitivity of 91% and AUC of 96%. Li et al. (2019) proposed an approach named AG-CNN that detected the glaucoma and pathological area localization using the fundus images. The model based on the attention prediction, localization of the affected area and glaucoma classification. The glaucoma prediction was done by the deep features highlighted by the visual maps of neurotic areas on the LAG and RIM-ONE datasets. The use of attention maps for localization of the pathological area was very effective. The model prediction for the glaucoma was better than the previous models with accuracy of 95.3%. Thakoor et al. (2019) developed model based on different CNNs architecture trained on OCT images and some pretrained on the ImageNet for the detection of glaucoma. The pretrained ResNet, VGG and InceptionNet were used in combination with RF and compared with CNNs trained on OCT images and achieved a higher accuracy 96.27% with the CNN trained on OCT image. Thakur et al. (2020) proposed a deep learning technique for the glaucoma detection before disease onset. The proposed technique diagnosed the glaucoma several years such 1–3 years and 4–7 years ago before the onset of the disease. In the proposed approach, two readers read the fundus image, if any abnormality was found then an endpoint committee further investigated it. Three datasets were designed for

Serial No.

Authors

Year

Dataset

Results

2014 2014 2015

Deep Learning Architectures ANN ART CNN

1 2 3

Gayathri et al. Yadav et al. Chen et al.

Local dataset of 30 images Local dataset of 20 images SCES, ORIGA

Asaoka et al. Sevastopolsk

2016 2017

FNN Modified U-Net

Abbas

2017

Accuracy 97.6% Accuracy 75% AUC 83.8% on ORIGA dataset, 89.8% on SCES dataset AUC 92.6% DICE 0.95 and IOU 0.89 (disk segmentation) DICE 0.85 and IOU 0.75 (cup segmentation) Sensitivity 84.50% PRC 84% Specificity 98.01% Accuracy 99% Specificity 85% Accuracy 90.8% Sensitivity 88.2% Accuracy 98.13%

4 5 6

AlexNet

2018

CNN

Local dataset of 1426 images

VGG16, Inception, ResNet50

Local dataset of 14,822 images

CNN, logistic regression, GoogleNet Inception v3 ResNet DenseNet

Local dataset of 1542 images

Soft-max linear

7

Al-Bander et al.

8 9

Raghavendra et al. Christopher et al.

10 11 12

2018a,b

2018 Ahn et al. Shibata et al. Al-Bander et al.

2018 2018

Local dataset of 3132 images RIM-ONE, ORIGA, DRISHTI -GS, ONHSD, DRIONS-DB

Sensitivity 88% Specificity 95% AUC 0.97 Accuracy 87.9% AUROC 0.94% (CNN) AROC 96.5% AUROC of 0.7776% (Continued)

229

2017

Local dataset of 159 images RIM-ONE v. 3, DRISHTI-GS, DRIONS-DB DRIONS-DB sjchoi86-HRF HRF PRV-Glaucoma RIM-ONE

AI Techniques for Glaucoma Detection

TABLE 8.3 Represents the Deep Learning Glaucoma Diagnosis Approaches

Authors

Year

13

Gómez-Valverde et al.

2019

14

Phan et al.

2019

15 16

LIAO et al. Serte and Serener

2019 2019

17

Juneja et al.

18

Deep Learning Architectures VGG19

Dataset

Results

Local dataset of 2313 images

AUC 0.94% Sensitivity 87.01% Specificity 89.01% AUC 0.9%

ResNet152, DenseNet201, VGG19 ResNet ResNet50, ResNet152 and GoogleNet

Local dataset of 3777 images

2019

U-Net

DRISHTI-GS

Yu et al.

2019

U-Net, ResNet

RIGA, DRISHTI-GS, RIM-ONE

19 20 21 22

Maetschke et al. Li et al. Thakoor et al. Thakur et al.

2019 2019 2019 2020

CNN CNN CNN MobileNet V2

23

Maheshwari et al.

2020

AlexNet

1110 LAG, RIM-ONE Local dataset of 737 images Local datasets of 45,301, 42,601, 42,498 images RIM-ONE

24 25

de Moura Lima et al. Hemelings et al.

2020 2020

CNN ResNet128

RIM-ONE r3 Local dataset of 1424 images

26

Saxena et al.

2020

CNN

ORIGA, SCES

ORIGA HRF, DRISHTI-GS1, RIM-ONE, sjchoi86-HRF, ACRIMA

Accuracy 0.88% Accuracy 53% AUC 83% Specificity 100% Accuracy 95.8% (OD segmentation) 93.0% (OC segmentation) DICE 97.38% (disk) DICE 88.77% (cup) AUC 0.94% Accuracy 95.3% Accuracy 96.27% AUC 0.97% Accuracy: 98.90% Sensitivity: 100% Specificity: 97.50% Accuracy 91% AUC 0.995% Sensitivity 99.2% Specificity 93% AUC 0.822 (ORIGA) AUC 0.882 (SCES)

Artificial Intelligence and Internet of Things

Serial No.

230

TABLE 8.3  (Continued) Represents the Deep Learning Glaucoma Diagnosis Approaches

AI Techniques for Glaucoma Detection

231

training of three different CNN models. These datasets will be used for the glaucoma diagnosis for 4–7 years, 1–3 years and after onset of the disease, respectively. The image preprocessing was performed on the fundus images. The CNN architecture MobileNet V2 was used to develop the model. Three MobileNet V2 models were trained on the three datasets. The transfer learning was used for training using ImageNet dataset. The data augmentation was used for training for equal number of classes in the dataset. The activation map was used to validate and verify the clinical region asses for diagnosis. The AUCs of the three models were of 0.88, 0.77 and 0.97, respectively. Maheshwari et al. (2020) proposed a novel method for the glaucoma detection using the fundus images in RIM-ONE dataset. The proposed approach used transfer learning approach and LBP data augmentation. The pretrained AlexNet model was used for transfer learning. The model separate the training and testing data and then the images were converted into the red, green and the blue channel. More LBP-based augmentation was applied on the training data. The LBPS was computed for each channel. The model was trained using the augmented training through the transfer learning. The decisions were combined from all modules using decision-level fusion technique. The proposed approach achieved accuracy of 98.90, sensitivity of 100 and specificity of 97.50 for the glaucoma detection. de Moura Lima et al. (2020) used a genetic algorithm (GA) to optimize CNNs architectures through evolution that can help in glaucoma diagnosis using eye’s fundus image from RIM-ONE-r2 dataset. The GA constructed the CNN with 25 layers for the best classification of the glaucomatous and the normal image with accuracy of 91%. The use of GA for the optimization solution of the problems showed the best results. Hemelings et al. (2020) proposed a deep learning approach for the diagnosis of glaucoma using OD centered fundus images. Pretrained Res-Net architecture with 128 layers using transfer learning approach was used for the training purpose. The use of active and transfer learning in deep networks made the system optimal for the training and diagnosis of the glaucoma. The saliency maps were used for the visual representation and helped the physicians in making final decision. The proposed approach achieved the AUC of 0.995, sensitivity of 99.2% and specificity of 93% on 1424 test images. Saxena et al. (2020) developed a CNN approach for the detection of glaucoma diagnosis ORIGA and SCES dataset. The ARGALI approach applied for removing the brighter part and extraction of the ROI was an input into the CNN for the classification. The data augmentation was used to avoid overfitting. The AUC of 0.882 was achieved on SCES dataset and 0.822 on the ORIGA dataset.

8.3 DISCUSSION In this chapter, various machine learning and deep learning approaches of the recent years for the automatic glaucoma diagnosis adopted by different researchers have been described. In machine learning studies, the basic glaucoma diagnosis framework contains the image preprocessing, feature extraction, feature reduction, normalization and the classification. The image is preprocessed by removing noise and performing enhancement to get better classification results as Mohamed et al. (2019) preprocessed

232

Artificial Intelligence and Internet of Things

FIGURE 8.8  Commonly used machine learning classifiers

the images using the CLAHE approach, which improves the contrast in the image and focuses on small regions of the images for better classification results. Simonthomas et al. (2014) used GLCM approach for the processing of the image, which is used to measure the texture of the image and extract many features. The handcrafted features are reduced, normalized and fed to different classifiers for classification. The machine learning algorithms lack of capability of automatic learning of features from the given data. The features are reduced using different machine learning approaches such as PCA and SFFS. The most commonly used classifiers are SVM, NB, RF, K-NN, logistic regression etc. for the glaucoma diagnosis. Figure 8.8 shows the frequently used classifiers for the detection of the glaucoma in machine learning. The SVM is the most frequently used classifier in the machine learning approaches. Table 8.2 shows all the features extraction, normalization techniques and classifiers used by the researchers in their studies. Different features in different approaches such as CDR, rim-to-disk ratio, spatial feature and spectral features (Akram et al., 2015), wavelet features (Singh et al., 2016), hybrid structural and textual (Khalil et al., 2017), GIST and PHOG features (Gour and Khanna, 2019) are extracted and Singh et al. (2016) used PCA methods for the feature reduction, Acharya et al. (2017) applied SFFS technique to decrease the features and t-test for the ranking of the best features. Sharma et al. (2019) reduced features using LSDA technique. Bisneto et al. (2020) used MLP, RF, SMO and achieved accuracy of 100%, sensitivity of 100% and specificity of 100%. Sharma et al. (2019) used SVM and achieved the accuracy 98.8%. The human extracted features do not guarantee to provide optimal results for the diagnosis. Now the deep learning models are widely used over the traditional machine learning due to their automatic feature learning and classification capability. In machine learning, the accuracy of the diagnosis of the glaucoma is affected by the features extracted by the ophthalmologists. The biasness is also involved at the ophthalmologists’ end. The CAD system are being replaced by the deep learning models for the accurate glaucoma diagnosis but a large repository of data is needed for the training of the deep learning. Different techniques can be used for this purpose such

AI Techniques for Glaucoma Detection

233

as pretrained models, data augmentation and transfer learning to provide a large amount of data for training. In deep learning, CNN is the most commonly used for the classification problems. The application of CNN in medical field is widely accepted. The convolution layer in CNN extracts the features from the image and learns during training. The pooling layer minimizes the features map and retains only the significant information. The classification is performed by the fully connected layer. Different CNN architecture is also used such as ResNet, U-Net, MobileNet V2, GoogleNet, DenseNet, AlexNet, VGG and Inception are used in the studies. Res-Net is used in various studies (Christopher et al., 2018b; Hemelings et al., 2020; Liao et al., 2019; Phan et al., 2019; Serte and Serener, 2019; Shibata et al., 2018) and showed significant results. The popularity of ResNet lies in the fact that it overcomes the “Vanishing Gradient” problem. Phan et al. (2019) used three deep convolution neural network ResNet152, DenseNet201 and VGG19, and the layers used are convolution, max pooling, activation layers and the full connected layer at the end. U-Net is also used in Juneja et al. (2019), Yu et al. (2019), Sevastopolsky (2017), Civit-Masot et al. (2020) and achieved the greater accuracy in the diagnosis of glaucoma. Juneja et al. (2019) used two neural networks (U-Net) for the segmentation of the disk and cup separately in conjunction. The model has 31 layers consisting of convolutional, max-pooling, up-sampling and merge layer. The filters are applied of size (3, 3), (1, 1) and the number of filters are 1, 32 and 64 on different layers. The filters in the deep learning models extract the features from the images. The MobileNet V2 (Civit-Masot et al., 2020; Thakur et al., 2020), GoogleNet (Serte and Serener, 2019), AlexNet (Al-Bander et al., 2017; Maheshwari et al., 2020) are used in the studies. Table 8.3 enlists all the CNN architectures applied by the researchers for the diagnosis of glaucoma in the recent years. Saxena et al. (2020) developed a six layers CNN model including four convolution layers and two fully connected layers. The image classification and the object detection are better performed by the CNN deep learning model. The following Figure 8.9 shows the most commonly used deep learning models in different studies.

FIGURE 8.9  The frequencies of different deep learning architectures

234

Artificial Intelligence and Internet of Things

In all studies, different types of images such as fundus and OCT are used. The structure of the eye can be clearly seen from the fundus image and abnormalities can be detected. It is less expensive source for the detection of the glaucoma. But the main drawback is that it depicts limited information characterizing glaucoma (Wang et al., 2020). The OCT are 3D images that are used for the accurate diagnosis of glaucoma but are more expensive than the fundus images. The automated glaucoma diagnosis from the OCT images is quite popular in clinical practice because the examination of OCT images is highly expertise and requires trained ophthalmologists. It is also always biased and more time-consuming process (Wang et al., 2020). Different datasets are publicly available and are being used for model training and testing that contains fundus and the OCT images. Some researchers collected the data from the public and the private hospitals and used them after processing. The images in the dataset are not enough for the training of the deep learning models (Sadad et al., 2018; Yousaf et al., 2019a,b). There is a dire need to build a large dataset for the training of the deep learning models for accurate prediction and detection of glaucoma. Different performance measures are used such as sensitivity, specificity, accuracy, AUC, ROC and DICE score. These measures showed the quality of work done by the researchers in the diagnosis of the disease. The best accuracy results are obtained by different authors (Maheshwari et al., 2020), (Abbas, 2017) and (Raghavendra et al., 2018). This chapter described the state-of-the-art machine learning and deep learning models of the recent years used for the classification of the glaucoma disease. All the studies showed that deep learning models performed the best for the glaucoma disease diagnosis. This chapter focuses on the milestones achieved by different researchers for the glaucoma diagnosis. The future work can be done on the development of the new methodologies for the early and accurate glaucoma diagnosis to save the time and need of the trained experts for screening of the large number of the affected people.

8.4 CONCLUSION The chapter has focused on many machine learning and deep learning studies for the early diagnosis of glaucoma where machine learning studies are discussed from the year 2010 to 2020 and deep learning are from the year 2014 to 2020. Many machine learning algorithms have been used that require effective feature space for the accurate diagnosis of glaucoma. Now the deep learning has significant superiority over traditional machine learning feature extraction methods due to automatic learning and accurate diagnosis of glaucoma. The deep learning models are the best substitute of machine learning models for the accurate diagnosis of glaucoma. A lot of progress in the glaucoma diagnosis using deep learning has been done. Still it is required to build very large datasets and new insights in deep learning architectures to diagnose the glaucoma in early stages with greater accuracy. As the glaucoma is an irreversible disease, early detection can help in reduction of chances of glaucoma occurrences. The deep learning based automatic glaucoma diagnosis systems will improve accuracy in early-stage diagnosis and assist the clinicians to overcome the shortage of experts for the screening of the masses.

AI Techniques for Glaucoma Detection

235

ACKNOWLEDGMENT This study is supported by Riphah Artificial Intelligence Research (RAIR) Lab, Riphah International University, Faisalabad Campus, Pakistan.

REFERENCES Abbas, Q. (2017). Glaucoma-deep: detection of glaucoma eye disease on retinal fundus images using deep learning. International Journal of Advanced Computer Science and Applications, 8, 41–45. Acharya, U. R., Dua, S., Du, X. & Chua, C. K. (2011). Automated diagnosis of glaucoma using texture and higher order spectra features. IEEE Transactions on Information Technology in Biomedicine, 15, 449–455. Acharya, U. R., Ng, E., Eugene, L. W. J., Noronha, K. P., Min, L. C., Nayak, K. P. & Bhandary, S. V. (2015). Decision support system for the glaucoma using Gabor transformation. Biomedical Signal Processing and Control, 15, 18–26. Acharya, U. R., Bhat, S., Koh, J. E., Bhandary, S. V. & Adeli, H. (2017). A novel algorithm to detect glaucoma risk using texton and local configuration pattern features extracted from fundus images. Computers in Biology and Medicine, 88, 72–83. Ahn, J. M., Kim, S., Ahn, K.-S., Cho, S. H., Lee, K. B. & Kim, U. S. (2018). A deep learning model for the detection of both advanced and early glaucoma using fundus photography. PLOS ONE, 13, e0207982. Akbar, S., Hassan, T., Akram, M. U., Yasin, U. U. & Basit, I. (2017a). AVRDB: annotated dataset for vessel segmentation and calculation of arteriovenous ratio. In Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV) (pp. 129–134). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), https://csce.ucmss.com/books/LFS/CSREA2017/IPC3337.pdf. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & ullah Yasin, U. (2017b). Decision support system for detection of papilledema through fundus retinal images. Journal of Medical Systems, 41(4), 66. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & Khan, S. A. (2018a). Decision support system for detection of hypertensive retinopathy using arteriovenous ratio. Artificial Intelligence in Medicine, 90, 15–24. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & ullah Yasin, U. (2018b). Arteriovenous ratio and papilledema based hybrid decision support system for detection and grading of hypertensive retinopathy. Computer Methods and Programs in Biomedicine, 154, 123–141. Akbar, S., Sharif, M., Akram, M. U., Saba, T., Mahmood, T. & Kolivand, M. (2019). Automated techniques for blood vessels segmentation through fundus retinal images: a review. Microscopy Research and Technique, 82(2), 153–170. Akram, M. U., Tariq, A., Khalid, S., Javed, M. Y., Abbas, S. & Yasin, U. U. (2015). Glaucoma detection using novel optic disc localization, hybrid feature set and classification techniques. Australasian Physical & Engineering Sciences in Medicine, 38, 643–655. Akram, M. U., Akbar, S., Hassan, T., Khawaja, S. G., Yasin, U. & Basit, I. (2020). Data on fundus images for vessels segmentation, detection of hypertensive retinopathy, diabetic retinopathy and papilledema. Data in Brief, 29, 105282. Al-Bander, B., Al-Nuaimy, W., Al-Taee, M. A. & Zheng, Y. (2017). Automated glaucoma diagnosis using deep learning approach. In 2017 14th International Multi-Conference on Systems, Signals & Devices (SSD) (pp. 207–210). IEEE. doi:https://doi.org/10.1109/ SSD.2017.8166974.

236

Artificial Intelligence and Internet of Things

Al-Bander, B., Williams, B. M., Al-Nuaimy, W., Al-Taee, M. A., Pratt, H. & Zheng, Y. (2018). Dense fully convolutional segmentation of the optic disc and cup in colour fundus for glaucoma diagnosis. Symmetry, 10, 87. Amin, J., Sharif, M., Rehman, A., Raza, M. & Mufti, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy Research and Technique, 81(9), 990–996. An, G., Omodaka, K., Hashimoto, K., Tsuda, S., Shiga, Y., Takada, N., Kikawa, T., Yokota, H., Akiba, M. & Nakazawa, T. (2019). Glaucoma diagnosis with machine learning based on optical coherence tomography and color fundus images. Journal of Healthcare Engineering, 2019, 1–9. Asaoka, R., Murata, H., Iwase, A. & Araie, M. (2016). Detecting preperimetric glaucoma with standard automated perimetry using a deep learning classifier. Ophthalmology, 123, 1974–1980. Bajwa, M. N., Singh, G. A. P., Neumeier, W., Malik, M. I., Dengel, A. & Ahmed, S. (2020). G1020: A Benchmark Retinal Fundus Image Dataset for Computer-Aided Glaucoma Detection. arXiv preprint arXiv:2006.09158. Bisneto, T. R. V., De Carvalho Filho, A. O. & Magalhães, D. M. V. (2020). Generative adversarial network and texture features applied to automatic glaucoma detection. Applied Soft Computing, 90, 106165. Chen, X., Xu, Y., Yan, S., Wong, D. W. K., Wong, T. Y. & Liu, J. (2015). Automatic feature learning for glaucoma detection based on deep learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 669–677). Springer. https://doi.org/10.1007/978-3-319-24574-4_80 Christopher, M., Belghith, A., Bowd, C., Proudfoot, J. A., Goldbaum, M. H., Weinreb, R. N., Girkin, C. A., Liebmann, J. M. & Zangwill, L. M. (2018a). Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Scientific Reports, 8, 1–13. Christopher, M., Belghith, A., Weinreb, R. N., Bowd, C., Goldbaum, M. H., Saunders, L. J., Medeiros, F. A. & Zangwill, L. M. (2018b). Retinal nerve fiber layer features identified by unsupervised machine learning on optical coherence tomography scans predict glaucoma progression. Investigative Ophthalmology & Visual Science, 59, 2748–2756. Civit-Masot, J., Domínguez-Morales, M. J., Vicente-Díaz, S. & Civit, A. (2020). Dual machine-learning system to aid glaucoma diagnosis using disc and cup feature extraction. IEEE Access, 8, 127519–127529. de Moura Lima, A. C., Maia, L. B., Dos Santos, P. T. C., Junior, G. B., De Almeida, J. D. & De Paiva, A. C. (2020). Evolving convolutional neural networks for glaucoma diagnosis. Brazilian Journal of Health Review, 3, 9224–9234. Dey, A. & Bandyopadhyay, S. K. (2016). Automated glaucoma detection using support vector machine classification method. Journal of Advances in Medicine and Medical Research, 11(12), 1–12. Fukushima, K. & Miyake, S. (1982). Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. Competition and Cooperation in Neural Nets. Springer, https://link.springer.com/chapter/10.1007/978-3-642-46466-9_18. Fumero, F., Alayón, S., Sanchez, J. L., Sigut, J. & Gonzalez-Hernandez, M. (2011). Rim-One: an open retinal image database for optic nerve evaluation. In 2011 24th International Symposium on Computer-Based Medical Systems (CBMS) (pp. 1–6). IEEE. doi:https:// doi.org/10.1109/CBMS.2011.5999143. Gayathri, R., Rao, P. & Aruna, S. (2014). Automated glaucoma detection system based on wavelet energy features and ANN. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 2808–2812). IEEE. doi:https://doi.org/10.1109/ICACCI.2014.6968654.

AI Techniques for Glaucoma Detection

237

Gómez-Valverde, J. J., Antón, A., Fatti, G., Liefers, B., Herranz, A., Santos, A., Sánchez, C. I. & Ledesma-Carbayo, M. J. (2019). Automatic glaucoma classification using color fundus images based on convolutional neural networks and transfer learning. Biomedical Optics Express, 10, 892–913. Gour, N. & Khanna, P. (2019). Automated glaucoma detection using GIST and pyramid histogram of oriented gradients (PHOG) descriptors. Pattern Recognition Letters, 137, 3–11. He, K., Zhang, X., Ren, S. & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778). Doi: 10.1109/CVPR.2016.90. Hemelings, R., Elen, B., Barbosa-Breda, J., Lemmens, S., Meire, M., Pourjavan, S., … & De Boever, P. (2020). Accurate prediction of glaucoma from colour fundus images with a convolutional neural network that relies on active and transfer learning. Acta Ophthalmologica, 98, e94–e100. Human Eye Anatomy. (2020). Available from: https://www.thoughtco.com/how-the-humaneye-works-4155646 [accessed 20 October 2020]. Iftikhar, S., Fatima, K., Rehman, A., Almazyad, A. S. & Saba, T. (2017). An evolution-based hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research 28(8), 3451–3455. Ioffe, S. & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv preprint arXiv:1502.03167. Jamal, A, Hazim Alkawaz, M, Rehman, A & Saba, T. (2017). Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17), 799–811. Jammal, A. A., Thompson, A. C., Mariottoni, E. B., Berchuck, S. I., Urata, C. N., Estrela, T., Wakil, S. M., Costa, V. P. & Medeiros, F. A. (2020). Human versus machine: comparing a deep learning algorithm to human gradings for detecting glaucoma on fundus photographs. American Journal of Ophthalmology, 211, 123–131. Juneja, M., Singh, S., Agarwal, N., Bali, S., Gupta, S., Thakur, N. & Jindal, P. (2019). Automated detection of glaucoma using deep learning convolution network (G-net). Multimedia Tools and Applications, April(1), 1–23. Kausu, T., Gopi, V. P., Wahid, K. A., Doma, W. & Niwas, S. I. (2018). Combination of clinical and multiresolution features for glaucoma detection and its classification using fundus images. Biocybernetics and Biomedical Engineering, 38, 329–341. Khalil, T., Akram, M. U., Khalid, S. & Jameel, A. (2017). Improved automated detection of glaucoma from fundus image using hybrid structural and textural features. IET Image Processing, 11, 693–700. Kim, S. J., Cho, K. J. & Oh, S. (2017). Development of machine learning models for diagnosis of glaucoma. PLOS ONE, 12, e0177726. Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25(2012), 1097–1105. Li, L., Xu, M., Wang, X., Jiang, L. & Liu, H. (2019). Attention based glaucoma detection: a large-scale database and CNN Model. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 10571–10580). doi:https://doi. org/10.1109/CVPR.2019.01082. Liao, W., Zou, B., Zhao, R., Chen, Y., He, Z. & Zhou, M. (2019). Clinical interpretable deep learning model for glaucoma diagnosis. IEEE Journal of Biomedical and Health Informatics, 24, 1405–1412. Maetschke, S., Antony, B., Ishikawa, H., Wollstein, G., Schuman, J. & Garnavi, R. (2019). A feature agnostic approach for glaucoma detection in OCT volumes. PLOS ONE, 14, e0219126.

238

Artificial Intelligence and Internet of Things

Maheshwari, S., Pachori, R. B., Kanhangad, V., Bhandary, S. V. & Acharya, U. R. (2017). Iterative variational mode decomposition based automated detection of glaucoma using fundus images. Computers in Biology and Medicine, 88, 142–149. Maheshwari, S., Kanhangad, V. & Pachori, R. B. (2020). CNN-Based Approach For Glaucoma Diagnosis Using Transfer Learning and LBP-Based Data Augmentation. arXiv preprint arXiv:2002.08013. Mashood Nasir, I., Attique Khan, M., Alhaisoni, M., Saba, T., Rehman, A. & Iqbal, T. (2020). A hybrid deep learning architecture for the classification of superhero fashion products: an application for medical-tech classification. Computer Modeling in Engineering & Sciences, 124(3), 1017–1033. Mittapalli, P. S. & Kande, G. B. (2016). Segmentation of optic disk and optic cup from digital fundus images for the assessment of glaucoma. Biomedical Signal Processing and Control, 24, 34–46. Mohamed, N. A., Zulkifley, M. A., Zaki, W. M. D. W. & Hussain, A. (2019). An automated glaucoma screening system using cup-to-disc ratio via simple linear iterative clustering superpixel approach. Biomedical Signal Processing and Control, 53, 101454. Mookiah, M. R. K., Acharya, U. R., Lim, C. M., Petznick, A. & Suri, J. S. (2012). Data mining technique for automated diagnosis of glaucoma using higher order spectra and wavelet energy features. Knowledge-Based Systems, 33, 73–82. Normal Retinal OCT Image. (2020). Available from: https://entokey.com/normal-retinalanatomy-and-basic-pathologic-appearances [accessed 20 October 2020]. Orlando, J. I., Fu, H., Breda, J. B., Van Keer, K., Bathula, D. R., Diaz-Pinto, A., Fang, R., Heng, P.-A., Kim, J. & Lee, J. (2020). Refuge challenge: a unified framework for evaluating automated methods for glaucoma assessment from fundus photographs. Medical Image Analysis, 59, 101570. Phan, S., Satoh, S. I., Yoda, Y., Kashiwagi, K., Oshika, T. & Group, J. O. I. R. R. (2019). Evaluation of deep convolutional neural networks for glaucoma detection. Japanese Journal of Ophthalmology, 63, 276–283. Qureshi, I., Khan, MA., Sharif, M., Saba, T. & Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images. International Journal of Intelligent Systems Technologies and Applications, 19(1), 1–16. 10.1504/IJISTA.2020.105172. Raghavendra, U., Fujita, H., Bhandary, S. V., Gudigar, A., Tan, J. H. & Acharya, U. R. (2018). Deep convolution neural network for accurate diagnosis of glaucoma using digital fundus images. Information Sciences, 441, 41–49. Raja, H., Akram, M. U., Khawaja, S. G., Arslan, M., Ramzan, A. & Nazir, N. (2020). Data on OCT and fundus images for the detection of glaucoma. Data in Brief, 29(April), 105342. Ramzan, A., Akram, M. U., Shaukat, A., Khawaja, S. G., Yasin, U. U. & Butt, W. H. (2018). Automated glaucoma detection using retinal layers segmentation and optic cup-to-disc ratio in optical coherence tomography images. IET Image Processing, 13, 409–420. Saba, T. (2020). Recent advancement in cancer detection using machine learning: systematic survey of decades, comparisons and challenges. Journal of Infection and Public Health, 13(9), 1274–1289. Saba, T., Rehman, A., Mehmood, Z., Kolivand, H. & Sharif, M. (2018). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), 704–715. doi:10.2174/1573405613666170912164546. Saba, T., Mohamed, A. S., El-Affendi, M., Amin, J. & Sharif, M. (2020a). Brain tumor detection using fusion of hand crafted and deep learning features Cognitive Systems Research, 59, 221–230. Saba, T., Haseeb, K., Ahmed, I. & Rehman, A. (2020b). Secure and energy-efficient framework using Internet of medical things for e-healthcare. Journal of Infection and Public Health, 13(10), 1567–1575.

AI Techniques for Glaucoma Detection

239

Sadad, T., Munir, A., Saba, T. & Hussain, A. (2018). Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. Journal of Computational Science, 29, 34–45. Sakthivel, K. & Narayanan, R. (2015). An automated detection of glaucoma using histogram features. International Journal of Ophthalmology, 8, 194. Salam, A. A., Akram, M. U., Wazir, K., Anwar, S. M. & Majid, M. (2015). Autonomous glaucoma detection from fundus image using cup to disc ratio and hybrid features. In 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) (pp. 370–374). IEEE. doi:https://doi.org/10.1109/ ISSPIT.2015.7394362. Saxena, A., Vyas, A., Parashar, L. & Singh, U. (2020). A glaucoma detection using convolutional neural network. In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC) (pp. 815–820). IEEE. doi:https://doi.org/10.1109/ ICESC48915.2020.9155930. Sengar, N., Dutta, M. K., Burget, R. & Ranjoha, M. (2017). Automated detection of suspected glaucoma in digital fundus images. In 2017 40th International Conference on Telecommunications and Signal Processing (TSP) (pp. 749–752). IEEE. doi:https://doi. org/10.1109/TSP.2017.8076088. Serte, S. & Serener, A. (2019). A generalized deep learning model for glaucoma detection. In 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) (pp. 1–5). IEEE. doi:https://doi.org/10.1109/ISMSIT.2019.8932753. Sevastopolsky, A. (2017). Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network. Pattern Recognition and Image Analysis, 27, 618–624. Sharma, R., Sircar, P., Pachori, R., Bhandary, S. V. & Acharya, U. R. (2019). Automated glaucoma detection using center slice of higher order statistics. Journal of Mechanics in Medicine and Biology, 19, 1940011. Shibata, N., Tanito, M., Mitsuhashi, K., Fujino, Y., Matsuura, M., Murata, H. & Asaoka, R. (2018). Development of a deep residual learning algorithm to screen for glaucoma from fundus photography. Scientific Reports, 8, 1–9. Simonthomas, S., Thulasi, N. & Asharaf, P. (2014). Automated diagnosis of glaucoma using Haralick texture features. In International Conference on Information Communication and Embedded Systems (ICICES2014) (pp. 1–6). IEEE. doi:https://doi.org/10.1109/ ICICES.2014.7033743. Simonyan, K. & Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv preprint arXiv:1409.1556. Singh, A., Dutta, M. K., Parthasarathi, M., Uher, V. & Burget, R. (2016). Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image. Computer Methods and Programs in Biomedicine, 124, 108–120. Sivaswamy, J., Krishnadas, S., Joshi, G. D., Jain, M. & Tabish, A. (2014). Drishti-GS: retinal image dataset for optic nerve head (ONH) segmentation. In 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI) (pp. 53–56). IEEE. doi:https:// doi.org/10.1109/ISBI.2014.6867807. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–9). IEEE. doi:https:// doi.org/10.1109/CVPR.2015.7298594. Thakoor, K. A., Li, X., Tsamis, E., Sajda, P. & Hood, D. C. (2019). Enhancing the accuracy of glaucoma detection from OCT probability maps using convolutional neural networks. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 2036–2040). IEEE. doi:https://doi.org/10.1109/ EMBC.2019.8856899.

240

Artificial Intelligence and Internet of Things

Thakur, A., Goldbaum, M. & Yousefi, S. (2020). Predicting Glaucoma before Onset Using Deep Learning. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z. & Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique, 82(4), 361–372. Wang, X., Chen, H., Ran, A.-R., Luo, L., Chan, P. P., Tham, C. C., Chang, R. T., Mannil, S. S., Cheung, C. Y. & Heng, P.-A. (2020). Towards multi-center glaucoma OCT image screening with semi-supervised joint structure and function multi-task learning. Medical Image Analysis, 63, 101695. World Health Organization. (October 8, 2019). World Report on Vision [Online]. Available from: https://www.google.com/url?sa=t&source=web&rct=j&url=https://apps.who.int/ iris/rest/bitstreams/1257940/retrieve&ved=2ahUKEwjGy4js3pPrAhWMEBQKHWU QCFMQFjAAegQIARAB&usg=AOvVaw3DSQZJ6aFidsEgDH4nsz8X. Yadav, D., Sarathi, M. P. & Dutta, M. K. (2014). Classification of glaucoma based on texture features using neural networks. In 2014 Seventh International Conference on Contemporary Computing (IC3) (pp. 109–112). IEEE. doi:https://doi.org/10.1109/ IC3.2014.6897157. Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A. M., Alharbey, R. & Rashid, M. (2019a). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: a survey. BioMed Research International, 2019, 1–26. Yousaf, K., Mehmood, Z., Awan, I. A., Saba, T., Alharbey, R., Qadah, T. & Alrige, M. A. (2019b). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, 23(2), 1–23. Yousefi, S., Goldbaum, M. H., Balasubramanian, M., Jung, T.-P., Weinreb, R. N., Medeiros, F. A., Zangwill, L. M., Liebmann, J. M., Girkin, C. A. & Bowd, C. (2013). Glaucoma progression detection using structural retinal nerve fiber layer measurements and functional visual field points. IEEE Transactions on Biomedical Engineering, 61, 1143–1154. Yu, S., Xiao, D., Frost, S. & Kanagasingam, Y. (2019). Robust optic disc and cup segmentation with deep learning for glaucoma detection. Computerized Medical Imaging and Graphics, 74, 61–71. Zhang, Z., Yin, F. S., Liu, J., Wong, W. K., Tan, N. M., Lee, B. H., Cheng, J. & Wong, T. Y. (2010). ORIGA-light: an online retinal fundus image database for glaucoma analysis and research. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology (pp. 3065–3068). IEEE. doi:https://doi.org/10.1109/ IEMBS.2010.5626137.

9

Artificial Intelligence in Brain Tumor Detection through MRI Scans Advancements and Challenges Sahar Gull and Shahzad Akbar

CONTENTS 9.1 Introduction�������������������������������������������������������������������������������������������������� 242 9.2 Literature Review����������������������������������������������������������������������������������������� 245 9.2.1 Datasets��������������������������������������������������������������������������������������������� 245 9.2.1.1 Brain Tumor Public Dataset�����������������������������������������������246 9.2.1.2 BRATS 2015 Dataset���������������������������������������������������������246 9.2.1.3 BRATS 2016 Dataset���������������������������������������������������������246 9.2.1.4 BRATS 2017 Dataset���������������������������������������������������������246 9.2.1.5 BRATS 2018 Dataset���������������������������������������������������������246 9.2.1.6 BRATS 2019 Dataset���������������������������������������������������������246 9.2.2 Performance Metrics������������������������������������������������������������������������246 9.2.2.1 Accuracy����������������������������������������������������������������������������246 9.2.2.2 Sensitivity/Recall��������������������������������������������������������������� 247 9.2.2.3 Precision����������������������������������������������������������������������������� 247 9.2.2.4 Specificity�������������������������������������������������������������������������� 247 9.2.2.5 F1-Score����������������������������������������������������������������������������� 247 9.2.2.6 Dice-Coefficient����������������������������������������������������������������� 247 9.2.3 Overview of Machine Learning�������������������������������������������������������248 9.2.3.1 Brain Tumor Detection Using Classical Machine Learning Algorithms���������������������������������������������������������248 9.2.4 Overview of Deep Learning������������������������������������������������������������� 254 9.2.4.1 Brain Tumor Detection Using Deep Learning Algorithms��������������������������������������������������������� 257 9.3 Discussion����������������������������������������������������������������������������������������������������� 267 9.4 Conclusion���������������������������������������������������������������������������������������������������� 268 Acknowledgment��������������������������������������������������������������������������������������������������� 268 References�������������������������������������������������������������������������������������������������������������� 268

241

242

Artificial Intelligence and Internet of Things

9.1 INTRODUCTION Diseases that are overcome with human intelligence and scientific progress have struggled over the last decades, but tumor remains a risk to humanity because of its unpredictable existence. One of the most rising illnesses is cancer of the brain. The brain is the human body’s central and most complex organ, containing nerve cells and muscles to monitor the body’s main tasks, such as breathing, muscle movement and our senses (Amin et al., 2018b). Each cell has its capabilities, some cells are increasing by their functionality and some are losing energy, resisting and becomes aberrant. This mass collection of irregular cells makes up the tissue as a tumor. Brain tumors have unnaturally developed brain cells (Ejaz et al., 2018a,b, 2019, 2020; Iqbal et al., 2017, 2018, 2019; Razzak et al., 2019). The brain tumors are categorized into various types/grades depending on their size, shape or location (Huang et al., 2014). The brain tumor is a dangerous and fatal cancer. In 2015 (Siegel et al., 2015), almost 23,000 patients were identified with a tumor in the United States. According to Siegel et al.’s study in 2017, the brain tumor is an irregular cell development within the brain or skull. An irregular development found within the brain is a primary brain tumor that typically does not spread to other areas of the body. A primary brain tumor is an irregular development found inside the brain that usually does not spread to other parts of the body. There are either malignant or benign primary brain tumors. A benign brain tumor produces slowly, has several boundaries and never spreads. Since the cells are not malignant when located in a critical area, and the malignant brain tumor is quickly growing, has irregular boundaries and is found in other regions of the brain, benign tumors may be life-threatening. Secondary brain tumors (metastatic) begin anywhere in the body and prevalence in the brain. A classification grading system was created by the World Health Organization (WHO) to standardize connectivity and forecast results for brain tumors. There exist 120 types of brain tumors. Some popular types of brain tumors are the meningioma, medulloblastoma, schwannoma (neuroma), pituitary adenoma, craniopharyngioma, epidermoid, lymphoma, pinealoma (pineocytoma, pineoblastoma), glioma, pilocytic astrocytoma (grade I), anaplastic ependymoma (grade III), oligodendroglioma (grade II), diffuse astrocytoma (grade II), ependymoma (grade II), anaplastic astrocytoma (grade III), and glioblastoma multiforme (grade IV) (Ronald Warnick and Gozal, 2018). The occurrence rates of glioma are 45%, meningioma 15% and pituitary tumor 15% among all the other brain tumors (Swati et al., 2019). The Feb 2018 study from the WHO has recently shown that the death rate is the largest on the Asian continent caused by central nervous system (CNS) cancer or brain tumors. The brain tumor must be diagnosed early to save more of these lives. Brain tumor grading is an important factor for targeted treatment (Hemanth et al., 2019). T1-weighted magnetic resonance image (MRI) of the brain (Magnetic Resonance – Technology Information Portal, 2020) is shown in Figure 9.1. Medical imaging strategies like complete tumor (CT), MRS, SPECT, PET and MRI are fully utilized to give significant data about the shape, size and the area that helps to diagnose the brain tumor (Işın et al., 2016). MRI is a medical imaging technique used in 2D and 3D formats for images of human body organs. MRI is the most precise method to predict and classify brain tumors with high-resolution

Artificial Intelligence in Brain Tumor Detection Through MRIs

243

FIGURE 9.1  MRI scanning of brain image

images (Kumar et al., 2017). Raymond V. Damadian invented the MRI in 1969 and was the first person to utilize MRI to examine the human body (Damadian et al., 1977). MRI is a serious medical imaging procedure used to deliver high-resolution pictures utilized when treating brain tumors. These high-resolution pictures are utilized to look at human mental health and find irregularities. Nowadays, there are a few techniques for grouping MRIs. Among all clinical images preparing, picture division is introductory and significant work; for instance, evaluation of the predetermined region must be founded on precise division. A tumor is a mass of tissue that becomes wild of the typical powers that manage development (Al-Ameen et al, 2015; Roy and Bandyopadhyay, 2012). MRI is a defensive medical imaging procedure that uses magnetic resonance (MR) signals to stimulate target nerves. MRI provides useful structural knowledge and enables the identification and segmentation of brain tumors also with their subdivisions. MRI is known to be the standard procedure for the detection of a brain tumor. Four typical MRI methods/ techniques used for glioma were determined to contain T1, T2 and fluid-attenuated inversion recovery (FLAIR). During the MRI processing, approximately 150 parts of 2D images that differ from system to system are created to reflect the intensity of the 3D brain. Besides that, the information ended up becoming extremely filled and puzzled as the splits from the critical standard techniques are combined for the analysis. Concluding that, T1 images are used to perceive strong tissues, but T2 images are used to diagram the area of edema that carries the optimistic signals on the image. The edge of the tumor in the complex cell area of the tumor tissue is shown in T1-Gd images without any impressive extension of the magnificent signal obtained by the differentiation specialist (gadolinium particles). Since necrotic cells do not operate for the identifying administrator, the hypo-exceptional portion of the tumor patient indicates that it is possible to simply segment them on a comparable technique from the complex cell area. The sign of water molecules is repressed in FLAIR images that show the edema zone cerebrospinal fluid (CSF) (Işın et al., 2016). The image patches (Menze et al., 2015) of the brain tumor are shown in Figure 9.2. The applications of artificial intelligence (AI) have inspired novel detections over different controls as well as medicine. Much legitimate verification has created in

244

Artificial Intelligence and Internet of Things

FIGURE 9.2  Image patches frameworks of the brain tumor subregions

the medical procedure, suggesting that produced structure can convey basic moves up to medical treatment, including radiology. The researchers and medical experts are essential to support the benefits of AI development in the organization of various diseases (Akbar et al., 2017, 2018a,b, 2019), especially neurological issues and brain tumors. AI enhanced the diagnosis and segmentation of brain tumors (Amin et al., 2019a,b,c,d). AI techniques like machine learning (ML) and deep learning (DL) methods have been used for the segmentation, grading and classification of brain tumors (Afza et al., 2019; Hussain et al., 2020; Khan et al., 2019a,b,c,d,e; Qureshi et al., 2020; Rehman et al., 2018). In radiology, these procedures take insight into the comprehension of the diagnosis, treatment and perception with the probability to take efficiency to the medical practice (Adeel et al., 2020; Fahad et al., 2018; Husham et al., 2016; Iftikhar et al., 2017; Jamal et al., 2017). The brain tumor segmentation has a group framework of whole tumor (WT), CT, enhancing tumor (ET) and tumor core (TC) used in DL. The detection of brain tumor stages (Roy and Bandyopadhyay, 2012) is shown in Figure 9.3. • • • • •

Image preprocessing Image segmentation Feature extraction Feature selection Classification algorithms

The main goal of preprocessing is to convert inaccurate and inaccurate real-world data into feasible data for evaluation to achieve better performance (Jamal et al., 2017; Javed et al., 2019a,b, 2020a,b; Khan et al., 2020a,b). The primary stage for MRIs is preprocessing. This stage is taken first to minimize the noise, to rebuild an image and then to detect the tumor in the image by certain texture features (Khan et al., 2017; Liaqat et al., 2020; Lung et al., 2014). Separating and examining an individual pixel in the image to specifically classify each pixel by its pixel values is referred to as segmentation. This move is done to remove the image features through

FIGURE 9.3  Phases of brain tumor detection

Artificial Intelligence in Brain Tumor Detection Through MRIs

245

the pixel values. Segmentation is the method of separating the images based on the same features in the segments (Khan et al., 2020c,d). In feature extraction, raw data selection is minimized for processing the more accessible classes. The selection features are designed to filtering redundant or irrelevant features from the dataset and, finally, the tumors are categorized by classification. These steps include various strategies that can be implemented (Majid et al., 2020; Mittal et al., 2020). Now it is essential to demonstrate the use of existing methods/techniques in this field by looking at the literature of the last few years.

9.2  LITERATURE REVIEW 9.2.1 Datasets Researchers employ standard datasets for experiments and result comparisons in the state of art (Rad et al., 2013, 2016). The dataset contains typically three brain tumor types such as the pituitary, glioma and meningioma with T1-contrast enhanced (CE) images of various patients (Yang et al., 2012). In 2012, The brain tumor segmentation (BRATS) challenge was first implemented, and every year since then, participants register online for the competition to have access to the dataset as one of the Medical Image Computing and Computer-Assisted Intervention (MICCAI) conferences. The efficiency of the research dataset and the submission of a small report detailing both tentative effects and segmentation processes are needed to determine the segmentation of the tumor. The organizer each year provides two different datasets. For the creation of the segmentation model, a training dataset was used. Test evidence is only valid for a certain period during which participants demonstrate their results (Ghaffari et al. 2019). For medical brain scanning, the BRATS datasets were developed. Data for clinical instruction contained MRIs of low-grade glioma (LGG) and high-grade glioma (HGG) patients with brain tumors. Brain images are a combination of preprocessing and post-processing images obtained over several years using multiple MRIs with different imaging protocols with two distinct field strengths. With each dataset, four modalities were scanned: T1, T1c, T2 and FLAIR. With the T1c MRI and isotropic resampling at 1-mm resolution, these methods have been rigidly co-registered. The brain images were brain stripped (Menze et al., 2015). The other brain diseases datasets are limited (Menze et al., 2015). For example, Alzheimer brain disease was identified (Luo et al., 2017; Petersen et al., 2010; Ramzan et al, 2020a,b) by taking brain tumor as region of interests (ROIs) in the dataset given by Cheng et al. (2015). With the relevant datasets of brain MRIs, all the approaches analyzed in this chapter are tested for their outcomes. Researchers also declare that for various datasets, the approaches can be validated. For training and research purposes, there are different datasets available (Tiwari et al., 2020), which are publicly available dataset (Cheng et al., 2015), local dataset, MICCAI Challenge on benchmark BRATS 2012 to BRATS 2019, BRATS 2015 (Menze et al., 2015), BRATS 2016 (Zhao et al., 2018), BRATS 2017 (CBICA, 2017), BRATS 2018 (Frangi et al., 2018), BRATS 2019 (CBICA, 2019), Figshare brain tumor dataset, digital imaging and communications in medicine (DICOM) dataset, MRI Brain Challenge dataset.

246

Artificial Intelligence and Internet of Things

9.2.1.1  Brain Tumor Public Dataset Two different hospitals have taken this dataset for the period from 2005 to 2010 in China. It consists of 3064 T1-weighted CE-MRI slices, which were obtained from 233 patients. In this dataset, the size of each slice is 512 × 512 by 6 and 1 mm, manually segmented each part of the selected tumor region by experienced radiologists. There are three kinds of brain tumors used in this dataset (Cheng et al., 2015). 9.2.1.2  BRATS 2015 Dataset The BRATS 2015 dataset is developed to classify and for the brain tumor classification. In several contributions, this dataset contains two tumor types, LGG and HGG. The dataset includes 274 MRIs with 220 and 54 images for glioma. Four modalities are used for MRI scanning and with a 240 × 240 × 155-image size (Menze et al., 2015). 9.2.1.3  BRATS 2016 Dataset The BRATS 2016 dataset is used for the classification of brain tumors. BRATS 2016 shares 220 images of HGG and 54 images of LGG. There are 191 cases with unknown grades in their testing dataset. In the BRATS 2016 dataset, half algorithms were based on convolutional neural network (CNN) and the other half based on the random-forest (RF) and support vector machine (SVM) (Zhao et al., 2018). 9.2.1.4  BRATS 2017 Dataset The BRATS 2017 dataset is used for the classification of brain tumors. For this dataset, the full initial TCIA glioma samples of 262 GBM and 199 LGG were radiologically tested by expert neuroradiologists, and each image was categorized as pre- and postoperative. The preoperatives were ultimately annotated and included in the dataset by specialists for the separate subregions of glioma (CBICA, 2017). 9.2.1.5  BRATS 2018 Dataset In BRATS 2018, image was categorized as pre and post-operative and employs the preoperative MRIs and emphases on the brain tumor segmentation, including gliomas. Besides, BRATS 2018 also focuses on predicting brain tumor patient survival (Frangi et al., 2018). 9.2.1.6  BRATS 2019 Dataset The BRATS 2019 dataset is used for the classification of brain tumors. This dataset is obtained from preoperative multimodal MRIs of HGG and LGG for brain tumor classification. The participants are allowed to use private data on BRATS 2019 dataset (CBICA, 2019).

9.2.2 Performance Metrics 9.2.2.1 Accuracy It is the ratio of predictions which are correct to overall predictions. Accuracy is essential when having symmetric datasets (false negative (FN) and false positive (FP)).

Artificial Intelligence in Brain Tumor Detection Through MRIs

247

Accuracy is used because there are equivalent costs for false negatives and false positives (Toshniwal, 2020).

Accuracy =

( TP + TN )

( TP + FP + FN + TN )

9.2.2.2 Sensitivity/Recall It is the ratio of true positives in the data to the whole (actual) positive ones. Sensitivity is essential when identifying the positives. It is used when it is unacceptable for false negatives to occur. Instead of saving some false negatives, you would rather have some extra false positives (Toshniwal, 2020).

Sensitivity or Recall =

TP ( TP + FN )

9.2.2.3 Precision It is the ratio of the true positive to the overall expected positive. Precision is essential because you choose to be more assured about the predicted positives and it is used when the presence of false positives is unacceptable, for example, spam emails (Toshniwal, 2020).

Precision =

TP TP ( + FP )

9.2.2.4 Specificity It is the ratio in the data between true negatives and total negatives. Specificity is important because you seek to preserve all of the true negatives. It is used when you do not want false alarms to come up (Toshniwal, 2020).

Specificity =

TN ( TN + FP )

9.2.2.5 F1-Score It considers both precision and recall. The harmonic means of accuracy and recall is F1-score. The classes are important once they have unequal distribution. F1-score is used due to the costs of false-positive and false-negative variances. The balance between accuracy and recall is taken by F1-score (Toshniwal, 2020).

F1-Score =

2 × ( Recall × Precision )

( Recall + Precision )

9.2.2.6 Dice-Coefficient It is the statistical measure of the degree of resemblance between two sets of samples (Munir et al., 2019).

248



Artificial Intelligence and Internet of Things

Dice cof . =

2 × TP 2 × TP + FP + FN

9.2.3 Overview of Machine Learning A subset of AI is ML. In 1959, Arthur Samuel gave the name ML in which machine improves from its mistakes or trains itself. The categories of ML (supervised, unsupervised and reinforcement learning) introduce the fundamental concept of the trade-off bias-variation for supervised ML as an essential framework (Mashood Nasir et al., 2020; Mughal et al., 2017, 2018a,b; Nazir et al., 2019). In supervised, ML specifies the features and labels and trains the machine by providing the features with class labels. The common supervised ML algorithms are linear regression, Naive Bayes, logistic regression, K-nearest neighbor (KNN), SVM, RF and CNN. Classification and regression are the two types of supervised learning that lead to discrete/qualitative and continuous/quantitative objectives (Perveen et al., 2020; Rahim et al., 2017a,b). In unsupervised, ML does not specify the features and labels and provides the raw data to the machine to decide which are features and class labels. The common unsupervised ML algorithms are clustering, anomaly detection. In reinforcement learning, it is required to give feedback to the machine to increase the accuracy to predict the output. The stages of brain tumor segmentation through ML algorithms are shown in Figure 9.4. Classification is one of the most widely used applications of ML algorithms. An ML algorithm is commonly referred to as a classifier in this utilization. The objects in an image are segmented through the use of threshold, edge-based classification and an active contour model, a segmentation technique. Although the segmented lesions are extracted with the use of feature extraction, the separate highlights are entered at that stage as a contribution to the ML model (Saba et al., 2019; Saba et al., 2020a,b). Through a multilayer perceptron (MLP) and SVM, class labels and group of features, the ML model is trained. The learned ML model, after testing, identifies an unknown lesion which belongs to a new class. So, it is possible to refer to the ML class with the feature input, ML-based on attributes and object-based ML classifier (Saba et al., 2018; Suzuki, 2017). Fuzzy logic is an ML technique, which has a lot of mathematical standards for information image dependent on grades of enrollment as different from the participation of traditional twofold validation. In brain tumor segmentation, the fuzzy technique permits the improvement of strategies that play out the related to human behaviors (Gordillo et al., 2010; Sadad et al., 2018; Yousaf et al., 2019a,b). 9.2.3.1 Brain Tumor Detection Using Classical Machine Learning Algorithms Pravin R. Kshirsagar et al. (2020) proposed an ML system in which the tumor was detected through MRI with the usage of tool reading algorithms. The purposed images were divided into three elements. The texture skills were taken out of the use of a grey diploma occurrence matrix. The texture abilities of the images regard at some stage included evolution, homogeneity, co-relation and electricity.

Artificial Intelligence in Brain Tumor Detection Through MRIs

249

FIGURE 9.4  Stages of brain tumor segmentation through machine learning algorithms

The multilayer perceptron has an accuracy of 98.6% and the Naive Bayes algorithm accuracy of 91.6% with 212 samples of resonance images. The accuracy can be multiplied through outsized statistics set and pull-out intensity based on functions to the texture-based skills. Manogaran et al. (2019) proposed a technique that was completely automated in classifying the brain tumor images built on the gamma distribution ML method that was used to detect the segments of brain tumors to perceive the automatic abnormality in ROI. The OGDMLA features were the self-identity of ROI with an improved image segmentation technique which was different from the other techniques. To analyze the performance of automatic brain tumor detection, the benchmark database of the medical image was collected and the mean error rate was determined with the use of mathematical formula. The results showed that the orthogonal gamma distribution technique attained an accuracy of 99.55% with an ML approach. Aaswad Sawant et al. (2018) focused on the use of TensorFlow to detect the brain tumor with the use of MRI. CNN with five layers was implemented in TensorFlow.

250

Artificial Intelligence and Internet of Things

This model was built on CPU-based TensorFlow and graphics processing unit (GPU) version of TensorFlow that was faster to train the data. In the dataset of a total of 1800 patients, MRI was used, and 900 were cancerous and 900 were non-cancerous. This system was used as a radiologist and decision by surgeons to detect brain tumors. The result showed that the accuracy to detect the brain tumors was 99% and validation accuracy was 98.6% in 35 epochs. Ozyurt et al. (2020) proposed a technique that was built on fuzzy C-means (FCM) along with CNN and ML algorithms (super resolution (SR)-FCM-CNN) to detect brain tumors. The aim of this chapter is to segment brain tumors with high performance using the SR-FCM technique from brain MRI for tumor detection. The squeeze architecture and features were taken out from a neural network (NN) method with some parameters. The proposed technique achieved the classification through extreme learning machine (ELM) of the features that were provided. The rate of identification of brain tumors segmentation with FCM and without SR was 10% greater. The result of the proposed technique showed that the accuracy rate was 98.33% on the DICOM format LR MRI with the use of SR-FCM. Himaja Byale et al. (2018) developed a technique that was based on algorithms to improve the performance of the brain tumor classification. The purpose of the research was to create an automated method that has a significant role in evaluating whether a lump (mass of tissue) in the brain through classification may be benign or malignant. This model included four steps: preprocessing used adapted median filter for noise removal, segmentation used GMM for finding the ROI, feature extraction used gray-level co-occurrence matrix (GLCM) for taking out the features of diverse kinds of tumors and used NN for the classification of the malignant tumor. The 60 samples of the MRIs were used from MS Ramaiah Memorial Hospital, Bangalore in DICOM format. The accuracy of the purposed method is better than the other ML algorithms like Adaboost which classifies the images into three types of brain tumors (normal, benign and malignant) with 89.90%. The result of the proposed method showed that the accuracy was 93.33%, sensitivity 93.33%, specificity 96.6% and precision was 94.44%. Citak-Er et al. (2018) developed an ML technique that was based on SVM with a linear kernel. The main goal was to obtain the multi-parametric (mp) MRI features in ML based on the classification of gliomas with a multi-ROI technique. The brain tumor MRI protocol included diffusion tensor, MR perfusion, T1- and T2-weight, diffusion-weight and MR spectroscopic images. The mp-MRI features were used to construct the ML methods for distinguished LGG from HGG. To determine the quantifiable performance of mp-MRI classification methods, the SVM-based recursive feature removal method was implemented. The result of the proposed method showed that the accuracy, specificity, sensitivity were 93.0%, 86.7% and 96.4%, respectively, for the classification of gliomas used randomly partitioned with tenfold cross-validation dataset built on the mp-MRI features. F. P. Polly et al. (2018) proposed a method that used the K-means algorithm for clustering while discrete wavelet transform (DWT) as a segmentation technique and for the feature extraction key slices, PCA. The SVM which was based on supervised learning has an important part of the proposed method that classified the brain tumor in LGG and HGG after the extraction of the features. The proposed method was

Artificial Intelligence in Brain Tumor Detection Through MRIs

251

classified in LGG and HGG with accuracy, sensitivity and specificity, and the model was more reliable for a large amount of data. The 60 samples of training data and 100 samples of the test dataset 30 for LGG and 30 for HGG training data and 50 for LGG and 50 for HGG test data. The result showed that the accuracy was 99%, specificity 98.02% and sensitivity was 100% of the proposed method. R. Pugalenthi et al. (2019) developed a method that was based on the ML approach to classify the 2D brain tumor through MRI. The ML approach applied three events. The preprocessing procedure improved the tumor segment which was built on the social group optimization (SGO) algorithm. The post-processing procedure applied the level-set segmentation (LSS) and the performance was validated along with segmentation events like ACS and CVS methods. The SVM-radial basis function (RBF) kernel was implemented for the brain tumor classification and the performance was validated by the RF and KNN classifiers. The result of the proposed method along with SVM-RBF showed that the accuracy was greater than 94% on the benchmark BRATS 2015 dataset. Amin et al. (2018a) proposed an unsupervised clustering technique for brain tumor segmentation which was based on ML and features fusion. The GWF, histogram of oriented gradients (HOG), LBP and SFTA features were the combination of the fusion feature vector. RF classifier was used for the classification of three (nonET, CT and ET). To overcome the overfitting problem, fivefold cross-validation techniques were used. The five benchmark datasets were used in the purposed method. The results showed the CT 0.91, non-ET 0.89 and ET 0.90 on the BRATS 2015 dataset. The purposed method’s accuracy, specificity and sensitivity were 92.0%, 98.4% and 98.5% on the BRATS 2015 dataset; 98.9%, 98.7% and 98.6% on the BRATS 2012 dataset; 91.0%, 98.7% and 98.5% on the BRATS 2013 dataset; 95.0%, 90.2% and 85.3% on the BRATS 2014 dataset; and 93.3%, 90.2% and 100% on the ISLES 2015 dataset. Li et al. (2018) proposed an ML method which classified the p53 status in LGG based on features taken out from MRI. The MRI has attained 272 patients through grade II and III gliomas and patients were arbitrarily assigned in training or validation dataset and 431 features were extracted from every patient. The least absolute shrinkage and selection operator (LASSO) algorithm was used for the extraction of the features. The ML method was decided to classify the p53 status in LGG with the use of selected features and SVM. The p53 radiology signature was constructed with the use of the LASSO algorithm and this method contained four statistics (firstorder) and ten textural features. The result showed the accuracy in the training set was under the curved area 89.6% and 76.3% in the validation to predict the p53 status in LGG. Leonardo Rundo et al. (2018) proposed an automated technique, the use of which was an unsupervised clustering approach for necrosis extraction (NeXt), after GTV segmentation. The ML FCM approach perceives and defines the necrotic areas as well as the brain tumors. The processing pipeline was associated with the two segmentation techniques which were useful for the support neuro-radiosurgery. The proposed method allows for the selective approach to increase the radiation dose in hypoxic radioresistant regions. This method examined the 3064 T1-CE MRIs and does not need the MRI data. The proposed method segmentation accuracy was

252

Artificial Intelligence and Internet of Things

estimated with the use of matrices and NeXt attained the best performance. The Dice similarity coefficient was of 95.93% ± 64.23%, and 0.225 ± 60.229 was the mean absolute distance in pixels. Noman Qasem et al. (2019) developed a technique using an MRI brain scan which was built on the texture, content and image shapes of the brain tumor. The proposed technique was contained preprocessing, computing foreground and background through the least computation cost applied watershed and provided features. The watershed overcomes the problem of distorted boundaries and wrong edges. The KNN classification is used for the detection of brain tumors with minimum falsepositive rate and best accuracy. The 1532 images were used for the three brain tumor types (glioma, pituitary tumors and meningioma) on the Figshare dataset. The proposed technique achieved the best performance used by watershed segmentation and the KNN algorithm. The result showed that the KNN algorithm accuracy was 86% on the Figshare dataset. Kavin Kumar et al. (2018) developed a method that predicted the classification, denoising and feature extraction of the brain tumor. For the denoising, an image used Pure-Let transforms. The modified multi-texton histogram (MMTH) and multitexton microstructure descriptor (MTMD) methods were the combinations of the feature extraction. The GLCM and GLRLM were used for the extraction of the feature. SVM, KNN and ELM were trained through the take-out features and used for the classification of brain tumors. Three classifiers of the performance matrices were used for the performance of the feature extraction. The 67 normal MRI and 67 MR brain tumor images were taken from the Brainweb database and the equal quantity of images collected from the Jansons MRI Scan Center Erode, Tamil Nadu, India dataset. The MMTH and MTMD methods showed that with KNN, the accuracy was 80%, sensitivity 90% and specificity was 70%; with ELM, the accuracy 91%, sensitivity 81% and specificity 100%; and with SVM, the highest accuracy was 95%, sensitivity 100% and specificity was 91% on the dataset. Usman and Rajpoot (2017) proposed a method that classified the brain tumor and segmentation for MRI. This chapter classified the tumor into three sections (ET, WT and CT) and the integrated features provided to RF classifier to predict the five class labels (background, ET, non-ET, necrosis and edema). In preprocessing, the MRI with different classifiers utilized the intensity, intensity difference and wavelet features extracted. The texture features wavelet-based along with RF classifier improved the performance of the detection of brain tumor. The dataset MICCAI used for the segmentation of brain tumors attained with high performance. The result showed the 75% TC, 88% dice overlap and 95% ET on the MICCAI BRATS 2013 dataset. Shanthakumar and Ganesh Kumar (2015) proposed a technique that classified the brain tumor and extracts the features, anisotropic filtering through MRI. The histogram was used in wavelets and a grey-level co-occurrence matrix that was used as a feature. The brain tumor was detected with morphological operations and with the use of the SVM classifier, and the extracted features were trained and classified. The watershed segmentation along with morphological operations was utilized brain image to detect brain tumors and SVM used for the brain tumor classification. The result of the proposed technique showed 0.94% accuracy, 0.95% sensitivity, 0.96% specificity, 0.78% positive predictive value (PPV) and 0.87% NPV.

Artificial Intelligence in Brain Tumor Detection Through MRIs

253

Naik and Patel (2014) proposed a method which classified the brain tumor through MRI based on the ML approach. The preprocessing procedure used the median filtering process and with the use of the texture feature extraction method, the features were extracted. The decision tree (DT) was used for the brain tumor classification and compared with the Naive Bayes algorithm used for MRIs dataset. The result showed that the DT algorithm has good accuracy than the Naive Bayes algorithm. The result showed 91% precision, 91% sensitivity, 83% specificity and 88.2% accuracy with the Naive Bayes algorithm, and 96% accuracy, 100% specificity, 100% precision and 93% sensitivity with the use of a DT classifier. Ahmed Kharrat et al. (2010) proposed a hybrid technique for the detection and classification of brain tumors using genetic algorithms (GA) and SVM in MRI. The proposed method was useful to classify the two types of brain tumors with improved sensitivity, specificity and accuracy values. The features were used to identify the brain tumor as normal, benign and malignant. The texture features were extracted through the used SGLDM. The features were provided to the classification model as input. The selection of features that were a major challenge in classification strategies was resolved with GA. The proposed technique achieved the best performance than the existing approaches. The result showed that the proposed hybrid technique accuracy range was 94.44%–98.14% on the 83 MRIs of the Harvard Medical School dataset. Ali Pashaei et al. (2018) proposed a kernel extreme (KE)-CNN method that was consisted of three brain tumor types in T1-CE-MRIs. ELM was an algorithm for learning that contained one or sometimes more layers of hidden nodes. These frameworks like that of classification and regression have been used in multiple areas. The CNN model was used to remove the hidden features of the images. The result showed that the KNN method accuracy was 91.28% on the T1-CE MRIs dataset and the proposed method accuracy on this same dataset was 93.68% which indicates KE-CNN has a precise accuracy compared to the existing techniques. Sharif et al. (2020a) proposed a triangular fuzzy median filtering technique that was implemented for image improvement, which helps to precisely segment brain tumors based on the unsupervised fuzzy set process. Similar texture (ST) characteristics were evaluated and provided to regression ELM leaves and ELM that were obtainable for brain tumor classification. Through individual patient lesions, the Gabor features were removed. The proposed methodology has been assessed on challenging datasets for BRATS 2012, 2013, 2014 and 2015, and also on the 2013 leader board. The proposed method provides improved performance with less processing time. The result showed that with the ELM method, the accuracy was 99%, dice coefficient (DSC) 99%, 100% PPV, 98% sensitivity, 2% FNR, 100% specificity, 0% FPR and 98% JSI on BRATS 2012; the accuracy was 99% on the BRATS 2013; the accuracy was 97.2% on 2013 leader board; the accuracy was 87.6% on BRATS 2014; and the accuracy was 96.5% on BRATS 2015 dataset. G. Hemanth et al. (2019) proposed an automatic segmentation method based on the CNN algorithm described by the small kernels. CNN ML algorithm from NN has a layer-based classification of results. CNN algorithms by classification and identification were a variety of stages associated with the proposed processes. ML methods are being successfully used at an early stage for the identification and prediction

254

Artificial Intelligence and Internet of Things

of brain tumors. There are various techniques for predicting brain tumors but the present study uses the CNN model to detect brain tumors. The CNN method provides effective brain tumor identification. The proposed algorithm is applied to various images, and the obtained output is optimal and effective. The result showed that accuracy was 91% and efficiency 92.7% on the UCI dataset. Brain tumor detection through classical ML algorithms is given next. The authors proposed the different methods/techniques for the detection of brain tumors with the use of different datasets and show the results of the proposed method in Table 9.1.

9.2.4 Overview of Deep Learning A subset of ML is DL. The algorithms are inspired by the structure and functions of the human brain in the DL (Mashood Nasir et al., 2020; Ullah et al., 2019). The DL helps to classify the tumor, segmentation, grading, successively mapping of brain tumor shape and brain tumor texture and estimate the survival of patients with brain tumors based on MRI datasets (Zhao et al., 2019). It provides the nature of the qualitative and quantitative of the brain tumor (Zhao et al., 2019). DL was proposed for ML algorithms of the illustration of objects. After 2012, the DL methodology depends on a CNN that increased a significant result within the best ImageNet pc vision competition, and DL became popular within the field of pc vision (Krizhevsky et al., 2012). DL consists of deep belief nets (DBNs) (Hinton et al., 2006). Several researchers have built many CNN architectures in the past, (LeNet-5, AlexNet, ZFNet, GoogleNet/Inception V1, VGGNet, ResNet) (Munir et al., 2019). • A seven-level CNN, which is called LeNet-5, was launched in 1998. The classification was the key feature of the network used for customer sorting of written information by banks. They used 32 × 32-pixel images for the classification of brain tumors. The high resolution included more convolutional layers for analyzing the multiple images. • In 2012, AlexNet, an architectural leader, minimized the upper five error rates from 26% to 15.3%. For an expanded number of per-layer inputs, this strategy is similar but wider to LeNet. AlexNet provides convolutional kernels and ReLU activations. Each convolutional fully linked layer is associated with ReLU. The network on the GPU-580 divides the network into two pipelines. • The ILSVRC 2013 was won by ZF-Net. The researchers minimized the upper five error rates to 14.8%. By maintaining the AlexNet structure, but modifying the hyperparameters, they have achieved it. • This was the ILSVRC 2014 winner with a 6.67% top five error rate, which is similar to the designers of the network, and was then required to execute human output at the human level. The upper five error rate of 5.1% was achieved by human experts and it is originally called LeNet-based CNN. CNN network system with 22 deep layers, but from 60 million to 4 million parameters, can be reduced. • VGGNet was second place in ILSVRC 2014. It is composed of a uniform architecture and 16 layers of convolution. There are 3 × 3 convolutions, but

Serial No. 1

Year 2020

2

Manogaran et al.

2019

3 4

Sawant et al. Ozyurt et al.

2018 2020

5

Byale et al.

2018

6

Citak-Er et al.

2018

7

Polly et al.

2018

8

Pugalenthi et al.

2019

9

Amin et al.

2018a

Li et al.

2018

10

Technique Naive Bayes algorithm, multilayer perceptron and tool reading algorithms Orthogonal gamma distribution with ML approach (OGDMLA) TensorFlow Fuzzy C-means (FCM), SR-FCM-CNN NN, GMM, GLCM

Dataset 212 samples of resonance images of the local dataset

Results Multilayer perception accuracy 98.6%, Naive Bayes accuracy 91.6%

Benchmark medical image dataset

Accuracy 99.55%

1800 MRIs of the local dataset DICOM format LR MRIs dataset 60 samples of the MRIs form MS Ramaiah Memorial Hospital, Bangalore Support vector machine Randomly partitioned into ten (SVM) with a linear kernel folds local dataset and RFE based on ML K-means algorithm, SVM, 60 samples of training data and DWT segmentation 100 samples of the test dataset technique SGO algorithm, SVM-RBF, Benchmark BRATS 2015 K-nearest neighbor (KNN), dataset Random-Forest (RF) RF classifier BRATS 2012, 2013, 2014, 2015 and ISLES2015 dataset LASSO algorithm and SVM

272 patients of grade II and III gliomas validation dataset

Accuracy 99% Accuracy 98.33% Accuracy 93.33%, sensitivity 93.33%, specificity 96.6% and precision 94.44% Accuracy 93.0%, the specificity 86.7% and the sensitivity of 96.4% Accuracy 99%, specificity 98.02% and sensitivity 100%

Accuracy greater than 94%

Accuracy, specificity and sensitivity were 92.0%, 98.4% and 98.5%, respectively, on the BRATS 2015 dataset and 93.3%, 90.2% and 100%, respectively, on the ISLES2015 dataset Accuracy under the curved area 89.6% and 76.3% in the validation (Continued)

255

Author Kshirsagar et al.

Artificial Intelligence in Brain Tumor Detection Through MRIs

TABLE 9.1 Brain Tumor Detection Using Classical Machine Learning Algorithms

Author Rundo et al.

Year 2018

12

Noman et al.

2019

13

Kavin Kumar et al.

2018

14

Usman et al.

2017

15

2015

16

Shanthakumar and Ganesh Kumar Naik and Patel

17

Technique FCM

Dataset T1-weighted contrast-enhanced 3064 MRI dataset KNN algorithm 1532 images of the Figshare dataset Extreme learning 67 normal MRI and 67 MR machine (ELM), SVM and brain tumor image from MMTH, MTMD methods Brainweb dataset and Jansons with KNN MRI Scan Center Erode, Tamil Nadu, India dataset RF classifier MICCAI BRATS 2013 dataset SVM MRIs of the public dataset

Results Dice similarity coefficient of 95.93% ± 64.23%, and 0.225 ± 60.229 mean absolute distance Accuracy 86%

2014

Decision tree (DT), Naive Bayes algorithm

MRIs of the local dataset

Kharrat et al.

2010

18

Pashaei et al.

2018

Genetic algorithms (GA) and SVM KE-CNN, KNN

Accuracy 96%, specificity 100%, precision 100% and sensitivity 93% with DT and precision 91%, sensitivity 91%, specificity 83% and accuracy 88.2% with Naive Bayes Accuracy range 94.44%–98.14%

19

Sharif et al.

20

Hemanth et al.

83 MRIs of Harvard Medical School dataset T1-weighted contrast-enhanced Accuracy 91.28% with KNN and the accuracy 93.68% with MRIs dataset KE-CNN Leader board 2013, BRATS Accuracy 0.99 on BRATS 2012, accuracy 0.99 on the BRATS 2012, 2013, 2014, 2015 2013, accuracy 97.2 on 2013 leader board, accuracy 87.6 on datasets BRATS 2014 and accuracy 96.5 on BRATS 2015 dataset UCI dataset Accuracy 91% and efficiency 92.7%

2020a,b Triangular fuzzy median filtering technique, ELM 2019

CNN machine learning algorithm

Accuracy, sensitivity and specificity with KNN 80%, 90% and 70%; with ELM 91%, 81% and 100%; and with SVM 95%, 100% and 91%, respectively

Core tumor 75%, dice overlap 88% and enhancing tumor 95% Accuracy 0.94%, sensitivity 0.95%, specificity 0.96%, PPV 0.78% and NPV 0.87%

Artificial Intelligence and Internet of Things

Serial No. 11

256

TABLE 9.1  (Continued) Brain Tumor Detection Using Classical Machine Learning Algorithms

Artificial Intelligence in Brain Tumor Detection Through MRIs

257

there are a lot of filters. It was trained on four GPUs for 3 or 4 weeks. Due to the architectural uniformity of its extraction of features from images, it is the most desirable network for the mission. This architecture was made public with weighted configurations and was used as the basis for many applications and issues as the characters from the feature extraction. This network is difficult to handle with its 138 million parameters. • The ResNet at ILSVRC 2015 standardization utilizes batch capabilities and skips connections. ResNet allows an NN with 152 layers to be trained and minimized the complexity compared to VGGNet. The upper five error rate was 3.57%, but it reached the dataset results at the human level (Munir et al., 2019). CNN automatically uses the values of the pixel in images rather than the features. Thus, it is not always essential to extract features or segment objects. Over the long period, the enhancement of segmentation techniques was studied, and it is still difficult to segment the objects, particularly for complex items and complicated context items. It is also a challenging task to identify and extract features, while the features do not have the deciding skill that is necessary to classify objects because DL may prevent errors due to the incorrect estimation of the features. The overall performance of DL for such objects is greater than the common classifiers (ML through function enter or object-based ML). Together with an unnecessary stage representation of objects or features in images, DL requires more than one layer of non-linear computation. DL leaves the steps of object segmentation, the extraction of features from segmented objects and the option of features to evaluate “successful features”. DL is also referred to as finishing ML because the total protocol for mapping raw input images to the final form is activated (Suzuki, 2017). The CNN architecture layers for the classification of brain tumors are shown in Figure 9.5. 9.2.4.1  Brain Tumor Detection Using Deep Learning Algorithms Sajid et al. (2019) developed a method that was CNN architecture used for the patchbased technique to take out the local and contextual information for the brain tumor detection and predicted performance categories. The proposed method contracts with the overfitting method that was used for batch normalization and the data unbalancing issue deal with two-phase CNN training. The two-phase training method has been used in the proposed method. For the bias field correction in the preprocessing

FIGURE 9.5  CNN architecture layers for the classification of brain tumors

258

Artificial Intelligence and Internet of Things

stage, N4ITK technique was used and during the post-processing stage, the morphological operations were used. The results showed that on the BRATS 2013 dataset, the Dice score was 0.86, specificity 0.91 and sensitivity 0.86. Abdelaziz Ismael et al. (2020) proposed a technique that was classified as the brain tumor with the use of the residual network. A DL-type ResNet50 was used for the design of the model. The data argumentation approaches (horizontal flips, zooming, vertical flips, shift, shear, zero-phase component analysis (ZCA) whitening and bright manipulation) were used to improve the accuracy and increase the dataset size. The benchmark dataset was contained 3064 T1-CE MRIs used for the brain tumor types. The performance metrics accuracy, precision, F1-score and recall were used for the performance of the proposed method. The outcome shows that the accuracy for the patient level was 97%, precision 98%, F1-score 97% and recall 97%. For the image level on the T1-CE MRI dataset, the accuracy was 99%, precision 99%, recall 99% and F1-score 99%. Rehman et al. (2019) developed a technique that contained the various CNN architectures (AlexNet, VGGNet and GoogleNet) that were developed to exchange information that classifies brain tumors. Transfer learning techniques (fine-tune and freeze) were conducted to strip out the patterns and visual bias features from MRIs. To classify the three brain tumor types (pituitary tumors, gliomas, meningiomas), the Figshare dataset was used and includes T1-CE 3064 MRI brain images. Improved accuracy, increased dataset size and reduced possibility of overfitting were applied to the different data argumentation techniques for the MRIs. The result showed that in the proposed method, the accuracy was up to 98.69% of the VGG16 architecture on the Figshare dataset. Yogananda et al. (2019) proposed a fully automated DL method for the segmentation of brain tumors. It implemented the structure of the three classes, providing many advantages. Three segments of brain tumor (CT, TC and ET) were composed of each group. This proposed method was used to break down complex problems of multi-class segmentation into individual binary segmentation problems in brain tumors. For the proposed technique, the BRATS 2019 dataset has 125 cases for segmentation and 29 cases for survival patient prediction. The proposed method provides 0.55 accuracy and the Dice scores of the CT, ET and TC were 0.901, 0.801 and 0.844, respectively, on the BRATS 2019 dataset. Mostefa Ben Naceur et al. (2020) developed a method that was fully automatic based on the CNN model for glioblastoma segmentation with a high- and low-grade brain tumor. Via the occipito-temporal pathway, the proposed model was stimulated. In three variants of CNNs, the suggested CNN model used selective focus strategies to improve the take-out of unique features from high- and low-grade MRIs. The result of the proposed method showed that the WT was 0.90, TC 0.83 and ET 0.83, and the median Dice score of the proposed model was associated with the Dice score of radiologists increasing from 74% to 85% on the BRATS 2018 dataset. Mittal et al. (2019) developed a technique that was built on a DL approach from MRIs for brain tumor segmentation. The combination of the G-CNN and stationary wavelet transform (SWT) provided the proposed approach for brain tumor segmentation. The G-CNN method was compared to SVM and CNN and used the BRAINIX

Artificial Intelligence in Brain Tumor Detection Through MRIs

259

medical images for the segmentation of the brain tumor. The result showed that the G-CNN for TP was 1, precision 0.9881, FP 10.012, FN 0.018, TN 0.984 and recall 0.9823 on the MRI dataset. The G-CNN method indicates that the mean square error (MSE) decreases that was 0.001, SSIM 0.986 and the 2% increase in PSNR 96.64 to achieve the proposed system. Thaha et al. (2019) proposed an enhanced CNN with the optimization of loss function through the BAT algorithm for automated segmentation of brain tumors and the main goal was to represent the accurate MRIs segmentation. The small kernels were used in deep architecture that provided fewer weights allocated to the system with a good influence against overfitting. For the preprocessing, skull stripping and enhanced image algorithms were used. The proposed algorithm results showed improved performance than the existing methods. The accuracy of E-CNN was 92%, recall 90% and precision 87% on the BRATS 2015 dataset for the segmentation of brain tumors. Naser and Deen (2020) developed a technique with the use of DL approaches that associate U-Net-based CNN and transition of learning models focused on pre-trained Vgg16 convolution to ensure streamlined automatic brain tumor segmentation. Automatic tumor segmentation, diagnosis and grading of brain tumor segmentation use an MRI pipeline. The proposed U-Net approach has shown that the mean DSC value is 0.84 and the accuracy of brain tumor detection is 0.92. Grading model classification LGG in grade II and grade III using FLAIR MRIs, the accuracy was 0.89, sensitivity 0.87 and specificity 0.92 at MRI level, and at patient level with accuracy 0.95, sensitivity 0.97 and specificity 0.98. Laukamp et al. (2019) used an mp DL model to investigate performance in automated segmentation and detection of meningiomas (a type of brain tumor). The detection of meningiomas contained MRI datasets used for the treatment of meningioma tumors. For the glioma cases, the DL model (DLM) used an independent dataset and for the brain tumor, image segmentation used the BRATS benchmark dataset. The outcome of the developed method was compared with manual segmentation and the result showed that the average Dice coefficients for total tumor volume were 0.81 ± 0.10 and the range was 0.46–0.93 and contrast-ET volume 0.78 ± 0.19 and range was 0.27–0.95 in T1-CE. Hu et al. (2019) developed a method that was built on the MCCNN to take out more distinctive features and linked with conditional random fields (CRFs). The three segmentation models were built and attained from dissimilar viewpoints using 2D patches to obtain ultimate segmentation results and the proposed method was assessed by the three public databases. The result was the CT 0.88, TC 0.81 and ET region 0.76 for DSC; the CT 0.86, TC 0.81 and ET 0.69 for PPV; and CT 0.90, TC 0.84 and ET 0.86 for sensitivity on the dataset BRATS 2013. The outcomes on the BRATS 2015 datasets were the CT 0.87, TC 0.76 and ET region 0.75 for DSC; CT 0.88, TC 0.83, ET 0.75 for PPV; and CT 0.87, TC 0.74, ET 0.80 for sensitivity, and the result of the purposed method showed the TC 0.7481, ET 0.7178 and WT 0.8824 for DSC; the TC 0.7621, ET 0.8684 and WT 0.9074 for sensitivity; the TC 0.9969, ET 0.9947 and WT 0.9918 for specificity; and the ET 5.6864, WT 12.6069 and TC 9.6223 for HD on BRATS 2018 dataset.

260

Artificial Intelligence and Internet of Things

Sultan et al. (2019) proposed a CNN built on DL which classified the types of brain tumors. The proposed method used two public datasets and brain tumors classification in three types (glioma, meningioma, pituitary tumor). The classification of glioma is performed in the three grades with the use of DL networks. From the input layer that contains preprocessed images through the convolution layers and activation functions, this method created 16 layers. To avoid the overfitting problem, two dropout layers were used. Data augmentation aids to show the best results and overcome the problems. The proposed method achieved the best performance. For the two studies, the accuracy was 96.13% and 98.7% on the 3064 T1-CE and other public datasets. Alqudah (2019) proposed a CNN method built on DL used to classify the data of brain tumors. The 3064 T1-CE brain MRI dataset used for the proposed method has three brain tumor types (glioma, meningioma, pituitary tumor). The three brain datasets uncropped, cropped and ROI are used for three classes of brain tumors for high performance. The result showed that the proposed method has the best performance with the accuracy of 98.93%, sensitivity of 98.18% for the cropped lesions, and the accuracy was 99%, and sensitivity 98.52% for the uncropped lesions and 97.62% accuracy and 97.40% sensitivity for segmented lesion images. Sajjad et al. (2019) proposed a CNN method that was based on multigrade brain tumor classification. The segmented data was further improved by the use of parameters to maximize the number of data samples and for multigrade classification of brain tumors pre-trained VGG-19 CNN technique used. For the radiologist to take a precise decision on the classification of brain tumors, the CNN-based CAD method was used. The data augmentation techniques and existing techniques were used to improve accuracy. The result showed that the accuracy was 94.58%, sensitivity 88.41% and specificity 96.12% on the MRI dataset. Saba (2020) developed a GrabCut technique for the segmentation of the brain tumor although the DL features and handcraft features were obtained from the transfer learning technique VGG-19. Entropy for classification enhanced the characteristics and the fused vector was given to classifiers. The suggested methodology was tested for brain tumor segmentation and classification on the BRATS dataset. To predict glioma with multiple classifiers, the classification of a brain tumor was achieved. The suggested methodology is educated and tested on numerous benchmarks. The developed methodology was trained and evaluated on various benchmark (BRATS 2015–2017) datasets. The outcome showed that the accuracy was 0.9878, and DSC 0.9636 on the BRATS 2015 dataset. The accuracy was 0.9936, DSC 0.9959 on BRATS 2016 dataset, and the accuracy was 0.9967, DSC 0.9980 on BRATS 2017 dataset. Abiwinanda et al. (2019) proposed a method by which CNN identified the three brain tumor types (glioma, pituitary, meningioma). The CNN technique without region-based preprocess includes two convolution layers, ReLu, the max pool from one invisible 64-neuron layer. The CNN approach continuously decreased the trend in the lack of validity, which improved the amount of the period, resulting in the highest accuracy of validations. The CNN was trained on the 3064 T1-CE MRI datasets that can be accessed publicly via Figshare brain tumor dataset 2017. The result of the proposed method showed that the training accuracy was 98.51% without

Artificial Intelligence in Brain Tumor Detection Through MRIs

261

region-based segmentation, the validation accuracy was 84.19% and the accuracy range was 71.39%–94.68% on the same dataset relative to the accuracies of the traditional protocol with region-based segmentation. Amin et al. (2020) proposed a fusion process method that was the combination of structural and texture information of four MRI classifiers. For the fusion process, the DWT with wavelet kernel provides information about the tumor region. The DWT with wavelet kernel was applied and PDDF was applied to remove the noise. For the segmentation of the brain tumor, which was distinguished based on the CNN model, the threshold approach was used. For the proposed approach, the five datasets were used and achieved the best results. The proposed method result showed that on fused images, the accuracy was 0.97 on BRATS 2012, accuracy 0.98 on BRATS 2013, accuracy 0.96 on BRATS 2013, 1.00 accuracy on BRATS 2015 and accuracy 0.97 on BRATS 2018 datasets. Sharif et al. (2020b) proposed a DL method that was based on the feature selection that predicted the brain tumor segmentation and identified brain tumors. The proposed method worked in two stages. In the first one, SbDL approach for segmented and confirmed brain tumor by DS rate and the second was that DL and DRLBP (dominant rotated local binary patterns) fusion functionality improved via the PSO algorithm. The configured functions were validated for the classification of a brain tumor that used a Softmax classifier. The contrast enhancement phase helps to make the image segmentation more organized and the integration of DRLBP and CNN features has been outlined but raises inaccuracy but boosts the classification time. Brain tumor segmentation SbDL was tested on the datasets BRATS 2017 and BRATS 2018. The result showed that the Dice score was 83.73% for TC, 93.7% for WT and 79.94% for ET on the BRAST2017 dataset, and the Dice score was 88.34% for TC, 91.2% for WT, and 81.84% for ET on the BRAST2018 dataset. In the classification of a brain tumors, the average accuracy was more than 92% on BRATS 2013, 2014, 2017 and 2018 datasets. Zhao et al. (2018) proposed a DL method based on the brain tumor segmentation within an integrated framework through fully convolutional neural networks (FCNNs) and CRFs. The DL model was trained in the three stages using image patches and sections of images. The image patches with each category were used in the first step to train FCNNs to prevent the issue of data unbalance problem. In the second stage, the proposed process, which was trained on three segmentation methods using 2D image patches, obtained axial, coronal and sagittal views with the use of a fusion-driven approach focused on voting. The multimodal segmentation of brain tumor methods has been established upon challenge BRATS 2013, BRATS 2015 and BRATS 2016 datasets. The fusion method result showed that the Dice score of CT, ET and TC was 0.88, 0.77, 0.84 with the axial method, the PPV was 0.92, 0.77 and 0.87 on BRATS 2013 dataset, respectively. Seetha and Raja (2018) developed an automatic brain tumor detection that was based on the CNN classification method. The DL-based design architecture was achieved by small kernels. The brain tumor classification was achieved by the use of FCM and other existing techniques. The CNN method contained the order of the feed-forward layer. The image net database was used for classification and raw pixel value was extracted from CNN. The gradient descent-based loss function

262

Artificial Intelligence and Internet of Things

was applied to attain high performance. The proposed method reduces the computation time and enhanced accuracy. The result of the developed method showed that the accuracy was 97.5% with low complexity on benchmark BRATS 2015 dataset. Bangalore Yogananda et al. (2020) proposed a method that was fully automated based on DL used for brain tumor segmentation. This method designed the 3D-Dense-U-Nets to simplify the complex problems of multi-class segmentations to break down the problems into binary segmentation. The proposed method was used to demonstrate the outcomes of the Dice score for the TC 0.84, WT 0.92 and ET 0.80. The WT, TC and ET Dice scores on the dataset of 20 test cases were 0.90, 0.84 and 0.80, respectively. The proposed methodology showed dice scores for WT 90%, TC 80% and ET 78% on the BRATS 2017 dataset, respectively. It also showed dice scores for WT 90%, TC 82% and ET 80% on the BRATS 2018 dataset, respectively, and showed the dice scores in segmentation of brain tumor for WT 85%, TC 80% and ET 77% on the clinical dataset, respectively. Arunkumar et al. (2020) proposed a method for brain tumor segmentation, and classification for MRI has used artificial neural network (ANN) to correctly classify where the ROI is located. This approach included three strategies that optimized images, segmented images and filtered out non-ROI depending on the features of texture and HOG. According to the histogram analysis, filtering through non-ROI has recommended staying away from the non-ROI and selecting the exact item. The 200 MRI cases were applied for the comparison between automatic and manual segmentation techniques. The tumor classification was accessed by applying the ROI texture features. The result showed that the precision was 92.14%, sensitivity 89% and specificity 94%. Zexun Zhou et al. (2020) proposed a model that was a fully automated CNN method for the segmentation of the brain tumor. It has two key problems; the first problem was the absence of spatial information and the second problem was the poor ability of multiscale process. To decrease the first problem, the 3D Atrous convolution was used. To address the second problem, 3D Atrous convolution in the multiscale background system pyramid is used to combine the backbone. The result was compared with the other datasets to study the improvement attained in the TC and ET. The outcomes of WT, TC, ET were 0.83, 0.68, 0.60 for AFPNet and 3D CRF on the BRATS 2013 leader board dataset and for WT, TC, ET were 0.82, 0.68 and 0.60 on the BRATS 2015 dataset, and the performances of AFPNet and 3D CRF for the lesion structure on the BRATS 2018 dataset for the WT, TC, ET were 0.8658, 0.7688 and 0.74434, respectively. Pereira et al. (2016) proposed an automatic segmentation method that was based on CNN. CNN was built on convolutional layers with small kernels that allowed the design of deep architecture which has a positive effect in contradiction of outfitting. In the preprocessing step, the segmentation of brain tumors consisted of bias field correction, intensity and patch normalization. Through rotating, the number of training patches was artificially expanded and HGG samples were used to increase the number of rare LGG classes. The result showed that for the brain tumor segmentation, the CT was 0.88, TC 0.83 and ET 0.77 in digital speckle correlation method

Artificial Intelligence in Brain Tumor Detection Through MRIs

263

(DSCM) on BRATS 2013 dataset and the DSCM with the same model for the CT 0.78, TC 0.65 and ET 0.75 on the BRATS 2015 dataset. Ben Naceur et al. (2020) proposed an automatic segmentation of the glioblastoma brain tumor method through the high and low grade that was based on the deep CNN model. To build the CNN model that aims to enhance the extraction from MRI of similar attributes, the selective attention technique was used. In the preprocessing phase, the quality of MRIs was enhanced that improved the segmentation results. To solve the problems of class imbalance, the spatial relationship used the sampling of image patches. It shows the impact of overlapping patches on the adjacent patches where the segmentation has been enhanced by the overlapping patches. The Dice score of the radiologist ranges from 74% to 85%. The result showed that Dice score for the WT 0.90, ET 0.83 and TC 0.83 on the BRATS 2018 dataset. Deepak and Ameer (2019) developed a method that was CNN-SVM based on a deep ML approach with the use of GoogLeNet to extract the features from MRIs. The characteristics were used to enhance the method’s efficiency. The three types of brain tumors were discussed in this chapter for the classification of brain tumors. For the patient level, the fivefold cross-validation approach was utilized on the MRI Figshare dataset. The proposed method accuracy was 98% and the performance measurements of recall for meningioma, glioma and pituitary were 96.0, 97.9, 98.9; and precision for meningioma, glioma, pituitary 94.7, 99.2, 98.0; and specificity for meningioma, glioma, pituitary 98.4, 99.4, 99.1 on the mean Figshare dataset, respectively. Siar and Teshnehlab (2019) developed a CNN method through MRIs used for the detection of brain tumors. This strategy is based on the integration with CNN of feature extraction techniques for the prediction of brain tumors. After the preprocessing level, the images were applied to CNN to evaluate the efficiency of CNN classifiers such as RBF and DT used by other classifiers in the classification model. The images were obtained at the clinician site; 1666 images were chosen for training data and 226 images were taken for testing. The results showed that the accuracy was 98.67% of the Softmax connected layer with the use of classifying images. For the RBF classifier, the accuracy of the CNN model was 97.34% and the accuracy of the DT classifier was 94.24% achieved. The proposed method accuracy on the test data was 99.12%. Talo et al. (2019) developed a technique that was classified the MRI with the AlexNet and ResNet-50. The pre-trained DL models are used in the normal, neoplastic, cerebrovascular, degenerative, inflammatory classes and movement. For the detection of the brain tumor, MRI techniques were used. This technique was fully automatic and did not take part in the extraction of features and classification steps. The data used for 1074 MRI were collected from the dataset of Harvard Medical School. With large MRIs of brain tumors, the proposed approach was tested and obtained the highest results. The result of the proposed technique showed that for the ResNet-50 and five pre-trained models, the accuracy was 95.23% ± 0.6. Brain tumor detection through DL algorithms is given next. The authors proposed the different methods/techniques for the detection of brain tumors through the use of different datasets and show the results of the proposed method in Table 9.2.

264

TABLE 9.2 Brain Tumor Detection Using Deep Learning Algorithms Serial No. 1

Year 2019

2

Abdelaziz et al.

2020

3

Rehman et al.

2019

4

Yogananda et al.

2019

5

Ben et al.

2020

6

Zhou et al.

2020

7

Mittal et al.

2019

8

Deepak and Ameer

2019

G-CNN, stationary wavelet transform (SWT) GoogLeNet

Thaha et al. Naser and Deen Laukamp et al.

2019 2020 2019

BAT algorithm, CNN U-Net-based CNN, Vgg16 DLM based FLAIR and T1-CE

9 10 11

Technique The patch-based technique, N4ITK ResNet50

Dataset BRATS 2013 dataset

T1-weighted contrast-enhanced 3064 MRI dataset Transfer learning techniques T1-weighted contrast-enhanced (fine-tune and freeze), VGG16 3064 MRI dataset 3D-Dense-U-Nets BRATS 2019 validation dataset Occipitotemporal pathway function (selective attention) AFPNet, 3D CRF

BRATS 2018 dataset BRATS 2018 dataset

Results The Dice score was 0.86, specificity 0.91 and sensitivity 0.86, respectively Accuracy was 0.99 for image level and 0.97 for patient level Accuracy was up to 98.69% Accuracy 0.55, Dice scores of WT 0.901, TC 0.844 and ET 0.801 Median Dice score for WT 0.90, TC 0.83 and ET 0.83

Lesion structure for ET 0.74434, WT 0.8658 and TC 0.7688 BRAINIX medical images, MRI MSC 0.001, precision 0.9881, FP 10.012, TN 0.984, FN public dataset 0.018 and recall 0.9823 MRI Figshare dataset Accuracy 98%. Meningioma, glioma and pituitary tumor; precision 94.7, 99.2, 98.0, recall 96.0, 97.9, 98.9; and specificity 98.4, 99.4, 99.1 BRATS 2015 dataset Accuracy 92%, recall 90% and precision 87% FLAIR MRI datasets DSC 0.84, and accuracy 0.92 BRATS (T1-CE, FLAIR) MRI Dice coefficients for contrast-ET 0.78 ± 0.19 and range dataset 0.27–0.95 and for total tumor 0.81 ± 0.10 and the range 0.46–0.93

Artificial Intelligence and Internet of Things

Author Sajid et al.

Serial No. 12

Author Hu et al.

Year 2019

Technique MCCANN, CRFs

Dataset BRATS 2018

13 14

Sultan et al. Alqudah

2019 2019

CNN based on deep learning CNN based on deep learning

3064 T1-CE images dataset 3064 T1-CE images dataset

15

Sajjad et al.

2019

16

Saba

2020

17

2019

18

Abiwinanda et al. Amin et al.

VGG-19, CNN-based CAD MRI dataset system Transfer learning method BRATS 2017 dataset VGG-19 ReLu, Max pool CNN techniquesFigshare dataset 2017

19

Sharif et al.

2020b

20

Zhao et al.

2018

21

Seetha and Raja

2018

22

Bangalore et al.

2020

2020

Threshold method, discrete BRATS 2018 dataset wavelet transform (DWT) DRLBP fusion, SbDL approach, BRATS 2018 dataset PSO algorithm CRFs and FCNNs based on DL BRATS 2013 dataset.

Results ET, WT, TC for DSC 0.7178, 0.8824, 0.7481, for sensitivity 0.8684, 0.9074, 0.7621, for specificity 0.9947, 0.9918, 0.9969, and for HD 5.6864, 12.6069, 9.6223, respectively Accuracy 98.7% Accuracy and sensitivity for cropped lesions 98.93%, 98.18%, for uncropped lesions 99%, 98.52% and for segmented lesion 97.62%, 97.40%, respectively Accuracy was 94.58%, sensitivity 88.41% and specificity 96.12% Accuracy 0.9967 and DSC 0.9980 Accuracy 98.51% without region-based segmentation, validation accuracy 84.19% Accuracy 0.97

265

Dice score 88.34% for TC, 91.2% for WT, 81.84% for ET CT 0.92, TC 0.87 and ET 0.77 with the axial method and CT was 0.88, TC 0.84, ET 0.77 with the fusion method FCM-based segmentation, DNN, Benchmark BRATS 2015 dataset Accuracy 97.5% CNN-based classification 3D-Dense-U-Nets, threefold BRATS 2018 and clinical dataset Dice score for WT, TC, ET was 0.85, 0.80, 0.77 on a cross-validation clinical dataset and 0.90, 0.82, 0.80 on the BRATS 2018 dataset (Continued)

Artificial Intelligence in Brain Tumor Detection Through MRIs

TABLE 9.2  (Continued) Brain Tumor Detection Using Deep Learning Algorithms

266

TABLE 9.2  (Continued) Brain Tumor Detection Using Deep Learning Algorithms Author Arunkumar et al.

Year 2020

24

Pereira et al.

2016

25 26

Naceur et al. Siar and Teshnehlab

2020 2019

27

Talo et al.

2019

Technique Dataset ANN used to classify where the 200 MRI of a public dataset ROI is located Direct simulation Monte Carlo BRATS 2013, 2015 dataset (DSMC)

Results Precision 92.14%, sensitivity 89% and specificity 94%

CT, TC and ET was 0.88, 0.83, 0.77 on BRATS 2013 dataset and 0.78, 0.65,0.75 on BRATS 2015 dataset, respectively LGG, HGG-based on CNN BRATS 2018 dataset Median Dice score for WT 0.90, CT 0.83 and ET 0.83 CNN with RBF and DT classifier 1666 images selected for training Accuracy 99.12% data and 226 images for test data of the public dataset ResNet-50, AlexNet 1074 MRIs data of Harvard Accuracy 95.23% ± 0.6 with the ResNet-50 model Medical School dataset.

Artificial Intelligence and Internet of Things

Serial No. 23

Artificial Intelligence in Brain Tumor Detection Through MRIs

267

9.3 DISCUSSION In this review chapter, the automated brain tumor classification and segmentation methods using ML and enhanced DL classifiers are discussed. The DL techniques changed the brain tumor detection system scenarios. The multiple classifications and segmentation strategies of brain tumors from MRI are studied in this work. The main goal of this review chapter is to help researchers to detect the different automated segmentation and classification techniques that are successful in using multimodal MRIs to detect brain tumors. The key contribution of this research is the comparative study of various brain tumor models/techniques. Multiple experiments were performed to classify brain MRI data using various feature extraction and the different data augmentation techniques were used to increase the dataset size and enhance horizontal and vertical flips, shear, manipulation of brightness, rotating, rotating, zooming and ZCA whitening. The different brain tumor classification methods (K-means clustering, FCM, KNN, SVM, DT, G-CNN, ANN, conditional random field-recurrent neural network (CRF-RNN), deep neural network (DNN), CNN, etc.) and the different datasets are publicly available dataset, local dataset, MICCAI Challenge on BRATS 2012 to BRATS 2019 datasets, Figshare brain tumor dataset, 3064 T1-CE images dataset, DICOM dataset and MRI Brain Challenge dataset, etc. used in this chapter. Several measurement metrics were used to access and measure the performance of the segmentation and classification models, such as accuracy, precision, recall, Dice coefficient, F1-score and specificity. The comparative analysis of reviewed methods for brain tumor detection shows that the CNN architecture of DL is mostly used than the other ML and DL algorithms in Figure 9.6.

FIGURE 9.6  Comparative analysis of the DL and ML algorithms for the detection of brain tumors

268

Artificial Intelligence and Internet of Things

9.4 CONCLUSION This chapter has presented the trending methods of fully automated brain tumor classification and segmentation from MRIs associated with ML and DL algorithms. The main goal of this research is to provide a review of the techniques/ methods used commonly for brain tumor classification. This study helps the researchers choose the best method/technique with the use of different datasets. The research involves the use of different MRI sequences to carry out various applications related to the detection of a brain tumor. To minimize the limitations in the image, different preprocessing methodologies are used. The analysis demonstrates that the various ML and DL techniques, including the FCM, SVM, DT, ANN, K-means clustering, CRF-RNN, DNN, KNN, ANN, DWT, Naive Bayes, global thresholding, G-CNN and DL techniques. CNN architectures (ResNet, LeNet-5, VGGNet, AlexNet, ZFNet, GoogleNet) illustrated their segmentation performance in multimodal MRIs of brain tumor patients. Some famous techniques are widely used to derive important information from methods in medical imaging. For brain tumor segmentation, image thresholding is chosen where the images have high contrast levels. The DL techniques are used to detect the brain tumor that is less time-consuming, cost-effective and more precise than the manual recognition method instead of manually recognizing tumor image processing. The DL-based techniques have more attention due to the ability to extract features. Several techniques are used for brain tumor detection on different datasets which are seen to achieve good performance. This chapter also contains the available datasets that were used by the researchers for the outcome validation purpose, and MICCAI challenges on benchmark BRATS 2012 to BRATS 2019 are mainly used datasets. More variations of various classifiers may be used to improve the performance of the models to produce more improved results. It can be seen that if the brain tumor is detected by quick and cost-effective diagnosis techniques, many lives can be saved.

ACKNOWLEDGMENT This study is supported by Riphah Artificial Intelligence Research (RAIR) Lab, Riphah International University, Faisalabad Campus, Pakistan.

REFERENCES Abdelaziz Ismael, S. A., Mohammed, A. & Hefny, H. (2020). An enhanced deep learning approach for brain cancer MRI images classification using residual networks. Artificial Intelligence in Medicine, 102, p. 101779. Abiwinanda, N., Hanif, M., Hesaputra, S. T., Handayani, A. & Mengko, T. R. (2019). Brain tumor classification using convolutional neural network. In World Congress on Medical Physics and Biomedical Engineering 2018. Adeel, A., Khan, M. A., Akram, T., Sharif, A., Yasmin, M., Saba, T. & Javed, K. (2020). Entropy-controlled deep features selection framework for grape leaf diseases recognition. Expert Systems. https://doi.org/10.1111/exsy.12569

Artificial Intelligence in Brain Tumor Detection Through MRIs

269

Afza, F., Khan, M. A., Sharif, M. & Rehman, A. (2019). Microscopic skin laceration segmentation and classification: a framework of statistical normal distribution and optimal feature selection. Microscopy Research and Technique, 82(9), pp. 1471–1488. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & Khan, S. A. (2018a). Decision support system for detection of hypertensive retinopathy using arteriovenous ratio. Artificial Intelligence in Medicine, 90, pp. 15–24. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & Ullah Yasin, U. (2017). Decision support system for detection of papilledema through fundus retinal images. Journal of Medical Systems, 41(4), p. 66. Akbar, S., Akram, M. U., Sharif, M., Tariq, A. & Ullah Yasin, U. (2018b). Arteriovenous ratio and papilledema based hybrid decision support system for detection and grading of hypertensive retinopathy. Computer Methods and Programs in Biomedicine, 154, pp. 123–141. Akbar, S., Sharif, M., Akram, M. U., Saba, T., Mahmood, T. & Kolivand, M. (2019). Automated techniques for blood vessels segmentation through fundus retinal images: a review. Microscopy Research and Technique, 82(2), pp. 153–170. Al-Ameen, Z., Sulong, G., Rehman, A., Al-Dhelaan, A., Saba, T. & Al-Rodhaan, M. (2015). An innovative technique for contrast enhancement of computed tomography images using normalized gamma-corrected contrast-limited adaptive histogram equalization. EURASIP Journal on Advances in Signal Processing, 2015(1), pp. 1–12. Alqudah, A. M. (2019). Brain tumor classification using deep learning technique – a comparison between cropped, uncropped, and segmented lesion images with different sizes. International Journal of Advanced Trends in Computer Science and Engineering, 8, pp. 3684–3691. Amin, J., Sharif, M., Gul, N., Yasmin, M. & Shad, S. A. (2020). Brain tumor classification based on DWT fusion of MRI sequences using a convolutional neural network. Pattern Recognition Letters, 129, pp. 115–122. Amin, J., Sharif, M., Raza, M. & Yasmin, M. (2018a). Detection of brain tumor based on features fusion and machine learning. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-018-1092-9 Amin, J., Sharif, M., Raza, M., Saba, T. & Anjum, M. A. (2019b). Brain tumor detection using statistical and machine learning methods. Computer Methods and Programs in Biomedicine, 177, pp. 69–79. Amin, J., Sharif, M., Raza, M., Saba, T. & Rehman, A. (2019a). Brain tumor classification: feature fusion. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–6). IEEE. doi: 10.1109/ICCISci.2019.8716449. Amin, J., Sharif, M., Raza, M., Saba, T., Sial, R. & Shad, S. A. (2019c). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, pp. 1–9. https://doi.org/10.1007/s00521-019-04650-7 Amin, J., Sharif, M., Rehman, A., Raza, M. & Mufti, M. R. (2018b). Diabetic retinopathy detection and classification using a hybrid feature set. Microscopy Research and Technique, 81(9), pp. 990–996. Amin, J., Sharif, M., Yasmin, M., Saba, T. & Raza, M. (2019d). Use of machine intelligence to conduct analysis of human brain data for detection of abnormalities in its cognitive functions. Multimedia Tools and Applications, 79(15), pp. 10955–10973. https://doi. org/10.1007/s11042-019-7324-y Arunkumar, N., Mohammed, M. A., Mostafa, S. A., Ibrahim, D. A., Rodrigues, J. J. and de Albuquerque, V. H. C. (2020). Fully automatic model-based segmentation and classification approach for MRI brain tumor using artificial neural networks. Concurrency and Computation: Practice and Experience, 32(1), pp. 1–9. Bangalore Yogananda, C. G., Shah, B. R., Vejdani-Jahromi, M., Nalawade, S. S., Murugesan, G. K., Yu, F. F., … & Maldjian, J. A. (2020). A fully automated deep learning network for brain tumor segmentation. Tomography, 6, pp. 186–193.

270

Artificial Intelligence and Internet of Things

Byale, H., Lingaraju, G. M. & Sivasubramanian, S. (2018). Automatic segmentation and classification of brain tumor using machine learning techniques. International Journal of Applied Engineering Research, 13, pp. 11686–11692. Center for Biomedical Image Computing and Analytics (CBICA). (2017). Multimodal Brain Tumor Segmentation Challenge 2019: Data [Online]. Available: https://www.cbica. upenn.edu/sbia/Spyridon.Bakas/MICCAI_BraTS/MICCAI_BraTS_2017_proceedings_shortPapers.pdf [Accessed 20/10/2020]. Center for Biomedical Image Computing and Analytics (CBICA). (2019). Multimodal Brain Tumor Segmentation Challenge 2019: Data [Online]. Available: https://www.med. upenn.edu/cbica/brats2019/data.html [Accessed 20/10/2020]. Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang, Z. & Feng, Q. (2015). Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLOS ONE, 10, p. e0140381. Citak-Er, F., Firat, Z., Kovanlikaya, I., Ture, U. & Ozturk-Isik, E. (2018). Machine-learning in grading of gliomas based on multi-parametric magnetic resonance imaging at 3T. Computers in Biology and Medicine, 99, pp. 154–160. Damadian, R., Goldsmith, M. & Minkoff, L. (1977). NMR in cancer: XVI. FONAR image of the live human body. Physiological Chemistry and Physics, 9, pp. 97–100, 108. Deepak, S. & Ameer, P. M. (2019). Brain tumor classification using deep CNN features via transfer learning. Computers in Biology and Medicine, 111, p. 103345. Ejaz, K, Rahim, M. S. M., Bajwa, U. I., Chaudhry, H., Rehman, A. & Ejaz, F. (2020). Hybrid segmentation method with confidence region detection for Tumor identification. IEEE Access. doi:10.1109/ACCESS.2020.3016627. Ejaz, K., Rahim, M. S. M., Bajwa, U. I., Rana, N. & Rehman, A. (2019). An unsupervised learning with feature approach for brain tumor segmentation using magnetic resonance imaging. In Proceedings of the 2019 9th International Conference on Bioscience, Biochemistry, and Bioinformatics (pp. 1–7). https://doi.org/10.1145/3314367.3314384 Ejaz, K., Rahim, M. S. M., Rehman, A., Chaudhry, H., Saba, T. & Ejaz, A. (2018a). Segmentation method for a pathological brain tumor and accurate detection using MRI. International Journal of Advanced Computer Science and Applications, 9(8), pp. 394–401. Ejaz, K., Rahim, M. S. M., Rehman, D. A. & Ejaz, E. F. (2018b). An image-based multimedia database and efficient detection through features. FAST Transactions on Software Engineering, 14(1), pp. 6–15. Fahad, H. M., Khan, M. U. G, Saba, T., Rehman, A. & Iqbal, S. (2018). Microscopic abnormality classification of cardiac murmurs using ANFIS and HMM. Microscopy Research and Technique, 81(5), pp. 449–457. doi:10.1002/jemt.22998. Frangi, A. F., Schnabel, J. A., Davatzikos, C., Alberola-López, C. and Fichtinger, G. eds. (2018). Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part IV (Vol. 11073). Springer. Ghaffari, M., Sowmya, A. & Oliver, R. (2019). Automated brain tumor segmentation using multimodal brain scans: a survey based on models submitted to the BraTS 2012–2018 challenges. IEEE Reviews in Biomedical Engineering. doi: 10.1109/RBME.2019.2946868. Gordillo, N., Montseny, E. & Sobrevilla, P. (2010). A new fuzzy approach to brain tumor segmentation. In International Conference on Fuzzy Systems. doi: 10.1109/ FUZZY.2010.5584178. Hemanth, G., Janardhan, M. & Sujihelen, L. (2019). Design and implementing brain tumor detection using machine learning approach. In Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019). doi:10.1109/ ICOEI.2019.8862553. Hinton, G. E., Osindero, S. & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, pp. 1527–1554.

Artificial Intelligence in Brain Tumor Detection Through MRIs

271

Hu, K., Gan, Q., Zhang, Y., Deng, S., Xiao, F., Huang, W., Cao, C. & Gao, X. (2019). Brain tumor segmentation using multi-cascaded convolutional neural networks and conditional random field. IEEE Access, 7, pp. 92615–92629. Huang, M., Yang, W., Wu, Y., Jiang, J., Gao, Y., Chen, Y., Feng, Q., Chen, W. & Lu, Z. (2014). Content-based image retrieval using spatial layout information in brain tumor T1-weighted contrast-enhanced MR images. PLOS ONE, 9, p. e102754. Husham, A., Alkawaz, M. H., Saba, T., Rehman, A. & Alghamdi, J. S. (2016). Automated nuclei segmentation of malignant using level sets. Microscopy Research and Technique, 79(10), pp. 993–997. doi:10.1002/jemt.22733. Hussain, N., Khan, M. A., Sharif, M., Khan, S. A., Albesher, A. A., Saba, T. & Armaghan, A. (2020). A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications. https:// doi. org/10.1007/s11042-020-08852-3 Iftikhar, S., Fatima, K., Rehman, A., Almazyad, A. S. & Saba, T. (2017). An evolution-based hybrid approach for heart diseases classification and associated risk factors identification. Biomedical Research, 28(8), pp. 3451–3455. Iqbal, S., Ghani, M. U., Saba, T. & Rehman, A. (2018). Brain tumor segmentation in multispectral MRI using convolutional neural networks (CNN). Microscopy Research and Technique, 81(4), pp. 419–427. doi:10.1002/jemt.22994. Iqbal, S., Khan, M. U. G., Saba, T. & Rehman, A. (2017). Computer-assisted brain tumor type discrimination using magnetic resonance imaging features. Biomedical Engineering Letters, 8(1), pp. 5–28. doi:10.1007/s13534-017-0050-3. Iqbal, S., Khan, M. U. G., Saba, T., Mehmood, Z., Javaid, N., Rehman, A. & Abbasi, R. (2019). Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation. Microscopy Research, and Technique. https://doi.org/10.1002/ jemt.23281. Işın, A., Direkoğlu, C. & Şah, M. (2016). Review of MRI-based brain tumor image segmentation using deep learning methods. Procedia Computer Science, 102, pp. 317–324. Jamal, A, Hazim Alkawaz, M., Rehman, A. & Saba, T. (2017). Retinal imaging analysis based on vessel detection. Microscopy Research and Technique, 80(17), pp. 799–811. https://doi.org/10.1002/jemt. Javed, R., Rahim, M. S. M & Saba, T. (2019b). An Improved framework by mapping salient features for skin lesion detection and classification using the optimized hybrid features. International Journal of Advanced Trends in Computer Science and Engineering, 8(1), pp. 95–101. Javed, R., Rahim, M. S. M., Saba, T. & Rashid, M. (2019a). Region-based active contour JSEG fusion technique for skin lesion segmentation from dermoscopic images. Biomedical Research, 30(6), pp. 1–10. Javed, R., Rahim, M. S. M., Saba, T. & Rehman, A. (2020a). A comparative study of features selection for skin lesion detection from dermoscopic images. Network Modeling Analysis in Health Informatics and Bioinformatics, 9(1), p. 4. Javed, R., Saba, T., Shafry, M. & Rahim, M. (2020b). An intelligent saliency segmentation technique and classification of low contrast skin lesion dermoscopic images based on histogram decision. In 2019 12th International Conference on Developments in eSystems Engineering (DeSE) (pp. 164–169). doi: 10.1109/DeSE.2019.00039. Kavin Kumar, K., Devi, M. and Maheswaran, S., 2018. An efficient method for brain tumor detection using texture features and SVM classifier in MR images. Asian Pacific Journal of Cancer Prevention: APJCP, 19(10), p. 2789. Khan, M. A., Akram, T., Sharif, M., Javed, K., Raza, M. & Saba, T. (2020a). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, pp. 1–30. https://doi.org/10.1007/s11042-020-08726-8

272

Artificial Intelligence and Internet of Things

Khan, M. A., Akram, T., Sharif, M., Saba, T., Javed, K., Lali, I. U., Tanik, U. J. & Rehman, A. (2019d). Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion. Microscopy Research and Technique, 82(5), pp. 741–763. http://doi.org/10.1002/jemt.23220 Khan, M. A., Ashraf, I., Alhaisoni, M., Damaševičius, R., Scherer, R., Rehman, A. & Bukhari, S. A. C. (2020d). Multimodal brain tumor classification using deep learning and robust feature selection: a machine learning application for radiologists. Diagnostics, 10, p. 565. Khan, M. A., Javed, M. Y., Sharif, M., Saba, T. & Rehman, A. (2019a). Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 International Conference on Computer and Information Sciences (ICCIS) (pp. 1–7). IEEE. doi:10.1109/ICCISci.2019.8716400. Khan, M. A., Lali, I. U., Rehman, A., Ishaq, M., Sharif, M., Saba, T., Zahoor, S. & Akram, T. (2019c). Brain tumor detection and classification: a framework of marker-based watershed algorithm and multilevel priority features selection. Microscopy Research and Technique, 82(6), pp. 909–922. https://doi.org/10.1002/jemt.23238 Khan, M. A., Sharif, M. I., Raza, M., Anjum, A., Saba, T. & Shad, S. A. (2019b). Skin lesion segmentation and classification: a unified framework of deep neural network features fusion and selection. Expert Systems, p. e12497. https://doi.org/10.1111/exsy.12497 Khan, M. A., Sharif, M., Akram, T., Raza, M., Saba, T. & Rehman, A. (2020b). Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Applied Soft Computing, 87, p. 105986. Khan, M. W., Sharif, M., Yasmin, M. & Saba, T. (2017). CDR based glaucoma detection using fundus images: a review. International Journal of Applied Pattern Recognition, 4(3), pp. 261–306. Khan, M. Z., Jabeen, S., Khan, M. U. G., Saba, T., Rehmat, A., Rehman, A. & Tariq, U. (2020c). A realistic image generation of face from text description using the fully trained generative adversarial networks. IEEE Access. doi:10.1109/ACCESS.2020.3015656. Khan, S. A., Nazir, M., Khan, M. A., Saba, T., Javed, K., Rehman, A., … & Awais, M. (2019e). Lungs nodule detection framework from computed tomography images using a support vector machine. Microscopy Research and Technique, 82(8), pp. 1256–1266. Kharrat, A., Gasmi, K., Messaoud, M. B., Benamrane, N. & Abid, M. (2010). A hybrid approach for automatic classification of brain MRI using genetic algorithm and support vector machine. Leonardo Journal of Sciences, pp. 71–82. http://ljs.academicdirect. org/A17/071_082.pdf Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Paper presented at Advances in Neural Information Processing Systems. https://doi.org/10.1145/3065386 Kshirsagar, P. R., Rakhonde, A. N. & Chippalkatti, P. (2020). MRI image-based brain tumor detection using machine learning. Test Engineering & Management, 81, pp. 4431–4434. Kumar, S., Dabas, C & Godara, S. (2017). Classification of brain MRI tumor images: a hybrid approach. Procedia Computer Science, 122, pp. 510–517. Laukamp, K. R., Thiele, F., Shakirin, G., Zopfs, D., Faymonville, A., Timmer, M., Maintz, D., Perkuhn, M. & Borggrefe, J. (2019). Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI. European Radiology, 29, pp. 124–132. Li, Y., Qian, Z., Xu, K., Wang, K., Fan, X., Li, S., Jiang, T., Liu, X. & Wang, Y. (2018). MRI features predict p53 status in lower-grade gliomas via a machine-learning approach. NeuroImage: Clinical, 17, pp. 306–311. Liaqat, A., Khan, M. A., Sharif, M., Mittal, M., Saba, T., Manic, K. S. & Al Attar, F. N. H. (2020). Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Current Medical Imaging, 16, pp.1229–1242.

Artificial Intelligence in Brain Tumor Detection Through MRIs

273

Lung, J. W. J., Salam, M. S. H, Rehman, A., Rahim, M. S. M. & Saba, T. (2014). Fuzzy phoneme classification using multi-speaker vocal tract length normalization. IETE Technical Review, 31(2), pp. 128–136. doi:10.1080/02564602.2014.892669. Luo, S., Li, X. & Li, J. (2017). Automatic Alzheimer’s disease recognition from MRI data using deep learning method. Journal of Applied Mathematics and Physics, 05, pp. 1892–1898. Magnetic Resonance – Technology Information Portal (2020). MRI Images [Online]. Available: https://www.mr-tip.com/serv1.php?type=img&img=Brain%20MRI%20 Images%20T1 [Accessed 29/09/2020]. Majid, A., Khan, M. A., Yasmin, M., Rehman, A., Yousafzai, A. & Tariq, U. (2020). Classification of stomach infections: a paradigm of the convolutional neural network along with classical features fusion and selection. Microscopy Research and Technique, 83(5), pp. 562–576. Manogaran, G., Shakeel, P. M., Hassanein, A. S., Malarvizhi Kumar, P. & Chandra Babu, G. (2019). Machine learning approach-based gamma distribution for brain tumor detection and data sample imbalance analysis. IEEE Access, 7, pp. 12–19. Mashood Nasir, I., Attique Khan, M., Alhaisoni, M., Saba, T., Rehman, A. & Iqbal, T. (2020). A hybrid deep learning architecture for the classification of superhero fashion products: an application for medical-tech classification. Computer Modeling in Engineering & Sciences, 124(3), pp. 1017–1033. Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., … & Van Leemput, K. (2015a). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34, pp. 1993–2024. Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A. & Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), p. 1068. Mittal, M., Goyal, L. M., Kaur, S., Kaur, I., Verma, A. & Jude Hemanth, D. (2019). Deep learning-based enhanced tumor segmentation approach for MR brain images. Applied Soft Computing, 78, pp. 346–354. Mughal, B., Muhammad, N., Sharif, M., Rehman, A. & Saba, T. (2018a). Removal of pectoral muscle based on a topographic map and shape-shifting silhouette. BMC Cancer, 18(1), pp. 1–14. Mughal, B., Muhammad, N., Sharif, M., Saba, T., Rehman, A. (2017). Extraction of breast border and removal of pectoral muscle in the wavelet domain, Biomedical Research, 28(11), pp. 5041–5043. Mughal, B., Sharif, M., Muhammad, N. & Saba, T. (2018b). A novel classification scheme to decline the mortality rate among women due to breast tumors. Microscopy Research and Technique, 81(2), pp. 171–180. Munir, K., Elahi, H., Ayub, A., Frezza, F. & Rizzi, A. (2019). Cancer diagnosis using deep learning: a bibliographic review. Cancers (Basel), 11(9), pp. 1–36. Naceur, M. B., Akil, M., Saouli, R. & Kachouria, R. (2020). Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy. Journal of Medical Image Analysis. https://doi. org/10.1016/j.media.2020.101692 Naik, J. & Patel, S. (2014). Tumor detection and classification using decision tree in brain MRI. International Journal of Computer Science and Network Security (IJCSNS), 14(6), pp. 49–53. Naser, M. A. & Deen, M. J. (2020). Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Computers in Biology and Medicine, 121, p. 103758. Nazir, M., Khan, M. A., Saba, T. & Rehman, A. (2019). Brain tumor detection from MRI images using multi-level wavelets. In 2019, IEEE International Conference on Computer and Information Sciences (ICCIS) (pp. 1–5). doi: 10.1109/ICCISci.2019.8716413.

274

Artificial Intelligence and Internet of Things

Noman Qasem, S., Nazar, A., Qamar, A., Shamshirband, S. & Karim, A. (2019). A learning-based brain tumor detection system. Computers, Materials & Continua, 59, pp. 713–727. Ozyurt, F., Sert, E. & Avci, D. (2020). An expert system for brain tumor detection: fuzzy C-means with super-resolution and convolutional neural network with extreme learning machines. Medical Hypotheses, 134, p. 109433. Pashaei, A., Sajedi, H. & Jazayeri, N. (2018). Brain tumor classification via convolutional neural network and extreme learning machines. In 8th International Conference on Computer and Knowledge Engineering (ICCKE 2018). doi: 10.1109/ ICCKE.2018.8566571. Pereira, S., Pinto, A., Alves, V. & Silva, C. A. (2016). Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35, pp. 1240–1251. Perveen, S., Shahbaz, M., Saba, T., Keshavjee, K., Rehman, A. & Guergachi, A. (2020). Handling irregularly sampled longitudinal data and prognostic modeling of diabetes using machine learning techniques. IEEE Access, 8, pp. 21875–21885. Petersen, R. C., Aisen, P. S., Beckett, L. A., Donohue, M. C., Gamst, A. C., Harvey, D. J., …& Weiner, M. W. (2010). Alzheimer’s disease neuroimaging initiative (ADNI): clinical characterization. Neurology, 74, pp. 201–209. Polly, F. P., Shil, S. K., Hossain, M. A., Ayman, A. & Jang, Y. M. (2018). Detection and classification of HGG and LGG brain tumor using machine learning. In International Conference on Information Networking. doi: 10.1109/ICOIN.2018.8343231. Pugalenthi, R., Rajakumar, M. P., Ramya, J. & Rajinikanth, V. (2019). Evaluation and classification of the brain tumor MRI using machine learning technique. Control Engineering and Applied Informatics, 21, pp. 12–21. Qureshi, I., Khan, M. A., Sharif, M., Saba, T. & Ma, J. (2020). Detection of glaucoma based on cup-to-disc ratio using fundus images International Journal of Intelligent Systems Technologies and Applications, 19(1), pp. 1–16. doi:10.1504/IJISTA.2020.105172. Rad, A. E., Rahim, M. S. M, Rehman, A. & Saba, T. (2016). Digital dental X-ray database for caries screening. 3D Research, 7(2), pp. 1–5. doi:10.1007/s13319-016-0096-5. Rad, A. E., Rahim, M. S. M., Rehman, A., Altameem, A. & Saba, T. (2013). Evaluation of current dental radiographs segmentation approaches in computer-aided applications IETE Technical Review, 30(3), pp. 210–222. Rahim, M. S. M, Norouzi, A., Rehman, A. & Saba, T. (2017a). 3D bones segmentation based on CT images visualization. Biomedical Research, 28(8), pp. 3641–3644. Rahim, M. S. M, Rehman, A., Kurniawan, F. & Saba, T. (2017b). Ear biometrics for human classification based on region features mining. Biomedical Research, 28(10), pp. 4660–4664. Ramzan, F., Khan, M. U. G., Iqbal, S., Saba, T. & Rehman, A. (2020b). Volumetric segmentation of brain regions from MRI scans using 3D convolutional neural networks. IEEE Access, 8, pp. 103697–103709. Ramzan, F., Khan, M. U. G., Rehmat, A., Iqbal, S., Saba, T., Rehman, A. & Mehmood, Z. (2020a). A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. Journal of Medical Systems, 44(2), p. 37. Razzak, M. I., Imran, M. & Xu, G. (2019). Efficient brain tumor segmentation with multiscale two-pathway-group conventional neural networks. IEEE Journal of Biomedical and Health Informatics, 23, pp. 1911–1919. Rehman, A., Abbas, N., Saba, T., Mahmood, T. & Kolivand, H. (2018). Rouleaux red blood cells splitting in microscopic thin blood smear images via local maxima, circles drawing, and mapping with original RBCs. Microscopic Research and Technique, 81(7), pp. 737–744. doi:10.1002/jemt.23030.

Artificial Intelligence in Brain Tumor Detection Through MRIs

275

Rehman, A., Naz, S., Razzak, M. I., Akram, F. & Imran, M. (2019). A deep learning-based framework for automatic brain tumors classification using transfer learning. Circuits, Systems, and Signal Processing, 39, pp. 757–775. Ronald Warnick, M. & Gozal, Y. (2018). Mayfield Brain & Spine: An Introduction of Brain Tumor and its Types [Online]. Mayfield Clinic. [Accessed 15/9/2020]. Roy, S. & Bandyopadhyay, S. (2012). Detection and quantification of brain tumor from MRI of brain and it’s symmetric analysis. ICT Journal, 2(6), pp. 477–483. Rundo, L., Militello, C., Tangherloni, A., Russo, G., Vitabile, S., Gilardi, M. C. and Mauri, G., 2018. NeXt for neuro-radiosurgery: a fully automatic approach for necrosis extraction in brain tumor MRI using an unsupervised machine learning technique. International Journal of Imaging Systems and Technology, 28(1), pp. 21–37. Saba, T. (2020). Recent advancements in cancer detection using machine learning: systematic survey of decades, comparisons, and challenges. Journal of Infection and Public Health, 13(9), pp. 1274–1289. Saba, T., Mohamed, A. S., El-Affendi, M., Amin, J. & Sharif, M. (2020b). Brain tumor detection using a fusion of handcrafted and deep learning features Cognitive Systems Research, 59, pp. 221–230. Saba, T., Rehman, A., Mehmood, Z., Kolivand, H. & Sharif, M. (2018). Image enhancement and segmentation techniques for detection of knee joint diseases: a survey. Current Medical Imaging Reviews, 14(5), pp. 704–715. doi:10.2174/1573405613666170912164546. Saba, T., Sameh, A., Khan, F., Shad, S. A. & Sharif, M. (2019). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of Medical Systems, 43(12), p. 332. Saba, T., Sameh Mohamed, A., El-Affendi, M., Amin, J. & Sharif, M. (2020a). Brain tumor detection using a fusion of handcrafted and deep learning features. Cognitive Systems Research, 59, pp. 221–230. Sadad, T., Munir, A., Saba, T. & Hussain, A. (2018). Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. Journal of Computational Science, 29, pp. 34–45. Sajid, S., Hussain, S. & Sarwar, A. (2019). Brain tumor detection and segmentation in MR images using deep learning. Arabian Journal for Science and Engineering, 44, pp. 9249–9261. Sajjad, M., Khan, S., Muhammad, K., Wu, W., Ullah, A. & Baik, S. W. (2019). Multi-grade brain tumor classification using deep CNN with extensive data augmentation. Journal of Computational Science, 30, pp. 174–182. Sawant, A., Bhandari, M., Yadav, R., Yele, R. & Bendale, S. (2018). Brain cancer detection from MRI: a machine learning approach (TensorFlow). International Research Journal of Engineering and Technology, 5, pp. 2089–2094. Seetha, J. & Raja, S. S. (2018). Brain tumor classification using convolutional neural networks. Biomedical and Pharmacology Journal, 11, pp. 1457–1461. Shanthakumar, P. & Ganesh Kumar, P. (2015). Computer-aided brain tumor detection system using watershed segmentation techniques. International Journal of Imaging Systems and Technology, 25, pp. 297–301. Sharif, M., Amin, J., Raza, M., Anjum, M. A., Afzal, H. & Shad, S. A. (2020a). Brain tumor detection based on extreme learning. Neural Computing and Applications, 32, pp. 15975–15987. Sharif, M. I., Li, J. P., Khan, M. A. & Saleem, M. A. (2020b). Active deep neural network features selection for segmentation and recognition of brain tumors using MRI images. Pattern Recognition Letters, 129, pp. 181–189. Siar, M. & Teshnehlab, M. (2019). Brain tumor detection using deep neural network and machine learning algorithm. 2019 9th International Conference on Computer and Knowledge Engineering (ICCKE). doi: 10.1109/ICCKE48569.2019.8964846.

276

Artificial Intelligence and Internet of Things

Siegel, R. L., Miller, K. D. & Jemal, A. (2015). Cancer statistics, 2015. CA: A Cancer Journal for Clinicians, 65, pp. 5–29. Siegel, R. L., Miller, K. D. & Jemal, A. (2017). Cancer Statistics, 2017. CA: A Cancer Journal for Clinicians, 67, pp. 7–30. Sultan, H. H., Salem, N. M. & Al-Atabany, W. (2019). Multi-classification of brain tumor images using deep neural network. IEEE Access, 7, pp. 69215–69225. Suzuki, K. (2017). Overview of deep learning in medical imaging. Radiological Physics and Technology, 10, pp. 257–273. Swati, Z. N. K., Zhao, Q, Kabir, M., Ali1, F., Ali, Z., Ahmed, S. & Lu, J. (2019). Contentbased brain tumor retrieval for MR images using transfer learning. IEEE Access, p. 7. doi:10.1109/ACCESS.2019.2892455. Talo, M., Yildirim, O., Baloglu, U. B., Aydin, G. & Acharya, U. R. (2019). Convolutional neural networks for multi-class brain disease detection using MRI images. Computerized Medical Imaging and Graphics, 78, pp. 101673. Thaha, M. M., Kumar, K. P. M., Murugan, B. S., Dhanasekeran, S., Vijayakarthick, P. & Selvi, A. S. (2019). Brain tumor segmentation using convolutional neural networks in MRI images. Journal of Medical Systems, 43, p. 294. Tiwari, A., Srivastava, S. & Pant, M. (2020). Brain tumor segmentation and classification from magnetic resonance images: review of selected methods from 2014 to 2019. Pattern Recognition Letters, 131, pp. 244–260. Toshniwal, R. (2020). How to select Performance Metrics for Classification Models [Online]. Available: https://medium.com/analytics-vidhya/how-to-select-performance-metricsfor-classification-models-c847fe6b1ea3 [Accessed 2/10/2020]. Ullah, H., Saba, T., Islam, N., Abbas, N., Rehman, A., Mehmood, Z. & Anjum, A. (2019). An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection. Microscopy Research and Technique, 82(4), pp. 361–372. Usman, K. & Rajpoot, K. (2017). Brain tumor classification from multi-modality MRI using wavelets and machine learning. Pattern Analysis and Applications, 20, pp. 871–881. Yang, W., Feng, Q., Yu, M., Lu, Z., Gao, Y., Xu, Y. & Chen, W. (2012). Content-based retrieval of a brain tumor in contrast-enhanced MRI images using tumor margin information and learned distance metric. Medical Physics, 39, pp. 6929–6942. Yogananda, C. G. B., Nalawade, S. S., Murugesan, G. K., Wagner, B., Pinho, M. C., Fei, B., Madhuranthakam, A. J. & Maldjian, J. A. (2019). Fully Automated brain Tumor Segmentation and Survival Prediction of Gliomas Using Deep Learning and MRI. https://doi.org/10.1007/978-3-030-46643-5_10 Yousaf, K., Mehmood, Z., Awan, I. A., Saba, T., Alharbey, R., Qadah, T. & Alrige, M. A. (2019a). A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease (AD). Health Care Management Science, pp. 1–23. doi: 10.1007/s10729-019-09486-0. Yousaf, K., Mehmood, Z., Saba, T., Rehman, A., Munshi, A. M., Alharbey, R. & Rashid, M. (2019b). Mobile-health applications for the efficient delivery of health care facility to people with dementia (PwD) and support to their carers: a survey. BioMed Research International, 2019, pp. 1–26. Zhao, F., Xu, Y., Zhang, H. & Ren, Y. (2019). Ultrasonographic findings of uterine carcinosarcoma. Gynecologic and Obstetric Investigation, 84, pp. 277–282. Zhao, X., Wu, Y., Song, G., Li, Z., Zhang, Y. & Fan, Y. (2018). A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Medical Image Analysis, 43, pp. 98–111. Zhou, Z., He, Z. & Jia, Y. (2020). AFPNet: A 3D fully convolutional neural network with atrous-convolution feature pyramid for brain tumor segmentation via MRI images. Neurocomputing. doi:10.1016/j.neucom.2020.03.097.

Part II IoT Applications in Healthcare

10

An Empirical Study of Domain, Design and Security of Remote Health Monitoring Cyber-Physical Systems Albatool Al-katrangi, Shahed Al-kharsa, Einaas Kharsah, Anees Ara

CONTENTS 10.1 Introduction�������������������������������������������������������������������������������������������������� 279 10.2 Domain Analysis������������������������������������������������������������������������������������������ 281 10.2.1 Asset Identification��������������������������������������������������������������������������� 281 10.2.2 Malicious Actors and Threat Identification�������������������������������������� 282 10.2.3 Vulnerability Identification and Quantization����������������������������������284 10.3 Design Analysis�������������������������������������������������������������������������������������������� 286 10.3.1 Architecture Overview��������������������������������������������������������������������� 286 10.3.1.1 System Architecture for Indigestible Pills������������������������� 288 10.3.2 CPS Security Model������������������������������������������������������������������������� 290 10.3.2.1 Assets to Security Model Mapping������������������������������������ 293 10.4 Security Analysis������������������������������������������������������������������������������������������ 293 10.4.1 Subdomain Analysis������������������������������������������������������������������������� 294 10.4.2 Security Analysis Results����������������������������������������������������������������� 294 10.5 Conclusion���������������������������������������������������������������������������������������������������� 298 References�������������������������������������������������������������������������������������������������������������� 299

10.1 INTRODUCTION Remote smart healthcare systems have various applications in the field of healthcare. Possible examples can be ingestible sensors which are pill-sized electronics that can be swallowed and consist of power supply, microprocessor, sensors and controllers. They can be used to monitor medical molecules and possibly diagnose other gastrointestinal ailments. They can also measure temperature, pH and blood pressure; indicate whether or not a patient has taken the medications; track chronic disease; and ensure medication consumption. 279

280

Artificial Intelligence and Internet of Things

Another example is a device made for diabetes patients consisting of two components, glucose monitor and an insulin pump. The glucose monitoring system sends readings to a sensor that is a small electrode attached under the skin—a small device that patient or their caregivers can use to make sensor insertion easier. After that, it measures glucose in the fluid found between the body’s cells and replaced after a few days of use. A transmitter sends information to the pump, which displays the glucose value and 24-hour trend graphs. Real-time glucose information on the pump allows patients to make immediate adjustments to their glucose control. Moreover, other healthcare applications provide a solution for Parkinson’s disease, a nervous system disorder that affects body movement. The promising “Movement disorder API” is a framework that will allow Apple Watch to monitor Parkinson’s disease’s symptoms constantly. It can detect any unusual footstep, instability of stride length in patients, analyze and draw a graph pattern (Godoi et al., 2019). The doctor checks data retrieved from the cloud and suggests solutions and alternatives for them. Such devices can be used by patients, doctors and nurses, and other healthcare providers. It varies depending on patients’ needs of the particular device, for instance, cardiovascular or heart patients, diabetes patients, paralytic patients or even patients with mental illnesses. Using remote monitoring devices makes it easy for those patients and their caregivers to monitor the vital signs and take immediate action if needed. Of course, this helps doctors and nurses a much easier since they receive information from patients’ devices directly, for example helping prescribe medications, updating diagnosis and efficiently providing real-time supervision. Besides, smart healthcare technology will be flourishing when both patients and healthcare providers will start using such devices, which improves better economy and revenues, and also cloud service providers if such a technology requires a cloud for uploading, updating or retrieving patients’ data. The purposes of such technologies are enormous; it could be for making the job of all users easier to monitor and control such sensitive data, for saving time and cash or even for involving and making use of technology and the digital revolution to reduce contagious diseases. The common objectives of healthcare smart technology businesses are to reduce the purchasing cost of such devices so that they can be affordable and easy to get, improve customer experience and help customers with good customer service and also increase the quality of the product as well as reduce the possible risks of information security incidents. This chapter will conduct an empirical study of the security of remote health monitoring system. These systems can create a vast market and ease of use to monitor the individuals remotely, as discussed earlier. The security and privacy of such systems are critical as it involves human lives. This chapter is organized as follows: 1. Section 10.2 will include asset identification and prioritization, malicious actors and threat identification (using STRIDE model) and vulnerability identification and quantization (vulnerability assessment model). 2. Section 10.3 will include the architecture overview of popular health monitoring systems in the literature. We discuss cyber-physical system (CPS)based security model briefly. Finally, a mapping between the assets to security model tracing is presented.

An Empirical Study of Domain, Design and Security

281

3. Section 10.4 will analyze selected respective subdomains of medical CPS such as sensing, actuating and cloud. Later, the security analysis results and some recommended domain concerns, subdomain concerns, design concerns, past vulnerabilities and availability concerns checklists for securing the remote health monitoring systems are discussed.

10.2  DOMAIN ANALYSIS This section precisely provides an overall assessment of the security risks posed by the CPS application domain. It includes asset identification and prioritization related to this CPS system, malicious actors and threat identification (using STRIDE model) and vulnerability identification and quantization.

10.2.1  Asset Identification In security, the asset identification and classification are very essential. Remote healthcare monitoring systems have a broad range of assets needed for their operation and must be protected, whereas some other smart remote healthcare monitoring systems are also relevant assets in traditional healthcare monitoring systems, as they are linked to the network and can make autonomous decisions (Sinhasane, n.d.). Among these, some of the critical assets are as follows: • Sensors: Sensors are devices, machines or modules that detect changes in the environment using physical parameters and translate them to digital and electronic signals. Sensors are assets because they collect real data from the environment and send it to other devices, resulting in vast and life-changing data in a system or application. Regarding health monitoring systems, sensors play a huge role in collecting data from the patients and sending it to intended parties, as they collect vital signs of a patient. • Data: The data and information collected from the sensors transmit through a network to the doctor’s devices and monitors. Such data can help monitor the patient’s status, arrange, modify and update operations. The collected blood pressure, pulse rate and temperature data can indicate the patient’s body state. • Remote meter devices: Meter devices (input/output devices) are devices that stay with the patients alongside the sensors and provide a consistent data transmission about the patients’ health status and help the patients by providing warnings, reminders and direct physical contact. The patient can either manually enter the data (e.g. insulin units) or the sensors can automatically enter them into the device. • Network: The establishment of a network cannot be without servers and communication protocols such as Zigbee. Zigbee is a widely used communication protocol in computing and wireless sensing networks due to its low power consumption, cost and bandwidth. Such protocols go under the network asset and are as important as any other asset in the system.

282

Artificial Intelligence and Internet of Things

The network helps connect electronic health records (EHR), such as insulin pumps with the patient’s input device (e.g. smartphone). Various types of networks are available, but it is necessary to check the device’s connectivity to communicate with EHR. Wi-Fi, mobile connection or Bluetooth connection can be used to achieve it. • Data Storage: For each diagnosed patient, it is essential to keep the patient data and health parameters in a local device or in a central repository in clinics and hospitals to be retrieved once needed. • Cloud: The cloud infrastructure is crucial because it provides extra flexible storage by cloud service providers so the patients’ data can be uploaded to or retrieved from the cloud anywhere, accessing mobile data. • Mobile Software: These are applications or software on the patient’s device. The mobile software such as diagnostic applications and record monitoring software interacts with healthcare-providing systems. In remote healthcare monitoring systems, it is essential to know that system assets hold different criticality levels for patients and service providers to perform their jobs. An asset is critical when any interruption or interception would affect the operation of the system negatively. Assets discussed are ranked and assessed based on the level of severity of an interruption or attack to the services. The most important physical assets in remote health monitoring are sensors that play a massive role in the remote health monitoring field as they collect data that is transmitted to healthcare providers. If any interruption happens to the sensors, the data may not be transmitted properly. Hence, they are a top priority. Sensors require a medium through which data can be transmitted. So, the network becomes the second-highest priority. Also, devices used by patients and healthcare providers come in priority after the network. The data collected by the sensors comes after that because the patients’ data is the main component in remote health monitoring systems as it allows doctors to diagnose and prescribe medication. This data needs storage space; prioritized assets can be stored in local storage, such as a computer hard disk or a USB driver. It also can be a data repository or a cloud service. Existing software/systems or applications have the lowest priority, and when an interruption happens, only privacy will be invaded but not the security. As mentioned earlier, not all assets have the same criticality for patients and services they need, so this will change in the future as more and more assets will be added due to the development in the field of technology and CPS, so reprioritization plans to the assets are needed, in a way, that balances and suits new changes in the remote monitoring system.

10.2.2 Malicious Actors and Threat Identification In general, several wireless devices are used and coupled with medical sensors to track them, but many proposed systems are not considering the required security controls. Since the sensitive information these systems handle, the security threats associated with these systems can be a major concern. To grant the full use of remote health monitors in the medical health-care domain, different qualities need to be identified and investigated, including vulnerabilities, countermeasures and critical

An Empirical Study of Domain, Design and Security

283

protection. Regularly, the patient environment consists of sensors and devices. These devices are needed to get sensor data measurements and promote input and feedback from health-care providers. This data stored in those devices helps the doctor make decisions and provide suggestions about health improvement, but the problem is that a third party also gets access to that data, which produces security concerns. Security has been a controversial topic concerning remote health-care systems. The lack of knowledge and inadequate implementation of healthcare protection expose patients’ data to attackers and threats. There are different security threats in remote healthcare monitoring systems that may occur. Attackers can attempt to steal patient’s records, block device resources or delete and alter patient’s data. They can exploit software of remote health monitoring systems in different ways, for instance, performing intentional bad acts like hijacking and skimming. Human errors can also occur while devices are being set up or run. They are often related to insufficient procedures or training. Besides, system failures can happen due to the high complexity of the systems. According to Butt et al. (2019), there are huge numbers of attacks in remote health monitoring systems, categorized as routing attack such as router attack, selective forwarding attack and location-based attack such as denial of service (DoS) attack and sensor attack. Often in the routing attacks, the attacker addresses the data path for sending or dropping data packets, whereas, in the locationbased attack, the attacker often targets the destination node to block the network services. The best procedure to identify and mitigate such attacks is threat modeling. It is an important part in security growth, which is a process aimed at creating stronger and safer systems by recognizing properties, assessing and mitigating potential threats. The threat modeling technique used in healthcare monitoring systems is STRIDE by Microsoft, where the initials stand for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service and Elevation of Privilege. The STRIDE threat model is used to identify most of possible threats in remote health monitoring systems as follows: • Spoofing is an act of attempting to access a particular system using a false identity. It happens when an attacker uses legitimate user credentials to access medical data. For example, the wireless communication between the pumps and the blood glucose meters are in clear text, a remote attacker can spoof the meter remote and access the patient information. It will invade the confidentiality of patents’ data. • Tampering occurs when an attacker modifies the data when in transit or at rest. For example, a hacker could connect wirelessly to a nearby insulin pump and change its settings leading to a drastic change in blood glucose. It will affect the integrity of the patient’s data and might lead to huge bad impact on the patients’ health. In addition to that, repudiation happens when an authorized system acts illegally and the system cannot track it. • Information disclosure attack exposes the patients’ information to any unauthorized entity or third party. For example, when an attacker gets access to medical records with harmful intentions. It is a big concern for the confidentiality of the data, especially since it is sensitive data.

284

Artificial Intelligence and Internet of Things

• DoS can put the system’s availability at stake because of DoS attacks hence the system denies to access the service for valid parties. For instance, when an insulin pump fails to send information to the blood glucose meter after sensing the busy channel. • Elevation of privilege grants higher privilege rights and tampering the patients’ information. It is violated when a user with lower privileges (e.g. HR employee in the hospital) takes the identity of a privileged user (e.g. the doctor) to gain higher privileged access to a resource (e.g. patient’s info). Any unprivileged access may lead to breaches of confidentiality (Cagnazzo et al., 2018). The main objective of security protection is to maintain data confidentiality, integrity and patient data availability (Alhirabi et al., 2019). Remote health monitoring devices are disclosed to confidentiality, which may be not easy for unauthorized users to attack. Also, integrity may be affected by unauthorized manipulation of patients’ information. However, the patients’ data availability is the most important security protection because the patient’s remote device should always be available to save the data sensed by the sensor and not to miss any data record (Hassanalieragh et al., 2015). This data is needed for the healthcare providers to monitor the patients’ health and give them the right instructions.

10.2.3 Vulnerability Identification and Quantization Vulnerability is a weakness to be exploited in API, firmware, hardware, operating systems, third-party software and medical devices as well as people, networks and processes. Body-attached medical devices are wirelessly reprogrammable devices, such as insulin pumps, neurostimulators and defibrillation devices, to track patient’s body status. These devices are vulnerable, which is damaging to the operation and the CIA of the associated data. Therefore, manufacturers, healthcare professionals and patients must ensure correctness of information and privacy and secrecy of patient, software and availability of information to prevent negative effects. Vulnerability is the result of causes related to technological, human and management. Incorrect configuration and security glitches with no mitigation controls are the vulnerabilities, overlooked by legacy OSs and systems, because of the lack of updates and poor fixes in the program. It is also a challenge where operational problems and delays to service are present. Another concern is that certain remote medical systems lack basic safety features that can interrupt clinical workflow when poorly implemented. The lack of knowledge in the domain of information security and related activities is also a challenge and can lead to bad practices. Such bad practices include sharing and distributing passwords specifically in devices that require passwords. Poor understanding and lack of training about cybersecurity risks and impacts often boost the ongoing vulnerabilities. It may also be difficult to maintain the balance between privacy plus protection side and quality plus safety of smart health-care systems. For instance, using advanced encryption techniques and access control measures improves security, but at the same time, it puts the patient at a risk in the case of emergencies. Encryption might hold the medical devices slow and

An Empirical Study of Domain, Design and Security

285

drain the battery life in case of limited power and resources. Such problems demonstrate the difficulty of cybersecurity risk control and management, which lead to the general lack of protection in the current field of remote health monitoring. The possible vulnerability allows for the unprotected storage of credentials, enabling an attacker to access private SSH keys in configuration files. Another vulnerability that affects platforms uses a poor encryption scheme for its remote desktop control, allowing a hacker to access remote code execution of the devices in the network. If an attacker gains access to critical networks and/or sharing information due to inadequate configuration or access to equipment, successful exploitation of such vulnerabilities may occur. An exploit may result in loss of monitoring and/or alarm failure during active monitoring of patients. If exploited, these vulnerabilities will allow an attacker to access PHI data and make device-level change to the operating system. As a result, an exploited device may be made unusable or interfere with the functionality of the device, it also may allow an attacker to change the alarm settings on the connected patient devices or even to access the UI for doctors and physicians. As a result, attack could cause a silent or wrong alarm. IT and security leaders should also ensure that such networks are set up to block all incoming traffic from outside the network, except for clinical data flows. Organizations can also limit physical access to the products and those networks which are impacted. Also, vulnerabilities usually exist when dealing with the cloud. The functionality of cloud-connected devices has allowed medical teams to communicate better. That’s because these devices will constantly send data to the cloud, and that data is integral to enhancing patient care but extends the risk of exposing new vulnerabilities for malicious hackers. Examples of such devices are implantable devices. These are embedded to transmit useful information to the cloud, but we know that anything with network connection is hackable. The vulnerability also occurs when there is no encryption to wireless commands and there is no mechanism to ensure patients are only able to upgrade their software from an official source. CISA advises that health organizations take protective steps to minimize the risk of exploiting these vulnerabilities, including locating and isolating medical system networks and remote devices behind firewalls. Also, establishing a full list of potential vulnerabilities can help to find some of them. The basic security key objectives are confidentiality, integrity and availability of information. Remote health monitoring devices are sensitive and open to confidentiality that may be compromised by unauthorized users because of bad control mechanisms. The implications of this result in noncompliance with regulations (Health Insurance Portability and Accountability Act (HIPAA), Australian Privacy Principles), reputational damage, litigation and financing. Furthermore, Integrity may be compromised because of bad configuration, data corruption or nonlegitimate modification. This will affect the patient safety from inappropriate clinical actions and wrong decisions, and be controlled by a non-legitimate user. Finally, Availability is compromised when access to data or a device is limited or completely unavailable. The impact of this on the patient safety from limited or blocked access to critical information, this would highly affect clinical decisions and patient safety especially real-time data are not received. Interception of data shared between an insulin pump and a body-connected device is not exactly a risky part. However,

286

Artificial Intelligence and Internet of Things

confidentiality may be compromised by eavesdropping, if third party accesses the data, it is not likely to result in any major safety issue. However, integrity is crucial, and this is particularly challenging when dealing with a wireless connection. As the transfer process is a radio wave signal, this signal cannot only be intercepted, but also an attacker can send his or her own signal. This is referred to as a MITM attack. This type of attack is extremely high risk, as an attacker could reprogram a device to operate in a way that could severely affect patient safety. The success of remote monitoring healthcare systems depends on network coverage and system compatibility, particularly when large data frames and packets are transmitted over the network, interruptions may block or interrupt the diagnosis. For example, in some cases, requiring constant monitoring of the patient’s condition, such as heart or brain disorders, interruptions in the network at crucial times may be unwanted. The input devices are also located near the patients, so patients need to ensure that the network is available to healthcare providers for persistent communication with the health recording system. Most vulnerability occurs in the availability category because patients may sometimes not be aware of the remote devices connected to the remote health monitoring systems. The vulnerabilities found are not only specific to remote health monitoring systems because most vulnerabilities related to the network, device, storage and other assets are available in other domains such as smart transportation and smart hotels. In smart hotels, if vulnerability is found in the storage, it can be exploited to access guest’s personal information and invade the hotel’s privacy. According to Peck (2011), the frequency of these vulnerabilities will increase in the future as more and more assets will be included. For example, Pacemakers, defibrillators and insulin pumps are wireless devices when communicating. Car companies are searching for ways to show glucose levels on dashboards. Insulin pumps should sooner or later be completely integrated with glucose monitors, taking instructions from an attached computer rather than from humans. Any improvement enhances the user experience but brings new vulnerabilities and concerns along with it. Paul mentioned that medical devices have been getting more complicated to the point where they have multiple devices. This makes the entire system more complicated to handle safety and security analysis.

10.3  DESIGN ANALYSIS This section assesses the high-level design of the system, which includes an overview of the architecture of the popular health monitoring systems in literature, a CPSbased security model and a mapping between the assets to security model tracing.

10.3.1  Architecture Overview In this section, we discuss popular remote health monitoring system architectures developed for various healthcare purposes. The system architecture for a remote health monitoring system normally has three main units: multiple wearable sensors evaluating physiological biomarkers allow the data acquisition, such as insulin pump, ingestible pill and sensor movement disorder API. The sensors connect to the

An Empirical Study of Domain, Design and Security

287

network in a specific medium, usually a mobile in a patient’s area. The system’s data transmission components are responsible for transferring patient information from the patient’s home (or any remote location) to the healthcare organization’s data center with guaranteed security and privacy, preferably in near real time. Typically, the wireless sensor network is linked through communication protocols such as Zigbee or Bluetooth to transfer the data from the sensor to the concentrator. Furthermore, aggregated data relates to a long-term storage organization utilizing Internet access, through Wi-Fi or cellular data connection of a smartphone. Sensors in the data acquisition component form a remote CPS-based healthcare architecture, because the data of each individual sensor can be accessed through the Internet. Often the processing device in location of a client is called a cloudlet or a cloud service, which is used to improve its storage or processing capabilities when the mobile resources do not meet the requirements of the application. The cloud service can be a local processing unit (such as a PC) that can be accessed through Internet network. It also can provide temporary storage in the cloud, or to be used as a transmitter to the data in case of lack of energy or access. The cloud processing has three main parts: analytics, visualization and storage. The system is designed to store valuable information for patients in the long term and assist doctors and nurses with diagnostic information. In addition, visualization is the main component for any system because it is inefficient to ask doctors to go through the enormous wearable sensor data. Visualization methods make the data available in a qualified format (see Figure 10.1). According to Mithun et al. (2013), pumping is done with the help of a (microelectromechanical systems) pump (sensors), a user interface for controlling, Bluetooth for communication with the meter devices, keypad, real-time clock, memory to save all data, audio, LED and alarms (actuators). Insulin pump has an insulin tank and syringe for injecting insulin into the body through cannula (a thin tube inserted into a vein or body cavity to administer medication). No air present in the cannula when the needle is inserting. Also, a function called prime is added to the platform. Patients can select the desire function with the help of switch. Insulin pump has

FIGURE 10.1  System architecture

288

Artificial Intelligence and Internet of Things

the ability for adjusting the amount of Basal dose by the doctor. There are different insulin pump profiles such as fasting day, holiday, party day and working day. For each profile, the dose will be different. Each event dose will be stored in the memory with the help of real-time clock. Alarms for any fail in the system are provided to add high accuracy and reliability to the system, for example low battery alarm and insulin empty. Doctors or patients can see events history through insulin pump’s menu options. Insulin pumps will also send this information to the doctor’s device through Wi-Fi and will be stored as an Excel sheet format in the computer. 10.3.1.1  System Architecture for Indigestible Pills The architecture of the ingestible pill can vary from one manufacturer to another. One can use additional components to enhance the overall operation of the pill and its connectivity to the healthcare providers. However, according to Weeks et al. (2018) and Ohta et al. (2015), the common subsystems among all types of pills are sensors and a Zigbee IC, controllers, microprocessors, battery and sometimes Bluetooth and NFC chips. Vital sensor data is transmitted to a smartphone linked to the cloud where hospital data can be accessed or to any Wi-Fi connection points with a passphrase. The overall inside subsystems of the ingestible capsule are flexible NFC, pH sensor, temperature sensor, battery, NFC chip (see Figure 10.2). Such subsystems combine in order to perform operations such as sensing of signals, alterations and temperature. Usually, sensors sense the level of pH and temperature. NFC chip lets users exchange information with other users with NFC and sends data to the health record systems and clinics. They combined battery to keep the pill alive and doing its job. The wearable sensor regularly detects the channel in the patient’s body to check for signals from the active ingestible sensors. The wearable sensor performs digital communications functions such as frame synchronization, frequency estimation and error correction to interpret messages sent by the active ingestible sensors when the channel is successfully sensed. As Brook (2020) mentioned, security is an important issue in the field of health monitoring systems. Networks are vulnerable, but healthcare providers are possible

FIGURE 10.2  Components of the ingestible pill

An Empirical Study of Domain, Design and Security

289

goal for attackers. The HIPAA makes sure that the patient health data is covered. To help keep the systems secure, companies who provide the remote health monitoring should be encrypting the collected data from patient’s data, backing up data regularly, training patients how to use the devices, monitoring the usage of patients, utilizing multifactor authentication and accessing vendor vulnerability. It was mentioned in an article by Bialas (2019) that in remote health monitoring systems, some subsystems are more susceptible to vulnerabilities due to their role in this activity. Most of the time, sensors like the MEMS pump are considered more vulnerable because they are related to the processes of producing the data, processing it, storing and transferring it to the device. Sensors are integrated with different network technologies, specifically the wireless ones. Threats that spread out from this upper layer and exploit weaknesses of sensors can also infiltrate sensor systems. We should also note that sensors often operate at the first frontier, directly communicating with human lives. Remote health monitoring systems are built based on the concept of CPSs. As we know, CPS is one of the latest technologies nowadays because of the huge amount of its implementation. This interdependence will make the system vulnerable to any type of cyber or physical attack, which makes security the main concern for both. There are several vulnerabilities and threats which may be used by an attacker to manipulate a physical or cyber component. Security measures should be implemented in both to ensure the sensed data, collected data and transferred data is protected and safe to save people’s lives. The data protection can be achieved by the three significant security features CIA as follows: • Data confidentiality is the act of ensuring that information is accessible only to authorized and legitimate parties. To view this information, authorized users must be authenticated in some way before access is granted. If the encryption and protocols are implemented incorrectly in the storage, the memory can be accessed by unauthorized people. • Data integrity aims to ensure that data is registered and displayed exactly as intended, and the data collected must be the same as what was measured in the case of a glucose meter or insulin dosing record. Any unexpected change in data as the result of storage, editing, retrieval operation or transmission is a violation of data integrity. For example, if the memory is attacked and the data is altered, the result that appears in front of the user is not the same as the measured one. • The aim of availability is for data to be immediately accessed. For instance, if the vibrator alarm is attacked and the readings are not received, the alarm will not work even if the glucose is high or low. The success of remote monitoring healthcare of diabetes patients depends on the interaction with the external systems. This can be achieved by using a wireless connection which will introduce a lot of risks. For example, wireless sniffing can happen when data is transmitted between the patient and the doctor, because malicious actors could use sniffing tools to obtain this sensitive data. In the future, the architecture will be more complicated, and more assets will be added, so the attacker will have a bigger landscape to attack.

290

Artificial Intelligence and Internet of Things

10.3.2 CPS Security Model The CPS security model is discussed based on the system architectures covered in the previous section. In these models, you can find most of the remote health monitoring components discussed earlier. The main controlling processes for all types of remote health monitoring systems are the same, but the way of sensing and type of information that is sensed is different. All the mentioned components will work together to achieve the remote health monitoring system objectives as follows: • Scenario 1: Ingestible capsule (see Figure 10.3): The patient swallows the sensor capsule and it will settle in the stomach. The sensor capsule will start sensing the around medium and send the data to a smartphone or meter device that is responsible for collecting all information and sending it to the cloud. The cloud job is to save and pass all the patient recorded information to the hospital database. The doctor can directly search for the needed data from the database. • Scenario 2: Insulin pump (see Figure 10.4): The patient will inject the insulin pump and attach the sensor to his/her body. The sensor responsibility is to sense the glucose level in the blood and send the data to a meter device or smartphone. The insulin pump is responsible for injection until insulin based on the sensor reading. The meter devices that are responsible to stay turned for any emergency that the patients suffer from like low blood sugar to automatically call the hospital. Also, it is responsible for collecting all information and sending it to the cloud. • Scenario 3: Sensor Movement disorder API (see Figure 10.5): The patient will wear the smart watch with a built-in sensor. The sensor responsibility is to sense any unusual footstep and instability of stride length in patients’ walk. The sensed data will be sent to the meter device or smartphone. The meter devices that are responsible to stay turned for any emergency that the patients suffer from like a sharp drop in blood pressure due to some types of medicine and automatically call the hospital. Also, it is responsible for collecting all information and sending it to the cloud.

FIGURE 10.3  Model of the ingestible

An Empirical Study of Domain, Design and Security

291

FIGURE 10.4  Model of insulin

System boundaries, machine or trust boundaries play a huge role in terms of security. Machine boundaries in remote healthcare monitoring systems can be defined depending on each component of the system. Originally, the main components of the remote health monitoring system are devices, field gateways, cloud gateways and different services. Devices in this category are the remote machines that are connected to sensors, actuators and other components in the remote application systems. Field gateways are devices or may be software components that connect the remote device to the cloud or the cloud to other components or servers creating a single connection point between two components to provide additional security to the system. Cloud gateways are the route that runs and connects any component to the cloud of the hospital instead of local connection. Different services in the system are the backend components that run in the system, such as patient records and medication prescription databases and communication protocols. Such boundaries in the case of remote healthcare monitoring devices can vary sometimes from one deployment to another. Some components may be added to or removed from the

FIGURE 10.5  Model of movement disorder

292

Artificial Intelligence and Internet of Things

system depending on the purpose of the device, to which kind of illnesses it will be used, and its manufacturers. To ensure that security in the system is enough, trust boundaries should be identified. Trust boundaries between different components of the system are crucial with respect to both hardware and software parts, especially when designing a secure CPS system. Different components will be segmented into different trust zones based on the level of “trust” and protection mechanisms to be used to prevent different attacks. “Trust boundaries may be connected to authority, rights, or networks, and may also mean attack areas where attackers can interfere” (Riel et al., 2017). Trust zones are categorized in the case of remote health monitoring devices as follows: • Local boundary: This zone may include hospital and clinic devices and computers that receive the monitoring records from patients’ devices. Securing this zone is important either physically or cyber-wise. End users (in this case doctors, physicians and nurses) can interact with them by granting them access privilege only if they need access to perform a task such as diagnosing or monitoring the patient’s state. Physically, those computers and devices should be put in a safe place and the use of them should be extremely restricted. • Device boundary: It is the environment where the health monitoring systems exist with the patient, the network each device is connected to other components in the system such as Wi-Fi. Such devices are the actual ingestible pill, the insulin pump and the “Movement disorder API” installed in a smart watch. Those devices should be secured by securing the wired and wireless network to which the device is connected using encryption techniques. • Field gateways boundary: It is the zone where the gateways in the system are established. Segmenting and separating each gateway by its own is also possible if needed. In the case of remote systems, sensors should be placed on the patient’s skin or even swallowed and set inside their bodies. Security measurements should be considered to provide a secure medium for transmitting data from the patient’s body to the cloud or other servers, including SSL/TLS encrypted communications. • Cloud gateways boundary: Cloud service could be a public one, such as Microsoft Azure, Amazon AWS or even an independent medical cloud provider. Countermeasures for protecting the cloud data both from unauthorized cyber and physical access are needed, especially if the system is totally cloud-dependent. Additionally, the independent cloud should implement encryption to prevent any data disclosure or modification that can invade the patient security. • Gateway and services boundary: It is where all the other backend services reside, including databases and APIs. This trust zone will vary based on used services in the remote devices. Because of this, security controls should be taken for each service and component to ensure the security implemented in all services. For instance, the database to which the glucose

An Empirical Study of Domain, Design and Security

293

levels are sent by the sensor should be separately encrypted so unauthorized users cannot reach it and apply any modification that could possibly change the diagnosis of a patient. • Remote user boundary: Any other remote systems that can be integrated with the remote system, such as the smart watch integrated with the API for monitoring the movement of the patient. Also, meter devices that are integrated with body-attached sensors that could help the patient and the physician track and monitor the data. All trust boundaries are important for the system to successfully function. Therefore, they should be well-protected against physical and cyberattacks and intruders. But most importantly, zoned with sensors, actuators and cloud storage must be very restricted due to their importance to the overall systems’ functionality. 10.3.2.1  Assets to Security Model Mapping The CPS models we previously discussed are insulin pump, ingestible pill and API movement disorder. They share most of the elements that can be mapped to the assets discussed previously. However, the types of sensors used in each may differ. In Chapter 1, we discussed seven important assets to our system. First, sensors are the main element in our CPS security model because they collect real-time data from the environment and send it to other devices. In insulin pumps, a glucose monitor sensor is used that measures the glucose level in the patient’s blood. As for ingestible pills, the sensors used are pH and temperature, which are responsible for measuring the level of pH and temperature of the body. Also, in API movement disorder, what’s used is a built-in sensor that is originally in the smart watch to sense the movement of the patients. Additionally, data collected by the sensors are transmitted through a network and reside in the database. All three models share this element and consider it as an asset, because any modification might lead to catastrophic results. The meter remote device is also considered an asset because it stays alongside the patients’ sensors to provide reminders, alarms and direct communication with doctors. This element mostly comes with the insulin pump, the network through which data is transferred from the patients to the doctor in the hospital. This element is important to be with any remote health monitoring systems. Besides, data storage and cloud which contain all the sensor-collected data are assets and reside in all CPS security models. Mobile applications that help patients to know their health states are also assets. They must be there in all CPS security models. So, all assets in our CPS system are now mapped and connected to our elements in CPS security model. The location in the architecture won’t directly affect the assets, because all the elements in the CPS model are connected through wireless communication, which lets attackers exploit any asset remotely. The assets are evenly distributed in our architecture in equal amounts. Hence all the assets related to CPS models are covered.

10.4  SECURITY ANALYSIS This section includes a brief security assessment and analysis of the selected subdomains of the medical CPS such as sensing, actuating and cloud. Later, the security

294

Artificial Intelligence and Internet of Things

analysis results and some recommended domain concerns, subdomain concerns, design concerns, past vulnerabilities, availability concerns checklists for securing the remote health monitoring systems are also discussed at the end.

10.4.1 Subdomain Analysis After discussing the design and architecture of remote health monitoring systems, we will talk about the security analysis of CPS applications. In analyzing the security of a remote health CPS, it is essential to understand that it is not enough to only assess the security issues at the network. Instead, the security of the entire system, including network, communication on network, sensors, actuators, controls, clouds and even the devices and hospital network, should be analyzed. The different segments of a remote health monitoring system have several security requirements and vulnerabilities that threatening operators can exploit to launch attacks against remote health monitoring systems. Sensing, actuating and cloud are the most important subsystems in the domain of remote health CPS. In order to transfer patient data to the hospital in the full and correct form, it is always necessary to secure those three subsystems, because a simple failure may put the patient’s health or life at risk. After analyzing the selected subsystems in the remote healthcare monitoring systems, we believe that we have selected the right subsystems because their security is very crucial for the systems to function successfully and serve their purpose of monitoring the patients’ health states. Sensing the right data, taking right action based on the sensed data at the right time and the appropriate place for saving those data are the most important in this CPS application.

10.4.2 Security Analysis Results • The sensing subdomain is responsible for detecting changes in the environment (human body) using physical parameters and translating them to digital and electronic signals. Sensing subdomain is considered an important part because it collects real data from the environment (human body) and sends it to other devices to take the right actions, and any wrong actions in the sensing process can result in huge and life-changing data in a particular system or application. In remote health monitoring systems, the sensors vary from one application to another. But in the end, they play the same role in importance. One of the sensors is a blood glucose monitor (BGM) which is with the insulin pump for monitoring the glucose level in the blood of humans. The assets of sensing subdomain should be protected to sense in ways. The assets are sensed data, communication protocols (Zigbee) through communication with the meter device and battery. If the patient, doctor or insulin pump vendor made a mistake in BGM, it will lead to vulnerabilities that open the door for attacks. The attacks may include changes in glucose levels from the BGM to the pump via the communication channel, changes in glucose levels from the BGM to the PC via the communication channel and changes to the BGM software by a PC. BGMs currently interact with desktop computers regularly to allow a patient to use data analysis tools on their blood glucose values. The interface between

An Empirical Study of Domain, Design and Security

295

a BGM and PC could be compromised through different viruses. One of the solutions is having a pump that has a fail-safe physical interface for the patient then the patient will retain pump control if a remote programmer is lost, stolen, or wireless communication is interrupted. Also, a simple tactile button on the device itself could be used to enable wireless communication for a short time. When that time is out, the wireless communication cannot take place. By temporarily disabling wireless communication, this protects against abuses where attackers can modify the data collected by this sensor. One of the vulnerabilities is the wrong configuration of the insulin pump that will lead to wrong doses. Another vulnerability is putting the sensor in the wrong place in the body, so all readings will be wrong. Lack of encryption techniques is another vulnerability that will lead to unauthorized access. • The actuating subdomain is responsible for altering the vital measurements after getting some data from the sensor. It receives control feedback (usually in a type of digital and electronic signals) and performs a modification in the vital system by performing the right actions. The actuating subdomain is considered an important part because they do actions based on collecting real data from sensors and any opposite action can result in life-changing data in a system. In remote health monitoring systems, the actuators vary from one application to another, but in the end, they play the same role in importance. One of the actuators is a syringe for injecting insulin which is in the insulin pump. It’s for the response to any increase or decrease in the glucose that is sensed by BGM. For actuating subdomains to do its job correctly and effectively, its assets should be protected. The assets are the commands to be executed, the amount of medicine (insulin) and the life cycle of the syringe. Mistakes that are made by the stakeholders may lead to attacks due to the vulnerabilities caused by these mistakes. Attacks may include accessing the pump communication by third-party access to the pump communication, changing of wireless pump commands already given, generating unauthorized wireless pump commands, remotely adjusting the program or system settings and block contact with the pump unit. To mitigate those attacks, check the source of commands, secure the communication between the actuator and controllers and regularly maintain and update the actuators. One of the vulnerabilities is when there is a malicious program in the actuators that will lead to executing wrong commands. Similarly, vulnerability is not being updated and does not check if there are flaws in the actuators. • Cloud subdomain is the virtual storage that patients’ data is sent to or retrieved from. It’s important because both patients and healthcare providers collected or updated data can be uploaded to a cloud to allow data mobile ability. Data in the cloud is constantly being added, updated, retrieved or removed which makes it a critical environment. In remote healthcare monitoring systems, the cloud is the same for all applications but the data that is stored is different. For the cloud subdomain to protect and save the data correctly, its assets should be protected. The assets are the data stored in, the communication protocol used to receive or send the data, and storage.

296

Artificial Intelligence and Internet of Things

TABLE 10.1 Recommended Checklist for Securing Remote Health Monitoring CPS Systems No. 1.0 1.1

1.2 1.3 1.4 1.5 1.6

2.0 2.1 2.2

2.3 2.4

2.5

2.6

2.7 3.0 3.1 3.2 3.3

Project-Specific Checklist Domain-specific security concerns The patients’ vital information (e.g. pH level) transmitting through the ingestible pill is well-encrypted and conserved Remote meter devices in insulin pumps are safeguarded from modification “Movement disorder API” in Apple smart watch checks regularly for updates Human errors are prevented during the configuration or operation of devices System failures are well-handled, despite the increase of complexity and dynamics of the systems The data quality by combing the medical data with some sensed data such as body motion, location or temperature is well verified and immediately informed to the healthcare provider in case of emergencies Exclusive subdomain vulnerabilities The wireless sensor network (WSN) uses encryption and protection mechanisms against attacks Zigbee protocol used to communicate between sensors is well-protected against detection of Zigbee networks, recordings and displaying information about the found devices Actuators that work alongside the sensors are checked for maintenance on a regular basis Cloud systems keep data secret by restricting it only to authorized entities and no unauthorized access to data can be obtained (data confidentiality) Cloud systems keep data not modified by any means while it is stored or transmitted through the network (data integrity) Classified traffic based on authorization and blocked traffic that is identified as unauthorized and allowed traffic that is identified as authorized The patients and physicians are well-trained to use the technology Past vulnerability mistakes Public key encryption is used instead of symmetric encryption Any incoming traffic is well-filtered Physically plugged malware is prevented and instead wireless router is used, and thus, a firewall is used

Checked/Not Checked

297

An Empirical Study of Domain, Design and Security

TABLE 10.1  (Continued) Recommended Checklist for Securing Remote Health Monitoring CPS Systems No. 3.4

4.4

Project-Specific Checklist The Wi-Fi network is well-secured with non-default passwords The responsible teams for implementing the devices are aware of installation specifications Design concerns Machine and trust boundaries are identified and protected based on each boundary The patient is comfortable using/wearing the remote monitoring device The healthcare organization’s PCs are protected using antivirus software The network is secured by firewalls and proxies

4.5

The rapid accessibility of patient’s data is possible

4.6

Appropriate communication protocols between components are used The remote devices achieve usability concern and do not require heavy training/instruction for using Availability Concerns Cloud service ensures that data and services are always available for legitimate users anytime and anywhere (data availability) Only authorized accessibility to patient medical records within the organization is possible Prevention techniques against denial of service attack are used A backup plan is maintained and used when necessary

3.5 4.0 4.1 4.2 4.3

4.7 5.0 5.1

5.2 5.3 5.4 5.5

Checked/Not Checked

Alternative power supply for the organization resources is available once needed

Making a mistake by the cloud provider may lead to a storage that is vulnerable to attacks. The attack at cloud can be the DoS attacks, cloud malware injection attack, side-channel attacks, MITMA and authentication attacks like brute force attacks and dictionary attacks. In DoS attack, an attacker overloads the target cloud system with service requests so that it stops responding to any new requests and hence made resources (medical records) unavailable to its users (patient and doctors). For restricting DoS attacks, we can classify traffic based on authorization, so we can block traffic that is identified as unauthorized and allow traffic that is identified as authorized. In cloud malware injection, attack is when an attacker tries to

298

Artificial Intelligence and Internet of Things

inject a malicious piece of code or malware into the cloud. To prevent the cloud from malware injection attack, the integrity with hardware can be combined or hardware for integrity purposes can be used, because it is not easy for an attacker to snoop at the IaaS level. Inside channel attack is when an attacker tries to hack the cloud system by placing a malicious piece of code near the targeted cloud server system and launching a side-channel attack. A mix of virtual firewall tools may be used to protect the cloud from side-channel attacks. In a man-in-the-middle attack, the attacker intercepts and retransmits messages in a public key exchange. Using a one-time password to avoid this attack as the one-time password is resistant to MITM attacks. Authentication is a weak point in cloud computing services, which is frequently targeted by an attacker and usage of biometrics can be possible solutions against authentication attacks. The vulnerabilities that lead to these threats are insecure cryptography, Internet dependency and unencrypted communication channel. One of the vulnerabilities is the insecure encryption, where encrypting algorithms normally random number generators are used, using unpredictable information sources to generate actual random numbers. Another known vulnerability is the Internet dependency where the patient and doctor are required to have an Internet connection to access the data saved in the cloud and any failure in the connection will make the cloud services unavailable. Finally, insecure communication channels lead to the transfer of data from doctor/patient via insecure data transmission.

10.5 CONCLUSION To sum up, a remote health monitoring system is a critical CPS system that requires heavy protection techniques and mechanisms. Remote healthcare monitoring systems are a safety-critical system that deals with real-time patient data that can affect their lives and health. In this project, we discussed remote healthcare systems, which have various applications that are already implemented in healthcare. The previously discussed architectures are ingestible sensors, insulin pumps and “movement disorder API” for Parkinson’s disease patients. As dealing with a critical system, assets related to our domain were identified and classified. Also, possible potential attacks that occur due to exploiting the related vulnerabilities were discussed. In addition to that, the system architecture overview was elaborated, and the security model and system boundaries were explained. Eventually, system security was analyzed and selected the three most important subsystems, sensing, actuating and cloud, and summarized the security analysis results. A project-specific checklist, which is influenced by past vulnerabilities that have to be prevented, assets are involved and based on the system’s functionality, was created for future work and research purposes. Promoting patient care is a priority for all healthcare providers. Hence with remote healthcare monitoring systems, the overall purpose is to save patients’ lives and release a high degree of patient satisfaction. In future, we would like to extent our work by conducting a detailed vulnerability assessment and penetration tests, risk analysis and finally develop a risk assessment framework exclusively for remote health monitoring systems.

An Empirical Study of Domain, Design and Security

299

REFERENCES Alhirabi, N., Rana, O. F., Perera, C., & Rana, O. (2019). Designing Security and Privacy Requirements in Internet of Things: A Survey. Ebbs and Flows of Energy Systems (EFES) View Project 99 (Vol. 9). Bialas, A. (2019). Vulnerability assessment of sensor systems. Sensors (Switzerland), 19(11). https://doi.org/10.3390/s19112518 Brook, C. (2020, March 16). What is a Health Information System? | Digital Guardian. Retrieved July 29, 2020, from https://digitalguardian.com/blog/what-health-information-system Butt, S. A., Diaz-Martinez, J. L., Jamal, T., Ali, A., De-La-Hoz-Franco, E., & Shoaib, M. (2019). IoT Smart Health Security Threats. In Proceedings – 2019 19th International Conference on Computational Science and Its Applications, ICCSA 2019 (pp. 26–31). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ ICCSA.2019.000-8 Cagnazzo, M., Hertlein, M., Holz, T., & Pohlmann, N. (2018). Threat modeling for mobile health systems. In 2018 IEEE Wireless Communications and Networking Conference Workshops, WCNCW 2018 (pp. 31). Doi: 10.1109/WCNCW.2018.8369033. Godoi, B. B., Amorim, G. D., Quiroga, D. G., Holanda, V. M., Júlio, T., & Tournier, M. B. (2019, November 1). Parkinson’s disease and wearable devices, new perspectives for a public health issue: an integrative literature review. Revista Da Associacao Medica Brasileira (1992). NLM (Medline). https://doi.org/10.1590/1806-9282.65.11.1413 Hassanalieragh, M., Page, A., Soyata, T., Sharma, G., Aktas, M., Mateos, G., … Andreescu, S. (2015). Health Monitoring and Management Using Internet-of-Things (IoT) Sensing with Cloud-Based Processing: Opportunities and Challenges. In Proceedings – 2015 IEEE International Conference on Services Computing, SCC 2015 (pp. 285–292). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ SCC.2015.47 Mithun, H. T., Naveen, R., Ananthanarayanan, V., & Rajeswari, A. (2013, January). Reliable and affordable embedded system solution for continuous blood glucose maintaining system with wireless connectivity to blood glucose measuring system. In IJCA Proceedings on Amrita International Conference of Women in Computing-2013 AICWIC (2) (pp. 36–43). DOI: 10.5120/9872-1314. Ohta, H., Izumi, S., & Yoshimoto, M. (2015). A more acceptable endoluminal implantation for remotely monitoring ingestible sensors anchored to the stomach wall. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Vol. 2015-November, pp. 4089–4092). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/EMBC.2015.7319293 Peck, M. (2011). Medical Devices Are Vulnerable to Hacks, But Risk Is Low Overall - IEEE Spectrum. Retrieved July 29, 2020, from https://spectrum.ieee.org/biomedical/devices/ medical-devices-are-vulnerable-to-hacks-but-risk-is-low-overall Riel, A., Kreiner, C., Macher, G., & Messnarz, R. (2017). Integrated design for tackling safety and security challenges of smart products and digital manufacturing. 66(1), 177–180. https://doi.org/10.1016/j.cirp.2017.04.037ï Sinhasane, S. (n.d.). Remote Patient Monitoring: Benefits, Challenges, and Applications. Retrieved July 29, 2020, from https://mobisoftinfotech.com/resources/blog/ remote-patient-monitoring-benefits-challenges-and-applications/ Weeks, W. A., Dua, A., Hutchison, J., Joshi, R., Li, R., Szejer, J., & Azevedo, R. G. (2018). A lowpower, low-cost ingestible and wearable sensing platform to measure medication adherence and physiological signals. In Conference Proceedings : … Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2018, 5549–5553. https://doi.org/10.1109/EMBC.2018.8513593

11

IoT Security and Privacy Issues A Game of Catch-Up Atheer Almogbil

CONTENTS 11.1 Introduction to the Internet of Things (IoT).................................................. 301 11.1.1 What is IoT?....................................................................................... 301 11.1.2 There’s no “I” in “Team”...................................................................302 11.1.3 IoT Architecture.................................................................................302 11.2 IoT Applications in the Real World...............................................................304 11.2.1 Healthcare Sector...............................................................................304 11.2.2 Government Sector............................................................................ 305 11.3 The IoT Advantage........................................................................................306 11.4 IoT Security and Privacy Issues.....................................................................307 11.4.1 IoT Security Issues.............................................................................308 11.4.2 IoT Privacy Issues.............................................................................. 311 11.5 Potential Solutions......................................................................................... 312 11.5.1 Consumers......................................................................................... 312 11.5.2 Governments...................................................................................... 313 11.5.3 Manufacturers.................................................................................... 313 11.6 Summary....................................................................................................... 315 References............................................................................................................... 315

11.1  INTRODUCTION TO THE INTERNET OF THINGS (IoT) 11.1.1 What Is IoT? The term IoT is on the tip of everyone’s tongue and spoken about globally. You may have heard of this acronym once or twice before, may have taken an online course about it or attended a conference that revolved around it. IoT stands for the “Internet of things”. The term was coined by Kevin Ashton in 1999 as a network of physical devices communicating with each other via the Internet. You may have come across an IoT device, whether you know it or not. Any device you own that has Internet-connecting capabilities as well as the ability to share information with similar Internet-connected devices falls under the IoT category. 301

302

Artificial Intelligence and Internet of Things

This ranges from simple everyday devices that you use, such as smartphones and laptops, to smart kitchen appliances and door locks. Numerous devices that were once basic household appliances are now a part of an interconnected network of smart devices computing, storing and exchanging information. You may be wondering, Is my refrigerator (that connects to the Internet and sends me notifications of groceries I need to buy) an IoT device? The answer is yes, it is! Life, as we know, is changing in front of our eyes as IoT devices are improving our standard of living by automating and facilitating tasks, i.e., not having to write a list of groceries for our next shopping trip.

11.1.2 There’s No “I” in “Team” Now that you have a good sense of what IoT is, let’s move on to the nitty-gritty details of what an IoT environment system entails. To start off, you must understand that IoT devices are meant to connect and collaborate with other IoT devices to collect and exchange data. IoT devices can work well on their own; however, it is better to introduce your devices to each other and allow them to reach their full potential by working as a team of interconnected devices. This interconnection of IoT devices creates an entire IoT system. An IoT system is a network of connected IoT devices that exchange and collect data. Let’s consider a scenario in which you have a bottomless list of chores and vacuuming and mopping the floor are the very last things you want to do. You find out that there a robot is able to connect to your Wi-Fi and digitally map out your home’s floor plan to help ensure it covers every inch of your home, but it is not able to mop the floor. You also find out that there is another robot that mops floors but does not vacuum. Now you may be thinking, which one should I buy? Well, the answer is You’re asking the wrong question! In a world in which IoT prevails, you would get both IoT devices, connect them to each other and program the robot vacuum to start at noon, while you are work or out for coffee. Then, program the robot vacuum to tell the robot mop that it has finished, and it is okay for it to start mopping the floor. Better yet, buy a robot vacuum and mop that already have those configurations programmed! By doing this, you have set up your own IoT system at home.

11.1.3  IoT Architecture Similar to any system or device, an IoT device has several parts that work together to help it do what it is meant to do! It consists of embedded components such as actuators, sensors, software and network connectivity. An actuator is a component responsible for controlling the device or system after it receives a signal to do so. A sensor is a component responsible for “sensing” environmental changes, i.e., motion, temperature and heartbeat. Software is a set of computer instructions that dictate what the sensors and actuators do, and when to do it. You can think of it as the brain of the IoT device. It is in charge of sending signals to the actuator based on the results it receives from the sensors. Last but definitely not least is the network component. The network is the medium that allows the IoT devices to connect and communicate to each other. This component is important as network connectivity is what sets IoT devices apart from “regular” non-IoT devices (Thilakarathne, 2020). Do not worry

IoT Security and Privacy Issues

303

FIGURE 11.1  Components of an IoT device

if you do not understand how all of the components work together. It will become clearer as you learn more about IoT architecture. An IoT system generally has a three-layer architecture: perception layer, gateway layer and cloud layer. Although every IoT system is different, these three layers are the basic building blocks of an IoT system. Other layers can be added for additional measures. For instance, an edge layer can be added before the cloud layer to provide enhanced data analytics and quicker response times and data processing. The first layer, the perception layer, is the layer in which data is collected. This layer consists of the aforementioned sensors and actuators. The sensors in this case help the IoT devices in populating data from the outside environment. The actuators then act upon the data retrieved by the sensors (AVS System, 2020). For instance, consider a smart cooling system. You set your desired room temperature to be a cool 68°F. The sensors pick up that the room’s current temperature is 76°F, which causes the actuators to increase the flow of cool air to try to bring down the temperature to 68°F. Once the sensors pick up that the room’s current temperature is 68°F, the actuators will decrease the flow of cool air. It is a constant back and forth process between the sensors and actuators to get your home to that perfect temperature you desire. The second layer, the gateway layer, allows for IoT devices to connect to other devices. Additionally, this layer works closely with the perception layer to enable transmitting and processing data from sensors and other devices to the cloud layer. The gateway layer preprocesses data collected by sensors before sending it to the cloud layer. It converts, filters and reduces the sensor data into a format that facilitates the transmission and usability of the data, which in turn reduces the cost of network transmission and response times (AVS System, 2020). The preprocessed data can be further processed by adding another layer between the gateway and cloud layers. Sitting between the perception and cloud layers, the gateway layer ultimately controls information flow between the layers. Accordingly, appropriate security mechanisms must be placed to minimize or prevent data leaks and potential attacks.

304

Artificial Intelligence and Internet of Things

FIGURE 11.2  IoT three-layer architecture

Possible security and privacy issues that can be faced in this layer will be discussed in detail throughout this chapter. The third layer, the cloud layer, is responsible for storing, processing and analyzing the preprocessed data (AVS System, 2020). After processing the data, this layer offers a better representation and understanding of said data to individuals through the use of data analytics. By better understanding the data at hand, individuals can more confidently carry out informed decisions based on real-time data. The cloud layer is what allows businesses to monitor and control the IoT system, further enhancing the value and benefits of the data collected by the IoT devices. Based on the logic behind an IoT system’s application and intended use, additional layers and components can be added to the basic IoT three-layer architecture. Nonetheless, a solid IoT architecture must provide vital features required to maintain a sustainable IoT system. These features are flexibility, availability, maintainability, reliability, security, privacy, interoperability, Quality of Service (QoS), functionality, scalability and cost-effectiveness (AVS System, 2020).

11.2 IoT APPLICATIONS IN THE REAL WORLD IoT applications are not limited to home appliances only. They can be implemented in various industries to help automate and improve processes. A few IoT applications in diverse sectors will be discussed in this section. Nevertheless, the possibilities of IoT applications are endless.

11.2.1  Healthcare Sector One the most prominent industries that have adopted and implemented IoT technology is the healthcare sector. You may have noticed that numerous IoT technologies

IoT Security and Privacy Issues

305

are targeting healthcare as there have been limitless ways in which IoT has done and will do to improve patient care, monitor vital signs and aid in diagnosis. The reason behind healthcare’s adoption of IoT technology is IoT’s reliability and quick analysis of real-time data, as well as the reduction of unnecessary costs provided by using such technology. In the past, a healthcare provider could only monitor their patient’s health during visits or by check-ups over the phone. This limited the provider’s ability to continuously monitor their patient’s health, in turn depriving the patient of recommendations and treatments based on real-time data. IoT has made it possible for doctors and physicians to remotely monitor patients on a continuous basis, improving patientdoctor interaction and engagement. A patient can wear a fitness wearable device or any health monitoring device that populates data regarding the user’s heart rate, blood pressure and glucose levels. This data is sent in real time to the patient’s physician, who can use the information to closely monitor the patient and create a better visualization of the patient’s progress or lack thereof. This provides patients with a higher quality of care, while unleashing a healthcare provider’s full potential to help treat and diagnose their patients in an efficient manner. IoT applications have improved many aspects in the healthcare industry ranging from patients and physicians to hospitals and medical insurance companies. For instance, IoT devices can be used to improve inventory control in hospitals and monitor and control the hospital’s environmental conditions (Karjagi & Jindal, n.d.). Furthermore, innovation in the field of IoT healthcare devices and systems has proven to be boundless. Consequently, the healthcare sector is one of many that has gained countless benefits from implementing IoT technology.

11.2.2 Government Sector Governments around the world are beginning to realize the benefits of implementing IoT technology. One such benefit is the use of IoT to better improve government resources and budget allocations. City councils are responsible for planning and distributing funding for public amenities and services, as well as the maintenance of such areas and services. Mismanaging resources and misdirecting money to a park rather than a public school in dire need of renovation and improvements can affect the city’s economy and resident satisfaction. IoT solutions can be used to advise city officials on how to properly distribute resources across various areas by taking into account the number of people that visit that location or attend certain schools. This will help officials understand and prioritize areas that are in dire need of funding and improvements. Moreover, in the past, if the town council wanted to proceed with roadwork without causing traffic congestion, a lot of time and effort had to be put into choosing the greatest time slot to do so. The tasks of collecting data, processing the data and analyzing the data were manually done by individuals. For instance, individuals were required to manually collect information on how many cars used a certain road throughout the day. This required a person, or multiple people, to sit at the specified road and manually record traffic logs for several weeks at a time. The logs were then analyzed manually to find a time period in which traffic was minimal. This time slot was then assigned to construction workers to complete road construction. Otherwise,

306

Artificial Intelligence and Internet of Things

FIGURE 11.3  Examples of industries that have adopted IoT technology

a major traffic congestion can be caused by the roadwork construction and result in undesirable outcomes such as traffic delays and accidents. IoT can have a great impact on reducing the time, cost and effort spent on finding the optimal time periods roadwork constructions can take place, with little to no impact on traffic. This can be done by implementing detection mechanisms that record and monitor traffic then display findings and logistics on a dashboard in a user-friendly manner. City council members will be able to easily make wellinformed decisions based on the presented data with little-to-no effort, allocating more time and effort to much more pressing matters. All in all, several industries have already joined or are joining the IoT train. The applications mentioned in this section are only a few examples of how industries can use IoT technology to solve challenges or even create new opportunities. By comparing how things were done before IoT and how they can be done using IoT, the benefits of the latter are very prominent. The following section will mention several benefits individuals, businesses and other parties can gain from the implementation of IoT technology.

11.3  THE IoT ADVANTAGE The examples of IoT applications from the previous section all share something in common: the parties invested in implementing IoT technology today to reap benefits tomorrow. The advantages of using IoT are numerous, but the main advantages a person or organization will gain include, but are not limited to, the following aspects: 1. Costs – Operational costs and expenses are reduced by automating processes which can replace multiple expensive professionals.

IoT Security and Privacy Issues

307

2. Decision-making – Data is visualized in a user-friendly manner, which improves analysis and reporting, hence improving decisions made. 3. Data collection and analysis – Large volumes of data can be collected and analyzed in an efficient manner, without hindering a system’s performance. 4. Real-time monitoring – Users can now monitor what was formerly unmonitorable on a continuous basis in real time to improve productivity, efficiency and effectiveness, in turn increasing business value. 5. Data accessibility – Data is processed and stored in the cloud, making the data remotely accessible to authorized users from anywhere across the world. This facilitates the sharing of information and improves collaboration and productivity among teams. These are just a few ways in which IoT can enhance an organization or business. Simplifying and automating what needs to be done is what IoT does best. Now, you may have realized why so many industries around the world are buying into IoT. Not only can IoT lower costs, but it can also provide a hands-off solution to most challenging problems in a timely manner. Although IoT may seem like the answer to all your problems, there are some issues that IoT devices and systems may present, which can cause great damages and major losses.

11.4 IoT SECURITY AND PRIVACY ISSUES Imagine a world where IoT devices can happily connect to one another creating a massive IoT network without any problems. It is a world where data can be transmitted and processed from around the world without any security or privacy concerns. Unfortunately, that ideal is just a mere figment of our imagination. Instead, the real world of IoT is made up of a constant fear of data exfiltration, data leaks and numerous cyber-attacks… Need I say more? IoT is meant to provide solutions to problems and help people live a convenient life; however, there are some instances in which IoT was the source of the problem. In 2016, North America and Europe experienced a massive distributed denial of service (DDoS) attack which hindered accessibility to major social media platforms and government services causing multiple outages. It was later discovered that the outages were caused by traffic coming from baby monitors, printers, cameras, even electric gates. The Mirai malware compromised several IoT devices around the globe that had a Linux operating system, creating a network of bots that was remotely controlled to conduct major network attacks. The malware was able to target insecure IoT devices using default authentication credentials, compromising them without alerting the unsuspecting owners of the devices. This unfortunate event showcases the importance of accounting for security vulnerabilities in any IoT device, even something as little as a smart doorbell can wreak havoc among nations. In 2015, a Jeep Cherokee’s smart system was hacked remotely using a zeroday exploit. A zero-day exploit is one in which hackers exploit a vulnerability that is unknown to the developers. The attackers were successful in cutting the car’s brakes while it was going 70 mph. The vulnerability exploitation additionally gave the attackers remote control of the Jeep’s acceleration, steering wheel, radio, air

308

Artificial Intelligence and Internet of Things

conditioning and windshield wipers. Fortunately for the driver, the attack was an experiment conducted with the driver’s knowledge to test whether or not attackers can remotely control a car. This experiment illustrates the importance of implementing security controls and mechanisms in everything, as the lack of security can cause a fatal accident (Greenberg, 2018). Changing technology requires security experts to continuously try to keep up with new liabilities. These liabilities must be accounted for in a time-efficient manner, as to not put people’s lives at risk. Whether an IoT device is just a baby monitor or something as big as a car, security should not be overlooked. CISCO predicts that in 2022, there will be 28.5 billion IoT devices (CISCO, 2018). The examples given above illustrate how attackers can weaponize the IoT devices and cause a devastating outcome. Imagine if all 28.5 billion devices had security vulnerabilities that attackers would be able to exploit to create an army. I would not want to live in that version of the world. As an emerging trend, IoT has defined the way our future will look like. Real-time analysis, remote monitoring, enhanced decision-making and automated processing can benefit various industries, as well as governments. However, being connected to the Internet allows a plethora of vulnerabilities waiting to be exploited by hackers across the globe. Imagine something as powerful and essential as a country’s power grids being disrupted. Mere minutes of this disruption can cause fatalities, considering that everything depending on electricity will be shutdown. The country will be left defenseless, vulnerable to any attack whether through sea, air, land or cyberspace. Accordingly, one may simply wonder why create such vulnerable devices or systems in the first place? Well, the answer is that developing an IoT device with maximum security is costly; therefore, IoT security is usually considered an add-on feature that is more of an after-thought rather than an essential aspect of the IoT device’s design. Inspecting current IoT literature will show a theme of a game of catch-up between security experts and IoT developers. Companies are continuously pushing out new IoT products into the market, and security experts are constantly experimenting and analyzing these products to show how easily sensitive information can be stolen, or how the devices can be weaponized. Consumers are left to decide for themselves whether the benefits of using an IoT device or system, such as a smart home, outweigh their concerns for privacy and security. Regrettably, it is often left to consumers to educate themselves of the issues in order to implement mechanisms to protect their data. It is logical to consider that privacy stems from comprehensive security and that security stems from the need to maintain privacy. For this section, we will use Norton’s definition of security as the protection of confidential information from unauthorized access, and privacy as the right to control who has access to your personal data and how your data is used (Gervais, 2020). This section will discuss major security and privacy issues in IoT devices.

11.4.1  IoT Security Issues One of the main issues that challenge IoT development is security. As every device requires different security needs, simply implementing a uniform security solution

IoT Security and Privacy Issues

309

for all IoT devices is not applicable. Therefore, security mechanisms must be tailored to each type of device. This makes IoT security a tedious job, causing most developers to forgo required security controls for a simple username and password authentication. IoT security issues include, but are not limited to, authentication, identification and device heterogeneity. This section will map out security issues to the architectural layer in which they exist. The perception layer is the lowest layer in the IoT architecture and consists of devices, sensors, actuators and radio frequency identification (RFID). As this layer is made up of several components, potential security risks exist due to the vulnerabilities found in each device within this layer. Some concerns include (Thilakarathne, 2020): 1. Denial of Service (DoS) – Sensors and other devices within the perception layer have limited computation capabilities and capacity. This makes it fairly easy for attackers to conduct a DoS attack, resulting in disrupted service. 2. Brute force – This attack is one in which an exhaustive search is conducted to guess possible combinations of credentials, i.e. username and password, to gain access to a device, account or network. Since the devices within the perception layer have low computation power, a brute-force attack on the devices is feasible. 3. Malicious node insertion – This attack consists of inserting a fake, malicious node among actual nodes to gain access to and control of the IoT network. 4. Hardware jamming – Unlike a DoS attack, a hardware jamming attack is when an attacker physically alters a node’s hardware components, thus damaging it and disrupting service. 5. Embedded passwords – A majority of IoT devices have embedded passwords stored to help technicians in remotely troubleshooting or installing updates. If the passwords are vulnerable or unencrypted, it may be utilized by hackers to gain control of the device. 6. Default credentials – IoT devices usually are given default credentials, i.e. username and password, that are meant to be changed by users once in their possession. Regrettably, it is common for users to keep the default credentials as is. This increases the chances of a hacker gaining access to the device by iterating a list of commonly used usernames and passwords such as “admin”. The gateway layer is the middle layer and it acts as a gateway between the perception and cloud layers. Security issues may arise as this layer is responsible for securing the communication between the devices and the cloud. These risks include the following (Thilakarathne, 2020): 1. DoS – Network connectivity makes this layer susceptible to DoS attacks that can hinder both devices and servers in the lower and higher layers, respectively.

310

Artificial Intelligence and Internet of Things

2. Session hijacking – This is a type of phishing attack in which an attacker obtains the session ID or token, enabling them to take control of a session. By doing so, the attacker has now obtained network access. 3. Man-in-the-middle – As communication between two nodes is being transmitted, an attacker can intercept it and steal the information being transmitted. Furthermore, if the traffic is not encrypted, the attacker will easily be able to read the obtained information. Being the top layer, the cloud layer offers users a means of controlling and monitoring the IoT devices. Common security issues viewed in cloud-based services are present in this layer. Some of these security issues include the following: 1. Data breach – A data breach occurs when external, unauthorized entities exfiltrate, transmit or even view confidential data. This is an intentional and malicious attack on a system that maintains and stores sensitive data, i.e. credit card numbers. Attacks such as SQL injection and cookie poisoning can lead to data breaches. 2. Data leak – This term is often used interchangeably with data breach; however, there is a slight distinction between the two terms. A data leak is often unintentional and not malicious and mostly caused by human error. It occurs when information is transmitted to external, unauthorized entities, i.e., an employee accidentally attaching sensitive documents to an email being sent to an authorized individual. 3. Insider threat – An individual that currently works for, or had worked for, an organization and has malicious intention poses as a threat to the organization’s security. This includes, but is not limited to, current or former employees, contractors and partners. An individual having unmonitored access to the cloud can steal confidential information and alter data, while remaining unnoticed. 4. Application vulnerability – Applications hosted on the cloud layer for user interface purposes can increase cyber risks by introducing unaddressed vulnerabilities and multiple access points that can be exploited. Attackers can gain access to the cloud network using methods such as backdoors, address resolution protocol (ARP) poisoning or cross-site scripting (XSS). 5. Malware – Introducing malware to the cloud layer can exponentially increase the spread of the malware across the network and to the IoT devices, hindering service and holding the sensitive data for ransom. IoT devices have limited computation power, memory and battery life. Seeing as security controls are resource expensive, it is clear why developers choose the easy route rather than discover ways to implement security while accommodating device capabilities. For a list of updated IoT security vulnerabilities, check out the Open Web Application Security Project (OWASP).

IoT Security and Privacy Issues

311

11.4.2  IoT Privacy Issues In our world today, many people enjoy sharing personal information online such as a post ranting about how hard work was today, a video of them and their friends hanging out, a picture of what they had for lunch, and so on. Sharing personal information with the world is the foundation that social media is built upon. Nevertheless, people do not view the actions mentioned previously as publicly sharing personal information. If social media users thought of the ease of someone stealing their identity or simply how effortless they have made online stalking become, they might not willingly post anything publicly ever again. One must keep in mind that anything that connects to the Internet poses privacy issues whether they are aware of it or not. Seeing as IoT devices connect to both the Internet and other devices, they are not exempted from the previous statement. Major IoT privacy issues can be categorized into the following four categories: 1. Abundant data generation – IoT devices generate data in unimaginable amounts at a high rate. The data collected by IoT devices within your household is not limited neither in quantity nor sensitivity. In a report titled “Internet of Things: Privacy & Security in a Connected World” by the Federal Trade Commission, it was found that 150 million data points are generated by less than 10,000 households (FTC, 2015). Just imagine all the possible attack points your devices have bestowed upon evil-doers. 2. Consumer behavior profile – Many companies that employ IoT devices use them to understand how their consumers behave to carry out decisions. For instance, a car insurance company can use the data collected by an individual’s car to understand their driving habits. This will help the insurance company in deciding what rate they should charge the individual, i.e., a higher insurance rate would be charged if bad driving habits were demonstrated when analyzing the car’s smart system. Although consumer profiles aid companies in assessing rates and fees, the same profile can be used to harm the consumer. For instance, data collected regarding a house hold’s electricity peaks throughout the day can indicate household members’ lifestyles, i.e., when they are likely to be awake or at home. If this information fell in the wrong hands, imagine how advanced a home burglary could be. 3. Unwanted solicitation – Data used to create the aforementioned consumer profiles are collected and stored in servers that span across the globe. This makes it difficult to keep track of who has your data and what is being done with it. Unfortunately, some businesses use this to their advantage and sell consumer information to other companies for monetary incentives. You may be wondering what value personal data has, and the answer is that it helps businesses better target market segments by understanding which products and services they should offer specific individuals in that segment. This is reason behind the unsolicited promotions and advertisement you get in your mail, email and text messages.

312

Artificial Intelligence and Internet of Things

4. Eavesdropping – Anything and everything connected to the Internet is highly vulnerable of being remotely accessible by unauthorized individuals. Nowadays, hackers are more tech savvy than they were before, making it comparably easier to virtually invade homes. This is evident in the fact that there are numerous cases in which a hacker was able to maintain access and control of IoT devices within an unsuspecting individual’s home. Similar to points two and three, this is yet another confidentiality issue. For instance, one such case is a viral video in which a hacker is speaking to a toddler through a baby monitor. Consider how many devices you own that have a microphone, camera and GPS-tracking capabilities. Can you say that you know for a fact that nobody has access to them but you? IoT devices were created with the purpose of interconnecting and communicating with other IoT devices, although, due to vulnerabilities instigated by security being an after-thought, security and privacy will always be a concern until considered a vital aspect in the IoT development life cycle. According to Statista (2016), there will be more than 75 billion IoT devices by 2025. Envision all the data being collected by the billions of IoT devices and consider the cyber risks discussed previously. Insecure IoT devices around the globe would collectively create a ticking time bomb.

11.5  POTENTIAL SOLUTIONS For IoT to proceed in making a great impact on the world, consumers must be able to trust the devices to allow them to achieve their full potential. Nowadays, consumers are more aware of the discrepancies in IoT security and how these discrepancies threaten their personal privacy and safety. The public’s perception of IoT matters in whether or not the adoption of IoT will progress across the world and at what rate it will do so. It is vital that consumers’ concerns regarding the security and privacy issues mentioned previously are addressed. One way to address IoT security and privacy is to view how different entities can contribute to better securing IoT devices. This section will discuss what consumers, governments and manufacturers can do to attain a world with secure IoT devices.

11.5.1 Consumers Consumers, whether individuals, businesses or even government entities, currently carry the responsibility of applying their own security measures to safeguard their IoT devices. This is problematic in the sense that IoT developers do not feel pressured neither by law nor by consumer demand to accommodate security in the development of IoT devices. Nevertheless, this state of passing the responsibility to consumers is coming to an end as consumers are beginning to educate themselves on the numerous security and privacy issues that arise from developers’ lack of interest in securing the devices. Consumers are always told to create strong passwords and change default configurations to ensure the security and maintain the privacy of their data; yet, this

IoT Security and Privacy Issues

313

can only go so far. It is now time for consumers to pass the responsibility back to the developers and to the government by demanding that security and privacy concerns are addressed before IoT devices are introduced to the market. Consumers must push the government to pass IoT security laws and publish standards and measures that companies must comply to. Additionally, consumers must pressure companies to adhere to these laws and recommended standards released by federal institutes, such as the National Institute of Standards and Technology (NIST) and the International Organization for Standardization (ISO). Moreover, consumers can ask companies if and how they have addressed security and privacy vulnerabilities published in OWASP’s list of vulnerabilities. This list includes hardcoded passwords, insecure update mechanisms, outdated components, insufficient privacy protection, insecure network services and unencrypted data storage and transmission. A more comprehensive list can be viewed on OWASP’s website at owasp.org.

11.5.2 Governments Governments are responsible for the safety and security of their citizens, even when it comes to cybersecurity. They are the entities that have the power to take action and pass laws that have the ability to enforce the compliance of companies in merging security into IoT devices. Australia and the United Kingdom have taken measures in providing IoT security standards that companies should adhere to. For instance, the United Kingdom requires that manufacturers provide means for reporting vulnerabilities. In the United States, both California and Oregon have made it compulsory for companies to provide an adequate level of security in IoT devices. Additionally, the United States’ FTC and the European Union released guidelines that increase the responsibilities and liabilities of failing to implement protective security measures. Mandating security is the first step in what can become a major change to the way governments develop and progress IoT security legislation. Furthermore, across the globe, a new trend of regulating IoT is emerging. Although difficult, the end goal of protecting consumers and national security is highly valuable as this will support and sustain the growth of IoT. Following the guidance of security researchers and experts, governments can pass laws that require manufacturers develop IoT devices in compliance to global or national security standards and to promote cybersecurity practices. Accordingly, IoT devices will be introduced to the market with security built-in, which will improve consumer perception of the trustworthiness of said devices.

11.5.3 Manufacturers Sensing the demand for secure IoT devices from the consumers and the government, developers will be forced to take on the challenge of implementing security in the resource-limited devices. With clear guidelines and standards to guide them, developers will have to check off the minimum requirements needed to comply for their devices to be introduced to the market. Better yet, this might push developers to compete in the extent at which their devices are secure!

314

Artificial Intelligence and Internet of Things

Under normal circumstances, the prevalent confidentiality, integrity and availability (CIA) triad might have sufficed when it came to guiding information security policies. Though IoT devices are unlike any other devices the world had dealt with before. Therefore, just considering CIA will not successfully address the novel risks or threats that may exist within an IoT environment. Due to this, a more comprehensive set of security goals called the information assurance and security (IAS) octave is recommended for dealing with IoT devices. The IAS octave fills the gap created by the CIA triad by further introducing accountability, non-repudiation, privacy, trustworthiness and auditability (Cherdantseva & Hilton, 2013). Table 11.1 defines the meaning and illustrates examples of each IAS-octave security requirement. TABLE 11.1 IAS-Octave Security Requirements Security Requirement Confidentiality

Definition Ensuring only authorized individuals have access to data

Integrity

Preserving data accuracy and completeness

Availability

Being readily available and accessible when authorized users require services Tracing actions to a specific individual responsible for said action

Accountability

Non-repudiation

Validating the occurrence of, or lack thereof, an incident or event

Privacy

Allowing users to control their own data

Trustworthiness

Confirming the identity of users and third-parties; considered an extension of confidentiality requirement Continuous monitoring and logging of network and user activity

Auditability

Examples Encrypting data so only intended recipients can decrypt and obtain the data Digitally signing data using hashing methods prevents message alteration while in transmission Fail-over and disaster recovery backups help maintain access to data when needed in critical times Employees are not allowed to download software onto company machines, if not followed, the individual who performed the action must be identified and held accountable Digitally signing a message when opened can prove that the intended recipient has seen the message and prevents them from denying so A list of authorized users in which the user wants to share or limit access to information Continuous authentication throughout sessions

Constant maintenance, review and analysis of security of IT infrastructure will identify vulnerabilities and help mitigate risk of cyber attacks

Source: Information obtained from Cherdantseva & Hilton (2013).

IoT Security and Privacy Issues

315

11.6 SUMMARY The Internet first started as a limited interconnected network of computers and has now grown into a global network consisting of interconnected devices that collect and exchange information. The advances made toward the Internet have given way to IoT, a network of smart devices connected to the Internet that can exchange, compute and store information. IoT has changed the way the world perceives and uses technology. Microwaves, refrigerators, dog collars, vehicles, thermometers, pacemakers and other common appliances, devices and machines have been converted into IoT devices to improve the way we live, work and innovate. Although the benefits of adopting IoT technology are countless and undoubtable, there are some major concerns regarding IoT security and privacy. These issues are due to device resource limitations, as well as companies’ lack of motivation to build security within IoT devices. Unfortunately, the security vulnerabilities found in IoT devices and systems have made them a prime target for hackers across the worldwide. Symantec reported that the number of attacks on IoT devices from 2016 has skyrocketed nearly eightfold in 2017. The report additionally states that in 2018, the number one password used by attackers to gain access to IoT devices was “123456” and the second most common password was the lack of one (Davis, 2019). Knowing this, it is evident that users may not understand the extent to which their privacy and security is at risk. Personal and sensitive data is collected and shared across multiple devices and stored in servers spread across multiple countries. Malicious entities, which are able to gain access to insecure data, can threaten and risk individual, business and national security. It is up to consumers, governments and manufacturers to realize this threat and to take action before the world is consumed by billions of vulnerable IoT devices.

REFERENCES AVS System. (2020, May 11). What is IoT Architecture? Retrieved July 09, 2020, from https:// www.avsystem.com/blog/what-is-iot-architecture/. Cherdantseva, Y., & Hilton, J. (2013). A Reference Model of Information Assurance & Security. In 2013 International Conference on Availability, Reliability and Security. doi:10.1109/ares.2013.72. CISCO. (2018). VNI Complete Forecast Highlights. Retrieved July 05, 2020, from https:// www.cisco.com/c/dam/m/en_us/solutions/service-provider/vni-forecast-highlights/ pdf/Global_2022_Forecast_Highlights.pdf. Davis, D. B. (2019, April 4). ISTR 2019: Internet of Things Cyber Attacks Grow More Diverse. Retrieved July 10, 2020, from https://symantec-enterprise-blogs.security. com/blogs/expert-perspectives/istr-2019-internet-things-cyber-attacks-grow-morediverse?om_ext_cid=biz_social3_AMS_NAM-IV_twitter_. FTC. (2015, January). Internet of Things: Privacy & Security in a Connected World. Retrieved June 20, 2020, from https://www.ftc.gov/system/files/documents/reports/ federal-trade-commission-staff-report-november-2013-workshop-entitled-internetthings-privacy/150127iotrpt.pdf. Gervais, J. (2020). Privacy vs. Security: What’s the Difference? Retrieved July 07, 2020, from https://us.norton.com/internetsecurity-privacy-privacy-vs-security-whats-thedifference.html.

316

Artificial Intelligence and Internet of Things

Greenberg, A. (2018, November 20). Hackers Remotely Kill a Jeep on the HighwayWith Me in It. Retrieved July 11, 2020, from https://www.wired.com/2015/07/ hackers-remotely-kill-jeep-highway/. Karjagi, R., & Jindal, M. (n.d.). IoT in Healthcare Industry: IoT Applications in Healthcare. Retrieved June 14, 2020, from https://www.wipro.com/en-US/business-process/ what-can-iot-do-for-healthcare-/. Statista. (2016, November 27). IoT: Number of Connected Devices Worldwide 2012– 2025. Retrieved July 05, 2020, from https://www.statista.com/statistics/471264/iotnumber-of-connected-devices-worldwide/. Thilakarathne, N. N. (2020). Security and privacy issues in IOT environment. International Journal of Engineering and Management Research, 10(01), 26–29. doi:10.31033/ ijemr.10.1.5.

12

Internet of Things for Mitigating Climate Change Impacts on Health Rehab A. Rayan, Imran Zafar, Christos Tsagkaris, Iryna Romash

CONTENTS 12.1 Introduction................................................................................................... 317 12.2 Major Climate Change Impacts on Health.................................................... 319 12.3 Health IoT...................................................................................................... 321 12.4 Examples of IoT-Based Environment Solutions............................................ 322 12.5 Limitations..................................................................................................... 324 12.6 Future Insights............................................................................................... 325 12.7 Conclusions.................................................................................................... 325 References............................................................................................................... 326

12.1 INTRODUCTION After the 2015-millennium development goals (MDG), the United Nations (UN) introduced sustainable development goals (SDGs) to tackle sustainably health, climate, and ecological problems for fostering the future ecosystem for better health in a sustainable community (Martin, n.d.; UNDP, n.d.). Sustainable health in a community requires following systematic approaches in investigating the effects of climate change and designing indicators, frameworks, and measures to observe and interpret various risk factors on health, hence ensuring the growth of the society. Therefore, integrating IoT with tools for sensing, communicating, monitoring, and decisionsupport is needed for sustainable health. Climate changes threaten significantly and diversely human health now and maybe for longer times (Bell et al., 2007; Shonkoff et al., 2011), for example, the rapidly ever-changing ecological emissions, ozone exposure, temperature, air, and water quality among other weather-related events (Ebi & McGregor, 2008). The growing carbon dioxide levels elevated temperatures associated with modifications in plant life, including induction of allergens (Idso & Idso, 2001). The growing emissions elevated temperatures and sea levels, altered precipitation patterns, and hence led to 317

318

Artificial Intelligence and Internet of Things

FIGURE 12.1  Climate change effects on health

a more extreme climate. Polluted surface water and food resources and degraded air quality had provoked disease spread, hence additional challenges for human health (Maibach et al., 2010). Figure 12.1 shows the different effects of changing climate on health, which results in worsening existing conditions and creating additional needs (USGCRP, 2016). The extent of the extreme climate effect on human health differs both temporarily and spatially (Fewtrell & Kay, 2008; Nitschke et al., 2011) where it strikes mostly youngsters, elderly, poor, and ill subjects (Balbus & Malina, 2009; Sheffield & Landrigan, 2011). Fungus and mold, heat waves, extreme temperatures, and precipitation are associated with indoor air quality (Committee on the Effect of Climate Change on Indoor Air Quality and Public Health, 2011; Elliott et al., 2013; Fisk et al., 2007; Hajat & Kosatky, 2010). While the indoor moist environment elevated the symptoms of the upper respiratory tract and asthma prevalence (Akinbami et al., 2011; Fisk et al., 2007), the high temperatures increase atmospheric pollutants and allergens, and hence growing rates of hospital admissions, emergency room (ER) visits, and moralities of children because of asthma (Anderson & Bell, 2011; Delfino et al., 2009; Wolf et al., 2010). Likewise, heavy precipitation derives serious floods, dangerous algae growth (Ahern et al., 2005; Gobler, 2020), and waterborne diseases. The growing wildfires adversely affect the outdoor air quality causing respiratory problems because of breathing smoke. Similarly, higher ozone levels badly affect outdoor air quality (Johnston et al., 2012; Li et al., 2018, pp. 1960–2010). Climate change has derived growth in several vectors for diseases such as ticks with Lyme disease (Keesing et al., 2009; Kolivras, 2010; Reiter, 2008; Semenza et al., 2012b) along with more water and foodborne diseases (Semenza et al., 2012a). Climate change is associated with poor mental health increasing stress, especially following crises and displacements (Hayes et al., 2018). To identify the risks of climate change in advance and, if possible, to minimize their consequences for the future of the planet, scientists all around the world are looking for ways to control the transformation of the environment. The latest trend

IoT for Mitigating Climate Change Impacts on Health

319

in the development of world industry is its movement to the level of “Industry 4.0” and the penetration of information technology in all aspects of human activity. One of them is the concept of the Internet of things (IoT). Connected to a global network, “smart devices” allow you to monitor and analyze the state of the ecosystem as a whole and to solve specific problems to eliminate the negative impact of human activities. Of the 17 global SDGs approved on Summit of the UN in 2015, most relate to environmental issues: directly – goal 13 – Combating Climate Change and indirectly – goals: 6, 7, 9, 11, 12, 14, and 15. Two years later, the National Intelligence Council (NIC), the center of mid- and long-term strategic thinking within the United States Intelligence Community (IC), identified seven global trends, and for following that, they must be resolved by 2035. And point 7 concerns climate change, the environment, and health issues. Its regulations state that more extreme weather conditions, water and soil problems, melting glaciers, environmental pollution, and food insecurity are problems that society will face in the near future and that will change living standards and living conditions. According to global forecasts, tensions over climate change will only increase, so action must be taken immediately and wisely (National Intelligence Council, 2017). This chapter describes the climatic effects on human health. It discusses detecting, communicating, and tracking techniques with ecological real-world applications. It also explores the influence of critical health and ecological elements on developing novel IoT technologies along with limitations and future potentials.

12.2  MAJOR CLIMATE CHANGE IMPACTS ON HEALTH The Montreal Protocol, signed back in 1978, is the first acknowledgment of climate change as a severe threat to humanity. The Protocol came into being grace to observational data from space, paving the way to international collaboration for the sake of the environment. More than three decades later, numbers speak for climate change, urging for a standpoint to move the world away from its consequences. The emission of gases has multiplied since 1990, the global temperature has increased by 1.0°C since 2017 and sea levels have risen by about 20 cm since 1880 (Velders et al., 2007). From the last months of 2019 onward, humanity has witnessed Australia and Siberia in flames. Local-scale yet unprecedented weather conditions have troubled the economy. The novel coronavirus disease (COVID-19) pandemic has a close link to the environment and since its beginning scientists come to realize that environmental deregulation pays back in human health. According to the World Health Organization (WHO), climate change affects human health by means of extreme heat, natural disasters, and variable patterns of rain as well as infectious diseases. Extreme heat consists of a considerable burden of respiratory and cardiovascular health, especially when it comes to elderly people (WHO, 2018). Chronic conditions such as asthma or chronic obstructive pulmonary disorders (COPD) may get aggravated due to the higher level of pollen and allergens in the atmosphere. At the same time, elderly patients who suffer from heart failure or are under diuretics are more prone to dehydration and electrolytic disorders. In this frame, it is not surprising that the heat wave of summer 2003 in Europe has been associated with no less than 70,000 excess deaths (Robine et al., 2008).

320

Artificial Intelligence and Internet of Things

Natural disasters inflict short-term and long-term consequences. Their number has followed an increasing pattern globally since the 1960s. When a tsunami or a hurricane lands on an area, homes, essential services, and medical facilities are destroyed. Large populations are forced to relocate surviving in poor hygiene and minimal resources conditions for a long time. The consequences of natural disasters result in over 60,000 deaths every year. The related mortality and morbidity is higher in developing countries (Yang et al., 2005a,b; Tcm et al., 2018). Rainfall pattern deregulation can have devastating effects on health as well. Communities rely on rainfalls, in order to cover their hydration and hygienic needs. The primary sector of financial activity, and especially farming and agriculture greatly depends on rainfalls as well. Every year it is estimated that half a million children die because of diarrheal disease due to lack of clean water (Yang et al., 2005a,b). Apart from the lack of water, floods can contaminate freshwater supplies, devastate residencies, or create breeding grounds for disease-carrying insects, increasing the risk of infection. Recent evidence suggests that rainfall patterns may be further disturbed till the end of the 21st century resulting in droughts and famine. Currently, malnutrition or undernutrition leads to about 3.1 million deaths every year, and this figure is expected to increase (Martiello et al., 2008). Waterborne diseases mainly arise from viruses, bacteria, protozoa along with harmful algae, toxins produced by cyanobacteria, and man-induced chemicals via ingestion or inhalation of polluted water, consuming contaminated seafood, and doing recreational activities in polluted water (Benedict et al., 2017). The waterborne diseases’ risk grows and disseminates according to various climatic conditions like storms, rains, floods, and tornados coupled with subjective adaptation capabilities (Semenza et al., 2012a). Universally, the availability of safe drinking water is highly challenging billions of people lacking access to it. In cities, waterborne diseases are a leading problem where most of the gastrointestinal illnesses are resulting from the seriously contaminated surface water and wells by several microorganisms. The rising temperatures worldwide promote the growth of blooms of poisonous algae that threaten health (USGCRP, 2016). Concerns about climate-change-related infections have been raised because of COVID-19. Although many questions are yet to be investigated and answered in this field, numerous infectious diseases have close ties with climate change. We have already elaborated on waterborne infections associated with poor sanitation especially in the developing word. As a matter of fact, the 6th Goal of the UN’s vision of sustainability focuses on water and sanitation. In China, snail-borne schistosomiasis has become more widespread and prevalent because of recent climatological alterations (UN, n.d.). At the same time, climate change seems to favor the development of insects such as the Anopheles and the Aedes mosquitoes – vectors of malaria and dengue fever accordingly. Malaria is currently accounting for the deaths of more than 400,000 people per year in Africa, and there are fears that this figure may keep increasing (Tcm et al., 2018; Yang et al., 2005a,b). The growing humidity leads to electrical storms that might induce respiratory and cardiovascular illnesses. Natural crisis adds to psychological health problems such as post-traumatic disorders. Longer hurricanes, tropical storms, and other extensive climate conditions might provoke stress that interferes with daily living and hence

IoT for Mitigating Climate Change Impacts on Health

321

influences health and well-being (D’Amato et al., 2015). Ultraviolet (UV) radiation is challenging the public and environmental health since added to impairing plastic, wood, and other structures; encountering the harmful UV could induce sunburns, dermal tumors, cataracts, and corneal injuries or immunocompromising effects (Barnes et al., 2019). Existing evidence, taking into account health science progress and economic growth models, suggests that the death toll of climate change may account for 250,000 additional deaths per year between 2030 and 2050; 38,000 due to heat exposure in elderly people, 48,000 due to diarrhea, 60,000 due to malaria, and 95,000 due to childhood undernutrition (WHO, 2018; Yang et al., 2005a,b). Although all populations are at risk of climate change, the WHO suggests that children and communities residing in remote areas and especially islands are more vulnerable (WHO, 2018). Tackling the impact of climate change on health requires rigorous research, effective policymaking, strategic partnerships, considerable financial resources and widespread awareness.

12.3  HEALTH IoT Applying IoT to healthcare could sustainably develop both health and the environment in parallel to economic and societal growth supporting a strong health IoT architecture for facing climate changes, including better air quality, more green urbanization, and fewer floods (Ebi & Semenza, 2008; Kjellstrom et al., 2010; Peel et al., 2013). Health IoT is promising to improve sustainable health and well-being via monitoring health outcomes over the entire range of changing climate, rendering data on health and environment describing such outcomes and facilitating prediction, determination, and proper address of risk factors. Examples of IoT applications in health include modern communication technologies such as the novel mobile health, innovative biosensors, remote and rapid diagnostics, disease-predicting models, telehealth, daily self-monitoring, and tracking techniques for better managing conditions and advanced therapeutics (Ebi & Semenza, 2008). The concept of IoT is one of the most promising and far-sighted areas for the implementation of environmental goals. At present, IoT technologies already exist that make it possible to analyze the ecological situation of many parts of our planet. Many of them are already adapted to the management processes of eliminating the negative impact of nature in places of high concentration of people, in particular, in large- and medium-sized cities. Developed and implemented special “smart” sensors constantly collect data on the basis of which the necessary decisions are made, and measures are taken to prevent threats associated with certain abnormal natural phenomena, the consequences of negative human impact. Currently, monitoring of the aquatic environment, air quality, seismic activity and illegal deforestation, peatland research, the study of the Great Barrier Reef, garbage management, and even the rational use of public lighting have become widespread. Environmental protection involves different types of tasks, and in the case of the right approach to their solution in most instances, they are under the power of the concept of IoT. However, many complex and insufficiently studied environmental problems remain. Their solution requires time to study, comprehend, and develop a

322

Artificial Intelligence and Internet of Things

method of neutralization. Therefore, collecting the necessary data is the first step in this direction. That is why millions of “smart” devices connected to a single network today monitor the environmental impact of mineral processing products, human waste, the state of forests, rivers, seas, and other ecosystems. In accordance with the requirements, scientists develop and improve special environmental sensors for collection, and mobile applications for reading and analyzing information. The range of their capabilities is quite wide from measuring environmental parameters (air quality, temperature, humidity, and carbon dioxide content) and determining the content of nitrates in food to registering the level of radiation. Operation via RFID, Wi-Fi, Bluetooth, GPS, Zigbee, LoRaWAN, and NB-IoT modules increases the degree of accuracy of the received data. Thus, personal sensors give the chance to change ways of reception of the information, and they transfer both on the personal computer and the smartphone with further processing (Tawfik et al., 2017).

12.4  EXAMPLES OF IoT-BASED ENVIRONMENT SOLUTIONS The IoT sensing technologies are promising in timely monitoring water quality and flow, alerting systems, and other curing techniques since they could improve water quality and mitigate or prevent water-pollution-related illnesses, hence informing choosing a suitable purifying approach according to the discovered contagious microorganisms. For instance, India uses IoT-integrated intelligent meters, reverse osmosis (RO) technique, and sensor networks to supply clean water via curing water in rural areas (Schmidt et al., 2016). China uses IoT-integrated sensors built in various spots of the water supply system for monitoring water flow (Staedter, 2018), while Bangladesh uses arsenic biosensor networks to monitor water quality (Dewan & Yamaguchi, 2009). Kenya uses an intelligent water hand pump, instead of the defective conventional ones, where sophisticated accelerometers with installed 3G radios are applied for real-time monitoring, hence limiting interruptions and delivering stable service of water supply (Koehler et al., 2015). Indonesia uses water flow and movement sensors to detect and improve individual hygiene behaviors such as washing hands following the toilet (Thomas et al., 2018). As an example of “smart” environmental monitoring devices that have proven themselves and become popular among users is the Air Quality Egg sensor, designed to check air quality (Air Quality Egg, 2018). The information collected by all devices connected to the network displayed on a special site in real time allows you to assess the level of air pollution directly in the user’s home or office and in the city as a whole. This technology is used in both America and Europe, and is gradually gaining ground in developing countries. Devices such as Speck, Sensordrone, and iGeigie have also proven themselves well (Jo et al., 2020). The real catastrophe of modern cities is household waste. To solve this problem, “smart” containers were invented a few years ago – Bigbelly. In the process of their technological development, they are constantly being improved and the fifth generation of seals is working for cleanliness around the world. The HC5 Bigbelly is a smart, solar-powered waste compactor. It is equipped with a sensor that monitors its filling and transmits via wireless to the relevant services of the city. With enough power and knowing when to collect, streamline waste management operations

IoT for Mitigating Climate Change Impacts on Health

323

increase productivity and keep public areas clean and green. The information collected from Bigbelly allows special teams to plan garbage collection and quickly clean containers from it (Bigbelly, 2018). Using the IoT, many countries around the world are trying to make the infrastructure of their cities not only more reliable but also more environmentally friendly. According to scientific studies, street lighting “supplies” about 6% of carbon dioxide to the atmosphere. Therefore, some countries are trying to reduce its share of emissions by improving electric lighting systems. For example, Denmark is installing “smart” streetlights on the streets of Copenhagen. With the help of sensors, they monitor the occupancy of a certain section of the street by cars or people, assess weather conditions and, based on these data, adjust the brightness of the lighting, and accordingly the level of carbon dioxide emissions. In addition, the new lighting network offers many opportunities to connect new services in the future: video security cameras, noise, and air quality sensors can further improve the safety and quality of life of citizens and turn Copenhagen into a real “Smart City” (C21 France La redaction, 2018). In turn, the American company TCS Digital Software & Solutions Group (Tata Consultancy Services) has launched an innovative cloud software called Intelligent Urban Exchange (IUX) to optimize street lighting. This solution is designed for both ordinary streetlamps and LED lamps used indoors. With the help of special sensors, “smart” lights can respond to sudden human movements, signaling about danger by increasing the brightness, as well as reduce it in case of low pedestrian activity. Smart lighting can also be automatically adjusted depending on weather conditions and air pollution levels. Thus, the new intelligent street lighting software should be an effective solution to the issue of energy optimization. A special project has been conceived to protect the forests of the Brazilian Amazon from illegal logging. Individual trees in the protected area of the Brazilian Amazon in a certain order are noticed by special cellular device called Invisible Tracck. This device is equipped with a communication module that sends a warning to the Cargo Tracck Operation Center about its location when it is within 20 mi of a cellular base station. Information can be obtained that one of the adapted trees is on the way received by the Brazilian Environmental Protection Agency. Law enforcement officers receive real-time location information, so they can intercept and arrest thieves – either in the rainforest or in a sawmill. This device is specially adapted to the weather conditions of the Amazon and can operate autonomously for more than a year without recharging. Thanks to radio communication technology (RED), it works in areas with low signal levels, as well as small in volume, which makes it almost invisible. Cargo Tracck geolocation algorithms, along with RED technology, provide high location accuracy even in extremely remote areas. From the very beginning of its implementation, this technology has shown its effectiveness. For its commitment to supporting and creating innovative IoT solutions that address such global challenges in the world, the company of world leaders in digital security Thales was recognized by the Internet development group (IDG) as the winner of the “Computer World” in the category “World Good”. The positive experience strengthens the hope for the preservation of one of the most cherished and important resources on earth – forests, lungs of the planet (THALES, 2019)!

324

Artificial Intelligence and Internet of Things

Declared by UNESCO as World Heritage Site, the Great Barrier Reef, located along the Australian continent, is an ecosystem for a huge number of living organisms and has a significant impact on the overall environment in the region. To ensure maximum preservation, Dr Eric Wang developed the IrrigWeb-automatic farm irrigation system, further improving it by connecting it to a WiSA system. This program was successfully developed and implemented as part of a pilot study on a Burdekin sugar cane farm by scientists E. Wang et al. (2019). It is designed to automate the irrigation system of fields and reduce the amount of run-off and the possibility of their entry into the ocean, as they adversely affect the flora and fauna of the Great Barrier Reef. In turn, equipped with special sensors buoys, data are collected on the biochemical state of the Great Barrier Reef. The data obtained from the sensors are used by Australian conservation organizations to analyze the degree of damage to coral reefs, the movement of fish and the state of their population by various microorganisms (Wang, 2019). In 2017, world-renowned companies Bayer and Bosch joined forces to develop a new digital solution – Smart Spraying. Such a joint research agreement would turn farms into digital farms and help make more effective use of plant protection products and use herbicides and insecticides only in areas of the field where they are really needed. The company’s research focuses on high-efficiency sensor technology, “smart” analytical devices and a selective spray system (Belgique, 2017). What if the farmland is located in places far from cities and sources of communication? The construction of traditional communication channels is quite expensive, and classic mobile technologies are only partially suitable for solving this problem. LoRaWAN-based technology comes to the rescue. New possibilities and features of its application were studied by Kovalchuk et al. (2019). In their work, they explored the possibility of connecting moisture, temperature, pressure, direction, and wind speed sensors based on the LoRaWAN protocol to Internet gateways without paying for cellular communications, additional power supply, and deployment of complex Wi-Fi networks in the field. IoT is promising to actively monitor the levels of UV radiation and ozone in large scale, hence designing future frameworks for making decisions. For instance, the dosimeter is a sensor that could identify exposure to the UV by quantifying the level of the absorbed ionizing radiation (Herndon et al., 2018).

12.5 LIMITATIONS Unfortunately, the widespread introduction of IoT technologies, both in general and in the field of ecology, is hampered by certain technical problems, such as different device protocols, lack of common standardization of IoT technology components, imperfect wireless infrastructure, and lack of a single stand-alone IoT platform to manage these devices. IoT technologies are very diverse: some of them are high-tech, with high memory capabilities, high-speed processors (smartphones, tablets), and others, on the contrary, have a low-level architecture, limited memory, and computing capabilities (temperature sensors). The relationship between these devices makes IoT a very complex system in general. Research and implementation of IoT security in recent years have become a leading issue due to denial-of-service (DDOS) attacks.

IoT for Mitigating Climate Change Impacts on Health

325

According to Mohammed Tawfik et al. (2017), the following categories have been identified for various IoT-related threats: denial-of-service, physical attacks, eavesdropping and passive monitoring, traffic analysis, and data output.

12.6  FUTURE INSIGHTS So, the development of IoT is designed to have a positive impact on the ecological state of the planet. While not yet fully understood, the potential of IoT technology can provide humanity with new solutions to environmental problems. According to scientific data, the more IoT devices are connected to the network, the faster they socialize. Cloud technology plays an important role in the IoT ecosystem. With the help of cloud computing, computing capacities and storage volumes can be increased. In addition, sensors can be used anywhere, and data from them can be processed through cloud computing services. In turn, cloud technologies will make the data easily accessible to many users. IoT devices and consumers will be connected by a common social network IoT (Social IoT, SIoT). Such environmental monitoring based on the capabilities of the SIoT platform is more accurate and effective than existing methods and will be able to provide an analysis of the state of the environment. This will allow you to receive real-time truly valuable information about the natural processes that take place on our earth. Quantifying the impact of climate change on health is quite challenging. It is clear that in the future it will be necessary to conduct many researches and analysis. Longitudinal studies and international collaboration are necessary to measure the risk factors aggravated by climate change, measure their effect, and take into account potential cofounders. Attention should be paid to the introduction of semantics and standards in the world of IoT, the settlement of its privacy and security, ensuring autonomy, and the development of an in-depth learning system for IoT platforms.

12.7 CONCLUSIONS This chapter described the climatic effects on human health, discussing detecting, communicating, and tracking techniques with ecological real-world applications. It also explored the influence of critical health and ecological elements on developing novel IoT technologies along with limitations and future potentials. Climate changes threaten significantly and diversely human health now and maybe for longer times. Sustainable health in a community requires following systematic approaches in investigating the effects of climate change and designing indicators, frameworks, and measures to observe and interpret various risk factors on health hence, ensuring the growth of the society. Tackling the impact of climate change on health requires rigorous research, effective policymaking, strategic partnerships, considerable financial resources, and widespread awareness. The concept of IoT is one of the most promising and far-sighted areas for the implementation of environmental goals. At present, IoT technologies already exist that make it possible to analyze the ecological situation of many parts of our planet. Integrating IoT with tools for sensing, communicating, monitoring, and decisionsupport is needed for sustainable health. However, having analyzed the information

326

Artificial Intelligence and Internet of Things

on the IoT, it is safe to say that despite their widespread use and good achievements, there are still relevant issues of different countries to focus their efforts on their speedy resolution.

REFERENCES Ahern, M., Kovats, R. S., Wilkinson, P., Few, R., & Matthies, F. (2005). Global health impacts of floods: epidemiologic evidence. Epidemiologic Reviews, 27(1), 36–46. https://doi. org/10.1093/epirev/mxi004. Air Quality Egg. (2018). Air Quality Egg—Science is Collaboration. https://airqualityegg. com/home. Akinbami, L. J., Moorman, J. E., & Liu, X. (2011). Asthma prevalence, health care use, and mortality: United States, 2005-2009. National Health Statistics Reports, 32, 1–14. Anderson, G. B., & Bell, M. L. (2011). Heat waves in the United States: mortality risk during heat waves and effect modification by heat wave characteristics in 43 U.S. communities. Environmental Health Perspectives, 119(2), 210–218. https://doi.org/10.1289/ ehp.1002313. Balbus, J. M., & Malina, C. (2009). Identifying vulnerable subpopulations for climate change health effects in the United States. Journal of Occupational and Environmental Medicine, 51(1), 33–37. https://doi.org/10.1097/JOM.0b013e318193e12e. Barnes, P. W., Williamson, C. E., Lucas, R. M., Robinson, S. A., Madronich, S., Paul, N. D., … & Zepp, R. G. (2019). Ozone depletion, ultraviolet radiation, climate change and prospects for a sustainable future. Nature Sustainability, 2(7), 569–579. https://doi. org/10.1038/s41893-019-0314-2. Belgique, B. (2017). Bayer and Bosch Develop New, Digital Smart Spraying Solutions— Bayer Belgique. https://www.bayer.be/fr/medias/communications/bayer-and-boschdevelop-new-digital-smart-spraying-solutions.php. Bell, M. L., Goldberg, R., Hogrefe, C., Kinney, P. L., Knowlton, K., Lynn, B., Klein-Rosenthal, J. E., Rosenzweig, C., & Patz, J. A. (2007). Climate change, ambient ozone, and health in 50 US cities. Climatic Change. https://doi.org/10.1007/s10584-006-9166-7. Benedict, K. M., Reses, H., Vigar, M., Roth, D. M., Roberts, V. A., Mattioli, M., … & Hill, V. R. (2017). Surveillance for waterborne disease outbreaks associated with drinking water—United States, 2013-2014. MMWR. Morbidity and Mortality Weekly Report, 66(44), 1216–1221. https://doi.org/10.15585/mmwr.mm6644a3. Bigbelly. (2018). Smart Waste Management for Cities—Bigbelly World Leading Smart Waste. Bigbelly. https://bigbelly.com/solutions/city/. C21 France La redaction. (2018, December 7). Smart Lighting in Copenhagen (Denmark): Sustainable Infrastructure Grand Prize Winner—Green Solutions Awards 2018— Construction21. Construction21.Org. https://www.construction21.org/articles/fr/smartlighting-in-copenhagen-denmark-sustainable-infrastructure-grand-prize-winnergreen-solutions-awards-2018.html. Committee on the Effect of Climate Change on Indoor Air Quality and Public Health. (2011). Climate Change, the Indoor Environment, and Health. https://doi.org/10.17226/13115. D’Amato, G., Vitale, C., De Martino, A., Viegi, G., Lanza, M., Molino, A., Sanduzzi, A., Vatrella, A., Annesi-Maesano, I., & D’Amato, M. (2015). Effects on asthma and respiratory allergy of Climate change and air pollution. Multidisciplinary Respiratory Medicine, 10(1), 39. https://doi.org/10.1186/s40248-015-0036-x. Delfino, R. J., Brummel, S., Wu, J., Stern, H., Ostro, B., Lipsett, M., … & Gillen, D. L. (2009). The relationship of respiratory and cardiovascular hospital admissions to the southern California wildfires of 2003. Occupational and Environmental Medicine, 66(3), 189–197. https://doi.org/10.1136/oem.2008.041376.

IoT for Mitigating Climate Change Impacts on Health

327

Dewan, A. M., & Yamaguchi, Y. (2009). Land use and land cover change in Greater Dhaka, Bangladesh: using remote sensing to promote sustainable urbanization. Applied Geography, 29(3), 390–401. https://doi.org/10.1016/j.apgeog.2008.12.005. Ebi, K. L., & McGregor, G. (2008). Climate change, tropospheric ozone and particulate matter, and health impacts. Environmental Health Perspectives, 116(11), 1449–1455. https://doi.org/10.1289/ehp.11463. Ebi, K. L., & Semenza, J. C. (2008). Community-based adaptation to the health impacts of climate change. American Journal of Preventive Medicine, 35(5), 501–507. https://doi. org/10.1016/j.amepre.2008.08.018. Elliott, C. T., Henderson, S. B., & Wan, V. (2013). Time series analysis of fine particulate matter and asthma reliever dispensations in populations affected by forest fires. Environmental Health: A Global Access Science Source, 12, 11. https://doi.org/ 10.1186/1476-069X-12-11. Fewtrell, L., & Kay, D. (2008). An attempt to quantify the health impacts of flooding in the UK using an urban case study. Public Health, 122(5), 446–451. https://doi.org/10.1016/j. puhe.2007.09.010. Fisk, W. J., Lei-Gomez, Q., & Mendell, M. J. (2007). Meta-analyses of the associations of respiratory health effects with dampness and mold in homes. Indoor Air, 17(4), 284–296. https://doi.org/10.1111/j.1600-0668.2007.00475.x. Gobler, C. J. (2020). Climate change and harmful algal blooms: insights and perspective. Harmful Algae, 91, 101731. https://doi.org/10.1016/j.hal.2019.101731. Hajat, S., & Kosatky, T. (2010). Heat-related mortality: a review and exploration of heterogeneity. Journal of Epidemiology and Community Health, 64(9), 753–760. https://doi. org/10.1136/jech.2009.087999. Hayes, K., Blashki, G., Wiseman, J., Burke, S., & Reifels, L. (2018). Climate change and mental health: risks, impacts and priority actions. International Journal of Mental Health Systems, 12(1), 1–12. https://doi.org/10.1186/s13033-018-0210-6. Herndon, J. M., Hoisington, R. D., & Whiteside, M. (2018). Deadly ultraviolet UV-C and UV-B penetration to earth’s surface: human and environmental health implications. Journal of Geography, Environment and Earth Science International, 1–11. https://doi. org/10.9734/JGEESI/2018/40245. Idso, S. B., & Idso, K. E. (2001). Effects of atmospheric CO2 enrichment on plant constituents related to animal and human health. Environmental and Experimental Botany, 45(2), 179–199. https://doi.org/10.1016/S0098-8472(00)00091-5. Jo, J., Jo, B., Kim, J., Kim, S., & Han, W. (2020, January 14). Development of an IoT-based indoor air quality monitoring platform [research article]. Journal of Sensors; Hindawi. https://doi.org/10.1155/2020/8749764. Johnston, F. H., Henderson, S. B., Chen, Y., Randerson, J. T., Marlier, M., DeFries, R. S., Kinney, P., Bowman, D. M. J. S., & Brauer, M. (2012). Estimated Global Mortality Attributable to Smoke from Landscape Fires. University of British Columbia. https:// doi.org/10.14288/1.0074716. Keesing, F., Brunner, J., Duerr, S., Killilea, M., LoGiudice, K., Schmidt, K., Vuong, H., & Ostfeld, R. S. (2009). Hosts as ecological traps for the vector of Lyme disease. Proceedings of the Royal Society B: Biological Sciences, 276(1675), 3911–3919. https:// doi.org/10.1098/rspb.2009.1159. Kjellstrom, T., Butler, A. J., Lucas, R. M., & Bonita, R. (2010). Public health impact of global heating due to climate change: potential effects on chronic non-communicable diseases. International Journal of Public Health, 55(2), 97–103. https://doi.org/10.1007/ s00038-009-0090-2. Koehler, J., Thomson, P., & Hope, R. (2015). Pump-priming payments for sustainable water services in rural Africa. World Development, 74, 397–411. https://doi.org/10.1016/j. worlddev.2015.05.020.

328

Artificial Intelligence and Internet of Things

Kolivras, K. (2010). Changes in dengue risk potential in Hawaii, USA, due to climate variability and change. Climate Research, 42, 1–11. https://doi.org/10.3354/cr00861. Kovalchuk, V. P., MatIash, T. V., Knysh, V. V., Voitovich, O. P., & Kruchenyuk, A. V. (2019). Internet of things (IoT) applications using the LoRaWAN protocol for monitoring irrigated land. Land Reclamation and Water Management, 2, 130–139. https://doi. org/10.31073/mivg201902-187. Li, F., Newman, P., Pawson, S., & Perlwitz, J. (2018). Effects of greenhouse gas increase and stratospheric ozone depletion on stratospheric mean age of air in 1960–2010. Journal of Geophysical Research: Atmospheres, 123(4), 2098–2110. https://doi.org/ 10.1002/2017JD027562. Maibach, E. W., Nisbet, M., Baldwin, P., Akerlof, K., & Diao, G. (2010). Reframing climate change as a public health issue: an exploratory study of public reactions. BMC Public Health, 10(1), 299. https://doi.org/10.1186/1471-2458-10-299. Martiello, M. A., Baldasseroni, A., Buiatti, E., & Giacchi, M. V. (2008). Health effects of heat waves. Igiene E Sanita Pubblica, 64(6), 735–772. Martin. (n.d.). Health. United Nations Sustainable Development. Retrieved June 13, 2020 from https://www.un.org/sustainabledevelopment/health/. National Intelligence Council. (2017). Global Trends: The Paradox of Progress. https://espas. secure.europarl.europa.eu/orbis/document/global-trends-paradox-progress. Nitschke, M., Tucker, G. R., Hansen, A. L., Williams, S., Zhang, Y., & Bi, P. (2011). Impact of two recent extreme heat episodes on morbidity and mortality in Adelaide, South Australia: a case-series analysis. Environmental Health, 10(1), 42. https://doi. org/10.1186/1476-069X-10-42. Peel, J. L., Haeuber, R., Garcia, V., Russell, A. G., & Neas, L. (2013). Impact of nitrogen and climate change interactions on ambient air pollution and human health. Biogeochemistry, 114(1), 121–134. https://doi.org/10.1007/s10533-012-9782-4. Reiter, P. (2008). Climate change and mosquito-borne disease: knowing the horse before hitching the cart. Revue Scientifique Et Technique (International Office of Epizootics), 27(2), 383–398. Robine, J.-M., Cheung, S. L. K., Le Roy, S., Van Oyen, H., Griffiths, C., Michel, J.-P., & Herrmann, F. R. (2008). Death toll exceeded 70,000 in Europe during the summer of 2003. Comptes Rendus Biologies, 331(2), 171–178. https://doi.org/10.1016/j. crvi.2007.12.001. Schmidt, S.-A., Gukelberger, E., Hermann, M., Fiedler, F., Großmann, B., Hoinkis, J., Ghosh, A., Chatterjee, D., & Bundschuh, J. (2016). Pilot study on arsenic removal from groundwater using a small-scale reverse osmosis system—towards sustainable drinking water production. Journal of Hazardous Materials, 318, 671–678. https://doi.org/10.1016/j. jhazmat.2016.06.005. Semenza, J. C., Herbst, S., Rechenburg, A., Suk, J. E., Höser, C., Schreiber, C., & Kistemann, T. (2012a). Climate change impact assessment of food- and waterborne diseases. Critical Reviews in Environmental Science and Technology, 42(8), 857–890. https:// doi.org/10.1080/10643389.2010.534706. Semenza, J. C., Suk, J. E., Estevez, V., Ebi, K. L., & Lindgren, E. (2012b). Mapping climate change vulnerabilities to infectious diseases in Europe. Environmental Health Perspectives, 120(3), 385–392. https://doi.org/10.1289/ehp.1103805. Sheffield, P. E., & Landrigan, P. J. (2011). Global climate change and children’s health: threats and strategies for prevention. Environmental Health Perspectives, 119(3), 291–298. https://doi.org/10.1289/ehp.1002233. Shonkoff, S. B., Morello-Frosch, R., Pastor, M., & Sadd, J. (2011). The climate gap: environmental health and equity implications of climate change and mitigation policies in California—a review of the literature. Climatic Change, 109(1), 485–503. https://doi. org/10.1007/s10584-011-0310-7.

IoT for Mitigating Climate Change Impacts on Health

329

Sousa, T. C. M., Amancio, F., Hacon, S. S., & Barcellos, C. (2018). Climate-sensitive diseases in Brazil and the world: systematic reviewEnfermedades sensibles al clima en Brasil y el mundo: revisión sistemática. Revista Panamericana de Salud Publica = Pan American Journal of Public Health, 42, e85. https://doi.org/10.26633/rpsp.2018.85. Staedter, T. (2018). Sensor network monitors 1,400-km canal [news]. IEEE Spectrum, 55(4), 10–11. https://doi.org/10.1109/MSPEC.2018.8322033. Tawfik, M., Almadani, A., & Alharbi, A. A. (2017). A review: the risks and weakness security on the IoT (SSRN Scholarly Paper ID 3558835). Social Science Research Network. https://doi.org/10.2139/ssrn.3558835. THALES. (2019). Cargo Tracck Protects Rainforest Trees (with IoT). https://www. thalesgroup.com/en/markets/digital-identity-and-security/iot/customer-cases/ iot-protects-rainforest. Thomas, E., Alberto, L., Christian, A., Borja-Vega, & Sturzenegger, G. (2018). Innovations in WASH Impact Measures: Water and Sanitation Measurement Technologies and Practices to Inform the Sustainable Development Goals. The World Bank. https://doi. org/10.1596/978-1-4648-1197-5. UN. (n.d.). Home: Sustainable Development Knowledge Platform. Retrieved August 5, 2020, from https://sustainabledevelopment.un.org/index.html. UNDP. (n.d.). Goal 3: Good Health and Well-Being. UNDP. Retrieved June 13, 2020, from https://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-3good-health-and-well-being.html. USGCRP. (2016). The Impacts of Climate Change on Human Health in the United States: A Scientific Assessment (pp. 1–312). U.S. Global Change Research Program. executivesummary.html. Velders, G. J. M., Andersen, S. O., Daniel, J. S., Fahey, D. W., & McFarland, M. (2007). The importance of the Montreal Protocol in protecting climate. Proceedings of the National Academy of Sciences, 104(12), 4814–4819. https://doi.org/10.1073/pnas.0610328104. Wang, E. (2019). How IoT Can Help Save the Great Barrier Reef [Text]. https://www.jcu.edu. au/this-is-uni/articles/how-the-internet-of-things-can-help-save-the-great-barrier-reef. Wang, E., Attard, S., McGlinchey, M., Xiang, W., Philippa, B., Linton, A. L., & Everingham, Y. (2019). Smarter irrigation scheduling in the sugarcane farming system using the Internet of Things. Proceedings of the 41st Annual Conference of the Australian Society of Sugar Cane Technologists, 41, 164–170. https://assct.com.au/component/assct/search-result?search_cat=title&filter_search=Smarter%20irrigation%20 scheduling%20in%20the%20sugarcane%20far&publisher=any&Itemid=0. WHO. (2018). Climate Change and Health. https://www.who.int/news-room/fact-sheets/ detail/climate-change-and-health. Wolf, J., O’Neill, N. R., Rogers, C. A., Muilenberg, M. L., & Ziska, L. H. (2010). Elevated atmospheric carbon dioxide concentrations amplify Alternaria alternata sporulation and total antigen production. Environmental Health Perspectives, 118(9), 1223–1228. https://doi.org/10.1289/ehp.0901867. Yang, G., Vounatsou, P., Zhou, X.-N., Tanner, M., & Utzinger, J. (2005a). A potential impact of climate change and water resource development on the transmission of Schistosoma japonicum in China. Parassitologia, 47(1), 127–134. https://europepmc.org/article/ med/16044681. Yang, G.-J., Vounatsou, P., Zhou, X.-N., Utzinger, J., & Tanner, M. (2005b). A review of geographic information system and remote sensing with applications to the epidemiology and control of schistosomiasis in China. Acta Tropica, 96(2–3), 117–129. https://doi. org/10.1016/j.actatropica.2005.07.006.

13

Automated Hybrid Recommender System for Cardiovascular Disease with Applications in Smart Healthcare Zahid Mehmood, Fouzia Jabeen, Muhammad Tahir, Rehan Mehmood Yousaf, Noor Ayesha

CONTENTS 13.1 Introduction................................................................................................... 332 13.2 Literature Review.......................................................................................... 333 13.3 The Proposed Disease Prediction Model...................................................... 334 13.3.1 Data Preprocessing............................................................................ 334 13.3.2 Feature Selection Process.................................................................. 335 13.3.3 Classification...................................................................................... 335 13.3.3.1 Support Vector Machine (SVM)......................................... 336 13.4 Proposed Recommender Model.................................................................... 337 13.4.1 Community-Based Recommender System........................................ 338 13.4.1.1 K-Nearest Neighbors (KNN)..............................................340 13.4.1.2 Multilayer Perceptron......................................................... 341 13.5 Experimental Results and Discussions.......................................................... 341 13.5.1 Dataset............................................................................................... 342 13.5.2 Performance Evaluation Criteria....................................................... 343 13.5.2.1 Precision and Recall........................................................... 343 13.5.2.2 Accuracy.............................................................................344 13.5.2.3 Mean Absolute Error (MAE)..............................................344 13.5.3 Results Analysis.................................................................................344 13.6 Conclusion and Future Work......................................................................... 350 References............................................................................................................... 350

331

332

Artificial Intelligence and Internet of Things

13.1 INTRODUCTION Cardiovascular disease (CVD) is a long-term disease that needs long-term treatment and in some cases, difficult to cure. The use of advanced artificial intelligence (AI) techniques could be useful to provide healthy lifestyle suggestions to cardiac victims, to decrease the social burden due to CVD. This research intends to build a recommender system for CVD. Dataset is collected with a cardiologist from Asfandyar Hospital, Attock, Pakistan (Jabeen et al., 2019). The system consists of two models, (1) the CVD detection model and (2) the hybrid recommender model, as shown in Figure 13.1. CVD is directly lethal to human life. It includes a list of diseases that disturb the proper working of the heart. Among the users who seek information about their health, users of chronic diseases like CVD need more support than normal users. Cardiovascular sickness (CVD) is a dangerous malady. It incorporates the rundown of sicknesses that exasperate the heart’s usefulness. The eventual coronary illness outcomes are severe, extending from arrhythmia, high hypertension (HTN) to strokes, myocardial infarction (MI), and even demise. According to the American College of Cardiology, 33% of deaths over the globe are because of heart attacks (Damen et al., 2016). Cancer, HIV/AIDS, and prematurity are the most common diseases which cause deaths, and CVDs are primary in that list (Hasan, Jasim, & Hashim, 2017). As stated by Pakistan’s premium news agency ‘Times of Islamabad’ on 30th October 2017, the Pakistani populace has a high danger of CVD. In Pakistan,

FIGURE 13.1  Workflow diagram of the proposed methodology

Automated Hybrid Recommender System

333

the percentage of people dying from cardiovascular malady is 30%–40%. Heart ailments are considered hazardous maladies, which, if not found in time and is not dealt with appropriately, can cause entanglements, including deaths (Zheng et al., 2017). The major risk factors causing CVD are as follows: Smoking: The risk of CVD is double for smokers as compared to nonsmokers. Cholesterol: The cholesterol occupied in veins results in narrowing the blood vessels that lead to CVD. Diet with low cholesterol and saturated, trans fats helps in reducing cholesterol levels and improves heart health. Blood pressure: Hypertension is also a reason for heart attack. Diabetes: The diabetic patient has a high risk of heart disease; therefore, it is necessary to treat diabetes in time to avoid severe subsequences. Sedentary lifestyle: Simple recreation time activities like gardening and strolling can lower our danger of coronary illness. Eating habits: Healthy eating routine and low consumption of salt, soaked fat, trans fat, cholesterol, and refined sugars will bring down our chances of getting a coronary illness. Stress: Inadequately controlled pressure and anger can prompt coronary failures and strokes. It is a long-term disease that needs long-term treatment. Sometimes, it turns out to be increasingly confounded and hard to fix. An efficient hybrid recommender system could be useful in such cases. The remaining sections of this study are organized as follows: Section 13.2 describes the literature review in recommender systems and disease detection. The proposed methodology is described in Sections 13.3 and 13.4. Dataset is collected for multi­ class identification because available dataset for CVD is binary labeled, that is Yes and No, used to identify the presence and absence of disease. The collected dataset and results are discussed in Section 13.5, and Section 13.6 contains the conclusion and future work.

13.2  LITERATURE REVIEW Utilization of present-day innovation could help give a solid way of life proposals to heart patients, to decrease the social burden due to CVD. Utilization of AI (information mining and machine learning (ML)) systems can decrease the expense and time of treatment (Sudhakar & Manimekalai, 2014). These AI and information mining methods help analyze long-standing illnesses and guide to improve patients’ way of life, which could help them raise their well-being. In the old days, people suffering from heart disease were of declining years. But now the situation is changing and younger people are also facing the risk of CVD. Although many medical facilities are available nowadays, people living in the countryside have fewer medical facilities and fight more to continue their lives. To diminish the risk associated with heart disease, mortality, and costs of clinical test, eHealth gives the best solutions. However, the eHealth system’s problem is managing huge amounts of data produced because of monitoring patients and maintaining their medical history. Remote checking frameworks are viewed as powerful on the off chance that they

334

Artificial Intelligence and Internet of Things

productively examine the huge information produce by patient’s therapeutic past events or medical tests (Lloyd-Jones et al., 2010). These medical tests records of past events are helpful to choose move ought to be made to maintain a strategic distance from or decrease CVD. A large portion of remote observing frameworks are inadequate in such efficiencies to manage huge information viably (Lan et al., 2012; Lloyd-Jones et al., 2010). Nowadays, efficient remote monitoring systems are developed. Advanced AI methods provide remote monitoring techniques that are more informative, while they were simple and less efficient in the past. Now, they can provide not only simple information about a person suffering from a chronic disease but also more informative data like physical activities to the end user (Chen, Zhu, Zhang, & Wang, 2012). Moreover, a ton of work has been done on information confidentiality to transmit precise information from one spot to the other without compromising (Hamza, Muhammad, Nachiappan, & González, 2017). In ongoing investigations, progressively complex data identified with CVD is introduced utilizing AI systems. These procedures speak to information in expectation, peculiarity location, and characterization. Order is the procedure to choose sickness or distinguish the malady that has a place with which class (Baig & Gholamhosseini, 2013; Bellazzi, Ferrazzi, & Sacchi, 2011). Information securing for illness expectations, effectively and accurately, is still a challenging task. Obtaining precise data is extremely critical in making a quality decision to predict CVD. The eHealth framework intends to identify heart disease in beginning periods to reduce malady and mortality hazards. It also means identifying the ailment and its stage to generate fitting advocacy to upraise the health of the patient (Bellazzi et al., 2011; Koh & Tan, 2011). There is a need to generate an appropriate suggestion for a cardiac patient to improve their well-being in countryside zones or without the presence of cardiologists. Smart visual sensors, modern processing, and data transmission techniques are required to provide healthcare in remote areas. IoT-based systems provide these facilities to collect surveillance data using multiple sensors (Muhammad et al., 2018). Survive recommender systems use ML classification techniques to classify CVD in one of the label classes. Recommendations are also provided using ML classification techniques (Ijas et al., 2018; Kadi, Idri, & Fernandez-Aleman, 2017). Data mining and classifications techniques are also used in recommender systems to generate recommendations for disease treatment (Nasiri, Rezghi, & Minaei, 2014).

13.3  THE PROPOSED DISEASE PREDICTION MODEL 13.3.1 Data Preprocessing Pretreating the data to put in the classifier is very important to perform a classification task. ML techniques can only be applied after processing data. These techniques are applied to preprocessed data for strategic analysis and ideal results. The well-known data preprocessing technique is data cleansing, which includes removing spelling mistakes, identical records, and unbelievable information. Missing numbers or values and anomalies are additionally treated in the same manner. To start with, a numerical cleaner channel demonstrates the missing value. The channel washes

Automated Hybrid Recommender System

335

down the information that is out of default extend, for example, from the top or bottom scope of information is set to the predefined esteem. When the missing worth is identified, the missing value is filled up by the mean estimation of circulated information. The preprocessed information is spotless, commotion free, and prepared to use for feature determination. Irrelevant highlights are disposed of from the dataset to improve the execution of the preparing model.

13.3.2 Feature Selection Process Decreasing the number of features and precisely disease prediction with restricted properties is a testing task in data mining. Noise is expelled from information and important data is separated from the clinical information, which helps diagnose CVD. The highlighted features recognize those properties which are increasingly explicit about analyzing sickness. Chosen features have various accessible classes (Rathore et al., 2017; Tang, Alelyani, & Liu, 2014). Dataset is available with 12 attributes, but all these attributes are not crucial to diagnose disease. Just a few available features are noteworthy to classify disease in one of the available classes. The process of feature selection is therefore applied to select those highly prominent features. Selecting more specific features reduces the sample size and improves the results. The feature dimension is reduced using principal component analysis (PCA) (Aljawarneh, Aldwairi, & Yassein, 2018). Experimental results show that the PCA technique used at the feature selection step gives more accurate results. It is a linear dimension reducing technique that finds the components in maximum variability direction, as shown in Equation (13.1).

∑=

1  {( x − x )( x − x )T (13.1) N

The covariance matrix is calculated in Equation (13.1). It is of N × N order where N is the number of dimensions in the sample, x is the given matrix with N dimensions, and x ̅ represents the mean vector. The next step computes the eigenvalues (Equation (13.2)) and eigenvector. The eigenvector is obtaining by arranging the eigenvalues in descending order; hence, the projection of actual data in the eigenvector direction is achieved (Wang, Ma, & Qian, 2018).

V −1 ∑ V = D

(13.2)

where, V represents the eigenvector and D represents eigenvalues of the covariance matrix ∑.

13.3.3 Classification After selecting features from all available features, the classification technique is applied to classify CVD in any accessible cardiac class. Support vector machine (SVM) is an ideal classifier and gives great outcomes in grouping issues (Joachims,

336

Artificial Intelligence and Internet of Things

1998; Witten et al., 1999). Different classification algorithms like K-nearest neighbors (KNN), decision tree (DT), etc. are compared with SVM. 13.3.3.1  Support Vector Machine (SVM) An SVM is a classifier utilized for supervised learning in the field of AI. It has the unique property to limit the grouping mistake and increment the geometric edges. Subsequently, these are likewise called the greatest edge classifiers. SVM is ideal for malady characterization issues because SVM is skilled in dealing with multimeasurement inputs. As per SVM’s suppositions, just a couple of unimportant features are contained by info space. Besides, they are best in the information scanty issue. SVM settles on the choice limits far from one another. The learning model of SVM determines the hyperplane between two classes: positive and negative. Hyperplane boosts the separation between two close purposes of the various classes. Equation (13.3) indicates how a double-class SVM allots class to new vector V′. msv

∑ ( x y v v'



k k k

k

+ a ) (13.3)

k =1

where msv represents the number of support vectors; x k denotes support vectors that are responsible for making boundaries used to distinguish one class from another; yk indicates the class names, which can be ‘Yes’ or ‘No’; and vk indicates vectors utilized for training. This percept for deciding Equation (13.3) assigns a class to v 'k . The assigned class is positive if the result of the decision rule is positive. Non-straight limits can likewise be adapted to SVM. It uses kernel trick (Burges, 1998) to deal with these limits. The thought is helpful to change over contributions to high dimension attribute space. This transformation converts linear operation of feature space into nonlinear operation. Multifaceted nature is diminished along these lines, and classification turns out to be simple. This transformation is represented as follows:  : A :→  A



where,  is the input space, and  denotes feature space. Equation (13.4) shows the example of polynomial kernel transformation.

(

)

a ( 1 , 2 ) :→ 12 , 2 2 ,  12 ,  X1 ,  X 2 , 1 (13.4)

After applying a transformation to Equation (13.3), it can be defined as follows: msv



∑ ( x y a k k

k =1

T

)

 v'k ) + a (13.5) ( vk ) a(

T  v 'k ) is used to denote kernel function. It is two-variable where, K ¸ ( vk ,  v 'k ) = a ( vk ) a( symmetric positive semi-definite function. SVM’s other kernel functions are linear kernel function, RBF, and polynomial kernel function (Huang, Maier, Hornegger, & Suykens, 2017).

Automated Hybrid Recommender System

337

13.4  PROPOSED RECOMMENDER MODEL In this section, heuristic recommender techniques are discussed to generate recommendations for cardiac patients. Generated recommendations are based on lifestyle, diet plan, and physical exercises. Table 13.1 shows the list of recommendations TABLE 13.1 Characteristics of the Recommended Lifestyle for CVD Recommendation ID R1 R2

R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R18 R19 R20 R21 R22 R23 R24 R25 R26 R27 R28

R29

Recommendation Details Increment the sum and power of your physical movement to coordinate the number of calories you take in Go for something like 150 minutes of moderate physical movement or 75 minutes of intense physical exercise – or an equivalent blend of both – every week Eat an assortment of natural products Consume fat-free dairy items You are encouraged to eat whole grains You must eat skinless poultry and fish Your diet should contain nuts and legumes Eating nontropical vegetable oils can improve your health Patients ought to avoid the utilization of immersed fat in their weight control plans Limit the utilization of red meat Avoid the use of desserts and sugar-improved drinks Lessen the utilization of sodium (salt) from your eating routine You should control your diabetes You are encouraged to control BP You are encouraged to eat verdant green salad You have to include omega-3 unsaturated fats in your eating regimen You need to include dietary fibers in your daily eating routine; you ought to pick entire grains rather than refined starches for their fiber Keep yourself away from cigarettes You are prescribed to constrain alcoholic beverages Improve your health by decreasing the consumption of espresso coffee You are directed to walk 30 minutes daily Stress can affect your health. Release your stress by practicing yoga You are advised to avoid smoking Maintaining a healthy weight is recommended. Try to reduce weight A patient ought to pursue rule for self-care and ordinary wellbeing checks You should do 30 minutes of moderate exercise, five times a week Aerobic activity for the duration of 150 minutes per week is recommended You are prompted for muscle reinforcing movement at least 2 or 3 days per week, which works every real muscle group (legs, hips, back, midriff, chest, shoulder, and arms) The patient is advised to perform daily exercise which mainly covers flexibility exercises

338

Artificial Intelligence and Internet of Things

generated against each predicted disease. All advices are not effective for every cardiac patient, e.g. exercise to worsen the health condition suffering from unstable MI. In contrast, exercise is good for acute coronary syndrome (ACS).

13.4.1 Community-Based Recommender System The key part of the proposed recommender framework is the demonstrated suggestion. The point is to give a precise and brief suggestion as indicated by the seriousness of ailment and hazard to human life. A versatile and savvy recommender demonstration pursues the expectation to break down the sick patient’s information for getting seriousness; what’s more, the likelihood of the event of illness, which thusly is utilized to give precise medicinal proposals to future activities. Medical suggestions are given utilizing information about the patient’s age and gender, which is assembled with the help of a cardiologist. It recognizes the significance and needs for medicinal variables, which assume a noteworthy job foreseeing precise coronary illness. This recommender model would support decreasing outstanding tasks at hand and the cost of time of the patient and clinical professionals. This would essentially decrease the money-related load on patients having a place with remote topographical areas. To generate appropriate cardiac patients’ recommendations, some rules are defined that work together with ML classifiers to select the most effective advice for patients. Two important features, age and gender, are used to make rules. The reason behind selecting these features for better suggestions is that ECG’s feature values are different in males and females, as shown in Figure 13.3, the same is the case with variation in age as shown in Figure 13.5. Heartbeat produced by the male heart is different from the female heart of the same age. One heartbeat along with seven points, i.e. P, Q, R, S, T, U, and J and interval between each point are mentioned in the diagram. These intervals are different in normal ECGs of males and females, also they are different for normal ECGs of different age groups. Therefore, if any interval is abnormal, we need to identify patients’ gender and age to give proper advice. Meanwhile, in 12 leads named I, II, III, aVR, aVL, aVF, V1, V2, V3, V4, V5, and V6 as shown in Figure 13.2, class of arrhythmia or CVD is decided after identifying abnormality in them. ECG of a normal child is different from the ECG readings of a normal adult. In the case of gender, the females’ heart rate is faster at the baseline of electrocardiography compared to the heart rate of males. Long QTc and smaller QRS duration are also observed in females while a male has a shorter QTc wave and long QRS duration. Age is also important in diagnosing the exact problem in heart functionality because different age groups have different ECG readings. For example, the newborn’s heart rate is 125–145 bpm (beats per minute), whereas, in adults, the normal heart rate is 60–100 bpm. The Q wave amplitude is 0.15 mV in a 1-month-old child, whereas 0.10 mV in a 16-year-old person, as mentioned in Figure 13.5. Therefore, medical advices for different age groups and different gender are not the same. Rules are designed based on age and gender to make advice more suitable for patients. Rules are defined in the following steps: • Select gender. • Select the age group: • Group 1: age > 70

Automated Hybrid Recommender System

339

FIGURE 13.2  Twelve leads of ECG

• Group 2: age > 30 and age