Proceedings of the Future Technologies Conference (FTC) 2020, Volume 3 [1st ed.] 9783030630911, 9783030630928

Citation preview

Advances in Intelligent Systems and Computing 1290

Kohei Arai Supriya Kapoor Rahul Bhatia   Editors

Proceedings of the Future Technologies Conference (FTC) 2020, Volume 3

Advances in Intelligent Systems and Computing Volume 1290

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Nikhil R. Pal, Indian Statistical Institute, Kolkata, India Rafael Bello Perez, Faculty of Mathematics, Physics and Computing, Universidad Central de Las Villas, Santa Clara, Cuba Emilio S. Corchado, University of Salamanca, Salamanca, Spain Hani Hagras, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK László T. Kóczy, Department of Automation, Széchenyi István University, Gyor, Hungary Vladik Kreinovich, Department of Computer Science, University of Texas at El Paso, El Paso, TX, USA Chin-Teng Lin, Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan Jie Lu, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia Patricia Melin, Graduate Program of Computer Science, Tijuana Institute of Technology, Tijuana, Mexico Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro, Rio de Janeiro, Brazil Ngoc Thanh Nguyen , Faculty of Computer Science and Management, Wrocław University of Technology, Wrocław, Poland Jun Wang, Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing such as: computational intelligence, soft computing including neural networks, fuzzy systems, evolutionary computing and the fusion of these paradigms, social intelligence, ambient intelligence, computational neuroscience, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, Web intelligence and multimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results. ** Indexing: The books of this series are submitted to ISI Proceedings, EI-Compendex, DBLP, SCOPUS, Google Scholar and Springerlink **

More information about this series at http://www.springer.com/series/11156

Kohei Arai Supriya Kapoor Rahul Bhatia •

•

Editors

Proceedings of the Future Technologies Conference (FTC) 2020, Volume 3

123

Editors Kohei Arai Faculty of Science and Engineering Saga University Saga, Japan

Supriya Kapoor The Science and Information (SAI) Organization Bradford, West Yorkshire, UK

Rahul Bhatia The Science and Information (SAI) Organization Bradford, West Yorkshire, UK

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-3-030-63091-1 ISBN 978-3-030-63092-8 (eBook) https://doi.org/10.1007/978-3-030-63092-8 © Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Editor’s Preface

With the aim of providing a worldwide forum, where the international participants can share their research knowledge and ideas, the 2020 Future Technologies Conference (FTC) was held virtually on November 5–6, 2020. FTC 2020 focuses on recent and latest technological breakthroughs in the areas of computing, electronics, AI, robotics, security and communications and map out the directions for future researchers and collaborations. The anarchic spirit and energy of inquiry found in our community always help researchers to produce brilliant technological advances which continue to restructure entire computing community. FTC see participation from such researchers, academics and technologists from leading universities, research firms, government agencies and companies to submit their latest research at the forefront of technology and computing. We are pleased to review and select a volume of high-qualified papers from all submissions during the conference. We hope these papers which have been gone through the double-blind review process can provide helpful reference for all readers and scholars. In these proceedings, we finally selected 210 full papers including six poster papers to publish. We would like to express our gratitude and appreciation to all of the reviewers who helped us maintaining the high quality of manuscripts included in this conference proceedings. We would also like to extend our thanks to the members of the organizing team for their hard work. We are tremendously grateful for the contributions and support received from authors, participants, keynote speakers, program committee members, session chairs, steering committee members and others in their various roles. Their valuable support, suggestions, dedicated commitment and hard work have made FTC 2020 a success. We hope that all the participants of FTC 2020 had a wonderful and fruitful time at the conference! Kind Regards, Kohei Arai

v

Contents

Biofluid-Biostructure Interaction Analyses Using Comprehensive Patient-Specific Geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Milan Toma and Rosalyn Chan-Akeley Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gift Arnold Mugisha, Christine Muhumuza, Faith-Michael Uzoka, Chinyere Nwafor-Okoli, Joletta Nabunje, Melody Arindagye, and Justine N. Bukenya

1

17

Dynamic Causality Knowledge Graph Generation for Supporting the Chatbot Healthcare System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hong Qing Yu

30

Healthcare Emergency Room Optimization Using a Process Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soraia Oueida and Yehia Kotb

46

Federated Learning Approach to Support Biopharma and Healthcare Collaboration to Accelerate Crisis Response . . . . . . . . . . . . . . . . . . . . . Arijit Mitra, Abrar Rahman, and Fuad Rahman

64

Ethical Analysis on the Application of Neurotechnology for Human Augmentation in Physicians and Surgeons . . . . . . . . . . . . . . . . . . . . . . . Soaad Qahhar Hossain and Syed Ishtiaque Ahmed

78

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Edgard Vargas-Solís, Daniel Cárdenas-Salas, Juan Gutierrez-Cardenas, and Vilma S. Romero-Romero Detecting Invasive Ductal Carcinoma with Semi-supervised Conditional GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Jeremiah W. Johnson

vii

viii

Contents

Computation of Incline Barriers via Smartphone Sensors in the Mobile App WheelScout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Bettina Harriehausen-Mühlbauer Sensor Networks and Personal Health Data Management: Software Engineering Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Xiang Zhang, Jialu Zhang, Matthew Pike, Nasser M. Mustafa, Dave Towey, and Vladimir Brusic Utilizing Digitized Surveys for Data Collection: The Case of Developing Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Abdullah Sarhan, Omar Addam, Jone Rokne, and Reda Alhajj Towards Better Remote Healthcare Experiences: An mHealth Video Conferencing System for Improving Healthcare Outcomes . . . . . . . . . . 180 El Sayed Mahmoud, Edward R. Sykes, Blake Eram, Sandy Schwenger, Jimmy Poulin, and Mark Cheers Volume Visualization and Beams Towards Computational Cancer Treatment Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Ismail Bahkali and Sudhanshu Kumar Semwal Virtual Reality and Visualization of 3D Reconstructed Medical Imaging: Learning Variations Within Detailed Human Anatomies . . . . 217 Erik N. Gaasedelen, Alex J. Deakyne, Alexander R. Mattson, Lars M. Mattison, Mikayle A. Holm, Jorge D. Zhingre Sanchez, Megan M. Schmidt, Michael G. Bateman, Tinen L. Iles, and Paul A. Iaizzo Development of a Socially Assistive Robot Controlled by Emotions Based on Heartbeats and Facial Temperature of Children with Autistic Spectrum Disorder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Teodiano Bastos, Lucas Lampier, Christiane Goulart, Vinicius Binotte, Guilherme Baldo, Carlos Valadão, Eliete Caldeira, and Denis Delisle Framework for an Integrated Ehealth Platform with Smart Diagnostic Engine for Improved Healthcare Access for Rural Dwellers . . . . . . . . . 240 Francis E. Idachaba and Ejura Mercy Idachaba Medical Visualization Using 3D Imaging and Volume Data: A Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Ismail Bahkali and Sudhanshu Kumar Semwal Matching System for Animal-Assisted Therapy Based on the Levenshtein and Gale-Shapley Algorithms . . . . . . . . . . . . . . . . . 262 Giuliana Gutiérrez-Rondón and Juan Gutiérrez-Cárdenas The ICT Design for Modern Education Technology and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Stefan Svetsky, Oliver Moravcik, Dariusz Mikulowski, and Mariya Shyshkina

Contents

ix

RiWAArch Style: An Architectural Style for Rich Web-Based Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Nalaka R. Dissanayake and G. K. A. Dias Digitalization of the Oil and Gas Industry: Practical Lessons Learned from Digital Responses During the First Stage of the COVID-19 Outbreak . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Trung Nguyen, Raymond G. Gosine, and Peter Warrian Code Complexity Metrics Derived from Software Design: A Framework and Theoretical Evaluation . . . . . . . . . . . . . . . . . . . . . . . 326 Omar Masmali and Omar Badreddin Does Context Matter? Assessing the Current State of Quality Practice During Software Development in Small Software Companies . . . . . . . . 341 Micheal Tuape, Phemelo Ntebane, and Pulafela Majoo Research on Various Software Development Lifecycle Models . . . . . . . . 357 Nabeel Asif Khan Usability Engineering Process for Medical Devices . . . . . . . . . . . . . . . . 365 Momina Shaheen, Tayyaba Anees, Muhammad Junaid Anjum, and Aimen Anum Evolution of Push-Communication Towards the Rich Web-Based Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 Nalaka R. Dissanayake, Dharshana Kasthurirathna, and Shantha Jayalal SAI: Sports Analysis and Informatics . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Archit RamanaSai Kambhamettu and Chandra Kambhamettu Theoretically Validated Complexity Metrics for UML State Machine Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 Omar Masmali and Omar Badreddin The New Role of Cloud Technologies in Management Information Systems Implementation Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Patryk Morawiec and Anna Sołtysik-Piorunkiewicz Design and Experimental Study of a Device that Allows Urban Gardeners to Understand What Their Plants May Be “Feeling” . . . . . . 442 Osvaldo Luiz de Oliveira and Waldinei Bispo de Lima Optimal Polynomial Backoff for IEEE 802.11 DCF . . . . . . . . . . . . . . . . 460 Bader A. Aldawsari and J. Haadi Jafarian Aware Diffusion Routing Protocol with Reliable Data Delivery Provision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 Kamil Samara and Hossein Hosseini

x

Contents

Exploration of Network Slice Charging in 5G Networks . . . . . . . . . . . . 482 Chin Hsuan Chen, Fuchun Joseph Lin, Chia-Hsuan Yu, and Wan-Hsun Hu Pervasive Information Architecture in Service Design Blueprints: Walking Tours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 Camila Meira, Mariana Bittencourt, Julia Souza, Ana Ribeiro, Katharina Poll, and Adriano Bernardo Renzi Cluster Merging Scheme in VANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 Osama AlQahtani, Frederick T. Sheldon, Axel Krings, and Julie Beeston Use of High Mobility Nodes to Improve Connectivity in Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528 Lucas dos Santos, Paulo Nascimento, Lucila Bento, Raphael Machado, Paolo Ferrari, and Claudio Amorim Message Routing and Link Maintenance Augmentation for Robotic Network Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546 Mustafa Ayad and Richard Voyles A Machine Learning-Based Migration Strategy for Virtual Network Function Instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 Dimitrios Michael Manias, Hassan Hawilo, and Abdallah Shami Optimizing Connectivity for the Internet of Vehicles . . . . . . . . . . . . . . . 578 Sumanjit Gill, Robert Wong, Shahab Tayeb, Fletcher Trueblood, and Matin Pirouz On Testing Microservice Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597 Arne Koschel, Irina Astrova, Mirco Bartels, Mark Helmers, and Marcel Lyko Development of an IoT Based Smart Campus: Wide Shuttle Tracking System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610 Francis E. Idachaba Bringing the QR Code to Canada: The Rise of AliPay and WeChatPay in Canadian e-Commerce Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622 Geneva Nam 2D Vector Map Fragile Watermarking with RST Invariance and Region Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 Nana Wang, Xiaohui Guo, and Jilong Bian Quantifying Information Exposure by Web Browsers . . . . . . . . . . . . . . 648 Fadi Mohsen, Mohamed Shehab, Maxamilliano Lange, and Dimka Karastoyanova Deception Detection in Videos Using Robust Facial Features . . . . . . . . . 668 Anastasis Stathopoulos, Ligong Han, Norah Dunbar, Judee K. Burgoon, and Dimitris Metaxas

Contents

xi

Ethereum Blockchain Enabled Secure and Transparent E-Voting . . . . . 683 Vishakh Rao, Ankur Singh, and Bhawana Rudra RECO-DryGASCON: Re-configurable Lightweight DryGASCON Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703 Robert Herndon, Rafed El-Issa, Daniel Heer, Jinjun Xiong, Wen-Mei Hwu, and Mohamed El-Hadedy Techniques Implemented in Software Protectors: A Journey with DBI Through What Protectors Use to Detect Bad Guys . . . . . . . . . . . . . . . . 722 Federico Palmaro and Luisa Franchina A Self-reliant Trust Management Model for VANETs . . . . . . . . . . . . . . 738 Ibrahim Abdo Rai, Riaz Ahmed Shaikh, and Syed Raheel Hassan Edge Crypt-Pi: Securing Internet of Things with Light and Fast Crypto-Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 749 Mohamed El-Hadedy, Xinfei Guo, Kevin Skadron, and Wen-Mei Hwu Sybil Attack Detection and Prevention in VANETs: A Survey . . . . . . . . 762 Yunpeng Zhang, Bidit Das, and Fengxiang Qiao Lightweight Cryptography for the Internet of Things . . . . . . . . . . . . . . 780 Alaa Hassan A Study on Soft-Core Processor Configurations for Embedded Cryptography Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796 Benjamin Kueffler, Wen-Mei Hwu, and Mohamed El-Hadedy Classification Method for Malware Detection on Android Devices . . . . . 810 Arthur Fournier, Franjieh El Khoury, and Samuel Pierre Social Structure Construction from the Forums Using Interaction Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 830 Kazuaki Kashihara, Jana Shakarian, and Chitta Baral Relationship Between Facial Recognition, Color Spaces, and Basic Image Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844 Giuliano Millan, Anas Salah Eddin, Jinjun Xiong, Wen-Mei Hwu, and Mohamed El-Hadedy Proposing Recommendations for Improving the Reliability and Security of Information Systems in Governmental Organizations in the Republic of Kazakhstan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854 Askar Boranbayev, Seilkhan Boranbayev, Yerzhan Seitkulov, and Askar Nurbekov An Application of Metacyclic and Miller-Moreno p-Groups to Generalization of Diffie-Hellman Protocol . . . . . . . . . . . . . . . . . . . . . 869 Ruslan Skuratovskii

xii

Contents

A Systematic Literature Review of Smart Contract Applications . . . . . . 877 Eghbal Ghazizadeh and Tong Sun A Framework of Signature-Matching-Algorithms for IoT Intrusion Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 889 Menachem Domb Suspected Software-Code Restoration Using a Dictionary Led System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899 Menachem Domb and Guy Leshem Hypervisor Based IDS Solution Approach Using Hybrid Anomaly Detection Model in Cloud Computing Environment . . . . . . . . . . . . . . . . 909 Frances Osamor and Anteneh Girma Security and Performance Considerations of Improved Password Authentication Algorithm, Based on OTP and Hash-Chains . . . . . . . . . 921 Ivaylo Chenchev, Ognian Nakov, and Milena Lazarova Audiouth: Multi-factor Authentication Based on Audio Signal . . . . . . . 935 Muhammad Ali Fauzi and Bian Yang Zero Trust in the Context of the Utility Industry . . . . . . . . . . . . . . . . . . 947 Nikhil Kumar and Noah LaRoy Secure Shared Processing on a Cluster of Trust-Anchors . . . . . . . . . . . 968 Keith Mayes The Development Process of a Tooth Brushing App on Android Using Movement Direction Detection with OpenCV . . . . . . . . . . . . . . . . . . . . . 985 Radu-Stefan Ricman, Roland Szabo, and Aurel Gontean Co-creative Robotic Arm for Differently-Abled Kids: Speech, Sketch Inputs and External Feedbacks for Multiple Drawings . . . . . . . . . . . . . 998 Shama Zabeen Shaik, Vidhushini Srinivasan, Yue Peng, Minwoo Lee, and Nicholas Davis LightCloud: Future of Dynamic Lighting in the Shared Space . . . . . . . 1008 Elena C. Kodama, Nan Zhao, and Joseph A. Paradiso Developing Blockchain Supported Collective Intelligence in Decentralized Autonomous Organizations . . . . . . . . . . . . . . . . . . . . . . . 1018 Aelita Skarzauskiene, Monika Maciuliene, and Daniel Bar A Software Platform for Use Case Driven Human-Friendly Factory Interaction Using Domain-Specific Assets . . . . . . . . . . . . . . . . . . . . . . . . 1032 Felix Brandt, Eric Brandt, David Heik, Dirk Reichelt, and Javad Ghofrani

Contents

xiii

Asset Administration Shell: Domain Specific Language Approach to Integrate Heterogeneous Device Endpoints . . . . . . . . . . . . . . . . . . . . 1044 Felix Brandt, Eric Brandt, Javad Ghofrani, David Heik, and Dirk Reichelt Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1053

Biofluid-Biostructure Interaction Analyses Using Comprehensive Patient-Specific Geometries Milan Toma1(B) and Rosalyn Chan-Akeley2

2

1 Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA [email protected] Lang Research Center, NewYork-Presbyterian Queens, Flushing, NY 11355, USA http://www.tomamil.com

Abstract. A multitude of imaging modalities are used to acquire patient-specific geometries. The extracted 3D patient-specific models are then typically simplified. The most common reason is to increase the speed of subsequent calculations. However, it can be argued that these simplifications nullify the reason for calling these geometries patientspecific. This argument is supported by the fact that, once modified, the geometry no longer accurately represents the patient it was extracted from. Moreover, biomedical applications do not just include complex geometries, they also include large deformations of their solid domains. The present paper shows the use of a next-generation SPH method that allows the user to gain results in a timely manner without the need to simplify the geometry. SPH is used to simulate the fluid domain, which is combined with a high-order finite element method used to simulate the solid domain, thus creating an ideal combination of methods to simulate fluid-structure interaction. The utilization of these methods is especially advantageous when complex geometries are to be included. Keywords: Computer models · Patient-specific computational modeling · Fluid-structure interaction

1

Introduction

This section is two-fold. First, advantages and disadvantages of computational and experimental methods are discussed. Second, since not every numerical method is suitable for calculations that include patient-specific geometries, the process of choosing the right numerical methods is examined. 1.1

Simulations and Experiments

Human patient-specific geometries are usually of a complex nature. It is challenging to conduct fluid-structure interaction (FSI) analyses while preserving all c Springer Nature Switzerland AG 2021  K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 1–16, 2021. https://doi.org/10.1007/978-3-030-63092-8_1

2

M. Toma and R. Chan-Akeley

the small-scale features of the geometries used, e.g. the gyri and sulci of the brain (in layman’s terms, the folds and indentations in the brain that give it its wrinkled appearance). The gyri and sulci of the brain greatly increase the brain’s surface area, a phenomenon commonly observed in other parts of the human body. Cerebrospinal fluid (CSF) fills the skull and surrounds the brain. It acts as a shock absorber for the central nervous system (CNS), cushioning the brain within the skull [1–3]. Due the increased surface area created by the gyri and sulci, the actual contact area between the brain and CSF is also larger. The complicated anatomy of the brain and the physiology of its relationship with CSF, for example how they interact in trauma or how the relationship changes in response to the brain’s unique form, makes the task of conducting an FSI analysis of said relationship inherently formidable. Understandably, many FSI laboratories choose to circumvent these intrinsic issues by simplifying the brain geometry into a single smooth mass placed in the center of another simplified geometry representing skull. The reliability of results achieved utilizing this simplified format are questionable, especially in the context of practicality and clinical applicability. Results that are unable to reflect, at least in some part, a close mirror of the true physiologic process cannot be used by physicians in a clinical setting to inform clinical decision-making. Validating computational models and frameworks is a common practice, but numerical simulations are still not preferred above actual experiments. There has been a worldwide benchmark Food and Drug Administration (FDA) study to standardize computational fluid dynamics (CFD) techniques used to assess the safety of medical devices [4], which was replicated also by our group with a special focus on mesh sensitivity analysis [5]. The benchmark flow model used for this study consists of a nozzle with a concentrator and sudden expansion, i.e. the geometry used is simpler than those used in patient-specific simulations. Moreover, CFD is numerically also more straightforward than FSI methods. Over 40 groups (self-ascribed as beginner, intermediate or expert) delivered their results. The results of the FDA study show that CFD results always need to be validated even when produced by experts. It was found that even the worst ‘beginner’ was actually closer to the experimental measurements than the worst ‘expert’; and some ‘beginners’ matched the experimental results better than some ‘experts’ [4]. While validations are performed on every model published, the input parameters (e.g. material properties, boundary/initial conditions, and so forth) are usually ‘calibrated’ in order to reach the desired results. Our group at the Georgia Institute of Technology has previously shown that it is possible to achieve computational results equivalent to the experimental ones, utilizing comprehensive patient-specific geometries that function without the need to ‘calibrate’ the material models used (i.e. the material parameters acquired from the tissue samples can be used directly). We have shown this in simulations and experiments conducted to mimic mitral valve (MV) function during systole and diastole in order to study it in both healthy and diseased states [6–13]. Other groups have

BioFSI using Patient-Specific Geometries

3

Realistic

Ex

pe

rim

en tal

also reached the same conclusion when analyzing clinical data quality and how it affects FSI simulations of patient-specific stenotic aortic valve models [14]. Ideally, we would investigate the biofluids interacting with the solid deformable domain using experimental in-vivo methods. To make it affordable and realistic would require the use of an invasive method, which would not be morally feasible. To make it affordable and non-invasive would not yield realistic results. To make it realistic and non-invasive would not be affordable or, in this case, even possible. As such, in situations where experimental methods cannot be used, a computational model is a viable alternative. However, computational methods have drawbacks as well. The use of complex patient-specific geometry often requires simplifications in the numerical algorithms used. Similarly, a computation with high-precision numerical algorithms and comprehensive patientspecific geometry could take even months to complete. Furthermore, the risk and occurrence of numerical errors must be considered when complex long calculations are involved. Thus, as the three constraints are interdependent (none of them can be altered without affecting one or both of the others) the triple constraint model can be applied here, as depicted in Fig. 1.

Cost

Complex Geometry

Co

mp

uta

tio

na

l

Non-invasive

Precision

Time

Fig. 1. The triple constraint summarizing the drawbacks of both the experimental and computational methods.

4

1.2

M. Toma and R. Chan-Akeley

Choosing Numerical Methods

Computer simulations provide virtual hands-on experience when actual handson experience is not possible. To use these simulations in medical science, they need to be able to predict the behavior of actual processes using actual patientspecific geometries with no simplifications. The most common numerical method used to perform FSI analyses is the Arbitrary Lagrangian-Eulerian (ALE) formulation. The accuracy of ALE methods depends on the use of meshes with high density, which can be a problem when modeling small gaps between the solid and fluid domains, especially when the solid domain is meant to represent a comprehensive patient-specific geometry of high complexity. In models involving detailed anatomy, the use of ALE brings additional complications. For example, ALE methods are either strongly coupled or weakly coupled. Weakly coupled ALE methods are known for accuracy issues when compared to strongly coupled methods and have difficulty performing well when attempting to parallelize the algorithms. Strongly coupled ALE methods yield large ill-conditioned system matrices, i.e. system matrices that are prone to large numerical errors with each system-solving iteration within each non-linear iteration within each time-step. They require additional algorithms to be implemented alongside the ALE algorithm just to keep well conditioned. Consequently, increasing numerical operations yields more numerical errors, thus making it nearly impossible to perform effective parallelization of the entire framework [15]. As the ALE methods rely on the use of meshes with high density, the users tend to use simplified geometries to decrease the number of elements necessary for stable calculations. This is done in an attempt to cut processing time down to weeks instead of months, all the while running on supercomputers utilizing thousands of processors while facing convergence issues. The combination of Smoothed-Particle Hydrodynamics (SPH) methods, used to simulate the fluid domain flow, with a high-order finite element method used to simulate the solid domain deformations, is ideal for simulating FSI, especially when complex geometries are included. Using SPH methods provides numerical stability because the contact between the solid and fluid domains is easily treated numerically. Moreover, SPH is highly parallelizable. That being the case, it is possible to run FSI simulations that are numerically stable, precise, parallelized on a standard GPU workstation (as opposed to large supercomputers), which do not require the use of simplified geometries, and possess a runtime of hours or days rather than weeks and months.

2

Methods

Two critical points are addressed in this section. First, when preserving the smallest details of complex patient-specific geometries the proper image processing techniques need to be used. Second, for the computational methods to handle the complex geometries, next-generation algorithms need to be chosen.

BioFSI using Patient-Specific Geometries

2.1

5

Creating Patient-Specific Geometries

Medical imaging is mainly used to create visual representations of the organs for clinical analysis and/or a visual representation of their function. There are numerous software packages available to create 3D printable anatomic models from medical scans. Patient-specific models are used with increasing frequency to plan and optimize surgical procedures. Other uses include training simulators for educational purposes, verification and validation of new medical devices, and/or computational simulations and analyses. Non-invasive MRI, CT or 3D ultrasound testing generates patient-specific imaging scans. Digital Imaging and Communications in Medicine (DICOM) is a standard for storing and transmitting medical images. The DICOM output provides a series of 2D images (slices) that are then reconstructed in 3D. The first step is to isolate the anatomy of interest (i.e. segmentation) and generate a surface stereolithography (STL) file. The resulting STL file can be used for rapid prototyping, 3D printing, computer-aided manufacturing, and/or creating volume mesh. The volume mesh can then be used for computer simulations. A variety of commercial, freeware, and open-source software packages are available to perform the segmentation. The first step in developing patient-specific models is to obtain DICOM data. All medical imaging devices that support the DICOM standard will export files readable by the editing software of choice. The human body is geometrically complex with multiple layers of varying densities. Thus, several imaging modalities are typically used for diagnostics and prognostics. One or more of the following modalities are used: – MRI (Magnetic Resonance Imaging) uses a strong magnetic field that excites hydrogen atoms in the body. The scanner then detects the radio frequency emitted by the hydrogen atoms. It is used for imaging soft structures in the body because hydrogen atoms exist in large quantities in humans, especially in areas with high concentrations of water and/or fat. Additionally, it does not expose the patient to radiation. – CT (Computed Tomography) uses a series of X-rays taken from different angles. In the resulting images, the brighter areas are denser than dark areas, e.g. a bone is brighter than the surrounding connective tissues. This method is quick and provides high resolution, but it is not as accurate for soft tissues and results in radiation exposure. – 3D ultrasound uses high-frequency sound waves sent into the body at different angles. As they reflect back, the receiving device displays them to produce a live 3D image of the internal organs. The method is cheap, provides a live image, and has no radiation exposure. However it is of low resolution and does not show internal structures. The software packages used to display DICOM images range from opensource software to enterprise-level solutions with FDA clearance. There are dozens of options available. The data set must be segmented to separate the

6

M. Toma and R. Chan-Akeley

y

z

x

Fig. 2. Original (yellow) surface from the threshold segmentation compared to the final surface mesh (green) after smoothing techniques applied. Zoom-in to an attachment points between the chords and leaflets is shown to demonstrate the complexity of the geometry.

Fig. 3. The depiction of the entire head model with skull, cerebrum, cerebellum, pituitary gland and brainstem, respectively. The subarachnoid space and other cavities are filled with fluid particles (blue dots surrounding the brain model in the lower right corner). The entire model with half the skull is also displayed (lower left).

BioFSI using Patient-Specific Geometries

7

area of interest from the surrounding tissues. Segmentation is done by marking the relevant data and discarding the surrounding data. Cutting-edge software solutions use advanced algorithms to automatically, or semi-automatically, separate specific organs from the surrounding tissues. Some of the methods used for automatic segmentation include region growing [16], region competition [17], digital subtraction [18], and seed growing [19]. In recent years, the method of applying machine learning, especially deep learning, to medical imaging has risen in popularity, due to its ability to achieve state-of-the-art performance in image analysis and processing [20]. In numerous instances, AIbased segmentation algorithms have successfully outperformed human experts [21]. Such endeavors are actively transforming the field of medical image processing. In scans where a certain anatomy has a very distinct set of pixel values, thresholding can be used, especially where high contrast is present. For example, in the case of CT images, the pixel values represent the density. Hence, keeping only the brighter pixels leaves only the bone. The pixel values are set within a certain range, with a minimum value as black and a maximum value as white. Values in between make up the different shades of gray. Setting the threshold maximum and minimum manually to a given range isolates a user-defined region. Our groups used μCT to create and validate fully comprehensive FSI MV and Tricuspid Valve models with 3D chordal structures [6–13]. Using MRI data available online [22] we were able to create and validate a fully comprehensive FSI head/brain model [23–28]. Figure 2 shows an example of MV geometry created by thresholding segmentation from DICOM images and produced using μCT and subsequent smoothing. An example of head/brain geometry created from DICOM images and produced using MRI and subsequent surface processing is shown in Fig. 3. Figure 2 and 3 demonstrate how small-scale details need to and can be preserved if the patient-specific geometry is to reflect and retain the target’s original intricacy. To reiterate, in order to keep the simulations clinically relevant the patient-specific geometry cannot be simplified. With simplifications it does not represent the patient it was acquired from. 2.2

Computer Simulations

In the studies presented here, the fluid motion and boundary interaction calcuR (IMPETUS Afea AS, lations are solved with the IMPETUS Afea γSPH Solver Norway), while large deformations in the solid parts are taken care of by the R . The γSPH solver uses a next-generation SPH method IMPETUS Afea Solver with increased accuracy. Both solvers use a commodity GPU for parallel processing. All solid elements are fully integrated, thus removing the possibilities of hourglass modes and element inversion, occurrences that plague classic underintegrated elements. Both fluid and solid domains, as well as their interaction, are solved with an explicit integration scheme.

8

M. Toma and R. Chan-Akeley

The contact, i.e. particle to structure contact, is very simple, which is why γSPH is ideal for these complex applications. It can easily account for movement in any direction, unlike finite element fluid solvers which involve more complicated contact and usually require re-meshing of the fluid domain during the simulation. The ability of IMPETUS to achieve a very high resolution in terms of particle density results in a very accurate particle to structure contact, especially in regards to a structure as detailed and complex as the brain. The other critical part of the simuR Element Technology provides lation is the structural model. The IMPETUS Aset high order tetrahedron elements that are accurate for nonlinear dynamic response. This allows the automeshing of complicated structures, such as those found in biomedical applications. The accuracy of these elements has been demonstrated in many commercial applications when compared with hexahedron elements. In addition, they are also very accurate in bending and plasticity. All simulations shown are solved on a standard workstation. Parallel acceleration was achieved with a Tesla K40 GPU with 12 GB of Graphic DDR memory and 2880 CUDA Cores. To confirm that convergence was reached, h-refinement of the finite element mesh is performed, and the solution is found to be equivalent. Similarly, a smaller number of fluid particles are used to obtain results within 5% of the values obtained with the higher number. This confirmed that the results converged. Our prior publication describes the SPH equations in greater detail [9]. To reiterate, the SPH method is chosen for this study because traditional FSI techniques can be computationally expensive and challenging regarding their parallelization [15]. In order to use traditional FSI techniques, geometrical simplifications would be needed, thus necessitating the sacrifice of anatomical accuracy. Moreover, in recent years the SPH has been increasingly used in biomedical applications [29].

3

Results

Following the two examples shown above, i.e. MV and brain geometries, both scenarios are similarly discussed in the results section. MV closure validation is shown and subsequently the blood dynamics of MV closure are described. Additionally, an example of traumatic brain injury is simulated and shown. 3.1

Mitral Valve Closure

The results of the FSI simulations using the MV geometry and its material properties have been validated against experimental data in two ways. Firstly, the direction and magnitudes of papillary muscle forces were compared with experimental measurements [11] and, secondly, the coaptation line between the anterior and posterior leaflets at closure has been compared with the μCT images (Fig. 4) [10].

BioFSI using Patient-Specific Geometries

9

x

y

(a) µCT

z

(b) FSI µCT

FSI

Fig. 4. Closed leaflets reconstructed from µCT images compared to the results of FSI simulations. The curves represent the coaptation line where the posterior and anterior leaflets are in contact [10].

Since the FSI analysis can provide additional fluid-related results, it is possible to observe the movement of mitral valve leaflets in relation to the cardiac cycle. The Fig. 5 shows the regurgitant volume measured as the mass of the fluid particles crossing the area at the level of the MV annulus [9]. No regurgitation is observed from the point when both the leaflets come in contact together, i.e. T = 0.75·Tsys , whereas the highest level of regurgitation is observed at time point T = 0.3·Tsys . The closure of the MV leaflets is initiated at around T = 0.1·Tsys . Tracking the fluid particles up to this time point allows us to observe the mechanism that initiates the closure, i.e. the occurrence of the eddies demonstrated by Henderson and Johnson in 1912 [30].

0.08

z x

Mass [g]

0.06

y

0.04

0.02

0.00 0.00

0 .2 5

0 .5 0 0 . 75 Time/Tsys

1 . 00

Fig. 5. Regurgitant volume measured as the mass of the fluid particles crossing the area at the level of the MV annulus [9].

10

3.2

M. Toma and R. Chan-Akeley

Fluid Dynamics of Mitral Valve Closure

The Fig. 6 shows the trajectory of the fluid particles, and their evolution in time from the beginning of the systole to T = 0.1·Tsys , originating at the plane close to the annulus, denoted by the red dotted line. The black dotted line in Fig. 6(f) shows downward trajectory. Four pressure waves, behind each leaflet, develop quickly at the beginning of the cycle, before T = 0.020·Tsys , and provide the initial impulse to the leaflets causing their partial closure. This is directly followed by ventricular systole with the immediate rise of the intra-ventricular pressure, starting at T = 0.080·Tsys .

z (a) T = 0.001·Tsys

(b) T = 0.020·Tsys y

x

z

(c) T = 0.040·Tsys

(e) T = 0.080·Tsys

x

y

(d) T = 0.060·Tsys

(f) T = 0.100·Tsys

Fig. 6. Fluid particles traced in time originating at plane close to the annulus (red dotted line).

BioFSI using Patient-Specific Geometries

3.3

11

Traumatic Brain Injuries

Structurally, brain geometry is even more complex than that of the MV. A multitude of possible traumatic scenarios can be simulated. One such scenario is abusive head trauma (AHT), which includes the phenomenon commonly known as Shaken Baby Syndrome. AHT is the leading cause of fatal brain injuries in children younger than 2 years old [31]. Children who are victims of AHT can suffer irreversible neurological damage, resulting in development delay and disability. Therefore it is imperative that better modalities are created to both study its effects and predict patient outcomes. In this model we have replicated the CSF’s cushioning effect for multiple cycles [27]. In the first shake, CSF traveled to the sites of hyperextension and hyperflexion, providing the anticipated cushioning effect. However, during hyperflexion on the second shake, the fluid did not have enough time to reach the affected areas. In other words, following the first shake, the CSF was unable to prevent the brain from colliding with the skull, suggesting that the fluid offers no protection at repeated frequencies (Fig. 7). Frontal view

(a)

(d)

(b)

(e)

(c)

(f )

2nd Hyperflexion

Hyperextension

1st Hyperflexion

Occipital view

Fig. 7. Shaken baby syndrome principal stress (Max - red: 3 MPa, Min - blue: 0 MPa) shown at the peaks of the three phases, i.e. 1st hyperflexion, hyperextension and 2nd hyperflexion; occipital and frontal views of the right hemisphere [27].

12

4

M. Toma and R. Chan-Akeley

Conclusion

The above examples and discussion emphasize the importance of preserving the geometry of anatomy for the patient-specific organ being studied. For example, the study of fluid dynamics of MV closure is incomplete if simplified MV geometry is used. This is clear when observing the reaction of fluid particles to pressure in relation to MV closure. The way fluid particles react in order to cause MV closure will change with different MV geometries. When a geometry that does not represent the MV in its entirety is used, the fluid particles form different trajectories. Naturally, using a simplified MV geometry will yield different fluid particle trajectories than when using a comprehensive patient-specific MV geometry. Hence, it can be argued that the results of studies that do not use comprehensive patient-specific geometries cannot be conclusive enough to be considered truly representative of the processes intended to be studied. The brain is the most complex organ in the human body. Because of this, computational models of the human brain are inherently challenging. As most computational models available in the literature are not based on an FSI analysis, they do not simulate the interaction between the CSF and cerebral cortex. Those models treat the CSF as if it was solid. However, as seen in our results, the fluid particles do not remain confined in a stagnant state in the space between the skull and brain. The AHT results displayed above show that during the first shake there are fluid particles between the skull and brain (best visible in Fig. 7(a)) providing a cushioning effect to protect the brain. However, during the second shake (2nd hyperflexion), no fluid particles can be found in the occipital region between the brain and skull (Fig. 7(c)), thus resulting in direct contact between the skull and brain. If the CSF is modeled using solid elements, the model would fail to correctly yield such a basic yet crucial conclusion. Even if in models where FSI algorithms are used, the brain geometry is simplified in order to make the calculations less computationally expensive and/or to avoid convergence issues. However it should be emphasized that with simplified geometries the fluid particle trajectories are different and therefore not representative.

5

Discussion

The generally accepted concept regarding the movement of the MV leaflets is that when the left atrium contracts it forces a small quantity of blood into the relaxed left ventricle. This raises the intra-ventricular pressure by a very small value but the change is large enough to form eddies behind the leaflets, causing their approximation or partial closure. This is then followed by ventricular systole and an immediate rise in intra-ventricular pressure. When the intra-ventricular pressure exceeds the intra-atrial pressure, complete closure of the valve occurs and regurgitation is prevented [32]. At the beginning of the twentieth century, Henderson and Johnson demonstrated experimentally the importance of the “breaking of the jet” phenomenon at the end of atrial systole, as well as the occurrence of eddies, or vortex formation, behind the atrioventricular valves, to initiate of mitral valve closure [30].

BioFSI using Patient-Specific Geometries

13

They show that normal atrial activity can initiate the closure of the atrioventricular valves before the onset of ventricular systole [30,32]. Early work on mitral valve action indicated a relatively wide range of movement for the leaflets during the cardiac cycle, including with a pre-systolic flick and bulging back into the atrium during ventricular systole [32]. Now, a hundred years later, we too have observed the occurrence of the eddies behind the mitral valve leaflets [30] and the pre-systolic flick [32] in our FSI simulations as well. Other groups have already confirmed the occurrence of the eddies computationally long before us [33,34]. However, they used simplified geometries. The mitral valve is considered to have two primary leaflets: the anterior and the posterior. Located in the posterior part of the aortic root, the anterior leaflet has a semicircular shape and is both larger and thicker than the posterior leaflet [35]. It consists of two zones: the rough zone and the clear zone. During systole, the position of the rough zone is adjacent to the posterior leaflet [36,37]. The posterior leaflet, on the other hand, is crescentic with a long circumferential base. Like the anterior leaflet, it can be divided into lateral, central, and medial scallops (referred to as A1, A2, and A3, for the anterior leaflet, and P1, P2, and P3 for the posterior leaflet) [38]. Additional leaflet tissue, known as commissural tissue, can be found at the anterolateral and posteromedial commissures [38]. Previous studies generally confirm the occurrence of the two largest eddies behind the two largest leaflets. In our studies, due to the use of comprehensive patient-specific geometries, we have observed the occurrence of an additional two smaller eddies behind the commissural leaflet tissue as well. AHT is defined by the Centers for Disease Control (CDC) as “an injury to the skull or intracranial contents of an infant or young child (under 5 years of age) due to inflicted blunt impact and/or violent shaking” [31]. The incidence of AHT occurring in the first year of life is estimated to be approximately 35 cases per 100,000 [39]. From 1999–2014, AHT resulted in approximately 2,250 deaths amongst children under 5 years old in the United States [40]. AHT is a serious condition with significant morbidity and mortality. 65% of victims have significant neurological disabilities, and 5% to 35% of infants die from their injuries [39]. The majority of survivors suffer permanent cognitive and neurologic impairment [39]. However, the long-term effects of AHT are often difficult to diagnose and predict. Computational simulations can help physicians visualize the true impact of AHT which an then assist them in formulating an accurate prognosis. However, existing simulations are insufficient, as they portray the fluid as an elastic solid and fail to replicate the intricate brain anatomy and how it interacts with CSF. In our previous study, we address these deficiencies, using a more precise simulation that reveals that the protection of CSF may last only for a single shake [27]. We do not wish to claim that this SPH approach is better suited for modeling the CSF than the more commonly used solid elements found in other models. However, there may be cases where it might be necessary to study the behavior of the fluid in a model using an actual fluid domain. For example, when studying the effect of CSF drainage via ventriculoperitoneal shunt as a form of treatment

14

M. Toma and R. Chan-Akeley

for hydrocephalus [26], models that utilize solid elements to study the behavior of CSF would not be useful. However, in other more straightforward scenarios, such as traumatic brain injury where a single blow to the head is studied, the use of solid elements is justified when properly validated. It is certainly less computationally expensive than using traditional FSI techniques in order to represent the CSF using a fluid domain. Though, compared to the methods used in this study, the use of solid elements to model the CSF does not provide significant advantage in terms of computational cost. Funding This study was supported by a grant from the National Heart Lung and Blood Institute (R01-HL092926) and a donation from New York Thoroughbred Horsemen’s Association. No benefits in any form have been or will be received from a commercial party related directly or indirectly to the subject of this manuscript. Conflicts of Interest The authors declare no conflict of interest. Data Availability Data are available upon reasonable request. Ethical Approval This article does not contain any studies with human participants or animals performed by any of the authors.

References 1. Bothwell, S.W., Janigro, D., Patabendige, A.: Cerebrospinal fluid dynamics and intracranial pressure elevation in neurological diseases. Fluids Barriers CNS 16(1), 9 (2019). https://doi.org/10.1186/s12987-019-0129-6 2. Rengachary, S.S., Ellenbogen, R.G.: Principles of Neurosurgery. Elsevier Mosby, Edinburgh (2012). https://doi.org/10.1016/C2014-0-02313-1 3. Linninger, A.A., Tangen, K., Hsu, C.Y., Frim, D.: Cerebrospinal fluid mechanics and its coupling to cerebrovascular dynamics. Annu. Rev. Fluid Mech. 48, 219–257 (2016). https://doi.org/10.1146/annurev-fluid-122414-034321 4. Stewart, S.F.C., et al.: Results of FDA’s first interlaboratory computational study of a nozzle with a sudden contraction and conical diffuser. Cardiovasc. Eng. Technol. 4, 374–391 (2013). https://doi.org/10.1007/s13239-013-0166-2 5. Toma, M.: Computational fluid dynamics simulations using FDA’s idealized medical device demonstrating the importance of model validation. Biomed. Res. Rev. 1(1), 1–3 (2018). https://doi.org/10.31021/brr.20181104 6. Singh-Gryzbon, S., Sadri, V., Toma, M., Pierce, E.L., Wei, Z.A., Yoganathan, A.P.: Development of a computational method for simulating tricuspid valve dynamics. Ann. Biomed. Eng. 47, 1422–1434 (2019). https://doi.org/10.1007/s10439-01902243-y 7. Bloodworth, C.H., et al.: Ex Vivo methods for informing and validating computational models of the mitral valve. Ann. Biomed. Eng. 45(2), 496–507 (2017). https://doi.org/10.1007/s10439-016-1734-z

BioFSI using Patient-Specific Geometries

15

8. Toma, M., et al.: Fluid-structure interaction analysis of ruptured mitral chordae tendineae. Ann. Biomed. Eng. 45(3), 619–631 (2017). https://doi.org/10.1007/ s10439-016-1727-y 9. Toma, M., Einstein, D.R., Bloodworth, C.H., Cochran, R.P., Yoganathan, A.P., Kunzelman, K.S.: Fluid-structure interaction and structural analyses using a comprehensive mitral valve model with 3D chordal structure. Int. J. Numer. Meth. Biomed. Eng. 33(4), e2815 (2017). https://doi.org/10.1002/cnm.2815 10. Toma, M., et al.: High resolution subject-specific mitral valve imaging and modeling: experimental and computational methods. J. Biomech. Model. Mechanobiol. 15(6), 1619–1630 (2016). https://doi.org/10.1007/s10237-016-0786-1 11. Toma, M., Jensen, M.O., Einstein, D.R., Yoganathan, A.P., Cochran, R.P., Kunzelman, K.S.: Fluid-structure interaction analysis of papillary muscle forces using a comprehensive mitral valve model with 3D chordal structure. Ann. Biomed. Eng. 44(4), 942–953 (2016). https://doi.org/10.1007/s10439-015-1385-5 12. Toma, M., et al.: Fluid-structure interaction analysis of subject-specific mitral valve regurgitation treatment with an intra-valvular spacer. Prosthesis 2(2), 65–75 (2020). https://doi.org/10.3390/prosthesis2020007 13. Toma, M., et al.: Effect of edge-to-edge mitral valve repair on chordal strain: fluid-structure interaction simulations. Biology 9(7), 173 (2020). https://doi.org/ 10.3390/biology9070173 14. Luraghi, G., et al.: Does clinical data quality affect fluid-structure interaction simulations of patient-specific stenotic aortic valve models? J. Biomech. 94, 202–210 (2019). https://doi.org/10.1016/j.jbiomech.2019.07.047 15. Toma, M., Oshima, M., Takagi, S.: Decomposition and parallelization of strongly coupled fluid-structure interaction linear subsystems based on the Q1/P0 discretization. Comput. Struct. 173, 84–94 (2016). https://doi.org/10.1016/j. compstruc.2016.06.001 16. Zucker, S.W.: Region growing: childhood and adolescence. Comput. Graph. Image Process. 5(3), 382–399 (1976). https://doi.org/10.1016/S0146-664X(76)80014-7 17. Zhu, S.C., Yuille, A.: Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18(9), 884–900 (1996). https://doi.org/10.1109/34.537343 18. Balter, S., Ergun, D., Tschnoll, E., Buchmann, F., Verhoeven, L.: Digital subtraction angiography: fundamental noise characteristics. Radiology 152(1), 195–198 (1984). https://doi.org/10.1148/radiology.152.1.6374758 19. Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994). https://doi.org/10.1109/34.295913 20. Lakhani, P., Gray, D.L., Prett, C.R., Nagy, P., Shih, G.: Hello world deep learning in medical imaging. J. Digit. Imaging 31(3), 283–289 (2018). https://doi.org/10. 1007/s10278-018-0079-6 21. Prevedello, L.M., et al.: Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions. Radiol. Artif. Intell. 1(1), e180031 (2019). https://doi.org/10.1148/ryai.2019180031 22. Warner, A., Tate, J., Burton, B., Johnson, C.R.: A high-resolution head and brain computer model for forward and inverse EEG simulation, bioRxiv (2019). https:// doi.org/10.1101/552190 23. Toma, M., Nguyen, P.D.H.: Fluid-structure interaction analysis of cerebrospinal fluid with a comprehensive head model subject to a rapid acceleration and deceleration. Brain Inj. 32(12), 1576–1584 (2018). https://doi.org/10.1080/02699052. 2018.1502470

16

M. Toma and R. Chan-Akeley

24. Toma, M.: Predicting concussion symptoms using computer simulations. Adv. Intell. Syst. Comput. 880, 557–568 (2019). https://doi.org/10.1007/978-3-03002686-8 42 25. Toma, M., Nguyen, P.D.H.: Coup-contrecoup brain injury: fluid-structure interaction simulations. Int. J. Crashworthiness 25(2), 175–182 (2020). https://doi.org/ 10.1080/13588265.2018.1550910 26. Toma, M., Kuo, S.-H.: Computational Assessment of Risk of Subdural Hematoma Associated with Ventriculoperitoneal Shunt Placement, Computer Methods, Imaging and Visualization in Biomechanics and Biomedical Engineering, vol. 36. Springer Nature Switzerland AG (2020). https://doi.org/10.1007/978-3-03043195-2 4 27. Toma, M., Dehesa-Baeza, A., Chan-Akeley, R., Nguyen, P.D.H., Zwibel, H.: Cerebrospinal fluid interaction with cerebral cortex during pediatric abusive head trauma. J. Pediatr. Neurol. (2020). https://doi.org/10.1055/s-0040-1708495 28. Toma, M., Chan-Akeley, R., Lipari, C., Kuo, S.-H.: Mechanism of coup and contrecoup injuries induced by a knock-out punch. J. Math. Comput. Appl. 25(2) (2020). https://doi.org/10.3390/mca25020022 29. Toma, M.: The emerging use of SPH in biomedical applications. Signif. Bioeng. Biosci. 1(1), SBB.000502 (2017). https://doi.org/10.31031/SBB.2017.01.000502 30. Henderson, Y., Johnson, F.E.: Two modes of closure of the heart valves. Heart 4(69) (1912) 31. Parks, S.E., Annest, J.L., Hill, K.A., Karch, D.L.: Pediatric Abusive Head Trauma: Recommended Definitions for Public Health Surveillance and Research. Centers for Disease Control and Prevention, Atlanta (GA) (2012) 32. Dean Jr., A.L.: The movement of mitral valve cusps in relation to the cardiac cycle. Amer. J. Physiol. 40, 206–217 (1916). https://doi.org/10.1152/ajplegacy.1916.40. 2.206 33. Peskin, C.S.: Numerical analysis of blood flow in the heart. J. Comput. Phys. 25(3), 220–252 (1977). https://doi.org/10.1016/0021-9991(77)90100-0 34. Lau, K., Diaz, V., Scambler, P., Burriesci, G.: Mitral valve dynamics in structural and fluid-structure interaction models. Med. Eng. Phys. 32(9), 1057–1064 (2010). https://doi.org/10.1016/j.medengphy.2010.07.008 35. Sanchez Vaca, F., Bordoni, B.: Anatomy, thorax, mitral valve. In: StatPearls. StatPearls Publishing, Treasure Island (FL) (2020). https://www.ncbi.nlm.nih.gov/ books/NBK549884/. Accessed 21 Oct 2019 36. Schubert, S.A., Mehaffey, J.H., Charles, E.J., Kron, I.L.: Mitral valve repair: the French correction versus the American correction. Surg. Clin. North Am. 97(4), 867–888 (2017). https://doi.org/10.1016/j.suc.2017.03.009 37. Carpentier, A.F., et al.: The “physio-ring”: an advanced concept in mitral valve annuloplasty. Ann. Thorac. Surg. 60(5), 1177–1185(discussion 1185–1186) (1995). https://doi.org/10.1016/0003-4975(95)00753-8 38. Dal-Bianco, J.P., Levine, R.A.: Anatomy of the mitral valve apparatus: role of 2D and 3D echocardiography. Cardiol. Clin. 31(2), 151–64 (2013). https://doi.org/10. 1016/j.ccl.2013.03.001 39. Joyce T., Huecker M.R.: Pediatric abusive head trauma (shaken baby syndrome). In: StatPearls. StatPearls Publishing, Treasure Island (FL) (2020). https://www. ncbi.nlm.nih.gov/books/NBK499836/. Accessed 11 Feb 2019 40. Spies E.L., Klevens J.: Fatal abusive head trauma among children aged under 5 Years – United States, 1999–2014. Morb. Mortal. Wkly. Rep. (MMWR) 65, 505– 509 (2016). https://doi.org/10.15585/mmwr.mm6520a1

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners Gift Arnold Mugisha1 , Christine Muhumuza2 , Faith-Michael Uzoka3(B) , Chinyere Nwafor-Okoli4 , Joletta Nabunje5 , Melody Arindagye6 , and Justine N. Bukenya2 1 Shenyang Aerospace University, Shenyang, China 2 School of Public Health, College of Health Science, Makerere University, Kampala, Uganda 3 Department of Math and Computing, Mount Royal University, Calgary, Canada

[email protected] 4 Canadian Institute for Innovation and Development, Calgary, Canada 5 School of Medicine, College of Health Science, Makerere University, Kampala, Uganda 6 Faculty of Medicine, Mbarara University of Science and Technology, Mbarara, Uganda

Abstract. Several health complications arise from non-adherence to medications. Most existing tools lack the ability to facilitate communication between the patient and their healthcare worker, especially in critical situations. We developed a smart pill dispenser/medical assistant to help address some adherence changes. This innovation is equipped with intelligent remote communication capabilities between the patient and a medical professional, and can be used in many settings such as hospital and home facilities to assist in dispensing of medications. The purpose of this study was to explore the user perception and attitudes towards the smart pill dispenser in addressing drug dispensing challenges in hospitals in Kampala, Uganda. This was a pilot study that employed qualitative data collection methods among health providers in four hospitals, involving 17 key informant interviews. The findings revealed that the prototype was easy to use and very acceptable to many health workers. In particular, the respondents rated the prototype highly based on supporting features such camera, video enabled two-way communication and water dispenser. However, some modifications were proposed such as decreasing the size, and increasing the number of dispensing channels. This study provides the impetus for an agile improvement strategy that includes prototype refinement and further acceptance study among users in more health facilities. Keywords: Automated healthcare assistant · Pill dispenser

1 Introduction Medication administration is a crucial aspect of patient care, whether the medication is administered by self, or by health practitioners. Procedural and clinical errors could be dangerous to the patient, affecting overall treatment process, recovery, immune system, and possibly fatal effects [1–3]. Several health complications arise from non-adherence to medications, which is more prevalent among patients with cognitive impairments © Springer Nature Switzerland AG 2021 K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 17–29, 2021. https://doi.org/10.1007/978-3-030-63092-8_2

18

G. A. Mugisha et al.

and those from low-income, less educated families who may not afford existing medication adherence tools. In addition, most existing tools lack the ability to facilitate communication between the patient and their doctor, especially in critical situations that are time-bound such as emergencies. Medication administration is a crucial aspect of patient care, whether the medication is administered by self, or by health practitioners. A number of studies have pointed to the adverse consequences of procedural and clinical errors in medication administration, which could include hospital readmission [4] and antimicrobial resistance [5] among others. Poor medication adherence could be dangerous to the patient, affecting overall treatment process, recovery, immune system, and possibly fatal effects [1, 6]. In addition, it has the socio-economic consequence of increasing health costs and making higher demands on health emergency response systems [7]. This risk of improper medication adherence becomes more exacerbated in self-administered medications, especially in patients with some known distractive disorders, such as Dementia [8]. Medication adherence is critical for patient recovery and overall reduction in treatment costs. Disease-related medical costs and hospitalization rates could be significantly reduced with proper medication adherence [9]. The challenge of medication adherence cuts across various demographic strata, but tends to be more prevalent in the following groups: i) cognitively impaired [8]; ii) younger people [10, 11]; and iii) people in the lower socio-economic stratum [12]. [13] found a concave relationship between age and medication adherence, with adherence being the lowest with the very young and the very old. They also found cost elements, such as copayments, to have negative impact on medication adherence. Mechanical and automatic pill dispensers have been found to improve medication protocols and adherence. For example, [14] reported that automatic pill dispensers could reduce medication error by 64.7%. A number of researchers (e.g. [15] believe that while there is tremendous progress in designing interventions for medication adherence (especially pill dispensing systems), the level of adherence to medications is still not acceptable, especially in low income among less educated individuals. The burden of medical administration puts pressure on the thinly stretched medical workers in the low-income countries. The World Health Organization recommends a density of about 2.28 health workers per 1,000 people in order to make good progress towards Universal Health coverage [16]. This is reflected in the Global Health Workforce Alliance, Health workforce 2030 – towards a global strategy on human resources [16]. This translates into about 23 health workers to serve 10,000 people. According to [16], in 2013, there were average of 33 nurses/midwives and 5 pharmaceuticals personal per 10,000 population. Africa had a precariously low number of (12 nurses/midwives and one pharmaceutical personnel per 10,000 populations). This underscores the need to have technology to assist the few available health workers when delivering services [17]. Emphasized the need for pharmaceutical care practices during pandemics (e.g. Covid-19) to reduce the burden on already weakened health systems in low-to-middle income countries. The pill dispenser could help in pharmaceutical care and also assist in the reduction of infections during epidemics/pandemics. It is important to understand the challenges faced by individuals in developing countries when taking prescribed drug regimes, which include poor power supply, low-to- moderate education on medication sensitivity, and inability to afford expensive medical care services. These challenges underscore the need for

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

19

low cost, efficient and effective medical technologies (including medication dispensing system) that will suit the needs and economic abilities of low-income countries. There is need to develop usable and affordable medication intervention systems that would possess enhanced capabilities to facilitate patient-physician interaction. We developed an initial prototype of an automatic pill dispensing system that would address these needs [18]. Our current system architecture consists of the following functional components: audio module (programmed to send person-specific reminders); messaging module (uses GSM capabilities for bidirectional communication); video module (programmed to use a smart 360-degree camera for video communication); emergency module (to assist in distress response); and dispensing module (for dispensing pills and water. Our initial design was meant to dispense two different sets of pills from two independent chambers. This study aimed at evaluating the acceptability and usability of the prototype by medical practitioners such as physicians, nurses, and pharmacists. The focus of our research was to explore the user perception and attitudes towards the newly developed smart pill dispenser in addressing drug adherence challenges. This study was conducted in Kampala, Uganda, which is a developing country context. In Sect. 2, we show our pill dispenser and its features, while Sect. 3 presents the study methodology. The results are presented in Sect. 4, while discussion of results is in Sect. 5. Some conclusions are drawn in Sect. 6.

2 The Pill Dispenser The architecture of the pill dispensing system is shown in Fig. 1. It is based on the following design principles: a. Simplicity, which enhances upgradability and maintainability. b. Affordability, which is a crucial characteristic of any system that has the potential of high utility in low-income environments. c. Flexibility, a characteristic that enables the system to adapt to new user requirements through the addition of new functionalities and system characteristics. In addition, this system is powered by rechargeable batteries. d. Scalability, which extends the flexibility characteristic to include an ability to accommodate multiple users in future extensions of the current design. Our initial design was meant to dispense two different sets of pills from two independent chambers. Our initial system functioned effectively with crude materials, with the ability to provide user notification, and dispense pills at the appropriate time, in accordance with programming instructions. The implementation of our current design takes advantage of the ubiquity of micro-processor systems and utilizes antimicrobial plastic components for the pill dispensing unit, to make the system smarter and more hygienic. Sustainable medical practices are growing and re-usable plastics components help keep hazardous materials out of landfills, thereby protecting the environment. Antibacterial plastics also have the potential of keeping medical devices and instruments in service longer, and reduce the possibility of product failures attributable to corrosion. In addition, plastics

20

G. A. Mugisha et al. Medicine window speaker

LCD display bu ons speaker

trough

Water reservoir

pathway

Stand

Fig. 1. a. System Blueprint, b. Functional System

are affordable, flexible, and come in different sizes, colours and shapes, especially with the advent of 3-D printing technology [19]. Our current design includes a water reservoir, which is used in lieu of the water supply from the water mains. This helps improve portability. In addition, the water supply has a two-stage control. First, by automatic program control, then followed by pressing a button upon discretion of the user. This gives the user an ability to control the quantity of water supply. Other features include the LCD display and multiple pill holders with a single pathway. The LCD display is programmatically connected to most components to

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

21

provide status indicator for each component. Our system has some smart abilities through the following modules: i) Audio module, which relays information from the memory card, in the language of the patient, applying varying emotional structures that target different age groups, to reduce the level of anxiety associated with taking medicine, especially in children; ii) Messaging module, is equipped with GSM capabilities to provide notifications to patients as well as allowing bi-directional messaging between the patient and the medical practitioner; iii) Emergency module, which is activated by the press of a button, and sends a pre-recorded distress message to the medical practitioner in the event of an emergency iv) Visual module, consisting of a camera setup with audio talk back that enables the doctor to directly talk to the patient. The doctor receives live videos from the patient, which helps in determining the adherence behaviour of the patient, especially in a hospital setting. With on-board motion sensors, the device can send an alarm to the medical practitioner in the event of a fall or some type of emergency. In addition to the camera, the system has a colour coded light indicator for the auditory impaired. Figure 1(b) shows the first functional prototype of the system. The wooden case provides us with a reliable sandbox for the initial concept development, though we recognize weight challenges associated with wood, and are working on finding light weight affordable alternatives.

3 Research Methodology 3.1 Study Design and Population This was designed as a qualitative study to inform the design of a follow-on quantitative survey that will quantify the attributes identified in this formative study and dig into the design of the innovation using existing usability models. This study employed key informant interviews (KIIs) to gain opinions from experts involved in the dispensing of drugs. Key informant interviews are “qualitative, in-depth interviews of 15 to 35 people selected for their first-hand knowledge about a topic of interest. The interviews are loosely structured, relying on a list of issues to be discussed” [20]. The study was conducted in four hospitals including two public and two private, not-for profit referral hospitals in Kampala city in Uganda. These health facilities included Naguru general hospital, Mulago Hospital-cancer institute, Kibuli and Mengo hospitals. The hospitals provide care for patients with chronic conditions such as HIV, cancer, diabetes mellitus, and hypertension that require long term treatment. In this study, health workers who dispense drugs from both in- and out-patient departments were interviewed. These included pharmacists, dispensers and any other cadre who dispense drugs. 3.2 Data Collection and Analysis Key informant interviews were conducted with relevant health workers at the four health facilities in Kampala city to identify key issues related to the smart dispenser innovation. The initial study design proposed a purposeful sampling to conduct six KIIs per facility. The targeted key informants included at least two trained health workers in the

22

G. A. Mugisha et al.

following health professionals: pharmacist, nurse, and physician. The respondents were interviewed using an interview guide that captured data on dispensing experiences and challenges faced by health workers, non compliance, consequences of not adhering to prescribed medicines, strategies to improve drug dispensing and administration as well opinions about usability and feasibility of the dispenser when used in different settings. However, we did not obtain the expected number of participants from all the health facilities, as staff were busy and not available for the interviews during the study period. On a positive note, we obtained a good professional representation in the field of nursing, pharmacy and clinical medicine. Table 1 shows the locations and designations of the health practitioners that participated in our study. Table 1. Participants’ professional demographics. Medical Facility

Number Profession Number

Kibuli Hospital

5

Nursing

8

Mengo Hospital

6

Pharmacy

4

Naguru Hospital

5

Physician

5

Mulago Hospital Cancer institute Total

1 17

17

The prototype of the pill dispenser was demonstrated to each participant prior to the interview. The participant was also granted an opportunity to independently utilize the pill dispenser, to personally explore the features and ask relevant questions on the features and functionalities of the prototype. The KII interviews followed each prototype demonstration. The prototype demonstration and KII interview took place on the same day for each participant. A maximum of two participants were interviewed each day. The semi-structured interview questions focused mainly on the participants’ understanding of patient’s medication adherence behaviours and their assessment of usability and feasibility of the smart pill dispenser. The interviews were conducted in English, and covered questions on challenges during dispensing drugs, effects of non-compliances with dosage instructions, questions about whether the participants (or the health facility) had measures to ensure medication adherence, and questions about attitudes and medication preferences. All interviews were digitally recorded with consent of health workers, then transcribed verbatim by two trained research assistants. The transcripts were read several times and codes were summarised manually in excel per themes that were determined a priori as per the semi- structured interview guide. 3.3 Quality Control Measures were put in place to ensure good quality of data and increase the reliability of the study results. Research assistants were deliberately chosen, focusing on individuals with working knowledge of qualitative research methods. In addition, they underwent a half-day training on the testing of the pill dispenser and the procedure for conducting

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

23

post evaluation interviews. The interview questions were also pre tested and adjusted prior to data collection. 3.4 Ethical Considerations Ethical clearances were obtained from the Makerere University School of Social Sciences Research and Ethics Committee MAKSS REC 03.19.277. Written permission to carry out the research was also obtained from the management of the respective health facilities. The objectives, benefits and risks of the study were explained to the prospective study participants and informed consent sought before interviews. Confidentiality was observed when collecting and handling data and anonymous identifiers were used in the transcribed data.

4 Results The essence of the pill dispenser is to make medication administration more effective, thus reducing the risks associated with poor adherence, both in the hospital and at home. We examined the challenges faced by the medical practitioners in medication administration to determine the need or otherwise, for an aid to make medication administration more effective. Many (43%) of the participants indicated that time was a major constraint as they have so many patients to attend to, so a system that would assist them in reducing the time spent per patient would be desirable. In addition, 25% indicated that contamination of dispensers was a major issue due to multiple use of one dispenser. The solution to this could be sterilization or the use of some disposable materials. The following comments buttress some of the challenges faced by the healthcare providers as summarized in Table 2: Table 2. The Logistic challenges when dispensing medications. Challenge

Number of respondents

Time constraints – so many patients to attend to

7

Compliance issues

2

Contamination from use of same dispenser for multiple drugs

4

Drugs not available in the wards

2

None

1

“wrong calculation leading to wrong dose or even wrong drug. This could be as a result of work overload” (Health worker, Kibuli Hospital) “use one tray to count different types of drugs at the same time” (Health worker, Nagulu Hospital) “contamination of drugs from use of contaminated tools; mixture of similar drugs could occur if drugs look alike, (Health worker, Mengo Hospital)

24

G. A. Mugisha et al.

Patient’s medication adherence behavior was a major issue highlighted by the study participants. Virtually all participants highlighted patient non-compliance as a major behavioral challenge in medication administration. This non-compliance has adverse effects on the individual and on the health system. The study participants identified prolonged periods of recovery and resistance to drugs, which ultimately leads to higher cost of treatment – as expressed below: “non compliance, which will lead to prescription of more expensive drugs; wrong prescription due to patients attitude makes doctors contribute indirectly to development of drug resistance” (Health worker, Kibuli Hospital) The study showed that the primary means of encouraging adherence is the use of reminders; however, only about 35% of the study participants sent reminders to patients, others did nothing to encourage adherence. A common theme that emanated from the study is the focus of nurses on what is considered their primary function in medication administration, which is: “to create medication schedule which organizes drugs to be administered in the morning, afternoon, etc. depending on the SIG (signature, or “let it be labeled.”) of the prescription” (health worker from Naguru hospital) The participants indicated that their patients preferred herbal alternatives due to high costs of conventional drugs (Fig. 2).

9 8 7 6 5 4 3 2 1 0 Prefer herbal alterna ves

General apathy towards drugs

Concerned about Indifferent or good costs of drugs a tude

Fig. 2. Respondent’s perception of patients’ general attitude to medication

“…tired of taking drugs, financial constraints, then herbal alternative medicine.” (Health worker, Kibuli Hospital) In prescribing drugs to patients, the physicians and other authorized personnel provide patients with the choice of whether to take pills or injections; in most cases, patients (especially out-patient ones) prefer pills. On the overall, the pill dispenser had a very positive review from the study participants. Most of the study participants rated the pill dispenser very high in terms of

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

25

utility and features such as camera, two-way voice communication, and water dispensing capability. However, major area of concern was the size of the dispenser, which needs to be considerably reduced. In addition, respondents suggested increase in the number of pills dispensed and issues that relate to its smart abilities. The suggested areas of improvement are shown in Table 3 and in the quotes below: Table 3. Respondent’s opinions on aspects of the pill dispenser that require improvement Area of improvement

Number of respondents

Reduce size

4

Increase number of prescriptions dispensed

3

Improve user interface (e.g. touch screen)

1

Use air tight dispensing boxes

1

Improve emergency alert sensors

1

Improve audio capabilities

1

Improve software ability

1

Create reminders for healthcare providers (nurses, etc.)

1

Include solar power to mitigate lack of electricity

1

Water dispensing should be from the mains

1

Create a syrup dispensing ability

1

Nothing to improve upon

3

“power to include solar due to lack of electricity in some areas”. (health worker from Kibuli hospital) “make it less bulky; make it a touch screen; Some drugs are hydroscopic and need air tight containers so the storage chamber needs to be air tight. However, some jobs cannot be replaced by a machine so nurses/attendants still needed. (Health worker from Mengo hospital)

5 Discussion In this paper we explored the challenges faced by the medical practitioners while administering medicine. The key challenges were time constraints, heavy workload, contamination of same trays used for multiple drug dispensing and non-compliance issues with patients. We also sought opinions from health workers regarding the usability of the prototype. The respondents rated the prototype highly, especially with respect to the camera, water dispenser, user friendliness and high utility. Respondents also discussed risks associated with medication non-compliance. Poor medication adherence has been attributed to a number of reasons such as forgetfulness, memory impairment, confusion, heath status, socio-economic status and literacy [21]. Our study showed that the primary means of encouraging adherence is the use of

26

G. A. Mugisha et al.

reminders but few study participants reported to have sent reminders to patients. A number of medical practitioners do not consider reminders and other medication adherence efforts to be part of their responsibilities. This problem is compounded by the high cost of phone communication to out-patients, especially in developing countries. Our study also confirmed previous studies that forgetfulness and revealed that some patients do not comply with medication instructions due to forgetfulness and being medication weary. In addition, we found out that most patients value reminders as a means of overcoming non-compliance. In view of this the automated dispenser we believe comes in handy to address the challenge of sending prompt reminders. Medication compliance is a major unsatisfied need, especially in developing countries. The World Health Organization’s [6] life expectancy report indicates that life expectancy in Africa is growing at a fast pace (currently at 60), which implies that the population of the aged (65 and above) will likely increase significantly in the very near future. This increases the shift toward in-home and group home car, and opens opportunities for medication adherence assistance devices, especially to the aged and the cognitively impaired. The potential value derivable from automated pill dispensers, especially in the developing world cannot be overemphasized. According to Future Market Insights [22] the use of medication dispensers is expected to grow at a fast rate in developing regions of the world by the year 2020. This growth will be fuelled by rapid improvement in healthcare services, increase in awareness, and affordability of pill dispensers. There are only very few companies in Africa that focus on the manufacture of pill dispensing systems, and the penetration of automated pill dispensers is extremely low. The study further revealed that patients preferred herbal alternatives. This poses a challenging dimension to the adherence question. If patients show apathy toward hospital prescribed drugs and go for herbal alternatives instead, there might be risk of overdosage and use of wrong medications, the risks of which are high and unpredictable. The use of automated dispenser will assist in monitoring of taking of prescribed drugs and any variance will be promptly noticed by health workers to counsel the patient accordingly. The drug dispenser can be a remedy to some challenges associated with medication administration However, this brings us to the question of whether the pill dispenser could provide a cost-effective solution for medication administration in a manner that would address patient forgetfulness, drug weariness, apathy and cost concerns. Obviously, the cost element may only be controlled through pharmaceutical companies; however, the smart capabilities of our pill dispenser provide the patient with reminders, and userfriendly interface.

6 Conclusion Our study revealed the need to find ways to re-engineer distribution of medication in busy facility settings. One way this can be accomplished is through the use of a smart pill dispenser that provides the health workers with reminders and other value-added support that would reduce the anxiety and apathy toward medication adherence among patients. Nurses and other support medical practitioners are in short supply in the developing countries. Taking on the responsibility sending patient reminders would add to

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

27

the workload of the nurses and limit the number of patients they could attend to. A pill dispenser, equipped with automated reminder capabilities will increase the nurse’s efficiency in medication administration and also improve patient’s adherence to medications, which will in turn reduce re-hospitalizations, drug resistance and other negative effects of non-adherence. Based on the findings from our study participants, we intend to: 1) further refine the pill dispenser to make it more portable, while maintaining affordability and local content - exploring the possibilities of utilizing stainless steel or wood/plastic composites that are lighter and more durable; 2) utilizing the power of 3-D printing in producing some components – taking advantage of flexibility and scalability of production; 3) partnering with investors and industry to produce the first set of commercial grade dispensers based on our refined design; 4) testing and conducting further acceptability studies on the product among both health worker and patients within health facilities and home setting; and 5) engaging in full scale commercial production. In refining the pill dispenser to a number of factors would be considered, including but not limited to: 1). Clinical level quality consideration: Most studies have focused on individual level factors that could impact on medication adherence; however, [23] emphasized the need for payor, pharmaceuticals and clinical level system level considerations. The use of smart pill dispenser in medication administration could be considered at both the individual (in private homes) and system levels (in hospitals and other medical facilities); 2). Consideration for user physiological characteristics such as visual and auditory impairment, and age-related characteristics [24]; 3) Monitoring: every medication management system requires proper monitoring to ensure adherence and avoid complications resulting from non-adherence. The Internet of Things (IoT) age makes it possible for monitoring capabilities into healthcare systems [25]; building this capability into the smart pill dispenser would greatly enhance usability of the system. While it is possible to gain insight into patient medication adherence from the viewpoints of experienced health workers who have been attending to patients in big hospitals, it may not be a good representation of patient medication attitudes. We recommend additional patient consultation in the design stage of the follow-up prototype to ensure the consideration of patient-desirable features.

References 1. Westbrook, J.I., Woods, A., Rob, M.I., Dunsmuir, W.T., Day, R.O.: Association of interruptions with an increased risk and severity of medication administration errors. Arch. Internal Med. 170(8), 683–690 (2010) 2. Ghaleb, M.A., Barber, N., Franklin, B.D., Wong, I.C.: The incidence and nature of prescribing and medication administration errors in paediatric inpatients. Arch. Dis. Child. 95, 113–118 (2010). https://doi.org/10.1136/adc.2009.158485 3. Cousins, D.H., Gerrett, D., Warner, B.: A review of medication incidents reported to the National Reporting and Learning System in England and Wales over 6 years (2005–2010). British J. Clin. Pharmacol. 74(4), 597–604 (2012) 4. Roughead, E.E., Semple, S.J., Rosenfeld, E.: The extent of medication errors and adverse drug reactions throughout the patient journey in acute care in Australia. Int. J. Evidence-Based Healthcare 14(3), 113–122 (2016)

28

G. A. Mugisha et al.

5. Theuretzbacher, U.: Global antibacterial resistance: the never-ending story (Review). J. Global Antimicrobial Resistance, 1(2), 63–69 (2013) 6. World Health Organization. Antimicrobial resistance: global report on surveillance 2014. http://www.who.int/drugresistance/documents/surveillancereport/en/. Accessed 18 Dec 2016 7. Huang, J., Li, S., Li, L., Wang, X., Yao, Z., Ye, X.: Alarming regional differences in prevalence and antimicrobial susceptibility of group B streptococci in pregnant women: a systematic review and meta-analysis. J. Global Antimicrobial Resist. 7, 169–177 (2016). https://doi.org/ 10.1016/j.jgar.2016.08.010 8. Arlt, S., Lindner, R., Rösler, A., von Renteln-Kruse, W.: Adherence to medication in patients with dementia. Drugs Aging 25(12), 1033–1047 (2008) 9. Sokol, M.C., McGuigan, K.A., Verbrugge, R.R., Epstein, R.S.: Impact of medication adherence on hospitalization risk and healthcare cost. Med. Care 43(6), 521–530 (2005) 10. Krueger, K., Botermann, L., Schorr, S.G., Griese-Mammen, N., Laufs, U., Schulz, M.: Agerelated medication adherence in patients with chronic heart failure: a systematic literature review. Int. J. Cardiol. 184, 728–735 (2005) 11. Anderson, G.J.M., Farr, P.W., Kelly, A.M.: Medicament Dispenser: US Patent No. 2005/0172964A1 (2015) 12. Jasti, S., Siega-Riz, A.M., Cogswell, M.E., Hartzema, A.G., Bentley, M.E.: Pill count adherence to prenatal multivitamin/mineral supplement use among low-income women. J. Nutrition 135(5), 1093–1101 (2005) 13. Gast, A., Mathes, T.: Medication adherence influencing factors—an (updated) overview of systematic reviews. Syst. Rev. 8(1), 112 (2019) 14. Fanning, L., Jones, N., Manias, E.: Impact of automated dispensing cabinets on medication selection and preparation error rates in an emergency department: a prospective and direct observational before-and-after study. J. Eval. Clin. Practice 22(2), 156–163 (2015) 15. Santiago, J.M.: The need for theory in addressing nonadherence to treatment. J. Clin. Psychiatry 77(10), 1348–1349 (2016) 16. World Health Organization: Global Health Observatory Data: Life Expectancy (2015). http:// www.who.int/gho/mortality_burden_disease/life_tables/situation_trends_text/en/. Accessed 25 Feb 2017 17. Kretchy, I.A., Asiedu-Danso, M., Kretchy, J.P.: Medication management and adherence during the COVID-19 pandemic: Perspectives and experiences from LMICs. Research in Social and Administrative Pharmacy (2020) 18. Mugisha, G.A., Uzoka, F-M., Nwafor-Okoli, C.E.: A framework for low cost automatic pill dispensing unit for medication management. In: Cunningham, P., Cunningham, M. (eds.) Proceedings of the conference IST-Africa (2017). IIMC International Information Management Corporation (2017). ISBN: 978-1-5386-3837-8 19. Primec: Six advantages plastic has over metals. http://www.primexfits.com/hvacventing/6advantages-plastic-has-over-metal/. Accessed 15 Dec 2016 20. USAID Center for Development Information and Evaluation (1996) Conducting Key Informant Interviews Performance Monitoring & Evaluation TIPS. Washington DC, USAID. http:// pdf.usaid.gov/pdf_docs/PNABS541.pdf 21. Harrison, E.: The cost of not taking our medicine: the complex causes and effects of low medication adherence, the American J. Accountable Care, 6(4), e11–e13 (2003) 22. Future Market Insights, Medication Dispenser Market: Global Industry Analysis and Opportunity Assessment 2016–2026. http://www.futuremarketinsights.com/reports/medication-dis penser-market. Accessed 4 March 2017 23. Dean, L.T., George, M., Lee, K.T., Ashing, K.: Why individual-level interventions are not enough: Systems-level determinants of oral anticancer medication adherence. Cancer (2020). https://doi.org/10.1002/cncr/32946

Usability Evaluation of Low-Cost Smart Pill Dispenser by Health Care Practitioners

29

24. Sahlab, N., Sailer, C., Jazdi, N., Weyrich, M.: Designing an elderly-appropriate voice control for a pill dispenser. Proc. Automat. Med. Eng. 1(1), 30 (2020) 25. Ananda Kumar, S., Mahesh, G.: IoT in smart healthcare system. In: Chakraborty, C., Banerjee, A., Kolekar, M.H., Garg, L., Chakraborty, B. (eds.) Internet of Things for Healthcare Technologies. SBD, vol. 73, pp. 1–19. Springer, Singapore (2021). https://doi.org/10.1007/ 978-981-15-4112-4_1

Dynamic Causality Knowledge Graph Generation for Supporting the Chatbot Healthcare System Hong Qing Yu(B) Institution for Research in Applicable Computing, School of Computer Science and Technology, University of Bedfordshire, Luton, UK [email protected]

Abstract. With recent viruses across the world affecting millions and millions of people, the self-healthcare information systems show an important role in helping individuals to understand the risks, self-assessment, and self-educating to avoid being affected. In addition, self-healthcare information systems can perform more interactive tasks to effectively assist the treatment process and health condition management. Currently, the technologies used in such kind of systems are mostly based on text crawling from website resources such as text-searching and blog-based crowdsourcing applications. In this research paper, we introduce a novel Artificial Intelligence (AI) framework to support interactive and causality reasoning for a Chatbot application. The Chatbot will interact with the user to provide self-healthcare education and self-assessment (condition prediction). The framework is a combination of Natural Language Processing (NLP) and Knowledge Graph (KG) technologies with added causality and probability (uncertainty) properties to original Description Logic. This novel framework can generate causal knowledge probability neural networks to perform question answering and condition prediction tasks. The experimental results from a prototype showed strong positive feedback. The paper also identified remaining limitations and future research directions. Keywords: Causality analysis · Healthcare · Knowledge graph · Natural language processing · Chatbot · Artificial intelligent

1 Introduction Self-healthcare information system interventions became an important component to improve all range individual’s capabilities on self-health management, testing and awareness [1]. Our research focus is on self-awareness area, especially applying knowledge and NLP based AI technologies to provide a potential solution to self-education and self-testing. If every individual can gain pre-knowledge of health conditions, such as symptoms and potential risks to others, then the healthcare outcomes to the individual and publics in the community will be improved. Not surprisingly, research results have supported the above hypothesis, e.g. [2] suggested that health education could improve © Springer Nature Switzerland AG 2021 K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 30–45, 2021. https://doi.org/10.1007/978-3-030-63092-8_3

Dynamic Causality Knowledge Graph Generation

31

patient engagement and treatment outputs. Moreover, self-education could also play a unique role in chronic disease management [3]. From the scientific point of view, our research aims are to provide a novel framework for supporting efficient machine-oriented health knowledge study and causality reasoning. There are currently many open remaining issues around these two areas. For example, most of the machine knowledge study processes are treated as the classic machine learning processes to mostly studying relational raw data, which leads to learning probability distributions of the raw dataset. As a result, the learning results are difficult to transfer as knowledge to human users. In the knowledge presentation domain, the most advanced KG (Knowledge Graph) framework has a high capability to represent knowledge. However, KG data are difficult to be the direct inputs for machine learning algorithms without encoding steps. The encoding will destroy most the knowledge constructive properties. The major reason of adapting machine learning process with KG together is that the current KG standards have lacks of uncertainty and causality supported reasoning. Therefore, a novel framework that can support KG retrieval with uncertainty and causality reasoning facilities will build the foundation for us to develop an AI embedded self-health care awareness system. In this paper, we will illustrate a Causal Probability Description Logic (CPDL) framework to provide novel reasoning capabilities with uncertainty and causality. Then, a health domain Causality Knowledge Neural Network (CKNN) can be generated based on CPDL. Besides, we introduce a Chatbot interactive environment to allow users to set up the learning topics and accessing the knowledge as self-education. In the meantime, the Chatbot can provide symptom-based predictions according to the knowledge to support self-diagnosis process. The paper is structured as: Section 2 discusses the related research work in the healthcare domain. Section 3 explains the CPDL framework with key terminologies and computation algorithms. Section 4 introduces the working process of Chatbot application in detail. Section 5 presents the evaluation and future research directions. Section 6 concludes the research.

2 Related Work Machine Learning (ML) technologies are widely applied in healthcare and medicine domain recently. The areas covered from fatal disease early identification to drug discovery and manufacturing. There was a lot of promising results suggested that the ML approach provided huge benefits to healthcare professionals. For example, most of the ML algorithms can provide more than 90% accuracy on breast cancer detection tasks and MLP (Multilayer Perceptron) algorithm can achieve 99.4% accuracy [4] worked on the Wisconsin diagnostic dataset1 . The other example is Deep Learning-based ML approach e.g. a deep learning-based automatic detection algorithm (DLAD) was developed for detecting chest radiographs [5] that can achieve 95% accuracy as well as a neural network based deep learning classification algorithm for skin cancer detection by Stanford research team [6]. There are many other research works focused on individual 1 https://www.kaggle.com/uciml/breast-cancer-wisconsin-data#data.csv

32

H. Q. Yu

but different types of diseases [7–13]. However, at the current state of art, there are two major limitations of these ML and Deep Learning (DL) approaches according to our research goals: 1. These approaches only work on the raw experienced clinical data for a certain prediction or classification task without creating fundamental inferable knowledge. Therefore, the outcomes are purely the statistical fitting models that cannot provide explanations and traceable evidences. However, the explanations are extremely crucial for self-care system to enable individuals to understand insight of the health condition for achieving education and awareness goals. 2. The learning datasets are isolated to each different health model without any relations. In healthcare domain, there are many important relations can affect diagnosis such family health history, pre-conditions and life-style may all have relationships to each other, especially and importantly the causal relations among them. However, these relations cannot be expressed by isolated single dataset without a comprehensive knowledge linking framework. To address these two major limitations, the causality based knowledge generation framework is required as the base for developing a self-healthcare system. In fact, causal relations analysis is widely used in health, social and behavioural research since NeymanRubin causal inference theory was published in 1986 [14]. However, the concepts of causal and association or correlation are always been mixed or misunderstood until the formal mathematics models are represented in 2010 by Pearl in [15]. The model applies probability joint distribution computation on the directional graph that satisfying the back-door criterion and a changing function of do(X=x) rather than a random x to have a probability prediction on Y based on statistic knowledge. In simplified terms, the causal relation can be observed if one property is modified, then the other property of a probability distribution will also change. Therefore, we can distinct associational relations and causal relations. Most recently, this idea has been applied on top of the Reinforcement learning process by the DeepMind team [16]. On a separate note, introducing probability concepts into knowledge graphs is another path to express knowledge with belief rating thresholds. Based on this idea, a Probabilistic Description Logic (PDL) was explained in 2017 [17] to deal with subjective uncertainty. The PDL extended Tbox and Abox definitions in the classic Description Logic (DL) in Eq. (1) and (2) with probabilistic thresholds notations as P∼n over concepts and individual instances, where ~ can be operated from ≤, , ≥ and n defines the thresholds value. However, the extended PDL requires manually encoding the default probability thresholds that are hard to implement in a dynamic knowledge learning system. T :: = C, D | C  D | ∈ r. (C) | P∼n C |

(1)

A :: = A | A ∧ A | r(a, b) | P∼n A |

(2)

In this paper, we will introduce a novel knowledge description framework with a combination of causal probability analysis and KG for supporting the health Chatbot application. As a result, the user will have an interactive learning environment with the

Dynamic Causality Knowledge Graph Generation

33

Chatbot, build up the knowledge that can answer questions and performing predictive suggestions from machine knowledge rather than raw data.

3 Causal Probability Description Logic (CPDL) Framework In this section, we start to introduce Causal Probability Description Logic framework that mainly works on DL-ει semantic level in this paper. The definitions are presented as following: A Causal Probability Knowledge Base (CPKB) has three elements that are defined as Eq. (3): CPKB = {T, A, φ (T)}

(3)

where T is the Tbox (knowledge schema) and A presents the Abox (individual instances) that are inherited from the classic DL KB terms. The φ(T) defines the causal relations that can be placed in between any two concepts in Tbox, which is an extra roof (universal) concept comparing to the classic DL. Therefore, the Tbox is defined as Eq. (4): T :: = C, D | C  D | ∈ r.(C) | ∈ r.(C) ⇒∈ φ .(C) | Pμ(φ (C))

(4)

where C, D and r are the same terms used in classic DL Tbox. However, CPDL claims that if there is a relation r, then there may have a causal relation φ exist mapping to the r according to knowledge certainty. The φ may have an explicit direction (a causes b) or bi-direction (a causes b and b also causes a) that need to be defined in Tbox. The Pμ (φ (C)) represents a dynamic probability distribution over all possible φ(T) by given observation input. Based on Tbox definition, the Abox is defined as Eq. (5): A :: = A | A ∧ A | r(a, b) | φ(a | b) | φ(b | a) | φ(b | a) ∨ φ(a | b) | Pμ(φ) | (5) where A, A’ and r are the same terms used in classic DL Abox. φ (b | a) implies b is one of the reasons to cause an observation condition ‘a’ according to the Tbox specification and vice versa is for φ (a | b). In contrast, φ (b | a) ∨ φ (a | b) indicates that there are bi-directional causal relations between ‘a’ and ‘b’ instead of having a relation r defined between them that might hide a causality inside the two terms (direction is unknown), the Pμ(φ) distribution will be dynamically computed by the Discrete Uniform Distribution (DUD) algorithm that is a symmetric probability calculation. For example, if there are n numbers of subjects cause a specific observation condition, and then the causal probability for each of the subject is 1n . We also one major assumption: the observation conditions are fully independent events. Therefore, the probability of subjects by given an observation (O) is represented as Eq. (6): P(si |s1 , . . . , sn ⇒ O) =

1 n

(6)

The probability aggregating function (Eq. 7) is to produce the final ranking results for an individual subject.  p(Sk |oi ) P(Sk |o1 , . . . , on ) = (7) n

34

H. Q. Yu

Here, Sk is the individual subject that has DUD causal probability distribution to the observations and n is the number of observations. The framework also considers the probability chain. For example, subject A causes subject B and subject B causes input observation C, and then there is a causal probability from A to B. The probability chain will be computed as Eq. (8):    (8) p(SA |SB )p(SB |OC ) P SA |SB, OC =

4 CPDL Supported Healthcare Chatbot Learning System 4.1 Causality Knowledge Neural Networks Our novel learning process uses an unsupervised learning method to understand and represent knowledge as a causal knowledge graph. The data sources used for the learning are website data from UK NHS and Wikipedia. The learning process is more similar to the human learning process (get topic/question -> analysis -> look for learning resources -> understand and record -> think) than other supervised learning methods. The causality triples integrated knowledge graph can be treated as neural networks (CKNN) that hold causality knowledge. When a particular question or learning topic arrived, the inference process will be trigged to collect relevant neural together to generate a run-time subgraph of a causal knowledge neural network with runtime probability computation applying Eqs. 6, 7 and 8. Then, knowledge can be provided as answers or predictions. The CKNN is designed to learn and model seven major causality relations in the healthcare domain represented in Fig. 1. All different causal relations are the sub-relation type of cause φ in the CPKB definition (Eq. 3). For example, causeSymp is a subclass of causality relation φ. The means of defined classes are: (i) (ii)

(iii)

(iv) (v)

(vi) (vii)

Term: it specifies a medical domain definition that can be identified from semantic datasets e.g. DBpedia. Disease: it is a subset class of the Term definition that presents a specific disease name. In our case, all the instances of the Disease are derived from UK NHS common disease A-Z index. Symptom: it is a subset class of the Term definition that presents a symptom discovered from semantic annotation process and is described in one or more diseases description. Drug: it is a subset of the Term definition that presents drug name discovered from semantic annotation process and is described in one or more diseases description. Anatomical Structure: it is a subset of the Term definition that presents specific name of human organ or body part discovered from semantic annotation process and is described in one or more diseases description. Gender: it is a sex category of human AgeGroup: It presents human groups according to age. In our case, the age can be but not limited to baby, youth, adult, elderly or aging people that are identified from disease text semantic annotation process.

Dynamic Causality Knowledge Graph Generation

35

Fig. 1. Health causality knowledge ontology

4.2 Chatbot Learning Approach Our chatbot’s (ROBO) learning approach is designed to enable the machine to understand the causality knowledge through human interactive learning processes. The approach contains five major phases represented in Fig. 2.

Fig. 2. Chatbot learning and answering approach

In detail, the first step is to receive and understand the chatting questions. The Chatbot can take general questions (e.g. greetings) but majorly the questions related to a disease or symptoms such as “what is pneumonia?” and “What are the symptoms of pneumonia?”. The question will be processed using our specific NLP and semantic annotation based question sentiment analysis algorithm (see Algorithm 1).

36

H. Q. Yu

Algorithm 1: question understanding and semantic annotations Input: sentence (text based sentence) Output: inp_kt (key phrases); annoation (DBpeida key terms in the sentence); is_health_question (is about health or not) #from NLTK package //word tokenization, remove stop words, Word stemming sp = Sentence_preprocessing (sentence) // get key phrase of the sentence through 'NP: {?*}' pd = Patten_detection (sp) #from Spacy nlp package // find out the entity words and adjective words from the key phrase _entity, _adjective = nlp(pd) If _adjective: inp_kt = append(_adjectivev+_entity) else: inp_kt = append(_entity) # DBpedia API //gain semantic annotations for the sentence _annotation = DBpediaAnotation(sentence) // get rdf:type of each annotated words rdf_types = [DBpediaType(_annotation)] if (rdf_type) contains ’Disease or symptom or health related terms’: is_health_question = 1 else: is_health_question = 2 return inp_kt, _annoation, is_health_question

In the same step, the Chatbot able to accept prediction questions by giving conditions or symptoms. However, the prediction answers are truly based on the knowledge in hand. More knowledge the machine has there will be more accurate predictions. Of course, we can throw a bunch of questions of different diseases to the system directly without using Chatbot interface. Although the CKNN generating API can create knowledge much faster, there is never a complete list of disease names and questions. Therefore, machine learning process should be a continuing study approach as same as human learning. After understood the question, the second step is to gain the accurate and related information if the question has not been asked before. We use UK NHS (National Health Service) website and Wikipedia contents as our raw data to get related texts. We invoke (1) Wikipedia API2 and our own developed UK NHS crawler API to identify certain sections or subsections that match the question. If no certain section is allocated, the summary section of the health condition topic will be returned as result; (2) SPARQL query on DBpedia dataset to identify all related RDF triples about the question keywords such as disease and symptom terms. The text-based first time answer will be generated using these related texts.

2 https://pypi.org/project/wikipedia/

Dynamic Causality Knowledge Graph Generation

37

Fig. 3. The ROBO interface and a chat example.

With related texts in hand, the third step is to apply NLP and DBpedia sentiment analysis APIs to identify the term tokens. The details of the r step sentiment analysis are: 1. Information crawling: based on the HTML features of NHS website, a dedicate web crawler is developed to enable retrieving the contents by different sections. Figure 4 shows example of the partial text crawled from pneumonia webpage. 2. Natural Language Processing (NLP): the NLP pipeline has been implemented including sentence segmentation, word tokenization, text lemmatization, remove stop words/punctuations. Finally we got useful processed word tokens for each sentence as bottom half of Fig. 5. 3. Semantic tagging and merging: the word tags will be processed to recognize only nouns in the sentences. Two tagging methods are used, which are NLTK word pos_tag and DBpedia Spotlight semantic annotation. Figure 6 shows the merged noun word tokens of pneumonia symptoms from pos_tag and semantic annotation methods.

38

H. Q. Yu

Fig. 4. Pneumonia section text contents

Fig. 5. Generated pneumonia section word tokens

Fig. 6. Term tokens for symptoms of pneumonia by merging pos_tag and semantic annotations.

Using the term tokens, the CKNN causal triples are generated following the ontology specification. Algorithm 2 presents the 4th step of the overall Chatbot workflow - semantic lifting process. The newly generated CKNNs will be integrated into the existing CKNN triple repository in step 5.

Dynamic Causality Knowledge Graph Generation

39

The CKNN will eventually have the capability to compute probabilities for answering causality and predicting questions. Therefore, if the semantic repeated question is asked again, the answer will be more directs and clearer from the previous CKNN. Figure 3 displays the ROBO prototype interface with a chat example about the question of “what are the symptoms of pneumonia?”. You can see that if the question is asked at first time, then the answer will come from online resources such as NHS and Wikipedia directly. Once the question’s semantics and answers were processed into CKNN repository, the Chatbot will use its own knowledge to generate answer with the original source attached whenever a similar question is asked later. Algorithm 2: CKNN generation Input: inp_kt //1st output of algorithm 1 _annotation // semantic annotations from algorithm 1 Output: _content

//integrated texts related to the topic

#Applying Wiki and NHS search APIs for x in inp_kt // x is topic and y is section content =_content +search(pair(x, Lambda y: _annotation)) // y is topic and x is section _content =+search (pair(Lambda y: _annotation, x)) for z in annotation: _content =+ DBpediasummerize(z) //create a dictionary contains the rdf:type about each term annotated in the content semantic_terms = {DBpedia.annotation(_content) : type} // based on the type to generate CKNN disease_namespece = generateURI() for st in _semantic_terms: type=st.getValue() // generate causes relations if type==’symptom’ or ‘sign’ or ’anatomicalStructure’ or ‘age’ or ‘gender’ or ‘treatment’: triple.add(URI(st.getId()), rdf:type, URI(st.getVlaue())) triple.add(disease_namespace, rdfs:causes, URI(st.getId())) // generate causes relation between two diseases else: triple.add(URI(st.getId()), rdf:type, URI(st.getVlaue())) triple.add(URI(st.getId()), rdfs:causes, disease_namespace)

Figure 7 shows a CKNN subgraph representation of Pneumonia disease (partially). The graph shows knowledge learned on symptoms and diseases that lead to Pneumonia by processing the question of “Do you know symptoms of Pneumonia?”. CKNN graph can easily estimate uncertainty or probability model based on the matched rates to the observations of inputs to single condition. However, the integrated CKNN graphs with different conditions will use three unique probability aggregating methods (Eq. 6, Eq. 7 and Eq. 8) to predict complex scenarios (see Fig. 10).

40

H. Q. Yu

Fig. 7. Pneumonia symptom based causality instant triples

5 Evaluation and Further Discussions Since the nature of the research, we have to divide our evaluation into two folders. One is to evaluate question-answer (learning) accuracy and system performance. The other is to evaluate prediction accuracy (thinking) and system performance. Designing a fully automatic testing system is a very challenging task. Therefore, a semi-automatic evaluation method has been applied for the evaluation. The method majorly contains four parts. The first part is to manually select top 15 common diseases [18, 19] with 6 questions each. The second part is to invoke the Chatbot question-answering program to answer these 90 questions and manually checking if each of the questions has been answered satisfied comparing to original source. In addition, the numbers of CKNNsemantic terms that have been created will be reported. The third part is to manually select symptoms described for the selected 15 diseases and generate 40 scenarios based on the NHS information. The fourth part is to calculate the number of the correct predictions performed by the system. Figure 8 shows the example of the evaluation results. The full

Dynamic Causality Knowledge Graph Generation

41

90 question evaluation information, data, Chatbot and backend codes are all available on our GitHub repository3 .

Fig. 8. Evaluation Form

The evaluation results show that the answer return rate is 92.2%. The satisfaction rate is 82.6% for answered questions and 78.9% in total. Figure 9 shows the satisfaction rate for each type of questions and numbers of different terms are generated per disease condition. We clearly see that the question related to symptoms, causes and treatments can be answered efficiently over 80% satisfaction rate. The most difficult question to answer is related to what affected people groups are (53.3%) and how to diagnosis (66.7%). In addition, the related diseases, symptoms and affected anatomical structure semantic terms are generated as the most and again the people group and diagnosis terms are the least that obviously matches to the previous evaluation results. Although we only tested 15 conditions for evaluation, the generated causal relational dataset is linked to another 129 diseases in our CKNN repository. Most noticeably, we recognized that Pneumonia condition is the most complex disease among these 15 conditions because it relates to 21 other disease (the most) and associates to 33 symptoms (the most) as well as 8 affected anatomical structures. Maybe this is one of the reason enable reflecting to current difficulties situation of coronavirus disease, which also shows that machine can discover 3 https://github.com/semanticmachinelearning/HealthChatbot

42

H. Q. Yu

hidden knowledges and reasoning conditions through our proposed framework. The prediction task has a 91.2% correct rate taking symptoms, age, gender and uncomfortable body parts. Figure 10 explains an example that generates a runtime probability knowledge graph (partially screenshot) with 4 input conditions: Symptoms = [‘Cough’, ‘Chest pain’, ‘Fever’]; Unwell Body Position = [‘Chest’]; Age = 10; Gender = 0 (male) With these input conditions, the CPDL graph presents the disease namespace, cause relations to the input conditions and probability distributions to each of them. For instance, the Common Cold disease has 0.0625 and 0.027 probability for the Cough and Fever respect. The age and gender both have neutral probabilities of 0.5.

Fig. 9. Answer satisfaction rate for each different type of questions and term numbers per disease condition

At the moment, we cannot find an accurate baseline for comparing our evaluation result in the common disease domain. The most valuable comparing scenario is the SQuAD (Stanford Question Answering Dataset) based question and answering solutions. At moment, the top solutions [20, 21] for SQuAD can achieve around 90% accuracy by applying ALBERT (A Lite BERT for Self-supervised Learning of Language Representations) supervised Deep Learning method with different tunes with more 100,000 questions and answers training pairs. In certain, our current experimental Chatbot application and CKNN framework have their limitations and future research directions: 1. We are still working on adding negation syntax to the CPDL logic framework to enable expressing ‘not causes’ semantics. The difficulties include two sides. On the one side, the negate relation is difficult to detect and to compute the probabilities. On the other side, the mechanism to aggregate both positive and negative semantics into predictions needs more research and it can bring dramatic computation complexities.

Dynamic Causality Knowledge Graph Generation

43

Fig. 10. Prediction runtime dynamic causal probability knowledge graph (CPKG)

2. At this stage, DUD probability distribution function is applied to compute the probability distributions with a strong assumption of treating all observations as independent events. However, this assumption is not always true in healthcare or other application domains. For example, symptoms mostly have association relations with each other, e.g. a long period of high temperature and infection always appear together. On our early initial exploration, we applied the Apriori association rule-mining algorithm [22] trying to investigate the association relations among symptoms and enabled to identify 26 association connections. Therefore, we need to find a way to add such an association into probability distributions, which is one of the important future research topics for us. In the near future, we will explore some other probability functions e.g. Multinomial Naive Bayes [23] or Markov chain algorithm [24]. 3. Clearly understand the weights specification to different categorical observation concepts is also important. For example, which observation is more important among symptoms, anatomical structure (position), age and gender for a particular disease? We will explore some existing approaches to test the training scenarios try to find the dynamic weighting algorithm for different types of disease. Currently, we tried

44

H. Q. Yu

the k-means clustering algorithm to separate the diseases into 15 small groups that can be used to train the weight calculations in future work. 4. Opportunities combining Deep Learning algorithms and CKNN in a way that causality relation can be properly encoded. 5. Generating better human natured communicating sentences e.g. answers or education contents to engage human interactions with the system. Currently, our sentences or paragraphs are circulated from the original text of the online resources and statically fill into an answer template with semantic terms. Therefore, we will research on how to improve Chatbot conversations with a smart algorithm to compose the CKNN semantics into sentences and paragraphs. 6. Adding more interactive functions to the Chatbot for supporting self-healthcare management such as treatment process management and lifestyle management. To achieve it, the extended version of the Health causality knowledge ontology presented in Fig. 1 is required by adding more treatment knowledge terminologies such as type of treatment, daily management activities and alert system and so on.

6 Conclusion In summary, this paper presents an extended knowledge graph framework that added causality properties into the DL reasoning framework for holding the learning outcomes from other resources. This learning approach is different from current ML methodologies including Deep Learning from two sides. The first one is that our approach takes humankind learning procedural to dig the information and transfer it into thinkable knowledge rather than mining the raw and isolated datasets without any insight understanding. The second one is that our proposed framework can apply and reuse the knowledge model to different AI tasks rather than a certain duty for a specific solution. The conducted evaluations show the advantages of the proposed alternative ML approach. We also see the great potential of our AI approach can be applied to wider domains.

References 1. Manjulaa, N.: Self-care interventions to advance health and wellbeing: a conceptual framework to inform normative guidance. British Medical Journal (BMJ) 365, 1688 (2019) 2. Paterick, T.E., Patel, N., Tajik, A.J., Chandrasekaran, K.: Improving health outcomes through patient education and partnerships with patients. In: Proceedings (Baylor University. Medical Center), 30(1), 112–113 (2017) 3. McGowan, T.P.: Self-management education and support in chronic disease management. Primary Care: Clin. Office Practice 2(39), 307–325 (2012) 4. Agarap, A.F.M.: On breast cancer detection: an application of machine learning algorithms on the wisconsin diagnostic dataset. In: Proceedings of the 2nd International Conference on Machine Learning and Soft Computing (ICMLSC 2018), pp. 5–9. Association for Computing Machinery, New York, NY, USA (2018) 5. Kim, H., Park, C.M., Goo, J.M.: Test-retest reproducibility of a deep learning–based automatic detection algorithm for the chest radiograph. Eur. Radiol. 30(4), 2346–2355 (2019). https:// doi.org/10.1007/s00330-019-06589-8

Dynamic Causality Knowledge Graph Generation

45

6. Esteva, A., Kuprel, B., Novoa, R., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017) 7. Lampropoulos, A.S., Tsihrintzis, G.A.: Machine Learning Paradigms: Application in Reconmmender System, Springer (2015) 8. Verma, C., Ghosh, D.: Prediction of heart disease in diabetic patients using naive bayes classification technique. Int. J. Comput. Appl. Technol. Res. 7(7), 255–258 (2018) 9. Yekkala, I., Dixit, S.: Prediction of heart disease using random forest and rough set based feature selection. Int. J. Big Data Anal. Healthcare, 3(1), 1–12 (2018) 10. Papachristou, N., Miaskowski, C., Barnaghi, P., et-al.: Comparing machine learning clustering with latent class analysis on cancer symptoms’ data. In: Proceedings of the IEEE Healthcare Innovation Point-of-Care Technologies Conference, IEEE, New York (2016) 11. Kaur, C., Sharma, K., Sohal, A.: Disease prediction system using improved k-means clustering algorithm and machine learning. Int. J. Comput. Sci. Eng. 7(5), 1148–1153 (2019) 12. Harini, D.K., Natesh, M.: Prediction of probability of disease based on symptoms using machine learning algorithm. Int. Res. J. Eng. Technol. (IRJET) 5(5), e-ISSN: 2395-0056 (2018) 13. Laskar, Md., Rahman, T., Raihan, A., Rashid, N.: Automated disease prediction system (ADPS): a user input-based reliable architecture for disease prediction. Int. J. Comput. Appl. 133, 24–29 (2016) 14. Holland, P.: Statistics and causal inference. J. Am. Stat. Assoc. 81(396), 945 (1986) 15. Pearl, J.: An introduction to causal inference. The Int. J. Biostat. 6, 2 (2010) 16. Dasgupta, I., Wang, J., et al.: Causal Reasoning from Meta-reinforcement Learning (2019). arXiv preprint arXiv:1901.08162 17. Gutierrez-Basulto, V., Jung, J.C., Lutz, C.: Probabilistic description logics for subjective uncertainty. J. Artif. Intell. Res. 58(2017), 1–66 (2017) 18. Brown, H.: Ten most common conditions see by GP, British Journal of Family Medicine, 08 July 2019 19. Finley, C.R., Chan, D.S., Garrison, S., Korownyk, C., Kolber, M.R., Campbell, S., Eurich, D.T., Lindblad, A.J., Vandermeer, B., Allan, G.M.: What are the most common conditions in primary care? Systematic review. Canadian Family Physician Medecin de Famille Canadien 64(11), 832–840 (2018) 20. Zhang, Z., Yang, J., Zhao, H.: Retrospective Reader for Machine Reading Comprehension (2020). ArXiv, abs/2001.09694 21. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019) 22. Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Bocca, J.B., Jarke, M., Zaniolo, C. (eds.) Proceedings of the 20th International Conference on Very Large Data Bases (VLDB 1994) Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 487–499 (1994) 23. Kibriya, A.M., Frank, E., Pfahringer, B., Holme, G.: Multinomial naive bayes for text categorization revisited. In: Webb, G.I., Yu, X. (eds.) AI 2004: Advances in Artificial Intelligence. AI 2004. Lecture Notes in Computer Science, vol. 3339. Springer, Heidelberg (2004) 24. Gagniuc, P.A.: Markov chains: from theory to implementation and experimentation (2017)

Healthcare Emergency Room Optimization Using a Process Learning Algorithm Soraia Oueida(B) and Yehia Kotb College of Engineering and Technology, American University of the Middle East, Egaila, Kuwait {soraia.oueida,yehia.kotb}@aum.edu.kw

Abstract. In healthcare organizations, delivering high quality service to patients with affordable cost is usually a challenge in nature. This is where process mining comes into place to learn system bottlenecks, their deficiencies, how to optimize processes and how to avoid cost over estimation. In this paper, β algorithm, a process mining technique, is used to understand, depict and enhance emergency rooms through workflow modeling and simulation. A closed form Theorem that defines a valid learning system is introduced with a closed form proof to validate the process mining approach. The β algorithm is applied to a real life scenario where event log files are extracted to exploit the behavior of system processes and illustrate the system workflow structure. Results show an enhancement with system behavior, especially with metrics related to deadlocks. Keywords: Healthcare · Learning algorithm Optimization · Process mining · Satisfaction

1

· Log files ·

Introduction

Process mining is the practice of extracting knowledge from log files maintained by organizations [1]. One of the functionalities of information systems used in industry is the ability to create a log file for all events that change the state of the system. Logs are always kept for future use yet, they are rarely used to enhance or optimize the underlying process. Process mining techniques are used in organizations to achieve many tasks including but not limited to the following: learn process models from log files, learn different events from logs, check bottlenecks, enhance the current process, and optimize system performance and processes. This is done through the information systems of the organization under study [1,10]. Merging process mining and business process management achieves an autonomous smart system that could enhance itself by learning how to enhance the process, detect deadlocks and determine better resource allocation. Hence, the process mining plays the middle-ware role in order to combine the data analysis with modeling. c Springer Nature Switzerland AG 2021  K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 46–63, 2021. https://doi.org/10.1007/978-3-030-63092-8_4

Healthcare Emergency Room Management

47

Fig. 1. The β framework workflow [6]

The work presented in this paper is a continuation for the work that has been presented in [6]. In [6] a novel probabilistic process learning algorithm has been proposed and it is named β algorithm as it is a modification and continuation for the α algorithm proposed by Aalst et al. [15]. Process mining algorithms and techniques should be generic enough to handle log feeds from any type of organization as long as those log files follow certain predefined format by the algorithm. β algorithm is probabilistic and has an accumulative nature. In other words, the more logs the algorithm is fed, the more accurate the learnt process is [6]. Figure 1 illustrates the phases of the β algorithm. The β algorithm can be summarized as the events and environment states being recorded into log files, where log files are being fed to the framework. The β algorithm starts by first learning the events of the system. Events are considered as a universal set. Based on the learnt events, the system starts learning the dependencies among those events based on the pattern of occurrence. The system learns about the probabilities of occurrences of different events and then starts learning correlations among events. Finally, the system builds the model and converts it into a Petri net. The β algorithm is discussed in details in Sect. 4.1 where the process of the algorithm is being presented, the mathematical operators are being illustrated and comparison between β algorithm and many other known process mining algorithms is shown. In this Paper, the β algorithm is used in healthcare domain as it is used to enhance an emergency room process in a hospital. The emergency room model was proposed in [14]. The outcome of β algorithm is a process. This process is simulated against the original model proposed in [8,14] and different performance measures are

48

S. Oueida and Y. Kotb

being compared between the two models. The study shows that β algorithm introduced an enhancement to the process as will be seen in Sect. 7. Simulation is being conducted using Arena [7]. As a result of the simulation, a complete report shows the weaknesses and issues facing the system. The aim of any organization (but not limited to) is to increase customer satisfaction, increase revenue and reduce cost without dramatically changing the current process. Enhancing the system should not cause losing of any old data and modifications must not negatively affect the quality of service provided. This paper is organized as follows: Section 2 explains the contribution of this work. Section 3 shows set of latest research that is related to this paper. Section 4 presents the preliminaries needed to smoothly go through this work. Section 6 validates the model through a proposed closed form theorem and its proof. Section 7 presents simulation methodology and discusses the results of the simulation. Finally, Sect. 8 concludes the work presented in this paper.

2

Contribution

In this paper, we apply β algorithm in a hospital in the middle east where we study its process and we enhance the process using our previously proposed β algorithm. A closed form theorem to validate the outcome is presented where the proof shows the validity of the enhancement. The contribution can be summarized in points as follows: 1. β algorithm is applied to a healthcare real life scenario. 2. The process of the healthcare system under study is enhanced. 3. A verification closed form theorem for the outcome is proposed.

3

Literature Review

A workflow process model is the backbone of any workflow management system. In order to better understand a certain workflow process, a workflow design is a must. However, designing a workflow model is not easy and time consuming. A workflow model cannot be 100% imitated where some discrepancies arise between the real model under study and the model designed. Therefore, some techniques must be followed while extracting the workflow process model. A new algorithm was generated by Van der Aalst et al. in 2004 to define a process model and represent it as a Petri net model [15]. Petri net is a mathematical modelling technique that helps in describing the behavior and structure of a management workflow system and proven to be efficient in modelling and analyzing complex systems. With Petri nets the behavior of every process in the model under study can be analyzed [16]. Van der Aalst introduced in 1998 the application of Petri nets to workflow management highlighting state-of-the-art results and workflow verification [22]. Mining techniques and management information systems are correlated through the data and knowledge extracted from log files [10]. In order to extract

Healthcare Emergency Room Management

49

insights on the performance of each process of the system under study, mining algorithms rely on log event files. Event logs contain details about the process that allows the discovery of the system behavior [18]. Data mining, process mining, big data and business intelligence are scientific terms related to the processing of a big amount of data provided by the information systems in any organization. However, these techniques differ in many aspects. Data refers to the features or properties of the system under study. For example, data elements collected from a healthcare information system refer to the age of patients, their gender, their medical history and more. Whereas, a process means the method followed in order to accomplish a task such as providing care to a patient using available resources. Those both techniques, data mining and process mining, regardless of the way they are handled, have a lot in common and are part of business intelligence. Both techniques apply specific algorithms to the data handled in order to discover relationships among events. One of the advantages of data and process mining is to provide decision makers with better insights and come up with solutions for optimization and system enhancement. Process mining is an emerging area that can be useful in helping organizations understand the status of the system, check for compliance and plan for improving processes. The challenge here is to select the suitable algorithm to that specific system under study. The first step is to evaluate the available process mining algorithms which are used to mine business models using the process log files. The mined model is then compared to the existing process models of the system under study; which is considered the basic step of the process mining techniques [4]. One of the major advantages of process mining techniques is to discover workflow process bottlenecks. He et al. applied process mining to improve business processes after a major gas explosion accident in China [24]. The advantage of one process mining algorithm over another is to produce mined models that imitates the process of the original model under study without the need to interrupt the model under study and leading to the best performance. Some recent researchers and software developers have attempted to provide an evaluation framework to choose the best process mining algorithm against a specific system model since it is time consuming and costly to evaluate all the available process mining algorithms every time a business model need to be studied and optimized [5]. Garcia et al. presented an overview on process mining algorithms and their applications to different business processes. Healthcare systems are found to be the major case studies focused on due to its complexity in nature [25]. In a previous work, a comparison between different process mining algorithms is presented. Based on this comparison, a new framework namely the β algorithm is proposed and proved to be efficient and valid [6]. Antunes et al. used simulation to optimize the process flow in a Brazilian private hospital. A framework solution was proposed to identify, optimize and simulate the process. Process mining techniques were approached in order to identify the main process flow [23]. Aguirre et al. approached in their work a combination of process mining and simulation techniques for a process redesign project. This combination is used

50

S. Oueida and Y. Kotb

in order to analyze log files, provide insights on system bottlenecks and evaluate some improvement alternatives [29]. In this paper, the β algorithm framework is applied to a predefined healthcare system [14] in order to study the system, detect bottlenecks and suggest improvements. A quick view and comparison of the available process mining algorithms is presented below: 1. Using the heuristics miner algorithm, dependencies among events can be studied using a dependency graph. However, dealing with short loops, or the logic dependency relations between different events like XOR, AND, Split/Join, is considered a limitation of this algorithm [11]. 2. The genetic algorithm [2] is mainly used for mining noisy event logs. It is considered as an adaptive search method used to mimic the process model. This algorithm uses the casual matrix representation which detects the direct cause of an event by applying some routing techniques. For example, a sequential routing is when one event is the single cause of another specific event. The three kinds of routing produced by this algorithm are sequential, parallel or choice routing. The routing type depends on the relations between the events in the log file provided and which one causes the other event. However, the generic algorithm is limited where some event relations are not handled [3]. 3. Using the region miner algorithm, a connection can be established between Petri nets and transition systems [12]. The algorithm studies the states of the process and provides an insight of possible transitions between these states. The model can then be represented as a Petri net model which explicitly clarifies the concurrency, causality and any available conflicts between transitions. One of the limitations of this algorithm is when it assumes that the transition system shows all possible transitions between states, while in process mining, event logs do not contain all possible sequences of events. 4. The α algorithm is the basic process mining algorithm. The β algorithm applied in this paper is an extended form of the α algorithm. It handles process concurrency and is modelled using a structured workflow net where a complete even log is required. The α algorithm is limited to few logical relations and does not handle real life applications [1]. In this paper, the β algorithm is applied to a very complex and important industrial system, the healthcare. Healthcare systems, facing many challenges and an exponential growth in arriving patient number and disease types, are considered nowadays one of the most growing sectors in the service domain. Despite the differences between the structure of healthcare systems in different countries, they are always facing similar challenges such as surge of patient, lack of resources, critical medical errors and poor coordination [9]. Hence, the application of process mining techniques on healthcare systems is growing in order to analyze the complex behavior and the multi-disciplinary, variable structure of its processes [17]. Kovalchuk et al. introduced an approach for identifying healthcare models and simulating patient flow using mining techniques [19]. Batista and Solanas presented an efficient overview of process mining techniques and its application in the healthcare sector [20]. Rojas et al. presented a literature

Healthcare Emergency Room Management

51

review on process mining and its advantages applied to healthcare with promising results [21]. Fernandez-Llatas et al. presented in their work how process mining techniques can support healthcare professionals in analyzing emergency processes using log files, better understand the workflow of the system and improve patients’ quality of care [26]. The capability of process mining in giving insights on how to improve processes and the quality of health services provided through calculating emergency room metrics is illustrated in Rojas et al. [27]. Rebuge and Ferreira presented in their work how process mining is a promising approach in order to better understand a workflow process and analyze event data from log files found in the healthcare information system [28]. The emergency department of a healthcare system based in the middle east is studied in this paper. A comparison between performance measures is discussed before and after applying the β algorithm showing the effects of applying this process mining technique on the performance of the entire system under study.

4

Preliminaries

In this section, the basic information about the β algorithm is demonstrated. Deep details about the framework and its constrains can be found in [6]. A workflow model for healthcare is also presented and discussed. 4.1

β Algorithm

As discussed earlier in Sect. 1, β algorithm is a probabilistic accumulative framework that keeps learning from input log files. The more input files are fed, the more accurate the learnt process is. β algorithm defines relations among different events according to the following: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

“No relation And relationship denoted by ∧ Or relationship denoted by ∨ XOR relationship denoted by ⊕ implication relationship denoted by → Fellowship denoted by  Partial parallelism relationship denoted by ↔ Full parallelism relationship denoted by ⇔ A sequence denoted by → A repeated sequence denoted by  A loop denoted by ” [6]

Having two events a1 and a2 , they are and-ed (a1 ∧ a2 ) if and only if there is an intersection between their execution times. a1 and a2 are or-ed (a1 ∨a2 ) if and only if they are concurrent. In other words, one of them is occurring or the two of them occurring at the same time. a1 and a2 are xor-ed (a1 ⊕ a2 ) if and only if they are concurrent. In other words, they can never happen in the same time.

52

S. Oueida and Y. Kotb

a1 leads to a2 (a1 → a2 ) if and only if the occurrence of a1 yields the occurrence of a2 . In other words, it is a causality relation between a1 and a2 . a1 and a2 are partially parallel (a1 ↔ a2 ) if and only if a1 and a2 start in the same time but not necessarily end in the same time. a1 and a2 are fully parallel however, if and only if, they both start and end in the same time. Events ℵ = (a1 , a2 , . . . an ) are in sequence (ai → an ) if and only if ∀ai , aj ∈ ℵ, aj ∈ [ai =⇒ ai ∈ [aj and an ∈ [a1 and ∀ai ∈ ℵ, ai ∈ [a1 , an ∈ [ai . A repeated sequence a1 n is a sequence ai → an that repeats two or more times without being interrupted by an event that does not belong to the sequence [6]. As a comparison between β algorithms and the most common algorithms used in process learning, we introduce Table 1. Table 1. Process mining algorithms and their capabilities and limitations Item Noisy input data: Concurrency: Loops: Repeated sequences: Fuzzy processes: Variant detection: Workflow building:

β α Fuzzy Genetic Heuristic √ √ √ √ × √ √ √ √ √ √ √

× ×

× × √ √ × √ √ × √ √ ×

√ √ ×

× × √

×

×

×

×

√ where : means that the algorithm can accommodate the mentioned challenge, and ×: means that it cannot. As shown in Table 1, β algorithm deals with noisy input data and that is because the learning is accumulative and probabilistic. Those capabilities allows the algorithm to perform information correction through feeding more log files. The algorithm operators define concurrency, parallelism, causality, looping, and variant detection. The outcome of β algorithm is a workflow. This means the outcome defines a verified process for the organization to follow. The β framework needs one or many log files to learn the process in an accumulative manner. The more the log files are, the more accurate the probabilities of execution are produced and the less the noise introduced from learning events from log files is leading to an accurate model built without the limitations of noisy log files. For detailed information about β algorithm, please refer to [6]. 4.2

Emergency Room Workflow

A predefined model [14] of two emergency rooms (ERa and ERb ) was designed based on site observations, data collected from databases, and meetings with medical resources [14]. This model was then simulated using Arena in order to

Healthcare Emergency Room Management

53

study the system. Figure 2 shows the overall design of the system as it consists of three phases: 1. Phase 1: describes the requirements, the topology and resource allocation for triage and examination. 2. Phase 2: describes the requirements, the topology and resource allocation for radiology and billing. 3. Phase 3: describes the requirements, the topology and resource allocation for treatment and exiting.

Beginning of workflow

Phase 1

Phase 2

Phase 3

End of workflow

Fig. 2. Overall high level model

Phase 1 shows the arrival event of patients which is modelled as a Poisson distribution with λ = 4. The value of λ was approximated from the data gathered from log files. Figure 3 shows this arrival rate of patients to the emergency room modeled as a Poisson distribution with a λ = 4. From the plot, it is obvious that it is slow in the beginning of the process, then, it becomes hectic in the middle of the process, afterwords, it starts slowing down again closer to the end of the process. Phase 1 of the model is illustrated in Fig. 4 and it mainly focuses on the preparation and scheduling of the patient. It includes three main tasks: 1. It models the arrival pattern of patients to the emergency room. 2. It deals with patient triage. 3. It examines the patient to evaluate the emergency of the case and according to that evaluation it acts accordingly. Phase 2 of the model is illustrated in Fig. 5 and it mainly focuses on the radiology process and billing. It does the following: 1. It checks if a facility is needed. 2. If no facility is needed it jumps directly to phase 3. 3. If a facility is needed, a transporter accompanies the patient to radiology and then billing.

54

S. Oueida and Y. Kotb

f(k)

λ=4

0

1

2

3

4

5

6

7

8

9

k

Fig. 3. Poisson distribution (f (k) =

λk e( −λ) ) k!

Phase 1

Beginning of workflow

Arrival

Assignment

Data collection

Release a doctor

Examination

Assign a doctor

with λ = 4

Triage

Is it urgent? yes

no Phase 2

Release nurse

Waiting for a doctor

Go to waiting room

Treatment by nurse

Move to exam room

Fig. 4. Phase1 of the process model: arrival-triage-examination

Phase 3 of the model is illustrated in Fig. 6 and it mainly focuses on releasing patients from emergency room whether to lead them to admission if needed, or to exit the system. Resources used in the system are classified by phase and presented in Table 2. The current model needed plenty of time to be studied and designed due to the effort needed to observe the events, build a log file and then discover the workflow of that hospital. Some issues facing this system are listed as the following: 1. Number of input patients to the system is greater than the number of output patients which may cause deadlocks in case of high demand or catastrophe.

Healthcare Emergency Room Management

55

Phase 2

yes Facility needed?

no

Transporter seized

Move to radiology

Radiology process

Report writing

Move to ER B

Is patient B

Billing

Billing receptionist

Transporter release

Phase 3

Fig. 5. Phase2 of the process model: facilities (billing and radiology)

Phase 3

yes Waiting for a bed

Admission needed?

Admission process

Patient room placement

Workflow complete

no Patient LOS calculation

Patient discharge

Fig. 6. Phase3 of the process model: treatment-exit

Table 2. Resources utilized in the system classified by phase Resource

Count Phase 1 Phase 2 Phase 3 √ Registered Nurse: 1 × × √ Nurse: 2 × × √ Doctor: 1 × × √ Physician: 1 × × √ Technician: 3 × × √ Accountant: 8 × × √ Receptionist: 1 × × √ √ √ Transporter: 1

56

S. Oueida and Y. Kotb

2. The required time to serve each patient exceeds the accepted average time to meet patients’ satisfaction. 3. The current process shows the need for more resources to serve patients which will increase the cost dramatically, and this is not the aim of this study.

5

Petri Net Model

In this section, the Petri net model of the process under study is presented. Refer to this previous work for more details [30]. The model is divided into three phases: emergency room, radiology and billing and no facilities needed. Pools are represented with yellow circles, stages are represented with brown rectangles and blue circles represent the medical resources’ pools. Whereas, the green circles represent the start and end of the process model. Figure 7 illustrates phase 1 of the process model. Phase 1 consists of the emergency room where patients go through several stages during their journey in the hospital. When the patient arrives, he/she goes through the triage phase in order to check the severity of his/her medical case and then either will be directed to a waiting room or directly moves to see an available doctor. The doctor is then responsible for diagnosis and directing the patient to another unit for extra facilities if needed or just offering an immediate treatment. Figure 8 represents the second phase of the process model referring to the radiology and billing. Patients are directed by a medical transporter to another unit in the hospital in order to perform some extra facilities such as radiology, blood tests, billing, etc. Once process is complete, the transporter redirects the patient back to the emergency room to complete the treatment before exiting the system. Figure 9 is the last phase of the process model. In this phase patients complete their journey by moving to the treatment stage since no facilities are requested by the doctor. The doctor is responsible to decide here whether to admit or release the patient. In either cases, the patient is exiting the emergency room. It is trust worthy to mention here that patients who are directed to another unit for extra facilities will need to be redirected to this phase in order to accomplish the treatment and exit the system.

6

Learned Model Validation

β algorithm produces a model from a log file that includes timed events extracted from an information system in an organization. In this Section, a validation mathematical model is given to define a valid learned system. Given the following framework: Θ = {Λ, Δ, Ψ, τ, Ξ, Γ, I, O, P} (1) where, – Θ is the organization system. – Λ is the set of events occurring in that organization. – Δ is the set of dependencies among events.

Healthcare Emergency Room Management Arival process

Registration process Triage area

70 Registration area

Patient pool

Doctor seize process

Nurse pool

Registered nurse pool

2

1

Is severe

yes In need for a doctor

Check if severe Triage process

no

Waiting room process Waiting to be prepared

Waiting area Non severe process

Non severe examination

Waiting for a doctor Preparation by nurse

Doctor pool

1 Go to radiology

yes Examination room

Extra facilities needed

Radiology waiting

Examination process

Radiology process

Fig. 7. Phase1 of the process model: emergency room

– – – – – – –

Ψ is the constrains on the original system. τ is the time dimension of a process in an organization. Ξ is a mapping function that maps Λ to τ . Γ is a set of log files used to learn the process. I is the set of inputs to the system. O is the set of outputs from the system. P is the set of performance measures of the system.

57

58

S. Oueida and Y. Kotb Radiology waiting

Receptionist pool

P

1 Technician pool

Accountant pool

3

8 Billing process

Physician pool

Billing waiting

Patient attribute check

1

P

P

Radiology process

T

T

Billing transport waiting

Transporter release

Routing to ER process

Treatment process

1

P

Transporter pool

No facility needed

Fig. 8. Phase2 of the process model - facilities needed (radiology and billing)

Theorem 1. Having a system Θ1 , a learned system Θ2 , Θ2 is valid if: 1. 2. 3. 4. 5. 6.

∀λ ∈ Γ, λ ∈ Λ. where λ is one event from the set of events Λ. ∀λ ∈ Λ, ∃τ ξ(λ, τ ) = ∅ and ξ ∈ Ξ ∀λi , λj ∈ Λ, λi , Λj ∈ Γ =⇒ ∃λi ∩ λj ∈ Δ ∀δ ∈ Δ, ∃ψ ∈ Ψ, δ ∩ ψ = ∅ ∀o1 ∈ O1 , o2 ∈ O2 , o1 ≡ o2 ⇐⇒ ∃i1 ∈ I1 , i2 ∈ I2 , i1 ≡ i2 ∀I1 , I2 , O1 , O2 , I1 ≡ I2 , O1 ≡ O2 =⇒ P2 ≥ P1

Proof ∵ ∀λ ∈ Γ, λ ∈ Λ. ∴ ∃λ |λ ∈ Θ2 ∵ ∀δ ∈ Δ, ∃ψ ∈ Ψ, δ ∩ ψ = ∅ ∴ ∀i ∈ I2 , ∃o ∈ O, which yields that new system is sound. ∵ ∀o1 ∈ O1 , o2 ∈ O2 , o1 ≡ o2 ⇐⇒ ∃i1 ∈ I1 , i2 ∈ I2 , i1 ≡ i2 ∴ ∀o ∈ O2 , o is a valid output. ∵ ∀I1 , I2 , O1 , O2 , I1 ≡ I2 , O1 ≡ O2 =⇒ P2 ≥ P1 ∵ P2 ≥ P1 and I1 ≡ I2 and O1 ≡ O2 and Θ2 is sound and Δ ∪ Ψ ≡ Δ ∴ Θ2 is valid. 

Healthcare Emergency Room Management No radiology process

59

Treatment process

Extra Facilities Needed

Admission needed

no No facility needed

1 yes Doctor pool

Exit

Admission

no ER release

Admitted Waiting for bed

1 Transporter pool

OUT No admission needed

Fig. 9. Phase3 of the process model - no facilities needed

In other words, Theorem 1 states that the system is valid if: 1. All events in the system and log files are consistent. 2. All events in the system are timed events. 3. If two events exist in log files, there must be a form of dependency among them like how it is defined in the β algorithm. 4. The dependencies learned have to follow the predefined constraints of the system. 5. If outputs of the two systems are equivalent then the inputs of the two systems must be equivalent. 6. If inputs and outputs of the two systems are equivalent then the performance of the learned system has to be as good as or better than the original system.

7

Experiments and Simulation

Patient arrivals are modelled in Fig. 3 as Poisson distribution and are imported to the platform as event logs where each action in the hospital is represented by an event that starts at a specific time and ends at another time for the same patient ID. This is collected according to the code derived for the β platform written in Java. This β algorithm code observed the events, learned the process model, established the relationships between those events and resulted with a new process model. This new model is used to understand the causes of bottlenecks and thus start the optimization phase.

60

S. Oueida and Y. Kotb

Figure 10 shows the length of stay for patients in the system. The simulation is done for different number of patient arrivals as seen. The study shows that the length of stay increases linearly with the number of arrivals. Figure 11 shows that the transporter resource is over utilized compared to all other resources.

L.O.S

Before optimization After optimization

10 20 30 40 50 60 70 80 90 100 number of patients

Fig. 10. The length of stay (LoS) with respect to the number of patients

60

40

to Te r ch ni c A cc i a n ou nt a Ph nt ys i R ec cian ep ti Tr oni st an sp or te r

oc D

R

ur

se

0

N

20

N

utilization

80

Before optimization

After optimization

Fig. 11. Utilization per resource type

Healthcare Emergency Room Management

61

Applying β algorithm to the log file generated by the simulation suggests that transporter needs to be less utilized for the performance measures to be improved. This is because the dependency between the event named “transporter seize” and the “moving to radiology” event and their AND, Partial Parallelism relations cause the deadlock. As a one tested solution, that does not require any resource addition, is to break the current relations for those two events. This assumption is based on the fact that not all patients need a transporter. Therefore, as part of applying the β algorithm, only patients who are in need for a transporter will seize one in order to move to another unit in the hospital, whereas other patients will move without a transporter. This will definitely alleviate the bottleneck and decrease the transporter utilization rate. To prove this, the system is simulated again after adding this change and results are added to Fig. 10 and 11 represented by the blue color as part of optimization. The importance of maintaining alignment between event logs and process models was illustrated in [13]. After learning the process and applying the new changes on the learned process model using the implemented engine, simulation results show a proper improvement after the optimization phase in the LoS and transporter utilization rate. LoS and transporter utilization rate are decreased accordingly. From Fig. 11, it is obvious that the utilization rates of several other resources are increased and this is due to the increase in number of patients being seen after the decrease occurred in the transporter utilization rate and LoS.

8

Conclusion

As a summary, the β algorithm is proved mathematically and implemented on different set of data types especially data that are extracted from the healthcare sector. In our work, we integrated β algorithm using Java programming language to implement a data engine that can handle many challenges which face process mining methods such as, the challenge of handling the noisy log files or handling large sets of data, where the events and their logical relations are detected in an accumulative manner. The β algorithm is implemented and programmed to establish a complete framework based on Java programming language that is able to provide the required knowledge of any system and detect the process model with all its aspects, weaknesses and strengths as well. The β algorithm framework shows its robustness in this paper by implementing its concepts and techniques on log files related to a hospital located in the middle east. The process model was previously designed and studied through observations and site visits. Then, as part of applying the β algorithm, those same log files of that same hospital are analyzed using the β techniques in order to detect all events in the system, their correlations, dependencies and their frequencies and probabilities. As a result, a process model similar to the predefined one is detected automatically. This automated method of discovering the knowledge of the system under study provides valuable information of the current process, in addition to providing suggestions for improvements. The new scenario is implemented and tested again. After running the process with the new modifications

62

S. Oueida and Y. Kotb

suggested, the output reports show better performance by getting a lower transporter utilization rate, a lower patient waiting time (LoS) without increasing the overall cost or adding extra resources; thus, leading to an increase in the number of patients leaving the system and a good improvement in patient and management satisfaction. In this paper, we also propose a new validation theorem that shows the validity of the new learned system and its efficiency in optimizing a healthcare emergency room.

References 1. de Medeiros, A.K.A., van Dongen, B.F., van der Aalst, W.M.P., Weijters, A.J.M.M.: Process mining: extending the alpha-algorithm to mine short loops. Univ. Technol. Eindhoven 113, 145–180 (2004) 2. de Medeiros, A.K.A.: Genetic Process Mining. Cip-Data Library Technische Universiteit Eindhoven (2006) 3. Anbarasi, M., Anupriya, E., Iyengar, N.C.S.N.: Enhanced prediction of heart disease with feature subset selection using genetic algorithm. Int. J. Eng. Sci. Technol. 2(10), 5370–5376 (2010) 4. Mans, R.S., et al.: Application of process mining in healthcare? A case study in a Dutch hospital. In: International Joint Conference on Biomedical Engineering Systems and Technologies. Springer, Heidelberg (2008) 5. G¨ unther, C.W., Van Der Aalst, W.M.P.: Fuzzy mining? Adaptive process simplification based on multi-perspective metrics. In: International Conference on Business Process Management, pp. 328–343. Springer (2007) 6. Zayoud, M., Kotb, Y., Ionescu, S.: β algorithm: a new probabilistic process learning approach for big data in healthcare. IEEE Access 7, 78842–78869 (2019) 7. David Kelton, W.: Simulation with ARENA. McGraw-Hill (2002) 8. Oueida, S., et al.: Simulation models for enhancing the health care systems. FAIMA Bus. Manage. J. 4(4), 5 (2016) 9. Blendon, R.J., et al.: Common concerns amid diverse systems: health care experiences in five countries. Health Aff. 22(3), 106–121 (2003) 10. Pourmasoumi, A., Bagheri, E.: Business process mining. Encycl. Seman. Comput. Robot. Intell. 1(1), 1630004 (2017) 11. Li, C., Reichert, M., Wombacher, A.: Discovering reference models by mining process variants using a heuristic approach. In: International Conference on Business Process Management. Springer, Heidelberg (2009) 12. Carmona, J., Cortadella, J., Kishinevsky, M.: A region-based algorithm for discovering Petri nets from event logs. In: International Conference on Business Process Management. Springer, Heidelberg (2008) 13. Van der Aalst, W., Adriansyah, A., van Dongen, B.: Replaying history on process models for conformance checking and performance analysis. Wiley Interdisc. Rev. Data Min. Knowl. Discovery 2(2), 182–192 (2012) 14. Oueida, S., Kadry, S., Abichar, P.: Emergency department proposed model: estimating kpis. In: International Conference on Management and Industrial Engineering, vol. 8. Niculescu Publishing House (2017) 15. Van der Aalst, W., Weijters, T., Maruster, L.: Workflow mining: discovering process models from event logs. IEEE Trans. Knowl. Data Eng. 9, 1128–1142 (2004)

Healthcare Emergency Room Management

63

16. Oueida, S., Kotb, Y., Kadry, S., Ionescu, S.: Healthcare operation improvement based on simulation of cooperative resource preservation nets for none-consumable resources. Complexity (2018) 17. Erdogan, T.G., Tarhan, A.: Systematic mapping of process mining studies in healthcare. IEEE Access 6, 24543–24567 (2018) 18. Augusto, A., et al.: Automated discovery of process models from event logs: review and benchmark. IEEE Trans. Knowl. Data Eng. 31(4), 686–705 (2018) 19. Kovalchuk, S.V., Funkner, A.A., Metsker, O.G., Yakovlev, A.N.: Simulation of patient flow in multiple healthcare units using process and data mining techniques for model identification. J. Biomed. Inform. 82, 128–142 (2018) 20. Batista, E., Solanas, A.: Process mining in healthcare: a systematic review. In: 2018 9th International Conference on Information, Intelligence, Systems and Applications (IISA), pp. 1–6. IEEE (July 2018) 21. Rojas, E., Munoz-Gama, J., Sep´ ulveda, M., Capurro, D.: Process mining in healthcare: a literature review. J. Biomed. Inform. 61, 224–236 (2016) 22. Van der Aalst, W.M.: The application of Petri nets to workflow management. J. Circ. Syst. Comput. 8(1), 21–66 (1998) 23. Antunes, B.B., Manresa, A., Bastos, L.S., Marchesi, J.F., Hamacher, S.: A solution framework based on process mining, optimization, and discrete-event simulation to improve queue performance in an emergency department. In: International Conference on Business Process Management, pp. 583–594. Springer, Cham (September 2019) 24. He, Z., Wu, Q., Wen, L., Fu, G.: A process mining approach to improve emergency rescue processes of fatal gas explosion accidents in Chinese coal mines. Saf. Sci. 111, 154–166 (2019) 25. dos Santos Garcia, C., et al.: Process mining techniques and applications - a systematic mapping study. Exp. Syst. Appl. (2019) 26. Fernandez-Llatas, C., et al.: Analyzing medical emergency processes with process mining: the stroke case. In: International Conference on Business Process Management, pp. 214–225. Springer, Cham (September 2018) 27. Rojas, E., Sep´ ulveda, M., Munoz-Gama, J., Capurro, D., Seret, A.: Calculating emergency room metrics through process mining. In: Proceedings of the IEEE Engineering in Medicine and Biology Society (EMBC) (2016) 28. Rebuge, A., Ferreira, D.R.: Business process analysis in healthcare environments: a methodology based on process mining. Inf. Syst. 37(2), 99–116 (2012) 29. Aguirre, S., Parra, C., Alvarado, J.: Combination of process mining and simulation techniques for business process redesign: a methodological approach. In: International Symposium on Data-Driven Process Discovery and Analysis, pp. 24–43. Springer, Heidelberg (June 2012) 30. Oueida, S., Ionescu, S.: A resource aware extended workflow model for inconsumable resources: application to healthcare. In: International Conference on Management and Industrial Engineering, vol. 8, pp. 377–389. Niculescu Publishing House (October 2017)

Federated Learning Approach to Support Biopharma and Healthcare Collaboration to Accelerate Crisis Response Arijit Mitra1(B) , Abrar Rahman2 , and Fuad Rahman3 1 Apurba Ltd., Cheltenham GL53 0JQ, UK

[email protected] 2 The University of California, Berkeley, CA 94703, USA 3 Apurba Technologies, Santa Clara, CA 95051, USA

Abstract. During a pandemic, such as COVID-19, the scientific community must optimize collaboration, as part of the race against time to identify and repurpose existing treatments. Today, Artificial Intelligence (AI) offers us a significant opportunity to generate insights and provide predictive models that could substantially improve the opportunities for understanding the core metrics that characterize the epidemic. A principal barrier for effective AI models in a collaborative environment, especially in the medical and pharmaceutical industries, is dealing with datasets that are distributed across multiple organizations, as traditional AI models rely on the datasets being in one location. In the status quo, organizations must slog through a costly and time-consuming process of extract-transform-loading to build a dataset in a singular location. This paper addresses how Federated Learning may be applied to facilitate flexible AI models that have been trained on biopharma and clinical unstructured data, with a special focus on extracting actionable intelligence from existing research and communications via Natural Language Processing (NLP). Keywords: Artificial Intelligence · Federated Learning · Unstructured data

1 Introduction Cross-organizational collaboration has long been a facet of biopharma research and development, medical research, and healthcare practice. The adoption of cloud technology has further facilitated this, as greater connectivity is now possible, with entire environments set up specifically for collaborative work. In contrast, traditional Artificial Intelligence (AI) modelling and training techniques have been typically implemented with a single repository of data. For enterprise-level computing, a singular repository could be, for instance, a data warehouse or data lake [1]. There is a fundamental disconnect between these two organizational philosophies that has not been well-addressed. Setting up a specific shared data infrastructure for use by multiple organizations takes time, server space, and power, all of which have associated financial costs [2]. Even if the costs of a centralized approach for the storage of datasets could be justified, collaborative © Springer Nature Switzerland AG 2021 K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 64–77, 2021. https://doi.org/10.1007/978-3-030-63092-8_5

Federated Learning Approach to Support Biopharma

65

medical research is complicated, especially when a portion of that dataset involves patient data. Patient records, an invaluable source of real-world data for medical researchers, is often under strict legal scrutiny due to privacy and ethics concerns surrounding usage of individuals’ data, making it difficult for data scientists to use the data outside of the originating organization. This can be an insurmountable barrier to collaborative work. This paper proposes a Federated Learning (FL) approach, a decentralized machine learning technique, to address these issues. The concept emerged from a paper penned by a team of Google researchers in 2016, which resulted in “a reduction in required communication rounds by 10–100× as compared to synchronized stochastic gradient descent” [3]. Since then, the technique has gained popularity, especially after TensorFlow introduced TensorFlow Federated (TFF) in March 2019 [4]. To date, major corporations such as Nvidia [5], IBM [6], and of course Google itself [7] have published further research on the topic, as well as blog posts to popularize the concept and its potential. Notably, two of these write-ups specifically mention healthcare as a major intended use case for the technology [5, 6], an area that this paper expands upon to develop the first stages of something potentially useful. Federated Learning allows us to avoid moving sensitive data to a central location for processing [5]. Instead of bringing the data to the algorithm, we take the algorithm to the data. The data remains at rest and a version of the algorithm is trained locally. Finally, all the locally installed algorithms communicate their results to a central global model that updates itself by consensus. This system facilitates training of complex machine learning models with wider-ranging distributed datasets than ever before. This approach is ideally suited to biopharma, healthcare, and clinical practices, allowing us to create more scalable, representative ML models by taking advantage of disparate, large datasets that would otherwise be hidden by layers of red tape. It is worth noting that there is a lot of clinical data with extractable intelligence beyond simply numbers— natural (human) language data sources include doctors’ notes, journal publications, and even emails between researchers. These sources of data lend themselves to a Natural Language Processing (NLP) approach, which this paper elaborates on elsewhere. NLP relies extensively on AI and ML, which are the technologies that FL seeks to streamline.

2 Background Envision the possibilities of leveraging biopharma research and development, unstructured clinical notes, and Electronic Health Records (EHRs) to improve decision-making in the clinical and research spaces. These could be pharmaceutical companies, medical research charities, and non-governmental organizations monitoring population health— not to mention primary (i.e. personal physicians) and secondary healthcare providers (i.e. hospital care). We are envisioning employing FL to train AI models in a collaborative environment with multiple organizations, all with complimentary perspectives of healthcare and biopharma research data. We are building on our previous experiences with big-data architectures in the healthcare space, documented extensively in two previously published works. In the first [8], we developed a system to process multi-modal health signals in a big-data architecture.

66

A. Mitra et al.

In the second, we developed a biopharma tool, aiding those in the healthcare and life science sectors to more easily develop their AI models to run on cloud infrastructure [9]. 2.1 Motivation During a pandemic, collaborative biopharma and healthcare research must occur under extreme time pressure, while still being fraught with organizational and regulatory overhead. Traditional machine learning approaches generally expect training, testing, benchmarking datasets in a single location, which is not manageable for joint operations involving multiple organizations, where in addition to technical challenges, it is often hard to overcome legal barriers to the transport of sensitive data outside an organization. The associated costs [2] of creating a single universe of supersized data warehouses and data lakes for structured and unstructured data, respectively, may prove to be prohibitive for operations of a certain scale. This presents a serious barrier for coordinated population health predictive analytics. We envision the use of cloud infrastructure, big data analytics frameworks, and a tested AI technology stack for generating new therapies or repurposed leads, deep patient history, and population health predictive models from distributed data sets held in multiple virtual private clouds and data centers. 2.2 Technical Challenges Federated Learning is defined as a machine learning technique that trains an algorithm across multiple decentralized edge devices or servers holding local data samples, without exchanging their data samples [3]. The application of this technique [10] in a multiorganizational environment is complicated due these factors: • Security – maintaining a secure connection between each organizational data source and ensuring each organizational data source only exposes the data that is pertinent to the study. • Bandwidth – performant connection between organizations that could well be spread across geographical regions. • Compute at Source – level of compute power at each organization needs to be sufficient for timely processing of local data. • Orchestration – managing federated machine learning approach, in terms of coordination of compute resource, infrastructure, auditing, monitoring, and gathering the output of the distributed model. • Data Integrity – at each organization, there needs to be an internal audit for data quality to ensure that the resulting trained AI model is robust, ethical, unbiased, representative, legal, and accurate [11].

Federated Learning Approach to Support Biopharma

67

3 Implementation 3.1 Federated Learning Architecture We chose FL due to its particular suitability for the clinical domain. In order to make an inference or diagnosis, a doctor relies on evidence and their own experience. Consider what could happen when a clinician encounters a rare disease. Even in a career that may span thirty years or more, they may only have come across such a condition only a few times. The promise of AI, ML, and Deep Learning is in this domain is that, provided that they have been given a wide-ranging dataset, they can assist the clinician in identifying patterns and features that the clinician would have ordinarily missed. To build a wide enough, bias-free, robust dataset, is challenging, and collaboration across organizations is critical. As stated earlier, one of the key barriers to pooling data is privacy. The data cannot easily be moved due to its sensitive nature. FL makes it possible for AI algorithms to learn from data located in different sites and organizations. Our FL approach consists of a central cluster. This central cluster is responsible for maintaining the global deep learning neural network model. A local deep learning neural network model is also installed on each local environment. This structure is illustrated in Fig. 1.

Fig. 1. Federated Learning diagram.

Once the model is trained locally and then the trained network parameters are uploaded to the central server, we can keep the local data just where it is. The central server then aggregates the model. This approach differs significantly from traditionally models, where there is only a single trained model and a single data source. The benefits are that the combined global model is trained from a wider dataset, but it presents its own set of complexities. Use of single trained model per single data source is far easier to implement. The use of a globally trained model brings with it complexity in terms of deployment, maintenance

68

A. Mitra et al.

and orchestration. The first two, we will address with containerization, cloud computing and microservices. The final orchestration is addressed with the use of an Event Bus. A potential alternative to FL is Gossip Learning [12], which takes the concept of decentralization even further, removing the need for a single aggregation server. In fact, you could absolutely emulate our work using Gossip Learning instead! In the end, the relative ease of setting up an FL solution as well as the more established research community around FL lead us to choosing this option instead. We will explain how FL can be applied using a practical example. We took inspiration from an earlier work conducted by NVIDIA and King’s College London, which utilized FL for the analysis of brain tumor images from several hospitals [5]. 3.2 Containerization For our implementation of FL, we took a DevOps approach. To that end, we used established industry tools, such as Docker and Kubernetes [13], to construct a container to hold all the resources for our deep learning model. Containerization allows developers to deploy code with all the resources it needs to operate in their target environment. We took this approach as it ringfences the local deep learning model. Installing it directly onto a host operating system would mean that separate testing could be required, and we could not completely discount the risk of contamination in the local environment and the local deep learning model. Furthermore, using a container ensures that each local deep learning model is operating on an identical technology stack. 3.3 Cloud Computing and Data Science in Biopharma/Healthcare Both the biopharma and healthcare sectors operate at big data scales. Gradually over the last decade, the adoption of cloud computing and big-data analytics has revolutionized data processing in these sectors. To practically employ data science at the scale required by these sectors, use of cloud computing is a prerequisite to any enterprise-scale deep learning initiative. For this project, we used Amazon Web Services (AWS) [14] as the cloud platform to manage the entire core infrastructure and leveraged specialist services for containerization, orchestration, monitoring and auditing. 3.4 Software Architecture All data in this study was anonymized. In practice, an effective population health predictive model during a pandemic will draw insights from multiple Artificial Intelligence and machine learning models. If we overlay this complexity with application of FL to facilitate a multi-organization taskforce, then we need an appropriate architecture. To this end, we implemented a microservice architecture that allows for a looselycoupled, strongly-aligned approach to achieve a robust and maintainable system. The critical success factors that we are looking for by implementing a microservice architectural approach are as follows:

Federated Learning Approach to Support Biopharma

69

• Common Enterprise Frameworks that describe a set of standards used by organizations within the taskforce. • Agility that allows for small, independent teams delivering functionality into the ecosystem. • Process that enables continuous integration and delivery for ease of deployment, monitoring, auditing, and on-going maintenance. 3.5 The Case for Microservices Use of microservice architecture will enable a smoother increase of services gradually introducing different models, data sources, modalities, platforms etc. Microservice architecture allows a consortium of organizations to cope with the variety of vendors using this platform as it becomes more widely adopted. It facilitates deployment and technology flexibility while allowing for services to be scaled separately. The use of modern processes and tooling for containerization and orchestration eases deployment and architectural complexity. 3.6 Intercomponent Communication The potential for confusion, when operating a Federated Learning approach across organizations, is considerable. Ensuring that data sets are clearly labeled and correctly exposed, the proliferation of hyper-parameters between model instances, coordinating, training, and testing pose a significant problem of information exchange. To overcome this challenge, we built upon an earlier work presented at the Academy of Science and Engineering (ACE) 2014, Stanford University. In that paper [15] proposed the use of an event bus for orchestration of data processing in a distributed environment. To get all the pieces of this platform to communicate with each other, we used an event bus called AWS EventBridge for this implementation [16]. This product is a modern commercial off-the-shelf (COTs) product running as a serverless implementation, handling a great deal of data pipelining, security, and error-handling for our project. 3.7 Amazon EventBridge We have adopted Amazon EventBridge to connect applications with data from a variety of sources. EventBridge delivered a stream of real-time data from each node of the Federated Learning application. AWS EventBridge ran as a serverless offering, reducing maintenance overhead. In addition, we implemented each AI model on AWS Lambda [17], which eliminated the need to manage an independent server while streamlining scaling for the project. The structure of AWS EventBridge as it pertains to our implementation [16] (Fig. 2).

70

A. Mitra et al.

Fig. 2. AWS EventBridge [16]

3.8 Using Microservices for Federated Learning We utilized a microservice architecture, as seen in Fig. 3, as it brings these key advantages:

Fig. 3. Microservice Architecture

• Deployment Flexibility ensures ease of implementation and mitigates risk. • Technology Flexibility (consider the number of AI SDK’s, Programming languages – this approach lets us choose any one that fits the task). • Providing the option of encapsulating serverless functions. • Allow each service to be scaled separately (key advantage where a pre-trained NLPfocused unsupervised learning algorithm has a different GPU requirement to an image classification AI model).

Federated Learning Approach to Support Biopharma

71

To support Federated Learning, we designated one microservice to encapsulate the global deep learning model. Three microservices, one for each organization, were used to encapsulate each local deep learning model. An additional microservice was used to manage interactions with EventBridge, allowing us to consolidate updates and create the global deep learning model.

4 Results As a showcase of our work, we created a Graphical User Interface (GUI), which ran through a web client. In order to properly demonstrate all the features that we had implemented, we will walk through each and every step of the workflow for using the GUI. Firstly, we will have a short explanation setting the stage. Following the initial introduction, we have seven figures, illustrating all of the features in the context of our GUI. Finally, a brief discussion will be presented to discuss our results. It is important to point out that the application presented here servers two purposes, to illustrate how we solve a real problem and to demonstrate how the distributed implementation encompasses our underlying approach introduced in the earlier part of this paper. The application we are about to present is about identifying key phrases that may have commercial values in biopharma and healthcare in a train of emails. This is one example of the aforementioned natural language data sources prevalent in the world of medicine, which require Natural Language Processing (NLP) technologies. Our NLP engine extracts key words from these documents, which are then verified against a predefined set of constantly updating “target components.” The application then allows the user to look up additional details about the identified component and see its value with a predefined “process workflow.” All the required processes within this complex workflow have been implemented as micro-services and are being run on various AWS instances hosted across the globe. For example, the NLP servers are in North California, the databases are in United Kingdom, the GUI is hosted in South East Asia, and so on. Before we start our walkthrough of the system, keep in mind that the screenshots below have been edited with nonsense filler text in order to protect sensitive information in compliance with privacy regulations. The fact that our example database itself was riddled with these kinds of data-handling restrictions just goes to show the necessity of implementing FL in our solution, as poetically convenient as it may be. As seen in Fig. 4, clicking on any particular message opens up a new interface, as seen in Fig. 5. On the right-hand side, the user can see the subject line and contents of each message. In the top left, there are two buttons. The first one identifies prominent, relevant topic clusters in the message text and highlights them, as seen in Fig. 6. The second specifically identifies any chemical compounds known within the provided database, again highlighting relevant threads, as seen in Fig. 7.

72

A. Mitra et al.

Fig. 4. Each search result consists of an archived message between researchers.

Fig. 5. Interface upon opening up a message.

Right-clicking any of those purple annotations brings up a flowchart, as seen in Fig. 8, generated from topics of interest as provided by the client. To read the chart, the items on the left are the broadest, least specific category. Arrows indicate some association or similarity between various clusters of topics, in order to show the relationships between the various topics that are being analyzed. The cluster from which the user arrives at the flowchart is highlighted in green.

Federated Learning Approach to Support Biopharma

73

Fig. 6. Clicking “Identify Topics” causes the NLP algorithm to comb through the message looking for relevant topics, presenting options on the left hand side.

Fig. 7. Clicking any particular topic will display all references within the text.

74

A. Mitra et al.

Fig. 8. The complete topic flowchart, with the current reference highlighted in green.

Similarly, users can click “Identify Compounds” to generate annotations for each and every one of the provided chemical compounds the researcher is interested in, again material which must come from the user, as seen in Fig. 9. This gives the user a simple, at-a-glance look at the chemicals named, as seen in Fig. 10. With data from a largerscale pharmaceutical research lab, this section can easily be expanded to provide more comprehensive information regarding that chemical.

Federated Learning Approach to Support Biopharma

75

Fig. 9. Clicking any given compound will display all references within the text.

Fig. 10. Clicking the annotation to see pertinent details about the molecule at hand.

5 Conclusions We applied a FL approach as a proof-of-concept to provide tools to accelerate collaboration processes used for identification of promising antiviral drug candidates for repurposing for other disease areas. Using this technology, we were able to show how scientific concepts could be retrieved and visualized in real-time during live collaboration between research partners.

76

A. Mitra et al.

The datasets in question were distributed within different content management systems. We designated a processing node for each content management system. To each node, we projected an initially pre-trained NLP model with an associated taxonomy or semantic network, plus a topic-modeling algorithm. A central parent node coordinated all nodes. We ran Amazon AWS P3 instances for the parent node and each child. The result was that we were able to provide a topic-based categorization of the distributed content, based on lexical chains describing a biopharma or clinical concept. To ensure accuracy the concepts were matched against a curated taxonomy. Federated Learning allowed us to train AI models for NLP across a wider and often quite disparate data set. This resulted in a set of AI models that were capable of identifying biopharma and clinical concepts in unstructured data and identifying a narrative between them. This implementation centered around FL approach to Natural Language Processing models covering corpora located in virtual private networks on AWS cloud belonging to a taskforce of multiple organizations. Each corpus was composed of unstructured data from diverse domains, biopharma, clinical trials and clinical practice. All data was an anonymized.

6 Future Work For future work, we would use this FL via microservice architecture approach to include different biopharma and healthcare modalities such as medical images, video, genomics, and chemoinformatic data.

References 1. Cedrine, M., Laurent, A.: The next information architecture evolution: the data lake wave. In: MEDES: Proceedings of the 8th International Conference on Management of Digital EcoSystems, pp. 174–180, November 2016 2. Rosenthal, D.S.H., Rosenthal, D.C., Miller, E.L., Adams, I.F., Storer, M.W., Zadok, E.: The Economics of Long-Term Digital Storage. http://www.unesco.org/new/fileadmin/MULTIM EDIA/HQ/CI/CI/pdf/mow/VC_Rosenthal_et_al_27_B_1330.pdf 3. McMahan, B.H., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) (2017) 4. Ingerman, A., Ostrowski, K.: Introducing TensorFlow Federated. https://blog.tensorflow.org/ 2019/03/introducing-tensorflow-federated.html 5. Riek, N.: What is federated learning? https://blogs.nvidia.com/blog/2019/10/13/what-is-fed erated-learning/ 6. Baracaldo, N.: Federated Learning. https://community.ibm.com/community/user/ datascience/ blogs/nathalie-baracaldo1/2020/01/03/federated-learning-part-2 7. McMahan, B., Ramage, D.: Federated Learning: Collaborative Machine Learning without Centralized Training Data. https://ai.googleblog.com/2017/04/federated-learning-collabora tive.html

Federated Learning Approach to Support Biopharma

77

8. Rahman, F., Mitra, A., Slepian, M.J.: A Novel big-data processing framework for healthcare applications—big-data-healthcare-in-a-box. In: 2016 IEEE International Conference on Big Data, pp. 3548–3555, December 2016 9. Toh, T.S., Dondelinger, F., Wang, D.: Looking beyond the hype: applied AI and machine learning in translational medicine, EBioMedicine. 47, 607–615 (2016) 10. Chandiramani, K., Garg, D., Maheswari, N.: Performance analysis of distributed and federated learning models on private data. In: Procedia Computer Science, vol. 165, pp. 349–355 (2019) 11. Hagendorff, T.: The ethics of ai ethics: an evaluation of guidelines. Mind. Mach. 30(1), 99–120 (2020). https://doi.org/10.1007/s11023-020-09517-8 12. Heged˝us, I., Danner, G., Jelasity, M.: Gossip Learning as a Decentralized Alternative to Federated Learning. https://link.springer.com/chapter/10.1007/978-3-030-22496-7_5 13. Melendez, C.: The Advantages of Using Docker and Kubernetes Together. 19 March 19 2018. https://stackify.com/kubernetes-docker-deployments/ 14. About AWS (2020). https://aws.amazon.com/about-aws/ 15. Rahman, F., Mitra, A.: Distributed Processing of Large Scale Financial ‘Big Data’ from Disparate Sources of Data Academy of Science & Engineering. Stanford University, California, USA (2014) 16. Amazon EventBridge (2020). https://aws.amazon.com/eventbridge/ 17. AWS Lambda (2020). https://aws.amazon.com/lambda/

Ethical Analysis on the Application of Neurotechnology for Human Augmentation in Physicians and Surgeons Soaad Qahhar Hossain1(B) and Syed Ishtiaque Ahmed2 1

Department of Computer and Mathematical Sciences, Department of Philosophy, University of Toronto Scarborough, Toronto, Canada [email protected] 2 Department of Computer Science, University of Toronto, Toronto, Canada [email protected]

Abstract. With the shortage of physicians and surgeons and increase in demand worldwide due to situations such as the COVID-19 pandemic, there is a growing interest in finding solutions to help address the problem. A solution to this problem would be to use neurotechnology to provide them augmented cognition, senses and action for optimal diagnosis and treatment. Consequently, doing so can negatively impact them and others. We argue that applying neurotechnology for human enhancement in physicians and surgeons can cause injustices, and harm to them and patients. In this paper, we will first describe the augmentations and neurotechnologies that can be used to achieve the relevant augmentations for physicians and surgeons. We will then review selected ethical concerns discussed within literature, discuss the neuroengineering behind using neurotechnology for augmentation purposes, then conclude with an analysis on outcomes and ethical issues of implementing human augmentation via neurotechnology in medical and surgical practice. Keywords: Neurotechnology · Ethics · Augmentation · Enhancement · Brain-computer interface · Physicians · Surgeons · Patients · Harm · Global · Social · Justice · Malicious brain-hacking · Personhood · Brain data · Discrimination · Rights · Medical practice · Cognition · Senses · Action

1

Introduction

The demand for medical and healthcare professionals continue to rise as the overall world population continues to increase. This demand is further increased when pandemics such as the SARS-CoV-2 outbreak, also known as the coronavirus disease COVID-19, occurs. The problem is that there is and may always be a shortage of especially medical practitioners due to challenges and barriers that exist to make it difficult for individuals to become physicians or surgeons. Even if there were sufficient medical practitioners, the other problem is the lack c Springer Nature Switzerland AG 2021  K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 78–99, 2021. https://doi.org/10.1007/978-3-030-63092-8_6

Ethical Analysis

79

of resources that hospitals and institutions alike possess. Given that resources are already a problem for hospitals, having sufficient clinicians would only make that worse. With recent advances in neuroscience and neurotechnology paving ways to innovative applications that cognitively augment and enhance humans in a variety of contexts, a solution to this would be to use neurotechnology to augment the cognition and senses of medical practitioners to then augment their actions [13]. In doing so, this will allow them to make optimal decisions and actions in less time, enabling them to treat more patients. As such, this makes the solution along with the research and discussions pertaining to it highly important as not only does the solution impact directly impact physicians and surgeons, but it also directly impacts patients as well. Additionally, as with any technological intervention, it is important that not only are the technical aspects of the intervention are analyzed and discussed, but the ethical aspects are as well. In approaching this technological intervention, we performed an ethical analysis on the application of neurotechnology for human augmentation in physicians and surgeons, and found that even in the best-case scenario, applying neurotechnology for human augmentation in physicians and surgeons has a significant negative impact on individuals, communities and countries. In this paper, we argue that applying neurotechnology for human augmentation to augment physicians and surgeons, and can cause personal identity, discrimination and financial issues for physicians and surgeons, and lead to patients being harmed. The way that the paper is structured is as followed: we first describe the augmentations and neurotechnologies that can be used to achieve the relevant augmentations for physicians and surgeons, then review selected ethical concerns discussed within literature particularly focusing on human rights, human-computer interaction, data, brain-computer interface, global bioethics and drug development, and discuss the neuroengineering behind using neurotechnology for augmentation purposes. We then conclude with an analysis on outcomes and ethical issues of implementing human augmentation via neurotechnology in medical and surgical practice. The ethical analysis specifically focuses on social issues, global health inequality and health migration, and patient harm, and includes an assessment on personhood with respect to the neurotechnology users (i.e. the physicians and surgeons). In this paper, we assume that all neurotechnologies mentioned always succeeds in providing a person with augmentations, but the type of augmentation provided by a neurotechnology is based on what its capable of. The motivation behind this assumption is to allow the paper to address a possible best-case scenario with neurotechnology and human augmentation in physicians and surgeons, and focus on ethical issues that arise with this scenario.

2 2.1

Human Augmentation and Neurotechnology Types of Augmentations

Human augmentation can be formally defined as an interdisciplinary field that addresses methods, technologies and their application for enhancing cognitive

80

S. Q. Hossain and S. I. Ahmed

abilities, senses and actions of humans [32]. In enhancing these leads us to augmented cognition, augmented senses, and augmented actions. Augmented senses focus on enhancing one’s ability to see, hear, smell, taste and touch. Augmented cognition focuses on enhancing one’s memory (short-term and long-term), attention, learning capabilities, awareness, and knowledge. Augmented action can be broken down to two parts: motion augmentation and gesture augmentation. Motion augmentation would simply be improving one’s ability to move, enabling them to take actions that they may not normally be able to do such as carry very heavy objects or run at a faster speed. Gesture augmentation is similar to motion augmentation, except that it focuses more movement and positioning of one’s hand and head. For instance, being able to keep your hand in a specific position for a long amount of time without fatigue and shaking would be considered augmented gesture and not motion augmentation. The end goal of both augmentations is that the augmentation should allow the person to perform an action optimally. From all the augmentations mentioned, we will only focus on augmented senses (specifically augmented vision and augmented touch), augmented cognition, and augmented action (specifically augmented gesture). The motivation behind these augmentations in particular is because these augmentations best allow physicians and surgeons to better diagnose and treat diseases and disorders. 2.2

Augmentations Through Neurotechnology

From current technologies that are being developed, one such technology that can be used to obtain these augmentations is neurotechnology. Neurotechnology can be formally defined as an interdisciplinary field that combines neuroscience, engineering and technology to create technical devices that are interfaces with human nervous system. Within literature on neurotechnology, there has been a considerable amount of work carried out on neurotechnologies for cognitive enhancement, specifically focusing on brain-computer interface (BCI), also known as neuroelectronic interface [2], applications. Based on previous studies, neurostimulation techniques, such as transcranial electric stimulation (tES) and transcranial magnetic stimulation (TMS), can be used to improve performance in different cognitive domains; these cognitive domains include perception, learning and memory, attention, and decision-making [13]. Accordingly, this can be used to accomplish augmented vision and part of augmented cognition. Neuromodulation techniques, such as transcranial direct current stimulation (tDCS), can also be used for memory, learning and attention [13]. Specifically, for tDCS, studies have shown that it could improve performance in verbal problem-solving task and other areas within complex problem solving [13]. This allows for further enhancement of specific cognitive domains, as desired. To address augmented touch while also addressing augmented action, a combination of tactile sensors and neuroprosthetics. Tactile sensors allow for perceiving the location of the point of contact, discerns surface properties, and detection for eventual slippage phenomena during the grasping and handling of an object [11]. Neuroprosthetics are devices that can enhance the input and output of a

Ethical Analysis

81

neural system [5]. Combined together, they can allow for augmented gesture through enhancing the person’s dexterous interaction with objects by ensuring that there is a correct application of forces between the contact surfaces of the person’s hand and fingers and those of the object, and having the neuroprosthetic distinguish brain activity patterns corresponding to the intention to act from control signals of the hand itself [38]. To summarize, the neurotechnology that can be used to accomplish the augmentations of interest is neurostimulation, neuromodulation, tactile sensors and neuroprosthetics. How each of these neurotechnologies accomplish augmentations will be discussed later in the paper when We discuss how neurotechnology can be applied in medical practice.

3 3.1

Ethics in Neurotechnology and Human Enhancement Human Rights in Neurotechnology

While neurotechnology has the can help achieve human augmentation, it also has the potential to impact human rights [22]. Three of the areas which neurotechnology is said to impact is the right to mental integrity, freedom of thought, and freedom from discrimination. The right to mental integrity is a concern with neurotechnology due to the problem of malicious brain-hacking. Malicious brainhacking is defined as the neuro-criminal activities that directly influence neural computation in the users of neurotechnological devices in a manner that resembles how computers are hacked in computer crimes [22]. For BCI applications, malicious brain-hacking can take place within BCI through having the malicious agent (e.g. hacker) attack the level of measurement or decoding and feedback of the BCI, or through the malicious agent manipulating the person’s neural computation through the BCI application. In such cases, the malicious agent can add noise or override the signals sent to the neurotechnological device with the objective of reducing or removing the control of the user over the BCI application [22]. Furthermore, the malicious agent can also hijack the BCI user’s voluntary control. For instance, a malicious agent can override the signals sent from a BCI user, then hijack a BCI-controlled device (e.g. smartphone) without the BCI user’s consent [22]. Ultimately, what results from malicious brain-hacking is both the user’s mental privacy and their brain data being at risk due from the loss of right to mental integrity. The loss of right to mental integrity leads to the loss of freedom of thought as when malicious brain-hacking occurs, the BCI user is no longer able to freely think of things due to the restraint placed by the malicious agent. One other human rights issue with neurotechnology is one that overlaps with a health rights issue from human enhancement. The ethical issue from human enhancement pertains to how the ability to augment one’s physical or mental performance raises several issues about fairness and justice regarding how augmenting technologies, such as neurotechnologies, should be accessed or regulated [20]. What follows from this is the question of whether they are intended for mass consumption or restricted to humans with identifiable impairments and disabilities. In the case where neurotechnologies are intended for mass consumption and

82

S. Q. Hossain and S. I. Ahmed

becomes commercialized, as with any other technology, it will likely be expensive at the beginning [20]. Consequently, this could generate and even exacerbate societal divisions among the population or among inhabitants of different countries [20]. Those of a low socio-economic status would not be able to afford to purchase neurotechnologies. Subsequently, a digital divide is created between those that can afford neurotechnology, and those that cannot. This leads to health right issue as the digital divide creates an injustice to those of low socio-economic status that cannot afford neurotechnology; they are indirectly being deprived of something that is needed for their health and well being. With the relationship between poor mental health and the experience of poverty and deprivation being well studied and an association between the two factors being established, communities deprived of neurotechnology can cause those communities to experience higher levels of deprivation and unemployment rates, leading to them experiencing higher rates of psychiatric admissions for psychotic as well as non-psychotic conditions and suicidal behavior [27]. 3.2

Discrimination from Augmentation

The ongoing human rights concern regarding neurotechnology an freedom from discrimination is that the use of neurotechnology will lead to discrimination among individuals and groups. This concern is not only one that is valid, but also strong evidence to support it. Throughout history, humans have shown that existing moral code is often broken in practice, and that observable differences between people, such as differences in race, gender, ethnicity, religion, sexual orientation and/or ability, tend to lead to moral and social inequalities [7]. Given that there will be a clear difference between those that adopt augmentation technologies and those that do not and the actions that humans have taken in the past and continue to take to this day, it is very likely that human augmentation will lead to new, unjustified inequalities, and may even undermined the core notion of moral equality used in Western societies [7]. Expanding on the unjustified inequality, there is a growing concern regarding how new human augmentation technologies can weigh more than personal experiences and benefits [32]. Due to the cognitive enhancement that augmentation technologies provide, those that utilize them will have a stronger say on matters than those that rely solely on their personal experiences and benefits. Also, there is a concern that neglecting potential negative effects of such technologies to daily life and society can create problematic scenarios for the future [32]. One such potential negative side effect comes from the pressure in adopting augmentation technologies. For a long time and even now, people frequently experience prejudice if their bodies or brain function differently from those around them [40]. With the integration of augmentation technologies such as neurotechnology within society, this will lead to people feeling pressure to adopt those technologies, and is likely to lead to change societal norms. In turn, this will lead to issues of equitable access and generate new forms of discrimination [40]. To describe the forms of discrimination, we refer to self-identity, industry practice, and history. In self-identity, moral and social importance plays a crucial

Ethical Analysis

83

role in how one people feel about themselves, and it plays a strong role in determining people’s intentions, attitudes and behaviors [7]. Consequently, a poorly developed self-concept could bring about low self-esteem, level of confidence, purpose and reason to live, and level of motivation. With groups and communities determining collectively how one is treated by others, poorly formed self-concept could make one subject to discrimination. Within industries, the discrimination can take place as earlier as the job application process to later while the person is employed with the company. In industry practice, it is common for companies to hire the person that they think are best for the position. With neurotechnology providing its users with augmentations, a complaint regarding this is that this provides those people with an unfair advantage that will allow them to secure jobs more easily, and that those that are augmented may view those that are not augmented as inferior. This unfair advantage can lead to discrimination within industries through employment. The unfair advantage from augmentations can have serious implications in the workplace especially once employers, administrators and decision-makers realize that those with augmentation perform better than those without them [10]. The realization will likely lead to them having a preference for those with augmentations and a bias against those that do not have any augmentations, leading to things such as them inquiring beforehand (or on the job for current employees) whether an applicant has undergone an augmentation or utilizes neurotechnology, and making decisions simply based on this information along with selected other pieces of information from the applicant (or employee). This can lead to prejudice, resulting in non-augmented people being discriminated against, creating justice and equity issues. Note that this type of discrimination is not limited to employment settings, and extends as far as in areas such as families and academic institutions. Between children, parents will give preference to the child that performs best academically and professionally. Similarly, with academic institutions, the institutions will prefer and reward those that perform best academically, and treat those that perform best academically better than those that perform worse regardless of whether they have undergone any sort of augmentation or not. We know this as in many academic institutions, if students do not perform well enough academically, they are either removed from their program, suspended or both, and those that perform well academically are rewarded awards, research and other opportunities, and scholarships. As a result, in the long-term and possibly even in the short-term, the implications from the unfair advantage from augmentations can lead to a collapse in the social process of education, care, interactions, relationships and more as impartiality, open-mindedness, nondiscrimination, acceptance, and unbiasedness would be minimized or lost completely in some groups and societies. With neurotechnology and human enhancement possibly being enjoyed by the already privileged, if this possibility is realized, then this will lead to a failure of distributive justice [18]. Of all communities, marginalized communities will suffer the most from this failure. A major challenge experienced by underprivileged communities comes from the fact that not only are they are often

84

S. Q. Hossain and S. I. Ahmed

excluded from opportunities, health services, development programs, resources and more, but they often struggle to raise their voice in public discussions due to the lack of access and autonomy [4]. As a result, underprivileged communities, including women, LGBTQ groups, refugees, people with physical or mental disabilities, people of low socio-economic status, and those that are less educated are often ignored, deprived, or discriminated [4]. Among them, marginalized groups in those communities are even more deprived, which this deprivation are sometimes due to their physical or economic vulnerabilities, or various social and cultural practices [4]. For example, in multiple places in the Indian subcontinent, women are not allowed to use mobile phones due to social and cultural reasons [4]. Consequently, this deprivation prevents them from obtaining the benefits from it - one of them being that it can be used to cope with everyday stress [28]. In the case where the group of Indian women were members of the LGBTQ group, this deprivation of technological device can further negatively impact their mental health. LGBTQ people have and continue to experience various forms of oppression and discrimination worldwide [21]. The oppression is in the form of harassment and violence while the discrimination is experienced in areas such as employment, housing, access to education and human services, and the law. This oppressions and discrimination along with rejection, violence and harassment have been shown to have negative physical and mental health effects on both LGBTQ people [21]. Combining the primary societal stressors that many LGBT people experience, that lead to “minority stress” [21], with deprivation of mobile phones leads to those Indian women having less effective coping mechanism options which are needed to deal with both everyday stress and minority stress. Consequently, this results to there being major mental health disparities between those Indian women within the LGBTQ population and other individuals. From just the transgender individuals alone, the mental health disparities can be severe enough to the point where the difference is that those individuals have higher odds of depression symptoms and attempted suicides [39]. What the failure of distributive justice will likely do is aggravate the everyday stress and minority stress experienced by this marginalized group and others to an extent where the majority of the marginalized population will show depression symptoms, and a substantial amount of them may even commit suicide. As such, the already marginalized communities will suffer more than they already are. The repercussions of this will have negative impacts on both a local and global level. 3.3

Autonomy and Brain Data

Neurotechnology not only has the potential to impact human rights, but is can also impact autonomy, confidentiality and protection. Note that it is possible to obtain information through BCIs that can be extracted to reveal security breaches [26]. Neurotechnology such as BCI relies on multiple types of probabilistic inference to be operationalized, for one to use any sort of probabilistic inference requires a large amount of data [19]. This data, consisting of neural information, is of high value to companies and individuals as neurotechnology produces raw data in a way that enables a more direct detection pathway of

Ethical Analysis

85

the neural correlates of mental processes, such as interests, intentions, mood and preferences [23]. Additionally, the neural data includes rich and personally identifiable sources of information that could be aggregated by data handlers to apprehend or predict parts of health status, preferences and behavior [23]. An example of where neural information would be very useful is marketing and advertising. The field of marketing has been interested in BCI research as they are interested in utilizing a neuromarketing-based approach to marketing and advertising [3]. Algorithms that are designed and used to target advertising can better calculate things like insurance premiums through using neural information [40]. Accordingly, investment in neural data (and computer scientists) is something that marketing firms are striving to acquire, heavily investing time, effort and money to obtain it. Given the value of neural information, these motivate malicious agents, companies and corporations to perform malicious brain-hacking, or retrieve, aggregate disseminate and use the information of the neurotechnology users without their informed consent. In the case of malicious brain-hacking, after a malicious agent has successfully hacked a device, rather than changing signals or taking control, they can instead extract the data from the device, then abuse to them to as they desire. Similarly, for companies and corporations behind a neurotechnological application, they can either take the data collected from the neurotechnology users as hostage of the users or sell the data to another entity or individual, with the purpose of making more money. This has several consequences, including loss of ownership of one’s own data, loss of privacy, increase in level of distrust, data used against them (e.g. for insurance purposes), Furthermore, with the malicious hacker or entities not getting the consent from the users to take and sell their data, the autonomy of the users is not respected. 3.4

Global Bioethics and Drug Development

The aftermath from a situation of malicious brain-hacking, neural data hostage, or neural data sharing does not just impact an individual, but it impacts society altogether. However, of the three, We will focus on neural data sharing, continuing the discussion from the point where what can occur after an individual, corporation, or organization has obtained neural data as this is one of the biggest concerns expressed within literature due to the impact that neural data sharing has on society at large. What a person or entity will do with the neural data is difficult to determine. However, it is easier for us to get a sense on what some individuals and companies will do with the data compared to others. With the advances in artificial intelligence and machine learning, when combining learning algorithms with brain data obtained through BCI or other neurotechnologies, they can lead to fruitful results for things such as drug development. However, using BCI for drug development is not a simple task. Contemporary drug discovery strategies rely on the identification of molecular targets associated with a specific pathway and subsequently probe the role of a given gene product in disease progression [6]. Subsequent to the identification of a relevant bioactive compound, further compound screening can be achieved through phenotypic

86

S. Q. Hossain and S. I. Ahmed

screens. With phenotypic screens, they are capable of capturing complex cellular level behaviors that are physiologically relevant and central to a pathology without relying on the identification of a specific drug target or a corresponding hypothesis concerning its pathological role [6]. For that reason, phenotypic screens are gradually becoming more utilized in drug discovery as using a phenotypic approach can generate novel treatments for diseases and disorders which have either complex or unknown mechanisms [6]. When using a phenotypic approach onto a nervous system, it provides a way to find compounds and genes active in a nervous system without assumptions about relevant molecular targets, which can possibly lead to new therapeutic targets, new disease biology, and new compounds or genes [14]. The challenge is that phenotypic screens require a quantifiable phenotypic alteration, such as neuronal survival or changes in fluorescently tagged protein expression in order to generate a usable output [14]. Conveniently, BCIs can provide brain data containing neuronal survival [2]. With BCIs being able to extract quantifiable phenotypic alteration and classification algorithms being able to understand BCI’s messages, we can conclude that pharmaceutical companies that purchased neural data will most likely use it with a phenotypic approach to discover, develop and patent new generic drugs. The problems that follow this are societal and global health issues. In the past, pharmaceutical companies have consistently argued that high prices and multilateral patent protection afforded by Trade-Related Aspects of Intellectual Property Rights are reasonable rewards for highly expensive and often fruitless research and development of pharmaceutical products, which are ultimately of benefit to society [9]. Furthermore, with pharmaceutical research and development costs being substantial and given the low success rates, pharmaceutical companies argued that profits from the few successful products are essential [9]. Consequently, the problem with pharmaceutical companies researching and developing drugs using neural data and phenotypic approach is that it will likely be very expensive as not only will the companies have to pay a price for conducting and developing the drug, but they also have to pay a price for neural data based on the price set by the third party. For them to generate the data can be even more costly for them as on top of cost associated with conducting data collection, hardware costs from neurotechnologies and costs associated with data preprocessing and augmentation will need to be included in the total cost. Furthermore, with patented drugs preventing others from making or using the generic drug without obtaining a license from the patent holder, most pharmaceutical companies refuse to grant licenses so they can benefit from having a limited monopoly on the drug [33]. As a result, not only will they likely enforce their patents on those generic drugs, but there is a chance that they will also charge a hefty amount for those interested in purchasing the license for it for therapeutic purposes. In turn, this will lead to affordability, accessibility and health issues experienced by the portion of the population unable to afford the drugs. On a global level, with the pharmaceutical companies enforcing their patents, this will prevent developing countries from manufacturing the generic versions

Ethical Analysis

87

of those drugs, creating affordability, accessibility and health issues experienced by those countries [8]. While it was easy to attempt to determine how pharmaceutical companies will neural data, in the case of the military, for example, is not as obvious. The US Department of Defense, for instance, is currently exploring the use of neurotechnologies and neural data [26]. It is said that in pursuing neurotechnology, they are using it with the goal of restoring function following trauma from war and to develop programs involving improving human training and performance [26]. From a first glance, it seems that they are doing it to help better those within the military recover or become a better version of themselves. However, from a closer look, this can be seen as the military trying to get an advantage over other countries [26]. To make neurotechnologies’ use more extreme, yet more effective for the military and even politicians and corporations such as those that sell firearms, exploration of neurotechnologies such as brain implants by the military can be used in a way that create a potential for terrorism [32], leading to beneficence issues. This is an issue as this indirectly promotes warfare between countries, which the consequences from any sort of warfare which neurotechnology contributed to includes an increase in casualties, environmental damage, loss of resources and infrastructure, psychological and neurological disorder cases, loss of talent, and more. These casualties and other losses are a major injustice to the innocent civilians caught in the middle of the warfare [31].

4 4.1

Augmentation of Physicians and Surgeons Neuroengineering for Augmentations

Before moving onto how neurotechnology can be applied to medical practice, it is important to have a foundational understanding of how the neurotechnologies that were mentioned earlier accomplish their respective augmentations. How neurostimulation techniques accomplishes enhancing perception, learning and memory, attention, and decision-making within a person is by using invasive or non-invasive electrical stimulation systems on the person [17], to then apply electricity to affect the person’s central nervous system [37]. Similarly, how neuromodulation accomplishes enhancing memory, learning, attention and complex problem solving within a person is by sending electrical or pharmaceutical agents to a target area of the person’s brain to alter nerve activities [24]. While there are several types of tactile sensors, the tactile sensor that would be best for achieving augmented touch and augmented action is a tactile sensor using CNT-based nanocomposites because not only are they mechanically flexible, robust and chemically resistant needed to achieve the flexibility of human skin, but they also require lower conductive phase concentration, which leads to better mechanical properties of the composite [12]. The tactile sensors would be placed onto a sensor glove, then worn by the user. The way they work in achieving augmented touch is through optimizing the use volume changes in sensitive materials to detect pressure [12], in a way

88

S. Q. Hossain and S. I. Ahmed

that allows the tactile sensors to be capable of [11]: (1) evaluating mechanical compliance and surface texture properties of objects; (2) sensing an applied force by evaluating the magnitude and direction; (3) spatially discriminating the point of application of a force; and (4) Emulating the dynamic behavior of human mechanoreceptors while tracking tactile stimuli overtime. Furthermore, the design criteria for the tactile sensors must consider spatial resolution and range of applied force in order to properly achieve augmented touch [12]. Accordingly, neuroprosthetics accomplishes augmented gesture and augmented touch by utilizing tactile sensors with the BCI to receive relevant sensory information concurrently with letting the user directly control the output behavior of the prosthesis (e.g. artificial arm movement); in doing so, this recreates the controlfeedback loop [1]. The signals produced by the loop provide output for controlled muscle contractions, input and feedback from sensory organs (e.g. position), creating a bidirectional pathway through which we explore and manipulate our environment [1], enabling for augmented gesture and augmented touch to take place. Figure 1 illustrates the process of augmented gesture. The approach used in the figure incorporates the neurofeedback approach – an approach that is promising for enhancing the performance of the brain [3].

Fig. 1. Diagram displaying the steps taken to obtain augmented gesture. First, the tactile sensors that are places on the hands or fingers of the surgeon. Once measurements are obtained, then they are sent to a controller (e.g. BCI) that evaluates, refines and translates the measurements into stimulations that neurons can react to. Upon receiving the stimulations, the neural circuits within the neurons reorganize themselves in a way that allows the surgeon’s brain to update itself on how to better perform the gesture. The gesture is then realized through neuroprosthetics combined with either a prosthetic or the surgeon’s hand.

4.2

Augmentations in Surgery

Knowing how neurotechnology can realize augmentations within humans enables us to address how such technology and augmentations will impact medical practice. For this section, we will specifically focus on the practice of surgery. In this case, the users of the neurotechnology are surgeons. Within surgical settings, augmented vision, augmented touch and augmented gesture will enable surgeons to perform surgeries with a higher level of precision and accuracy by allowing them to see details more clearly, utilize surgical tools and methods better, and perform surgical procedures and operations better. As a result, the success rate

Ethical Analysis

89

of surgeries performed by surgeons utilizing neurotechnology will increase, which is good for multiple reasons; the main reason being that successful surgery will enable the patient to live longer and healthier life, which is helpful for the individual, their family and society. Augmented cognition for surgeons enables surgeons to be able to learn new surgical procedures that are less invasive, shorten operative times, lessens costs and reduce the likelihood of complications. Furthermore, it will allow surgeons to be more knowledgeable in surgical techniques and competent in the ability to recognize the limits of their professional competence [36]. The benefits of these can best be understood through a bioethics perspective, with specifically using the principle of beneficence, non-maleficence and justice, In terms of beneficence, in neurotechnology and the augmentations empowering surgeons, they enable surgeons to take actions, make suitable surgical judgments to assess the risks, burdens and benefits, and approach surgeries in a way that respects and is in the best the interest of their patients. In addition, their improved cognitive abilities will allow them to stay up to date with activities and update their knowledge base as required. With having surgeons keeping themselves professionally and educationally advanced, this will help ensure that they will provide the highest standards of patient care and the lowest rates of complications [36]. This leads to nonmaleficence as higher standards of patient care is accompanied with better communication and improvement in the type of care being offered (e.g. palliative care), and lower rates of complications leads to a decrease in the likelihood of a medical error occurring and minimization of harm caused to the patient. Lastly, with the surgeons being able to perform tasks better, this will decrease the time needed for surgeries, and can decrease the time of the patient’s stay in the hospital. From a justice standpoint, this is crucial as just as much as health burden is an issue for patients, so is economic burden. With surgeons successfully performing surgeries on their patients, patients will not need to stay in the hospital for long, enabling them to be able to live back their normal life and do their regular activities, such as working and spending time with their family. 4.3

Augmentations in Medical Practice

For this section, we will focus on medical practice that do not involve surgery. In this case, the users of the neurotechnology would be physicians. Within nonsurgical settings, augmented vision and augmented cognition will allow physicians to better perform diagnoses. For instance, given that the physician will have enhanced vision and cognitive abilities, analysis of scans from diagnostic radiology exams such as computed tomography (CT) and fluoroscopy would be better performed as physician will be able to see details more clearly and better spot indicators on the scans. As such, this will allow physicians to make more accurate diagnoses, which can result in better understanding assessment, planning and execution of treatment, communication between the physician and the patient, maintain of the physicians’ fidelity and responsibilities, and care for the patient. These help with achieving non-maleficence as effective diagnosis and communication significantly reduces the amount of harm within patients on both

90

S. Q. Hossain and S. I. Ahmed

a physical and mental level. Good diagnosis helps with increasing the chances that a correct treatment will be used, and prevents moral distress in patients and their family. Additionally, and most importantly, with neurotechnology, augmented vision and augmented cognition enhancing physicians’ diagnoses performance, this leads to a major step towards justice. Specifically, improvement in diagnosis will lead to reduction in discrimination, misclassification and prejudice experienced by patients, and better provide freedom from misdiagnosis to patients. Augmented cognition, augmented gesture and augmented touch from neurotechnology will allow physicians to perform treatments and therapy in a more efficient and effective manner. The benefits of performing successful treatments and therapy are similar to those of performing a successful surgery or diagnosis. Just as in performing a successful surgery provides justice to the patient by allowing them to leave the hospital and get back to their routinely activities, performing a successful treatment is no different. However, given the difference in nature of surgeries and treatments in terms of activities and procedures, there are particular benefits that neurotechnology and augmentations provides that applies more for physicians than for surgeons. One of these benefits is how they help physicians with meeting their fiduciary duty to their patients during treatment through enabling physicians to properly implement precision medicine within the treatment itself. What makes precision medicine desirable for treatments is that its approach consists of utilizing the patient’s genes, environment, and lifestyle. This is especially of interest for patients as they would much rather have a treatment that is personalized for them than a one-size-fits-all treatment.

5 5.1

Ethical Analysis Personhood Assessment

In the ethical evaluation of technological interventions, integrity and dignity of a person are the most relevant criteria [29]. Within philosophical literature, a portion of ethical questions raised in regard to neurotechnology are associated with what we call our “self” or “Soul”, which the debate itself usually draws on the concept of personhood as a modern notion that includes core aspects that we typically ascribe to our self or soul; these aspects include planning of the individual future and responsibility [29]. The concept of personhood always has normative implications due to the fact that we describe certain attributes and capabilities of a person, and want to have that person recognized, acknowledged and guaranteed [29]. Just as patients must consciously authorize a neurotechnological intervention before it is conducted, physicians and surgeons must also consciously provide authorization as well. As such, the concept of a person can provide an ethical benchmark applicable to persons such as physicians and surgeons, assuming that we do not want to impair the personal capabilities, such as autonomy and responsibility interventions in the brain [29]. With that being said, neurotechnological interventions are ethically unacceptable if remaining a person is at risk.

Ethical Analysis

91

This statement poses major problems for neurotechnology and human augmentation for physicians and surgeons for two reasons. To best understand those reasons, we will provide two cases. Note that the concept of personal identity refers to the query to which degree and under which circumstances a person remains the same over time, above and beyond physical identity [29]. In investigating and evaluating for the “ameness” for a person, we need to consider both the interaction that the person has with others and the appreciation of moral capabilities, such as the ability to make a promise and keep it [29]. This, especially, is important for the physicians and surgeons as both of them have a fiduciary responsibility to their patients. Consider the simple case where all physicians and surgeons provide authorization for augmentation via neurotechnology and neurotechnology successfully accomplishes its augmentations on them without any interference (i.e. no hackers or third parties involved). An undesired outcome that can be produced from this comes from the augmented cognition. The increase in knowledge does not guaranteed consistency or improvement in morality within a person. As such, there is a chance that the cognitive enhancement that they have undergone changes the way that they appreciate moral capabilities. A physician with increase in knowledge on the importance of using a specific biomedical approach to treating patients with a specific condition can reinforce their current belief that biomedical approaches are better than holistic approaches. In turn, this can lead to a lack of openness for alternative approaches such as a holistic approach, and empathy needed to understand why approaches like the holistic approach is needed for the treatment of those kinds of patients. This will lead to disputes and disagreements between the physician or surgeon and other physicians and surgeons, and with medical practitioners such as nurses and clinical social workers. Overtime, this will lead to the interaction that they have with each other as well as with those medical professionals to change. By the end of it, at the cost of wanting to improve their cognitive abilities for the sake of their patients, the physician or surgeon will have lost their personal identity. This possible loss of personal identity makes it problematic for neurotechnological interventions as the physician or surgeon is at risk of not remaining the person that he or she was prior to the augmentation. For the second case, consider the instance where simple case where all physicians and surgeons provide authorization for augmentation via neurotechnology and neurotechnology successfully accomplishes its augmentations on them, but an interference by a third party, by say a malicious brain-hacker, occurs later. After such an instance occurs, assuming that the negotiation with the malicious brain-hacker was successful resulting in no changes made to the BCI and brain of the physician or surgeon, it is pretty much guaranteed that the physician or surgeon will not remain after the incident. The physician or surgeon will likely suffer from sort of paranoia, trust issues, and/or psychological trauma. This is not surprising as often when an individual’s mental integrity and dignity has been compromised, they end up suffering mentally from it to some degree. Furthermore, with the advancement of technologies and areas within computer science, specifically artificial intelligence, machine learning and quantum computing, it

92

S. Q. Hossain and S. I. Ahmed

is justified for them to have fears and worries as there is no guarantee that the same issue will not be experienced again. What that degree is heavily depends on the person’s mental strength, willpower and self-control, the amount of support provided to them after the incident has occurred, and the environment where the person is situated after the occurrence of the incident. Regardless, their interaction with physicians, surgeons and others will likely change and their appreciation of moral capabilities will be altered as their view on them will be called into question. In worst case scenario, they become diagnosed with a psychological or neurological disorder as an aftermath or have their brain altered by the malicious brain-hacker, guaranteeing that their interaction with others and their appreciation of moral capabilities will change. This unintentional or intentional loss of personal identity makes it problematic for neurotechnological interventions as the physician or surgeon is at risk of becoming a new person that he or she was not prior to the augmentation. 5.2

Social Issues

What can be noticed in the previous section is a third reason, or case as we can call it, which is the case where some physicians and surgeons decides to either not provide authorization to the neurotechnological intervention altogether or to provide authorization to the neurotechnological intervention but not to undergo at least one type of augmentation. The types of groups and a possible distribution of these groups is shown in Fig. 2. This case leads to one of the previously raised concerns stemming from the consequence of human augmentation. As discussed earlier, there is a chance that augmentations can lead to a divide between humans, where the population of humans with augmentations will feel that they are more superior to those without augmentation, leading to different types of social issues. In our case with the physician or surgeon not providing authorization to neurotechnological intervention, this divide can occur in three different divides: 1. 2. 3. 4.

Division Division Division Division

among physicians and surgeons domestically. between physicians and surgeons, and healthcare professionals. between physicians, surgeons and healthcare professionals. among physicians and surgeons internationally.

For our case, we will only mainly focus on (1) and (4) as (2) and (3) are more distant to the scope of the paper. However, those are areas that can and should be further investigated. With there being four different groups, there is the change that those that physicians and surgeons that are fully augmented (i.e. have successfully undergone all augmentation via neurotechnology) mistreat those that do not have the augmentations that they have. This possibility is not farfetched and is very likely to happen. In every industry, including healthcare and medicine, we see prejudice and workers being mistreated due to their title, lack of experience or the lack of knowledge that (at least according to the administration or management within their organization). With those physicians and

Ethical Analysis

93

Fig. 2. Diagram displaying the types of groups that would exist in the case where undergo types of augmentations for physicians and surgeons are optional, and a possible distribution of each group. To avoid providing false information, population percentages were omitted.

surgeons that do not have a particular augmentation, such as and especially augmented vision, cognition and touch, fully and semi augmented physicians and surgeons can take advantage of that to enforce their agendas and ideologies and bypassing arguments and statements made by them on the grounds that given their inferiors cognitive abilities or senses, what they say and their opinions should not be weighted equally to those that are fully augmented, or things along those lines. As such, augmentations increase the chance of prejudice and mistreatment occurring among physicians and surgeons, reinforcing existing beneficence, non-maleficence and justice issues within healthcare and medicine. Overtime, this can lead to alteration of personality among physicians and surgeons. Furthermore, this can lead to moral distress, trust issues, and other psychological issues experienced by those doctors that are mistreated, resulting in a loss of personal identity. This is dangerous as this shift in personality and personal identity will not only impact the performance of those physicians and surgeons and the lives of patients, but can create or further contribute to suicidal thoughts within them. Depending on the profile, involvements, commitments and types of patients that the doctor was treating, the impact that their suicide can have can be one that is negatively impacts individuals on a one-to-one, domestic, and/or international level. The other way in which physicians or surgeons can be discriminated is in an instance where either malicious brain-hacking or a data hostage leading, with either of them leading to data sharing occurs. What makes doctors neural data valuable is that the doctors are known for being very well off financially. With proper preprocessing of the neural data and correct use of data mining and

94

S. Q. Hossain and S. I. Ahmed

machine learning algorithms, health status, preferences and behavior of physicians and surgeons, physicians and surgeons can be exploited financially by companies such as insurance companies. Note that data records are stored electronically, which the highly personalized nature of neural data – much like genomic data – may increase the identifiability of individuals [25]. With that being said, the neural data obtained from malicious brain-hacking or data sharing could be used for discrimination purposes against a particular physician or surgeon, or to blackmail them into doing something. The severity of the consequences from the blackmail will depend on what is being asked and whether the doctor carries out the act. Regardless of what is produced from this scenario, as the risk of psychological harms depends directly on the presence of other risks [16], it is guaranteed that the doctor will experience distress, anxiety and other psychological issues, which can lead decreased performance in completing medical and non-medical tasks. This decrease in performance increases the chances that they harm the patient. 5.3

Global Health Inequality and Health Migration

Given that there are major economical, technological, and infrastructural disparities among countries, it is inevitable that some countries will not be able to provide their physicians and surgeons with the neurotechnological interventions needed to obtain augmentations. As computing technologies can greatly contribute to the growth and development of countries [4], countries such as the United States, Germany and China that have the manpower, infrastructure and resources to realize computing technologies for economic, social and medical development will be able to provide their doctors with the augmentations quicker and with guarantees compared to countries that are struggling due to poverty, warfare and humanitarian crises such as Libya, Syria and Myanmar. Consequently, if only doctors in developed countries are augmented, then this will create a global health inequality. Patients in countries with augmented physicians and surgeons will receive better treatment than those with regular physicians and surgeons. Furthermore, in times where doctors from different countries have to collaborate together on medical and healthcare issues, conflicts will likely arise between augmented doctors from one country (i.e. developed country) versus regular doctors from another (i.e. underdeveloped country). With augmented doctors possibly having a lack of openness for alternative approaches and empathy towards patients from different countries, this will lead to disputes and disagreements between the doctors from the different countries. The disputes and disagreements can lead to physicians and surgeons experiencing distress, anxiety and other psychological issues, resulting in decreased performance in completing medical and non-medical tasks. This decrease in performance increases the chances that they harm their patients and patients internationally depending on the case. With developed and high-income countries being able to provide not just augmentations to physicians and surgeons but also better employment opportunities, education, safety and security for them and their families, these will push

Ethical Analysis

95

physicians and surgeons to migrate to those countries [34]. Their migration will further contribute to the current disparity in the health workforce between highand low/middle-income countries [35]. In Serour’s work, which was done back in 2009, she stated that Africa needs 1 million health workers, which includes physicians and surgeons, for Sub-Saharan Africa alone. This number has likely increased due to the SARS-CoV-2 outbreak that has and continues to take place in 2020. With countries like the United States, United Kingdom, Canada and Australia already benefitting considerably from the migration of nurses and doctors [30], the implementation of neurotechnology for human augmentation will further benefit those countries as well as other developed countries, leaving the underdeveloped countries to suffer more. What results from this is an increase in harm experienced by the patients and an increase in mortalities in the underdeveloped countries’ population. 5.4

Patient Harm

Whether it is data hostage situation of the data from a TMS or malicious brainhacking of a neuroprosthetic, given the risks associated with physicians and surgeons using neurotechnology for augmentations along with the impact that augmentation has on people in general, there is no doubt that there is a chance that the change in self-concept, personal identity or the discrimination, psychological and other issues experienced by physicians and surgeons will influence doctor-patient relationships. One of the reasons for why the relationship will likely become worse is because the voices of the patients will either be ignored or be of less value to their physicians and surgeons. For fully or semi augmented physicians and surgeons, especially those with augmented cognition, there is a higher chance that they will think that whatever decision they make will be the right decision for the patient. Consequently, when patients of theirs voice their concern to them, they will either ignore or listen only selected parts of the concern. In turn, this will make the patient feel as if their doctor does not care for them, leading to feelings of sadness, distress, and pain. This is problematic because even in the case where the patient is successfully treated, they would leave the hospital or clinic feeling mentally hurt. While the degree in which the mental pain from the experienced differs from one patient to the other, it is still a problem because the physician or surgeon did not accomplish non-maleficence. During times when they could have prevented the patient from experiencing mental harm, they did not prevent it. As a result, the physician or surgeon did not ensure to inflict the least amount of harm possible to reach a beneficial outcome. Rather, they inflicted an amount of harm onto the patient that either they were not aware of or did not care about enough to reach a beneficial outcome. The other reason for why doctor-patient relationships will likely worsen is because of the side effects experienced by physicians and surgeons from either undergoing all augmentations or selected few augmentations, and/or from being mistreated and discriminated by their colleagues [15]. For fully or semi augmented physicians and surgeons, in the case where they lose their personal identity, this can make it difficult for patients to communicate their concerns as

96

S. Q. Hossain and S. I. Ahmed

the new personality developed by their physician or surgeon can be one that makes it difficult to communicate with them, and vice-versa. Consequently, in the physician or surgeon having difficulty communicating with their patients, this increases the chances of and eventually lead them harm their patients either by accident or on purpose. In the case of harm occurring due to an accident, this accident could be due to the physician or surgeon making a wrong assumption based on the limited information verbally provided by their patients. Alternatively, this can also be due to after using tES to increase the physician or surgeon memory, rather than the memory being used for remembering details about the patients, the memory retrieval within the physician or surgeon’s brain retrieves unrelated information or even problematic information such as details of a traumatic event; in retrieving the wrong information, the physician or surgeon’s attention is not adequately concentrated on the patient, making it difficult for them to communicate with the patient, and eventually leading to them accidentally harming the patient. In the case of harm occurring on purpose, this could be due to first the personal identity change resulting in the physician or surgeon not being able to properly control their anger, then the patient criticizing the physician or surgeon, infuriating the doctor in a way that made them feel that the patient is attacking their expertise, experiences and augmentations. The problem of the voices of patients being ignored extends to issues of justice. What results from the patient’s voice being partially or fully ignored is unequal and unfair treatment by the fully or semi augmented physicians and surgeons to their patients. The way that they approached the medical intervention was not the same across all of their patients. Some received more care from the physicians or surgeons while others received less. Consequently, this unequal level of care is unfair to those that do not receive the same amount of care from the physicians or surgeons. This unequal treatment to patients can not only be experienced while undergoing the medical intervention, but it can be experienced prior to it as well. Especially with augmented cognition, augmented vision and BCIs, those make it possible for physicians and surgeons to easily take advantage of their patients during the diagnosis and consent phase prior to the treatment. With tDCS providing physicians and surgeons the ability to better solve complex problems, they can find ways to trick patients into providing their “informed” consent to treatments that are either costly, ineffective, harmful or all three combined. Consequently, this will lead to the patient further suffering from the disease and any complications that arise during the medical intervention, financial loss, and increased levels of distress from spending a larger than expected time in the hospital and from the worry of either dying or possibly not being able to live a regular life again or anytime soon.

6

Conclusion

A limitation with our work is that there were no medical professionals that directly contributed to the paper. As such, there are some notable ethical issues that were not included in this paper. Additionally, as our work did not include

Ethical Analysis

97

any discussions on health policy and cyber security, we were not able to discuss solutions that could be used to address the ethical and health issues. Future work can ethically analyze the impact of neurotechnology for human augmentation on physicians and surgeons with respect to the division between physicians, surgeons and healthcare professionals, and should focus on health policy and cyber security needed to address those ethical and health issues. Even with the best-case scenario where neurotechnology always succeeds in providing people with augmentations, in applying it on physicians and surgeons to augment them, there are high risk associated with it. This makes it challenging when it comes to doing a cost benefit analysis of it, especially because when doing such an analysis, we cannot make the big assumption that we made in this paper. A way to reduce the risks associated with the neuroengineering component of augmenting physicians and surgeons is to design and program the tactile sensors and neuroprosthetics so that in a worse case where a malicious brain-hacker takes over a surgeon’s BCI and tries to make the surgeon harm their patient, the neuroprosthetic detects the abnormality then communicates with the tactile sensors, activating the emergency magnet mechanism in the glove, forcing the gloves to tighten together like handcuffs. This will prevent the surgeon from harming the patient and medical practitioners in the room, and give the surgical team time to figure out what to do next. It is important that such safety mechanisms like these and others are rigorously tested prior to using them in clinical settings, and have them ready for use once they are ready to be used. Furthermore, after augmentations have been embedded within physicians and surgeons, have a designated medical and bioethics combined team dedicated to supporting them mentally to ensure that the transition is smooth and the physicians and surgeons’ personal identity and other crucial aspects of them are preserved. In doing so, this can help reduce and prevent the occurences of harm and ethical, medical, and personhood issues experienced by physicians, surgeons and patients.

References 1. Adewole, D.O., Serruya, M.D., Harris, J.P., Burrell, J.C., Petrov, D., Chen, H.I., Wolf, J.A., Cullen, D.K.: The evolution of neuroprosthetic interfaces. Crit. Rev. Biomed. Eng. 44(1–2), 123–152 (2016). https://doi.org/10.1615/ CritRevBiomedEng.2016017198 2. Adewole, D.O., Serruya, M.D., Wolf, J.A., Cullen, D.K.: Bioactive neuroelectronic interfaces. Front. Neurosci. 13(269), 1–4 (2019). https://doi.org/10.3389/fnins. 2019.00269 3. Abdulkader, S.N., Atia, A., Mostafa, M.M.: Brain computer interfacing: applications and challenges. Egypt. Inform. J. 16, 213–230 (2015). https://doi.org/10. 1016/j.eij.2015.06.002 4. Ahmed, S.I.: Designing for Voice: The Challenges of Access, Autonomy, and Accountability. Cornell Theses and Dissertations, pp. 4–9 (2018). https://doi.org/ 10.7298/X4P55KQJ 5. Bavishi, S., Rosenthal, J., Bockerbrader, M.: Neuroprosthetics. In: Rehabilitation After Traumatic Brain Injury, pp. 241–242 (2018)

98

S. Q. Hossain and S. I. Ahmed

6. Black, J.B., Atmaramani, R., Plagens, S., Campbell, Z.T., Dussor, G., Price, T.J., Pancrazio, J.J.: Emerging neurotechnology for antinoceptive mechanisms and therapeutics discovery. Biosens. Bioelectron. 126, 679–685 (2019). https://doi.org/10. 1016/j.bios.2018.11.015 7. Brey, P.: Human enhancement and personal identity. In: New Waves in Philosophy of Technology, pp. 169–185 (2009). https://doi.org/10.1057/9780230227279 9 8. Brennan, R., Baines, P.: Is there a morally right price for anti-retroviral drugs in the developing world? Bus. Ethics Eur. Rev. 15(1), 29–41 (2006) 9. Buckley, J., Tuama, S.: International pricing and distribution of therapeutic pharmaceuticals: an ethical minefield. Bus. Ethics Eur. Rev. 14(2), 127–139 (2005) 10. Caras, A., DeJesus, J.: Ethical analysis of brain augmentation through nanotechnology. Sound Decis. Undergrad. Bioeth. J. 4(1), 7–11 (2018). https://soundideas. pugetsound.edu/sounddecisions/vol4/iss1/4 11. Cutrone, A., Micera, S.: Implantable neural interfaces and wearable tactile systems for bidirectional neuroprosthetics systems. Adv. Healthc. Mater. 8, 1–11 (2019). https://doi.org/10.1002/adhm.201801345 12. Chi, C., Sun, X., Xue, N., Li, T., Liu, C.: Recent progress in technologies for tactile sensors. Sensors 18(9489), 7–12 (2018). https://doi.org/10.3390/s18040948 13. Cinel, C., Valeriani, D., Poli, R.: Neurotechnologies for human cognitive augmentation: current state of the art and future prospects. Front. Hum. Neurosci. 13(13), 1–16 (2019). https://doi.org/10.3389/fnhum.2019.00013 14. Cooper, D.J., Zunino, G., Bixby, J.L., Lemmon, V.P.: Phenotypic screening with primary neurons to identify drug targets for regeneration and degeneration. Mol. Cell. Neurosci. 80, 161–166 (2017). https://doi.org/10.1016/j.mcn.2016.07.001 15. Douglas, T.: The Harms of enhancing and the conclusive reasons view. Camb. Q. Healthc. Ethics 24, 23–27 (2015). https://doi.org/10.1017/S0963180114000218 16. Dufresne-Camaro C., Chevalier F., Ahmed S.I.: Computer vision applications and their ethical risks in the global south. In: Graphics Interface 2020 Conference, pp. 5–8 (2020) 17. Edwards, C.A., Kouzani, A., Lee, K.H., Ross, E.K.: Neurostimulation devices for the treatment of neurologic disorders. Mayo Clin. Proc. 92(9), 1–5 (2017). https:// doi.org/10.1016/j.mayocp.2017.05.005 18. Farah, M.J.: An ethics toolbox for neurotechnology. Neuron 86(1), 34–37 (2015). https://doi.org/10.1016/j.neuron.2015.03.038 19. Giordano, J., Kulkarni, A., Farwell, J.: Deliver us from evil? The temptation, realities, and neuroethico-legal issues of employing assessment neurotechnologies in public safety initiatives. Theor. Med. Bioeth. 35, 73–89 (2014). https://doi.org/ 10.1007/s11017-014-9278-4 20. Goering, S., Yuste, R.: On the necessity of ethical guidelines for novel neurotechnologies. Cell 167, 882–885 (2016). https://doi.org/10.1016/j.cell.2016.10.029 21. Harper, G.W., Schneider, M.: Oppression and discrimination among lesbian, gay, bisexual, and transgendered people and communities: a challenge for community psychology. Am. J. Community Psychol. 31(34), 243–248 (2003). https://doi.org/0091-0562/03/0600-0243/0 22. Ienca, M., Andorno, R.: Towards a new human rights in the age of neuroscience and neurotechnology. Life Sci. Soc. Policy 13(5), 7–23 (2017). https://doi.org/10. 1186/s40504-017-0050-1 23. Ienca, M., Haselager, P., Emanuel, E.: Brain leaks and consumer neurotechnology. Nat. Biotechnol. 36(9), 805–809 (2018). https://doi.org/10.1038/nbt.4240 24. International Neuromodulation Society (INS) (2019). https://www. neuromodulation.com/about-neuromodulation. Accessed 15 Mar 2020

Ethical Analysis

99

25. Kellmeyer, P.: Big brain data: on the responsible use of brain data from clinical and consumer directed neurotechnological devices. Neuroethics, 1–15 (2018). https:// doi.org/10.1007/s12152-018-937-x 26. Khan, S., Aziz, T.: Transcending the Brain: is there a cost to hacking the nervous system? Brain Commun. 1(1), 5–9 (2019). https://doi.org/10.1093/braincomms/ fcz015 27. Kuruvilla, A., Jacob, K.S.: Poverty, social stress & mental health. Indian J. Med. Res. 126, 273–278 (2007) 28. Kuss, D.J., Janjo, E., Crook-Rumsey, M., Kibowski, F., Wang, G.Y., Sumich, A.: Problematic mobile phone use and addiction across generations: the role of psychopathological symptoms and smartphone use. J. Technol. Behav. Sci. 3, 145–147 (2018). https://doi.org/10.1007/s41347-017-0041-3 29. Muller, O., Rotter, S.: Neurotechnology: current developments and ethical issues. Front. Syst. Neurosci. 11(93), 1–5 (2017). https://doi.org/10.3389/fnsys.2017. 00093 30. Naiker, S., Eastwood, J.B., Plange-Rhule, J., Tutt, R.C.: Shortage of healthcare workers in sub-Saharan Africa: a nephrological perspective. Clin. Nephrol. 74, S130–S133 (2010). https://doi.org/10.5414/cnp74s129 31. Noll, G.: Weaponising neurotechnology: international humanitarian law and the loss of language. Lond. Rev. Int. Law 2(2), 201–210 (2014). https://doi.org/10. 1093/lril/lru009 32. Raisamo, R., We, R., Majaranta, P., Salminen, K., Rantala, J., Farooq, A.: Human augmentation: past, presentation and future. Int. J. Hum. Comput. Stud., 131–143 (2019). https://doi.org/10.1016/j.ijhcs.2019.05.008 33. Resnik, D.B.: Fair drug prices and the patent system. Health Care Anal. 12(2), 91–97 (2004). https://doi.org/1065-3058/04/0600-0091/1 34. Roberts, J.H.: The impact of global inequalities on health professional migration. Policy Polit. Nurs. Pract. 9(4), 323–327 (2008). https://doi.org/10.1177/ 1527154408327289 35. Serour, G.I.: Healthcare workers and the brain drain. Int. J. Gynecol. Obstet. 106, 175–178 (2009). https://doi.org/10.1016/j.ijgo.2009.03.035 36. Shaha, K.K., Patra, A.P., Chaudhari, V.A., Das, S., Das, S.K.: A review on applications of principles of medical ethics in the practice of surgery. South-East Asian J. Med. Educ. 12(1), 13–17 (2018) 37. Starnes, K., Miller, K., Wong-Kisiel, L., Lundstrom, B.N.: A review of neurostimulation for epilepsy in pediatrics. Brain Sci. 9(283), 1–3 (2019). https://doi.org/ 10.3390/brainsci9100283 38. Schwartz, A.B.: Movement: how the brain communicates with the world. Cell 164, 1122–1131 (2016). https://doi.org/10.1016/j.cell.2016.02.038 39. Su, D., Irwin, J.A., Fisher, C., Ramos, A., Kelly, M., Mendoza, D.A.R., Coleman, J.D.: Mental health disparities within the LGBT population: a comparison between transgender and nontransgender individuals. Transgender Health 1(1), 12–19 (2016). https://doi.org/10.1089/trgh.2015.0001 40. Yuste, R., Goering, S., Arcas, B.A., Bi, G., Carmena, J.M., Carter, A., Fins, J.J., Friesen, P., Gallant, J., Huggins, J.E., Illes, J., Kellmeyer, P., Klein, E., Marblestone, A., Mitchell, C., Parens, E., Pham, M., Rubel, A., Sadato, N., Sullivan, L.S., Teicher, M., Wasserman, D., Wexler, A., Whittaker, M., Wolpaw, J.: Four ethical priorities for neurotechnologies and AI. Nature 551(7679), 159–163 (2017). https://doi.org/10.1038/551159a

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation Edgard Vargas-Solís(B) , Daniel Cárdenas-Salas, Juan Gutierrez-Cardenas, and Vilma S. Romero-Romero Universidad de Lima, Lima, Peru [email protected], {decarden,jmgutier, vromero}@ulima.edu.pe

Abstract. The high levels of ultraviolet (UV) radiation in Peru constitute a risk for the population, that does not give it the importance that it should and does not take adequate measures to protect against it and to prevent skin injuries. This research aims to educate the general population about the high radiation levels registered in our country. To accomplish this objective, a virtual reality application was developed to visualize real time UV index, the maximum exposition time before getting a sunburn according to the user’s skin type, the potential skin damage, and, lastly, it provides a Solar Protection Factor (SPF) recommendation. To validate the research, a survey was applied to 63 participants, who were mostly between 18 and 24 years old, in two parts: the first part (knowledge segment) was applied before the simulation took place in order to analyze the user’s knowledge level about the subject; and the second part (application segment) measured how valuable the application was in terms of education, usability and appeal. The survey results (p < 0.001) indicate that most of the participants do not know or are indifferent to high UV radiation (knowledge segment), and that the virtual reality application educated the participants about the UV radiation problem (application segment, education component). There is evidence that virtual reality can be an effective method to teach people about a problem, being part of it, and observe the consequences. Keywords: Minimal erythema doses · UV index · Virtual reality · Fitzpatrick

1 Introduction The high levels of solar ultraviolet (UV) radiation in Peru is a phenomenon that directly affects the population’s health, causing severe skin diseases [28]. Furthermore, according to Lozano and Odón [30], radiation levels in Lima, Peru, are high in almost all seasons, reaching during summer values higher than 15 (“extreme” risk level), which is particularly harmful to the general population (see Table 1). Yaipen et al. [19] showed that the general population does not give the proper attention to UV index levels and does not realize UV radiation levels they are exposed to daily. This indifferent behavior has partly caused a rise in the number of cases of skin disease © Springer Nature Switzerland AG 2021 K. Arai et al. (Eds.): FTC 2020, AISC 1290, pp. 100–112, 2021. https://doi.org/10.1007/978-3-030-63092-8_7

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation

101

Table 1. UV index scale used by the National Meteorology and Hydrology Service of Peru (SENAMHI) [30]. Index value Risk level 1–2

Minimal

3–5

Low

6–8

Moderate

9–11

High

12–14

Very High

14+

Extreme

in Peru; according to the National Institute of Neoplastic Diseases [28], between 2004 and 2005 there were 2333 new skin cancer cases detected, a number that significantly grew to 5975 between 2006 and 2010, reaching the fourth place among all cancer types nationwide. According to that report, just in the year 2011 there was a total of 1208 deaths caused by malignant skin neoplasms, which represented an adjusted mortality rate of 1.6 for every hundred thousand people. The main motivation for this research is to allow people to appreciate the damage that can be caused by UV radiation and, by doing so, to learn about the risk involved with overexposure to UV radiation without taking adequate protection measures. In order to achieve our research objectives, we used virtual reality, which is a technology that could be employed for education and for enhancing the user experience in contraposition with conventional methodologies. This advantage is mentioned in the research work of Limnio et al. [10], who employed virtual reality to teach chemistry to a group of students, achieving an increase in their marks in comparison with a traditional teaching methodology. Also, Schutte and Stilinovi´c [31] managed to increase the empathy of a group of persons towards the refugee camps in Syria by using virtual reality. The following research work comprises the following sections: In Sect. 2, we examine some related work in the topics of solar ultraviolet radiation and the use of virtual reality in a set of educational environments. In Sect. 3, we review the theoretical background concerning the statistical measures that we employed to measure the reliability of our proposed tests. In Sect. 4, we described the methodology followed for the application of VR for simulating the effects of the ultraviolet radiation caused by exposure to the sun rays without protection, along with some experiments in Sect. 5. Some results and discussions are described in Sect. 6, ending our article with conclusions and future work that can be done in this area.

2 Related Work 2.1 Ultraviolet (UV) Index Acquisition The UV index is the preponderant factor to consider when calculating the Minimal Erythema Dose (MED), which is defined as the “the smallest UV dose that produces

102

E. Vargas-Solís et al.

perceptible redness of the skin (erythema) with clearly defined borders at 16 to 24 h after UV exposure” [27]. Therefore, the UV index is a value that must be acquired with high precision levels to avoid erroneous results. One research [15] developed a way to measure UV index with a Genicom GUVA-S10GD sensor, using photodiodes to detect ultraviolet radiation subtypes A and B (abbreviated as UVA and UVB). The result was a very close approximation of more than 90% to UV index measurement using professional measurement devices. In another research [14], the UV index was measured using a ML8511 sensor, in a similar way to the research revised above, and it obtained a very similar precision, although its value could be adversely affected if the sun’s angle with respect to the device is less than 20°. A different way to measure UV radiation involved using CMOS technology found in a mid-range smartphone [13]. The researches obtained a 95% precision when compared to a professional digital UV measurement device. 2.2 Skin Damage Base on the Minimal Erythema Dose (MED) Several factors can be involved to suffer an erythema1 , but some, like the skin type or the exposure duration, also have great relevance to determine the damage that a person will receive from solar overexposure, because those influence the Minimal Erythema Dose (MED). UV radiation has been demonstrated to cause skin damage by Goettsch et al. [3], who looked to visualize the effects of controlled doses of UV radiation using FS40 tubes, a Kromayer handheld lamp, and laboratory mice and rats. Their results, with a Kromayer dose of 250 J/m2 and a FS40 tube dose of 1000 J/m2 , showed skin damage, with small signs of acanthosis, hyperkeratosis, parakeratosis and inflammation. As expected, the authors highlighted the fact that with higher doses those small symptoms could be aggravated. Another way to show the damage caused by UV radiation is to look for its effects at the cellular level, as Leite et al. [4] did. For their study, they used human myeloid leukemia cells HL 60 and a UV transilluminator as a radiation source. The results indicated that cells exposed to radiation in short periods of 3 s, 15 s, 30 s, 1 m and 3 m. showed morphological signs of apoptosis2 , cellular contraction, chromatin condensation and nuclear fragmentation, while longer exposures of 15 m, 25 m y 1 h generated inflammation at the nuclear level, uniformly dense chromatin and plasma membrane rupture, resulting in a lower definition of cell contours or necrosis. A separate research [6] was aimed at determining the skin response of subjects in Taiwan with respect to UV radiation based on their skin phototype. They used 61 volunteers, a MX18 mexameter to detect erythema and melanin in the skin, and four 1 Erythema: Abnormal redness of the skin or mucous membranes due to capillary congestion (as

in inflammation). Merriam-Webster Dictionary. 2 Apoptosis: a genetically directed process of cell self-destruction that is marked by the fragmenta-

tion of nuclear DNA, is activated either by the presence of a stimulus or removal of a suppressing agent or stimulus, is a normal physiological process eliminating DNA-damaged, superfluous, or unwanted cells, and when halted (as by gene mutation) may result in uncontrolled cell growth and tumor formation. Merriam-Webster Dictionary.

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation

103

parallel fluorescent tubes TL 20 W/12 inside an Ultraviolet Therapy System UV-801KL as a light source. After applying 12 gradual doses of 20 a 240 mJ/cm2 to the participants, with an increment of 20 mJ/cm2 per dose, they concluded that the MED value is varies based on the skin phototype. For example, volunteers with skin type II obtained an average MED value of 122.9 ± 29.3 mJ/cm2 , while skin type V volunteers obtained 165.0 ± 52.6 mJ/cm2 . It is usually thought that factors like genre and age are relevant to determine skin damage. However, this believe was denied by Gloor and Scherotzke [5] who looked to identify MED differences among a group of youth and senior people. For this study, they enrolled 20 young adults and 20 senior (age > 70 years old); they used an UVA/TL01 UV tester to determine the MED, a waldmann UV system UV 801 BK as a UV source, a CR 200 chromameter to evaluate skin color and a dopples laser to evaluate the participants’ cutaneous blood flow. The results concluded that MED does not significantly vary according to age, since there were 12 older subjects and 9 young subjects who had an erythemic reaction with 0.99 J/cm2 . They also discovered that genre in both groups did not show any significant statistical difference in skin redness, because the p-value was greater than 0.05 for all tests. 2.3 Effects of Virtual Reality (VR) on Human Beings Virtual Reality (VR) tools have been often used in the psychology field to treat mental disorders. A clear example of this was a study [7] that created a war environment in VR to treat post-traumatic stress disorder in active soldiers and obtained a 22.7% improvement of the symptoms after the treatment. VR can also be used to treat phobias, obtaining similar results to standard treatments. Michaliszyn et al. [8] compared a standard or “live exposure” treatment for arachnophobia (abnormal fear of spiders and arachnids in general) and a treatment based solely in VR. The results for both treatments were very similar, as the Behavioral Avoidance Test (BAT, a measure to determine the success factor of a phobia treatment) value was 3.17 for the standard treatment, and 3.56 for the VR treatment (0.39 difference). Another research in this field [9], using VR environments, demonstrated that there is a 40% probability that individuals could present unfounded paranoia. In terms of learning, VR can be used as a tool to obtain greater motivation. Limniou et al. [10] tried to increase student interest and motivation towards chemistry learning using a Full Immersive Virtual Environment (CAVETM) technology and compared it to a 2D standard presentation, with which they achieved 63% more correct responses than with the traditional method. Another study [11] prepared new nursing students from Midwest University with the sterilization process through a virtual environment on a computer; the students highlighted the secure environment provided by the virtual tool where they could afford to make mistakes without real consequences. Finally, Bisson et al. [12] aimed to validate the effects of a VR training with biofeedback to improve balance and reaction times in older adults. The results, once the training was finished (10 weeks, 2 sessions of 30 min per week), were positive since the participants increased their performance and registered significant changes in their mobility, balance and reflex.

104

E. Vargas-Solís et al.

3 Theoretical Framework 3.1 Cronbach’s Alpha Cronbach’s alpha is an indicator that provides an estimate about a test’s reliability. The result of this indicator should be greater than 0.6 to assure that the data gathered in the survey is reliable [23]. Cronbach’s alpha formula involves the number of items in the survey (K), the sum of the variance of each item ( Vi ), and the total variance (VT ).    Vi K (1) 1− α= K −1 VT 3.2 McDonald’s Omega McDonald’s Omega is another indicator used to evaluate a survey’s reliability. In this case, the value must be greater than 0.65 [24]. It uses the factorized loads λ of the variables or items in the survey to reflect the linear relation between the construct and the items it is composed of [25].  2 λj ω =  2  (2) λj + σε2j 3.3 Fitzpatrick Skin Type Classification According to Fitzpatrick [1] this classification is a standard measurement tool to auto evaluate skin sensitivity during the initial moments of sun exposure. It was based on personal interviews of sun burning and/or suntanning after 45 to 60 min of exposure to midday sun in the early summer in northern latitudes. It has been updated since its creation, and now it lists four skin types as shown in Table 2. Table 2. Fitzpatrick classification of skin type [1]. Type Attribute I

Always burns, never tans

II

Burns easily, tans with difficulty

III

Rarely burns, tans easily

IV

Never burns, always tans

3.4 Minimal Erythema Dose (MED) The Minimal Erythema Dose (MED) is also defined as the minimum dose of UV radiation that a person can tolerate before suffering from a first-degree skin burn [17]. MED varies depending on the type of the skin, as shown in Table 3.

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation

105

Table 3. MED by phototype [17]. Phototype UVB MED(J/m2 ) I

150–300

II

250–400

III

300–500

IV

400–600

V

600–900

VI

900–1500

3.5 Minimum Exposure Time and Solar Protection Factor (SPF) According to [18], UV index is a measure that represents the maximum erythemic irradiation received during the day. The UV index is calculated by multiplying the erythemic irradiation by the standard scalation factor (40 W-1 m2 ). For example, if the erythemic irradiation would be 0.29 W/m2 , the UV index would be 11.6 (see Eq. 3). W ∗ 40W −1 m2 = 11.6 (3) m2 To obtain the effective erythemic irradiation from the UV index registered at any given time, the UV index is divided by the standard factor escalation [26]. For example, using a UV index of 11.6, the erythemic irradiation would be 0.29 (see Eq. 4). VI = 0.29

11.6 = 0.29W /m2 (4) 40Wm2 To obtain the minimum time that a person can be exposed to the sun before getting a skin burn, the person’s MED (Table 3) is divided by the effective erythemic irradiation previously calculated from the UV index, and the result will be the time in seconds that the person can be exposed to a give UV index before getting a sunburn [26]. For example, for a person with a skin type I and a MED of 150 J/m2 , the maximum amount of time that he or she can be exposed to a UV index of 12 would be 516 s or 9 min (see Eq. 5). Eery =

T=

150J = 516 s. 0.29J /S

(5)

It is worth mentioning that the Solar Protection Factor (SFP) of commercial products provide longer exposure times, thus increasing the time one can be exposed to the sun before reaching the MED. The SPF is based on the MED with solar protector (Dp ) and the MED without protection (Du ) [26] (see Eq. 6). SPFi =

Dp Du

(6)

Therefore, if the SPF is 10 and Eq. 6 is used, the person could stand up to 1.43 h (5160 s) under the sun (see Eq. 7). Dp = 10 ∗ 516 s = 5160 s

(7)

106

E. Vargas-Solís et al.

4 Methodology 4.1 General View of the System The system developed as part of this research (see Fig. 1) requires the user to select a skin type and the sun exposure time. The application then request, via an API, the current UV index to a service called OpenUV3 . Once the data is calculated, the user is shown: a) the effect of the sun on his or her simulated arm on the screen, b) the maximum amount of time, according to the skin type previously selected, before the user could show signs of an erythema or skin redness, and c) the minimum SPF recommended to protect the exposure time indicated at the beginning of the simulation.

Fig. 1. Operation of the VR system.

4.2 Participants and Procedure The research was conducted with 63 university students between second and fifth year of their careers, who were previously informed about the objective of the study and signed an agreement form. The sample size was calculated with 90% confidence level and 10% error margin. The participants were initially asked to determine their skin type by completing a survey from the Skin Cancer Foundation [29]. Then, a second survey was distributed to determine their previous knowledge about UV radiation in the country. The participants were then put the VR visors and the simulation began for 5 to 7 min. Once the experience was over, they were given the final part of the second survey to measure the effect generated by the application and its quality.

5 Experimentation 5.1 Devices Used In this section, the design of a UV visualization system using virtual reality will be discussed. The equipment used was a Samsung Galaxy Gear version 2017. 3 Free plan includes 30 requests per day.

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation

107

For the development of the system, virtual reality goggles Gear VR compatible with the Oculus software libraries were used. These devices have a visual field of 101° (with an interpupillary distance of 62 mm and an eyes contour of 10 mm), a gyroscope, a proximity sensor, and a Bluetooth controller to interact with the virtual reality environment. As a complement of the goggles, a high-end smartphone that can reach a resolution of 1440 × 2560, with 4 GB of RAM and a storage capacity of 32 GB was used. It is important to mention that it has total compatibility with Samsung Gear VR when using the Oculus application. 5.2 Study Details After the participants were exposed to the two questionnaires mentioned before (the first one related to determining their type of skin and the second one to gather information about their knowledge of UV radiation levels in Perú), the experiment started with the VR simulation. After this step was finished, the participants completed the second part of a questionnaire. This contained three remaining themes: a) appealing, in which the quality of the simulated environment was evaluated; b) education, to evaluate if the application increased the awareness about the exposure to UV radiation problem, and c) usability, which allowed to detect if the product was of easy to use by the students. The techniques of Cronbach’s Alpha and the Omega of McDonald’s Omega were used for the validation part of the results of the questionnaires to ensure its reliability. It is important to mention that according to Brown [16], when a questionnaire is divided into categories, it is recommended to apply the reliability analysis to each section independently. Finally, separated evaluations for each segment of the questionnaire were used to obtain the p-value of each of them and to check if the null hypothesis hold. 5.3 Development of the Application by Using Virtual Reality We developed two scenarios in Unity; the first one displayed a start menu with different configuration options, while the second one contained the virtual environment. The main menu of our system obtained the different input parameters given by the user: the number of hours that the user wanted to simulate being exposed to the sunlight, his or her type of skin (which is key for knowing the user’s MED), and the speed of the simulation. The interaction with the user interface was controlled via Bluetooth on the Samsung Gear VR, while the UV index measure was obtained in real-time considering the user’s location (latitude and longitude) obtained via the API OpenUV. For the main scenario, we used Oculus libraries, which were compatible with Samsung Gear VR, and allowed us to use specific cameras for virtual reality applications. We can observe in Fig. 2, after the initial configuration of our system, the model of a right arm of a user. This showed to the students the effects of the UV radiation on their skin. To simulate the effect of skin redness or erythema, we employed the method of linear interpolation available through the lerp function from Unity. This method allowed us to manage the color transition at a determined time. The parameters used for this method were the initial and final color, and the time that it will take for this to change (see Fig. 3).

108

E. Vargas-Solís et al.

Fig. 2. Arm model of a participant showing skin redness at the beginning of the simulation (left image) and at the end (right image).

Fig. 3. User interface of the Virtual Reality application, showing skin type selection (left), the arm within the environment (middle), and a final report (right).

Regarding the colors used for the basic skin tone of the arm, we used, as a reference framework, the colors employed in the Fitzpatrick scale [1]. For the skin burned tone, we based our tones by looking at pictures of people with different skin phototypes who suffered first-degree burns. The environment of our simulation consisted of a beach coupled with sounds to ensure the immersion of the user in our system. 5.4 Questionnaire The questionnaire used for our study contained the following parts: knowledge, appealing, usability, and education; by using a Likert scale of 5 points [2]. In this scale, a value of 1 means totally disagree/never, two means disagree/rarely, 3 means no-disagree or no-agree/sometimes, 4 means agree/almost always and 5 strongly agree/always. Additionally, we formulate the hypothesis for each section of the questionnaire, which is seen in Table 4.

Virtual Reality Application to Teach Dangerous Over Exposure to UV Radiation

109

Table 4. Null hypothesis for each questionnaire section. Section

Null Hypothesis Description

Knowledge

μ0 >= 3

Shows that the population has an average knowledge about the problem of the UV radiation in the country

Education

μ0