Digital Eye Care and Teleophthalmology: A Practical Guide to Applications 3031240510, 9783031240515

Citation preview

Digital Eye Care and Teleophthalmology A Practical Guide to Applications Kanagasingam Yogesan Leonard Goldschmidt Jorge Cuadros Giselle Ricur Editors

123

Digital Eye Care and Teleophthalmology

Kanagasingam Yogesan • Leonard Goldschmidt • Jorge Cuadros • Giselle Ricur Editors

Digital Eye Care and Teleophthalmology A Practical Guide to Applications

123

Editors Kanagasingam Yogesan School of Medicine University of Notre Dame Australia Fremantle, WA, Australia

Leonard Goldschmidt Department of Veterans Affairs VA Palo Alto Health Care System Palo Alto, CA, USA

Jorge Cuadros Department of Optometry and Vision Science University of California Berkeley, CA, USA

Giselle Ricur Miller School of Medicine Bascom Palmer Eye Institute University of Miami Miami, FL, USA

ISBN 978-3-031-24051-5 ISBN 978-3-031-24052-2 https://doi.org/10.1007/978-3-031-24052-2

(eBook)

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Foreword

This is an ideal time for the third volume of Digital Ophthalmology and Teleophthalmology—A Practical Guide. The possibility of telehealth has captivated imaginations since the beginning of the Australian Royal Flying Doctor Service in the 1920s and television shows such as The Jetsons in the 1960s. Yet for decades, there have been significant barriers to large-scale adoption of telehealth such as technological limitations, unclear demonstration of diagnostic accuracy, difficulty of incorporation into clinical workflow, logistical questions such as licensure and medicolegal liability, and challenges involving reimbursement and cost-effectiveness. More recently, advances in information technology and artificial intelligence have created increased interest in telehealth research. Furthermore, the COVID-19 pandemic has rapidly stimulated real-world adoption of telehealth clinical practice because of travel restrictions and physical distancing requirements. Health systems around the world quickly developed and implemented telehealth solutions, and insurers expanded coverage to include telehealth services. In addition, this pandemic has exposed many underlying inequities in the accessibility of health care, particularly for people from urban areas, rural areas, and disadvantaged backgrounds—which might also be addressed using population-based telehealth strategies. Ophthalmology is an ideal field for applying technology because clinical diagnosis is heavily image-based and technologically oriented, particularly for common causes of visual loss such as diabetic retinopathy, age-related macular degeneration, and glaucoma. The editors of this textbook, Drs. Kanagasingam Yogesan, Giselle Ricur, Jorge Cuadros, and Lenorad Goldschmidt, are all highly accomplished and recognized experts. They have compiled an outstanding set of chapters that review the state of the art, describe advances in digital imaging and artificial intelligence, and provide many examples of teleophthalmology implementation globally. This book will be useful to provide readers with backgrounds in either clinical eye care or information technology with a broad understanding of the practical aspects of teleophthalmology. I hope this will help readers think of

v

vi

Foreword

new collaborative approaches toward knowledge discovery and practical implementation of teleophthalmology systems that will improve the quality, delivery, and accessibility of eye care for patients around the world. Michael F. Chiang, MD Director National Eye Institute National Institues of Health Bethesda, MD, USA

Preface

We are pleased to publish the third book in a series that surveys leaders in the global practice of telemedicine, telehealth, and remote diagnostics in eye care. Since our first book was published in 2006 and our second book in 2012 the world has experienced remarkable growth and transformation in technology and the delivery of healthcare. The past decade has brought us efficient, low-cost, reliable, and ubiquitous telecommunications and computation in our daily lives. Healthcare is leveraging cloud computing, big data, artificial intelligence, and increased acceptance of telehealth during the COVID pandemic to transform the delivery of services. Eye care has incorporated these new offerings and innovations more than other fields in medicine. The tremendous change in eye care has prompted us to release this new edition. Our first book presented nascent teleophthalmology programs from around the world and imaginative futuristic uses of remote technology for diagnosis and treatment. Providing eye exams and even remote surgery in very isolated areas, including outer space, were some of the highlights of the first book. The main content, however, presented examples of successful programs, technical and practice standards, and considerations for day-to-day use of teleophthalmology. Our second book described mature retinal screening programs that had scaled to millions of people in some cases. It showed how telehealth was broadened to detect glaucoma, cardiovascular disease, and other conditions, and it introduced automated image analysis for the detection of various conditions. Teleophthalmology had become an established practice and the intention of the second book was to guide readers in its use. Our intention for this third edition has remained the same. This book provides a practical guide and real-world examples to inform those who are developing their own programs in diverse settings. We are once again pleased that the leaders in this field have shared their knowledge for this book, which is organized into three sections. The first section describes the current state-of-the-art telemedicine programs, including guidelines and best practices as well as ethical considerations and many examples of diverse programs. The second section focuses on the rapidly evolving field of digital imaging and artificial intelligence for autonomous and assisted diagnosis of a wide variety of ocular and systemic conditions. The third section is a showcase of successful well-established and emerging programs throughout the world. This collection of experiences demonstrates the diverse and surprising challenges and opportunities that organizations have encountered. vii

viii

Preface

Little doubt exists that technology and health care will continue to change as we address emerging environmental and resource challenges. Our hope is that this book can be used as a foundation for future programs by contributing to a broad understanding of the increasingly important field of remote and digital eye care. Fremantle, Australia Palo Alto, USA Berkeley, USA Miami, USA

Kanagasingam Yogesan Leonard Goldschmidt Jorge Cuadros Giselle Ricur

Contents

Current State of the Art Teleophthalmology and COVID . . . . . . . . . . . . . . . . . . . . . . . . . . . . Terri-Diann Pickering and Sunita Radhakrishnan

3

A Practical Guide to Telehealth in Ophthalmology . . . . . . . . . . . . Vazquez-Membrillo Miguel, García-Roa Marlon, Anurag Shrivastava, Arias-Gómez Alejandro, López-Star Ellery, López-Star Bethania, Van Charles Lansingh, Vega-Lugo Jessica, Gonzalez-Daher Pamela, and Diaz-Flores Teresa

13

Smartphone Technology for Teleophthalmology . . . . . . . . . . . . . . . Nergis Khan and David Myung

37

Ethical Recommendations for Online Medical Consultation and Teleophthalmology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arturo E. Grau, Paulina Ramos Vergara, and Sebastián Valderrama The Use of Telehealth in Optometry: Present and Future Clinical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thomas A. Wong, Gary Chu, Timothy Bossie, Susy Yu, Delaram Shirazian, Ken Lawenda, and Munish Sharma

55

63

Low Vision TeleEye Rehabilitation . . . . . . . . . . . . . . . . . . . . . . . . . Carolyn Ihrig

73

Best Practices: Telemedicine-Diabetic Retinopathy . . . . . . . . . . . . . Mark B. Horton and Jerry D. Cavallerano

81

Teleretinal Diabetic Retinopathy Screening in Primary Care Settings—Considerations for Safety Net Organizations . . . . . . . . . Jorge Cuadros and Lauren P. Daskivich

89

Digital Imaging and Artifical Intelligence Image Processing in Retinal Imaging . . . . . . . . . . . . . . . . . . . . . . . . T. M. A. Rehana Khan, Vitthal Bhandari, Sundaresan Raman, Abhishek Vyas, Akshay Raman, Maitreyee Roy, and Rajiv Raman

99

ix

x

OCT Imaging and Applications in the Retina . . . . . . . . . . . . . . . . . 119 Ziyuan Wang, Delia Cabrera DeBuc, Mirza Faisal Beg, SriniVas Reddy Sadda, and Zhihong Jewel Hu Ultrawide Field Imaging in Retinal Diseases . . . . . . . . . . . . . . . . . . 145 Aditya Verma, Chitralekha S. Devishamani, Rekha Priya Kalluri Bharat, Sashwanthi Mohan, Rupak Roy, and Rajiv Raman Digital Glaucoma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Georg Michelson, Folkert Horn, Elisabeth Grau, Stefan Andrae, David Kara, Matthias Ring, Wolfgang Mehringer, Luis Durner, Sebastian Kohl, Milos Wieczoek, Philipp Gagel, Moritz Michelson, and Hans Schüll Digital Tools for Visual Acuity Self-Assessment . . . . . . . . . . . . . . . 175 Aline Lutz de Araujo, Cristina Cagliari, Daniel Diniz, and Paulo Schor Transfer Learning for Artificial Intelligence in Ophthalmology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Paisan Ruamviboonsuk, Natsuda Kaothanthong, Varis Ruamviboonsuk, and Thanaruk Theeramunkong Beyond Predictions: Explainability and Learning from Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Chih-Ying Deng, Akinori Mitani, Christina W. Chen, Lily H. Peng, Naama Hammel, and Yun Liu Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Bjorn Kaijun Betzler, Tyler Hyungtaek Rim, Carol Y. Cheung, Tien Yin Wong, and Ching-Yu Cheng Natural Language Processing (NLP) in AI . . . . . . . . . . . . . . . . . . . 243 J. K. Wang, S. K. Wang, E. B. Lee, and R. T. Chang Global Experiences Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America . . . . . . . . . . . . . . . . . . . . . 253 Alejandro Vazquez de Kartzow, Pedro J. Acevedo, Gabriela Saidman, Vanina Schbib, Claudia Zuluaga, Guillermo Monteoliva, Marcelo Carrascal, Adrian Salvatelli, Susana Patiño, Juan Marmol, Juan Lavista Ferres, and Maria Ana Martinez Castellanos Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later . . . . . . . . . . . . . . . . . . . . . 297 Giselle Ricur, Roger Zaldivar, Roberto Zaldivar, and Raul Guillermo Marino

Contents

Contents

xi

Teleophthalmology in Brazil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Alexandre Chater Taleb Veteran Affairs (VA) Ocular Telehealth Programs . . . . . . . . . . . . . 321 April Maa, Timothy Elcyzyn, Robert Morris, and Leonard Goldschmidt Retinal Screening of Patients with Diabetes in Primary Care Clinics Why Has Uptake of This Promising Idea Been So Low? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Kanagasingam Yogesan, Andrew Wilcock, and Ateev Mehrotra Tele-Ophthalmology for Diabetic Retinopathy in the UK . . . . . . . 355 Peter H. Scanlon Screening for Diabetic Retinopathy in Denmark . . . . . . . . . . . . . . 367 Jakob Grauslund Diabetic Eye Screening Using a Hand-Held Non-mydriatic Digital Retinal Camera: Experience from a Lower Middle-Income Country . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Prabhath Piyasena, Tunde Peto, and Nathan Congdon More Than Retinopathy. Has the Time Come to Recognize Diabetic Retinopathy as Neuro-vasculopathy? Would This Change Your Practice?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Stephen Cook Teleglaucoma: Tools for Enhancing Access to Glaucoma Care for At-Risk and Underserved Populations . . . . . . . . . . . . . . . 435 Stuti M. Tanya, Abeba T. Giorgis, Sheila Marco, and Karim F. Damji Teleophthalmology for Vision Centres . . . . . . . . . . . . . . . . . . . . . . . 463 Kim Ramasamy, Dhivya Ramasamy, and Usha Kim Teleophthalmology—LVPEI Eye Health Pyramid Program Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 Padmaja Kumari Rani, Ranganath Vadapalli, Nabeel Quadri, Beula Christy, Anthony Vipin Das, Rohit C. Khanna, and Pravin Krishna Vadavalli Teleophthalmology in Timor-Leste: A Journey . . . . . . . . . . . . . . . . 485 Manoj Sharma, Nathan Chin, Christian Mich, and Nitin Verma Teleophthalmology in Nepal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 Anil Parajuli, Sean Collon, David Myung, and Suman Thapa Developing a Comprehensive Diabetic Eye Service Model with Telemedicine—The CREST (Comprehensive Rural Eye Service and Training) Project in Rural China . . . . . . . . . . . . . . . . 505 Tingting Chen and Nathan Congdon

xii

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 Appendix B: Meeting Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

Contents

Current State of the Art

Teleophthalmology and COVID Terri-Diann Pickering and Sunita Radhakrishnan

Abstract

Introduction

Teleophthalmology rose to prominence during the unprecedented stressors of the COVID-19 pandemic. Ophthalmologists rose to the challenge of caring for our patients while under shelter-in-place mandates, and while under the risk of infection themselves. Teleophthalmology was adopted in numbers never seen before and with little to no advance preparation. Then, just as abruptly as it was widely adopted, teleophthalmology was largely abandoned. However, since its wide introduction in a variety of ophthalmic specialties, it is unlikely that it will disappear entirely. In this chapter, we address the practical successes, unmet needs, and future promise of teleophthalmology. Keywords







Teleophthalmology Telemedicine Glaucoma Retinopathy Digital health COVID-19 Technology











“Tele” is derived from the Greek word for distance [1]. Telemedicine is the use of digital means and information sharing to provide health care from a distance [2]. The need for remote health care can arise in various situations which may be patient-related, such as restricted mobility, or provider-related, such as lack of specialist physicians in rural areas. Telehealth has many benefits including expanded access to healthcare, increased convenience, and comfort for patients, and decreased exposure to communicable diseases for both patients and providers. The field of ophthalmology is well suited for imaging-based models of telehealth and thus far, the highest utilization of teleophthalmology has been for the monitoring of retinal disease in limited settings such as the Veterans Administration. Telemedicine in ophthalmology became more widely adopted during the COVID-19 pandemic when there was a drastic reduction in patient visits to the office necessitating the use of telehealth as a substitute for non-emergency ophthalmic care. In this chapter, we will discuss various aspects of teleophthalmology including practical tips based on our experience with adopting telehealth during the COVID-19 pandemic.

T.-D. Pickering (&)
S. Radhakrishnan Glaucoma Center of San Francisco, 55 Stevenson, St. San Francisco, CA 94105, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_1

3

4

Teleophthalmology Before COVID-19 Traditional ophthalmic telemedicine has mainly focused on retinal remote monitoring and portable imaging, typically in rural or underserved areas. Store-and-forward technologies are used to send information such as still images, patient history, and test results for specialist review, diagnosis, and management advice, similar to radiology [2, 3]. This process relies on the patient visiting a local facility where a skilled assistant (e.g., optometrist, ophthalmic technician, nurse, primary care provider) is available [4]. Teleophthalmology has also been employed internationally, with collaboration between ophthalmologists and optometrists, and other local care providers [5]. A 2019 review reported telehealth models for ophthalmology from 19 different countries in addition to several international programs [4]. The use of teleophthalmology has been well established for screening for diabetic retinopathy, retinopathy of prematurity, and age-related macular degeneration [2, 4]. Studies have found high positive predictive values and over 90% agreement between retinal imaging and clinical examination [4]. The advantage of teleophthalmology over the in-person eye exam is evident in the case of diabetic retinopathy. This disease remains the leading cause of vision loss in young patients under the age of 60 despite the availability of treatments. Although an annual dilated examination is recommended in all diabetics, only 35– 50% of managed care patients receive an annual eye examination, and a 2008 study showed that over 70% of those with diabetic retinopathy were unaware of their diagnosis [6, 7]. In response to this, the Department of Veterans Affairs set up the largest telemedicine diabetic retinopathy screening program in the United States which has resulted in decreased travel time for eye examinations, screening at younger ages, and the identification of more cases of diabetic retinopathy [6]. A 2019 study found that remote diagnostic imaging (fundus photography and

T.-D. Pickering and S. Radhakrishnan

OCT) using an FDA-approved non-contact retinal imaging device (iFusion; Optovue) versus standard examination by a retinal specialist were equivalent in detecting referable macular degeneration [8]. There are a few tele-retinopathy of prematurity (ROP) screening programs in the US, in collaboration with neonatal intensive care units [9]. A 6-year retrospective study at Stanford University’s tele-ROP program showed sensitivity and specificity both approaching 100% [10]. However, the AAO and the American Academy of Pediatrics still recommend at least 1 in-person ROP evaluation due to the high stakes involved, and limited view of the retinal periphery with fundus photography [9]. In addition to these examples from the United States, multiple global studies have also demonstrated the effectiveness and benefits of teleophthalmology screening [11–13]. One promising approach is the use of e-consults to connect experts around the world, such as through the Cybersight platform (https:// cybersight.org/consultation/) which integrates artificial intelligence-enabled consults for online mentoring and case discussion [12]. A retinal screening study from Nepal using a Paxos Scope ophthalmic camera system (DigiSight Technologies, San Francisco, CA, USA) attached to a 6th generation iPod Touch showed that of 101 patients referred to an ophthalmologist based on screening results, the ophthalmologist concurred with the appropriateness of the referral in 97% of cases [13]. Teleophthalmology benefits more than retinal disease and can aid in diagnosing and monitoring ocular emergencies, anterior segment conditions, strabismus, and glaucoma [2]. Although 2 million people seek ophthalmologic care in the emergency room (ER) setting in the United States, over 50% of ERs do not have ophthalmologists available [9]. There are only a few U.S.-based ER teleophthalmology consult programs. The U.S. Army uses teleophthalmology for consultations in military settings overseas. The University of Pittsburgh emergency department has used an

Teleophthalmology and COVID

iPhone and an ophthalmoscope adaptor to obtain consults from remote ophthalmologists. In a review of 50 consecutive patients, researchers found this technology successfully allowed remote ophthalmologists to make accurate triage decisions [14]. Several teleretinal programs have reported high rates of incidental detection of glaucomatous-appearing optic nerves, and suspected glaucoma is a major reason for clinical referrals in teleretinal diabetic screening [15]. A study comparing cup-to-disc ratio assessment using teleophthalmology compared to in-person examinations by glaucoma specialists showed a positive predictive value of 77.5% and negative predictive value of 82.2% [16]. A costeffectiveness analysis of screening for glaucoma using telehealth in a rural Canadian setting showed increased referral rates, decreased patient travel time, and higher cost-effectiveness compared to in-person examination [17]. However, diabetic retinopathy screening is the most common condition for disease-specific remote care in ophthalmology [4].

Teleophthalmology and COVID-19 Telemedicine in ophthalmology became more widely adopted after the COVID-19 pandemic was declared [18]. The need became urgent when Dr. Li Wenliang, an ophthalmologist from Wuhan, China, died of COVID-19 on February 7, 2020 while treating an asymptomatic glaucoma patient [19]. Once the pandemic was declared, most practices followed the American Academy of Ophthalmology (AAO) guidelines to stop all routine visits [2]. After the Centers for Disease Control and Prevention (CDC) recommended telemedicine instead of in-person exams, visits to ophthalmologists dropped by 80% [20]. Under these conditions, many practices adopted teleophthalmology despite never having considered it before. The goal of teleophthalmology was to mitigate transmission of the coronavirus while still providing non-emergency patient care [21]. Practices were encouraged when reimbursement policies for telehealth were expanded on March 13, 2020 under the

5

Coronavirus Preparedness and Response Supplemental Appropriations Act [3]. Additionally, early on in the pandemic, federal authorities in the US waived normal privacy guidelines for various telemedicine platforms [22]. The biggest change was allowing telemedicine video visits with patients at home. Prior to this, telehealth services were covered by Medicare only if the patient lived in a designated rural area and left their home to visit a medical facility for the telehealth service. Before the pandemic, the Centers for Medicare & Medicaid Services supported 13,000 telehealth visits per week [23]. By May 2020, the number had risen to 1.7 million per week. If these changes persist, teleophthalmology may continue to have a role in clinical medicine beyond the pandemic.

The Teleophthalmology Exam Unlike the traditional use of teleophthalmology where a skilled operator is available at the patient-end to obtain images or other information that is then forwarded to the ophthalmologist, virtual eye exams during the COVID-19 pandemic were predominantly conducted with no assistance available at the patient-end. Telehealth visits can be “synchronous” meaning the interaction between patient and physician is in realtime, or “asynchronous” meaning messages or images are sent by the patient and reviewed by the physician at a later time. A “synchronous” televisit with video and audio enables direct communication with the patient and allows a detailed history, and basic psychologic, neurologic, external adnexal, pupil, motility, alignment, anterior segment, iris, and corneal light reflex exams. With the currently relaxed HIPPA rules, multiple apps and online platforms may be used, including Zoom, Facetime, Skype, and Doximity. Many interpersonal skills and procedures that providers have mastered do not translate intuitively to best practices in virtual encounters [24]. It is vital to first obtain and document permission to conduct the visit. Both patients and physicians should work in well-lit areas with good internet

6

connectivity. The camera position and stability should be optimized to maintain “eye contact” [24]. Using larger devices with higher-resolution screens improves image acquisition and viewing. This also helps with accessibility, especially for patients with vision challenges. Based on our experience with adopting teleophthalmology during the COVID-19 pandemic, we offer the following practical suggestions. Scheduling: The front desk staff is critical for patient triage. Once it has been determined that a telehealth visit is appropriate and feasible, staff may schedule video meetings or telephone calls, or the physician may decide which type of visit is appropriate. Remember that emergency calls still must be handled immediately and these will most likely require in-person evaluation. History: It is crucial to take a detailed history during triage and during the teleophthalmology visit. The required data to support the visit coding must be documented. Patients may also fill out online questionnaires and email photos before the exam. The camera quality of most smartphones is excellent and “selfies” may be helpful. Visual acuity: The physician can guide visual acuity measurement via mobile app (e.g., www. mdCalc.com, www.peekvision.org), or online chart (e.g., www.farsight.care, www.eyes.arizona.edu, www.optoplus.com). Printed charts are another option and are easy to validate, e.g., www.aao.org (Teleophthalmology: How to Get Started), www.visionsource.com, www.allaboutvision.com. If serial telehealth visits are necessary, try to use the same method to check acuity each time to see if there are trends. Evidence for smartphone apps for testing Snellen visual acuity is limited, and currently, no app has been found to be accurate to within at least one line of formal visual acuity testing (and further validation is required). Variation in screen resolution and text size may also impact the accuracy of visual acuity testing [25]. Intraocular pressure measurement: Options include finger tensions performed by the patient

T.-D. Pickering and S. Radhakrishnan

(does the eye feel like a grape, tomato, or apple?), or home tonometry (which is expensive, not widely available, and not widely accepted in terms of accuracy). Home tonometry options include the iCare Home, and the Triggerfish contact lens. Unfortunately, reliable, easy to use, and inexpensive remote intraocular pressure measurement represents a major unmet need. Visual fields: Confrontation visual fields can be taught to the patient and are easier if there is an assistant at home to help. There are online and app-based perimeters, e.g., www.keepyoursight. org, Melbourne Rapid Fields, rarebit perimetry, and peristat [26, 27]. There are several portable visual field testing devices, e.g., Oculus Easyfield, PalmScan VR headset, VisuALL Field Analyzer, and VIVID Vision Perimetry, but these are not widely available [28]. Evaluation of macular disease: Amsler grids are available online (e.g., www.aao.org (Teleophthalmology: How to Get Started), www. farsightcare.com) and may be printed. There are also apps and devices to monitor the central 10 degrees, e.g., maculatester, Eyecare–Amsler Grid Eye Test, ForeseeHome. Retina exam: Smartphone retinal imaging exists (e.g., DigiSight Paxos Scope (San Francisco, USA), Peek Retina (London, UK) D-EYE (Padova, Italy), Remidio Fundus on Phone (Bangalore, India), and Welch Allyn Panoptic with iExaminer (Skaneateles Falls, USA); however, none are designed for widespread home use. Color desaturation: Red dots to check for red color desaturation are available online, e.g., www.farsightcare.com, or through apps. Billing: It is important to be aware of the telehealth billing codes and the documentation required to support each code. Overall, home-based remote monitoring devices for ophthalmology are still in their infancy and not widely available. However, teleophthalmology does provide some measure of care and counseling. These interactions provide reassurance to patients and may identify symptoms that require an in-person evaluation.

Teleophthalmology and COVID

Patient Acceptance of Teleophthalmology Teleophthalmology’s most direct benefit to patients is increasing access to health care [6]. Access to quality ophthalmologic care can be limited, especially in rural areas in the United States where specialists are few [11]. Teleophthalmology has been shown to lower barriers to screening, help screen is known at-risk patients that otherwise would be lost in the traditional health system, and help identify those individuals that are truly in need of a sub-specialist’s services in order to maximize limited ophthalmologic resources in a community [2]. Teleophthalmology can fill in many of these gaps in care by allowing patients to receive care closer to their homes and avoiding the significant burden of travel and delays in care. This potential convenience factor to patients cannot be emphasized enough: it takes an average of 20 days to get a 20-minute appointment with a physician in the United States. With the addition of travel and wait time considerations this may consume over 2 hours of time and may require a day off from work [1]. Patients in an urban, low-income U.S. population with high prevalence of diabetes greatly valued the convenience of primary carebased teleophthalmology, would recommend the service to others, and would even pay at least the amount of their typical co-payment to receive the service [6, 29]. During the COVID pandemic, for the first time telehealth services were covered when performed at the patient’s home. With many people working from home and with the rapid expansion of video meeting platforms, patients became more accepting of telemedicine [30]. Even though the technology for remote ophthalmic monitoring is not scalable or ready, patients are willing to try this approach. A televisit is like having a house call and many patients love it. Patients also find virtual clinic visits easier to attend, while still meeting their medical needs [23]. A televisit eliminates barriers related to transportation such as an inconvenient or unsafe public transit system or lack of affordable

7

parking. Now that patients are aware and accepting of telemedicine they, in fact, request it. When coronavirus cases were surging, patient demand for telemedicine outstripped the ability of healthcare providers to supply it [31]. As people have become accustomed to their new routine established during the COVID-19 pandemic, some patients are finding that they prefer to stay at home. Schools or summer camps are still not operating at their usual schedule in many places, so many parents have no choice but to stay home. New research suggests that immune-suppressed patients may not develop a robust antibody response despite vaccination and these patients can decrease their risk of exposure to COVID-19 by staying at home [32]. Telemedicine has become an attractive option for many patients—it is time-efficient and reduces the cost, inconvenience, and danger of travel to an office. Telehealth has many advantages but there are also important issues to address such as equitable access to telemedicine. Some patients may have difficulty using the technology needed for telemedicine, e.g., those with low vision or hearing, the elderly, the cognitively impaired, or those with poor or no internet access or no device to connect to the internet [40]. Language and cultural barriers may need to be overcome so that minority populations can effectively access telehealth. As Parrish and Higginbotham stated in a recent editorial, we have to be cautious that teleophthalmology will not become just another tool that inadvertently divides those who already have access to health care from those who do not [18]. It is important to manage patient expectations, and explain that since the exam is virtual, it is not equivalent to an in-person exam [33]. In a survey conducted by the Glaucoma Research Foundation, 87% of patients said, given the choice, they would rather wait 6 weeks for an in-office appointment than see their glaucoma doctor virtually in 2 weeks [34]. Although this may indicate that patients seeking ophthalmologic care do understand the limitations of a virtual visit, it is still important to specifically mention that a

8

teleophthalmology examination cannot replace an in-person eye examination. Gaining patients’ trust may be a hurdle [1]. In a 2020 study conducted in Rochester, New York, teleophthalmology via the primary care provider was compared to in-person dilated examination for diabetic retinopathy screening [6]. While patients overall had a strongly positive perceived value for teleophthalmology, older adults were more comfortable and more trusting of their inperson annual dilated examination. These patients also reported valuing their relationship with their eye doctor [6]. Additionally, some artificial intelligence (AI) algorithms in development have been found to have high falsenegative rates of detection, incorrectly identifying an eye with pathology as being normal [1]. The stakes are always high in health care and such a scenario could be clinically disastrous for a patient—highlighting the ongoing need for improvement and further development of AI technology.

Physician Acceptance of Teleophthalmology Telemedicine in ophthalmology may be used for screening, triage, patient consultations, providerto-provider consultations, training and education of ophthalmologists, remote supervision of procedures, and perioperative care via video call [4]. COVID-19 disrupted traditional medical education and “shadowing”, which has led to the integration of teleophthalmology into medical training [35]. In a pre-COVID-19 survey of a mix of ophthalmologists’ and optometrists’ attitudes toward telemedicine for eye care delivery, 70% of respondents did not use telemedicine and 60% were not confident in their ability to effectively manage eye-related disease outside of a traditional office-based encounter [36, 37]. Ophthalmic clinicians initially reported low confidence in telemedicine-based eye care delivery, but this changed given its rapid expansion during the coronavirus 2019 pandemic. Teleophthalmology use increased from

T.-D. Pickering and S. Radhakrishnan

31% before the pandemic to 86% after the pandemic in ophthalmic clinicians at University of Michigan Kellogg Eye Center [36]. After using telemedicine as a necessity, clinicians’ confidence in their ability to use telemedicine increased, with 29% feeling confident/extremely confident, and 38% somewhat confident. However, in this group of 73 ophthalmologists and 15 optometrists, 33% remained not-at-all confident. A 2021 Israeli study showed that almost 90% of patients expressed high levels of satisfaction with video calls during the COVID-19 lockdown, regardless of the clinic in which they were treated [38]. Clinician satisfaction, however, was significantly lower (less than 40% expressed high levels of satisfaction), mainly because of technical problems leading to increased workload. The limitations of teleophthalmology became rapidly apparent during the COVID-19 pandemic, and its use declined once medical offices reopened. There are several challenges that make it difficult for telemedicine to be a viable option in ophthalmology. For most ophthalmic conditions, it is not equivalent to an in-person examination [3]. For glaucoma, in particular, the benefits of a virtual exam may not outweigh the risks of not being able to adequately monitor and change therapy as needed for a potentially blinding disease. On returning to in-person practice an ophthalmologist in Pennsylvania noted a much higher than usual proportion of patients who had either stopped their medications and developed progressive glaucomatous optic neuropathy or worsening of visual fields or both [18]. In our practice in San Francisco, we have noted a similar trend of vision loss. In the past, digital health reimbursement was substantially lower than in-person patient care visits, but this improved somewhat in the setting of the COVID-19 pandemic. The wide adoption of teleophthalmology is contingent on appropriate reimbursement [24, 39]. Other barriers to teleophthalmology adoption by physicians include the inadequacy of home monitoring devices and the inefficiency of these exams [6]. Critical aspects of the doctor-patient relationship are at risk—we should not trivialize the importance of human contact, non-verbal cues, touch,

Teleophthalmology and COVID

expressiveness, and traditional ways to express empathy and build rapport for diagnosis, treatment, and recovery [40]. We must be concerned with the potential for depersonalization and lack of intimacy, prioritizing efficiency and economics over quality care, and for lack of sensitivity to patients’ community, culture, and language.

Back to the Future The COVID-19 pandemic spurred many physicians and patients to embrace telemedicine. Historically, doctors visited the patient’s home, which was replaced by patients visiting doctors’ offices and hospitals. Telemedicine has the potential to bring healthcare from the clinic back into the patient’s home again [1]. The need for emergent telemedicine implementation was driven by COVID-19: during the pandemic, everyone became remote and underserved, not just rural residents. This made telehealth mainstream, and its potential advantages became apparent. Currently, telemedicine is more suited to fields that rely heavily on digital photography and imaging such as radiology and pathology, but with improvements in remote testing and better integration of the patient’s record, teleophthalmology should become more widely adopted. Teleophthalmology is one of the fields leading the way for telemedicine, including the integration of novel devices enabled with AI to assist in remote patient evaluation and screening [1]. Given the potential to transform health care, venture capital funding for digital health quadrupled from 1.1 billion dollars in 2011 to 4.3 billion in 2015 [41]. In 2018 this number increased to 9.5 billion [41]. Total corporate funding for digital health more than doubled in 2 years and reached $21.6 billion in 2020, because COVID-19 supercharged funding activity [42]. Within digital health, telemedicine topped funding at $3.2B in 2020 [43]. However, all is not rosy. Barriers to the widespread adoption of teleophthalmology and

9

teleglaucoma, in particular, remain significant. Because there is a lack of valid remote methods to evaluate intraocular pressure, visual fields, and optic nerves; the only way to adequately care for our glaucoma patients is to evaluate them inperson. Easy, reliable, and inexpensive remote intraocular pressure (IOP) monitoring remains elusive but is invaluable. The Philadelphia Telemedicine Glaucoma Detection Study emphasized the importance of (IOP) measurement in teleglaucoma [44]. For patients with both suspicious optic nerve head findings and elevated IOP (>21 mm Hg) at the first visit, the odds ratio for being diagnosed with glaucoma at the second visit was 4.48 when compared to patients with neither suspicious nerve findings nor elevated IOP at screening. For patients with suspicious nerve findings, but IOP 65” is 2 points, “past history of hypertension” is 1 point, etc. Add all points together to get the risk score

The points are fixed for each risk factor for a given patient population. A risk table can be consulted to understand the final total risk in terms of percentage

The point score for each risk factor is an approximation of the true risk increase, and the percentage risk increase may not be linearly associated with the point scores

Regression models e.g., logistic regression, linear regression

Risk factors are multiplied with specific weights and summed

“Decades of age * 2.3” + “1.3 if hypertensive” + …

The weights indicate the importance of associated risk factors

Depending on technique, the summed intermediate score is often mapped using a nonlinear function (such as an “s”-shaped sigmoid function), resulting in nonlinear risk increases

is 2 points and “past history of hypertension” is 1 point, sum all points to get the overall risk score). These risk scores are both easily calculated and enable mental shortcuts (e.g., “if the patient has more than 3 of 5 risk factors, treatment is warranted”). Some image-based models also have inherently explainable properties. For example, models can be trained to provide the precise locations of abnormalities or anatomy via a “bounding box” (Fig. 1a) or a pixel-wise segmentation delineating the region of interest (Fig. 1b). Outputs of the localization models can be used to make final

disease predictions, based on the numbers and the distribution of the detected abnormalities. Segmentation models can be used in a similar manner, e.g., based on the thickness of a specific layer. In these use cases, the final predictions are explained by these models’ outputs, though posthoc explainability methods can still be applied to better understand how these models arrived at these bounding boxes or segmentations. A disadvantage of both problem framings (localization and segmentation) is that they tend to require laborious labeling to indicate the location or extent of abnormalities of interest. Similarly,

Fig. 1 Types of image processing models that have intrinsically interpretable properties

202

because these approaches affect problem framing, they cannot be “added” to a model that was originally trained for another purpose, such as classifying images as “DR” versus “no DR”. Yet another explainability approach is to leverage “attention” mechanisms, which typically come in the form of special “neurons” that learn to explicitly capture the importance of different regions in each image (Fig. 1c). These importance estimates are used by the model to determine where to “pay attention” to—hence the name, and can be presented as a heatmap along with the final prediction. A popular type is “soft-attention” methods [49] that produce attention on a continuum (i.e., 0.45, 0.67, etc. instead of a binary 0/1 importance). One potential downside of adding the attention mechanism to the model is that it may reduce the classification performance of the model in return for easier explainability. Moreover, attention cannot be added after a model without the attention mechanism is already trained—and so may require access to the original data and retraining of the model. Post-hoc Explainability Methods Post-hoc methods help extract insights from models that are not directly interpretable, via a second mathematical processing step. One potential approach is to train a separate, simpler and explainable model. While there is no guarantee that the two models have learned the same features, one could try to improve the similarity by specifically training the second model to approximate the target model via model distillation and simplification techniques [50, 51]. Another, more common class of approach involves analyzing the target model more directly. The most prominent recent class of methods that require such post-hoc explainability aids is neural networks used to interpret images. Specifically, these studies usually leverage convolution neural networks (CNNs) that use millions to billions of mathematical operations such as multiplications and additions to process images. The calculations typically occur in layers, with the first layer using the image as input and each subsequent layer processing the output of

C.-Y. Deng et al.

the previous layer. The final layer processes the second-last layer’s output to produce the final prediction (e.g., DR present versus absent). Although every single calculation is well defined in the algorithm, it is hard to directly interpret the large numbers of parameters and mathematical operations in totality. These post-hoc methods are designed to help extract insights that are easier to understand, while leveraging the available techniques to improve performance without being limited to inherently explainable architectures. To illustrate additional subcategorizations of post-hoc XAI methods [46, 47, 52], we next present various post-hoc explainability methods for image-based models.

Explainability Methods for ImageBased Models One key distinction for interpretability methods for image-based models is whether the objective is to understand (1) why a given model produces a specific output for a specific input image or (2) what the model has learned more holistically. We will provide examples of each type together with the clinical questions that they can help answer. Methods for Specific Input Images We first describe methods that focus on understanding why a model provided a specific output for a specific input image. Many of these techniques present the final explanation as a heatmap. For example, for a DR detection model, these heatmaps may highlight lesions such as microaneurysms, hemorrhages, or hard exudates. We should caution that a “heatmap” visualizes an explanation technique’s importance estimates for each region in the image—interpreting the heatmap requires understanding of the original method used to extract each region’s relative importance. Quantifying importance of image regions on a prediction. One way to assess the importance of input image pixels is to randomly perturb the image and see if it affects the output.

Beyond Predictions: Explainability and Learning from Machine Learning

Changing the value of one pixel may not change the predictions substantially, but removing nearby related pixels (“superpixels”) together may affect the prediction. LIME [53] proposed to replace superpixels with predefined colors and identify superpixels that can affect the predictions if removed by fitting a linear model. One caveat is that interaction between features is not well captured, e.g., when coexistence of two abnormalities particularly increases disease risk. SHAP (SHapley Additive exPlanations) [54] was proposed to overcome this by calculating Shapley values of the contributions from the components to take coalition into account. Another popular set of techniques rely on tracing the “pathways” and the degrees to which neurons are activated in the network, with intermediate layers in the network typically capturing learned high-level features (Fig. 2a). Because CNNs often reduce the spatial resolution of the internal representation of the image as the layers of computation happen (via “spatial pooling” and “striding”), the (spatial) resolution of the intermediate layers usually decreases in the later layers. By analyzing how the activation of each neuron contributes to the final classification outputs, one can obtain a low resolution visualization of the spatial location of important features. This technique is termed Class Activation Mapping (CAM) [55], which was later extended in various ways to produce popular techniques such as GradCAM [56] and Guided GradCAM [56]. The specifics of the interpretation vary

203

based on the exact technique used, but most of these techniques aim to provide low-resolution estimates of the most important regions that contributed to a specific image’s prediction. Quantifying the influence of image pixels on a prediction. Another popular set of techniques similarly rely on understanding the activations in the network, by tracing these pathways back to the original input pixels, producing estimates of “importance” that are at the pixel resolution of the original image (Fig. 2b). This group of techniques typically quantifies the degree to which changing an image pixel changes the final prediction, resulting in highresolution visualizations. Because the simplest approach, “vanilla gradients”, often fails to highlight important regions, different strategies have been proposed to combat hypotheses for the failures of vanilla gradients: Guided BackProp [57], IntGrad [58], SmoothGrad [59], BlurIG [60], XRAI [61], Expressive Gradients [62], Expected Gradients [63], etc. One hypothesis is that a model could be “overly confident” in a prediction such that changing a small number of pixels does not sufficiently alter the predictions to be visible. To resolve this, several methods use a “baseline” image (e.g., a blank “image”) as a reference that lacks informative features, and interpolate between this baseline image and original image to extract important regions as they “appear” in the interpolation process. Importantly, the results may vary based on the baseline used [64].

Fig. 2 Explainability methods for image-based models that focus on understanding model outputs for specific input images. Images and heatmaps in panels a and b from [34] panel c from [30]; reused with authors’ permission

204

Understanding changes needed to an image to change the model’s prediction. Lastly, counterfactual explanations [65] revolve around developing a second “generative” model that learns to change a given image such that the original model (e.g., for DR detection) now makes a different prediction (Fig. 2c). For example, this second generative model could modify a “no-DR” image to a “DR” image by making a red spot more prominent, suggesting that the DR detection model partially relies on the presence of prominent red spots (microaneurysms) to detect DR. Similarly, the generative model could show that removing some lesions from an image with DR would change the model’s prediction to “no DR”. The biggest drawbacks of this technique are: (1) the requirement to develop a second, generative, model to make such transformations and (2) the generative model can leverage “noise” to change the prediction instead of interpretable transformations. Thus, a failure to learn interpretation information may represent a limitation of the original model (that it did not learn interpretable features) or a failure on the part of the generative model (that it was unable to learn interpretable transformations). Methods for Understanding a Model’s Learnings This next section focuses on a broader understanding of what a model has learned from the data, as opposed to why it makes specific predictions for a specific input image. Analyzing internal model parameters. Among techniques that help understand a model generally, a few classes of methods focus on visualizing the weights within the model. For example, visualizing the neurons in the first layer of neural networks for many image-based models tends to produce simple lines (“edges”) [67] in different orientations and colors (see Fig. 3a for a set of simplified black-and-white examples). These lines enable a model to detect edges in an image. An example of an “edge” in a fundus photograph is blood vessels that appear as lines running across the retinal fundus. Unfortunately, because neural networks tend to use many layers of neurons, and each layer after the first depends

C.-Y. Deng et al.

on the output of the previous layer, the complexity grows rapidly, and such visualizations are challenging for subsequent layers (Fig. 3b) [67]. Because the first layer tends to reveal simple patterns and the subsequent layers are difficult to visualize, direct visualization of the weights in neural networks is rarely conducted. Generating images to maximally activate neurons. To understand the later layers of a neural network, one class of techniques involves synthetically creating input images that optimize the output of each individual neuron (i.e., the neuron’s “activation”). When this is done for neurons in different layers, we often see simple patterns such as edges and stripes in the early layers, more complex patterns and textures in subsequent layers, and eventually parts of objects. An interesting property of this class of methods is that the created images tend to repeat features (e.g., noses and eyes for models developed to distinguish cars and buildings from cats and dogs, Fig. 3c), and these visualizations are thought to resemble psychedelic hallucinations. Hence, one technique for this purpose is called DeepDream [68, 69]. Extrapolating from this, applying DeepDream to deep learning models for detecting DR in fundus photographs may create images that contain many DR lesions. Exploring the model’s internal representation of images. Another high-level approach to exploring the final layers of the network revolves around first calculating the model’s internal numerical representation (called an “embedding”) for a dataset (Fig. 3d). This embedding is often extracted from the second-last layer of the network, i.e., the layer immediately preceding the final classification layer, which often contains thousands of numbers. The learned embedding can be thought of as the model’s numerical “summary” of the image, where each number is a complex learned concept (that is not always possible to interpret directly). To understand these embeddings, a variety of techniques are subsequently applied. For example, one can use established clustering methods such as k-means clustering to group images and look at representative examples of each cluster, or use principal component analysis or t-distributed

Beyond Predictions: Explainability and Learning from Machine Learning

205

Fig. 3 Explainability methods for image-based models that focus on understanding the model holistically (i.e., not for a specific input image). Patterns in panel c were generated using [66]

stochastic neighbor embedding (t-SNE) [70–72] to visualize patterns in the data. One cautionary note that these techniques compress information in thousands of numbers into two or three axes for visualization, which may create non-existent patterns or hide real patterns [71]. Thus, these visualization techniques should be used for hypothesis generation, not for hypothesis confirmation. Testing whether a model may have learned specific features. A related technique that relies on these embeddings include Testing with Concept Activation Vectors (TCAV) [73], which requires labeled images for specific features of interest (e.g., having microaneurysms versus not), and the target model (e.g., for detecting the presence of DR). By computing the DR detection model’s embeddings for the images labeled for microaneurysm presence, TCAV can be used to quantify the extent to which the DR detection model may have been relying on microaneurysm presence. A major limitation of TCAV is the need to define and label data for each such

feature (“concept”) of interest, though this may be partially mitigated by efforts toward automated concept generation [74]. Finding representative and challenging examples. Another set of techniques called “example-based explanation” seeks to find images that most influenced the model’s learnings (Fig. 3e). A simple approach is to compute the model’s predictions for many examples, and manually compare examples that are confidently predicted as one class (i.e., close to 0 or close to 1) versus examples for which the model indicated greater uncertainty (e.g., predictions close to 0.5 if that is the model’s decision boundary between the two categories). Intuitively, if the model is highly confident for an image, then that example is likely to contain representative features that the model learned. A potential downside of applying this method to training examples is the potential of the model to have learnt features unique to the training set, but that does not contribute to the final performance on other datasets. Other methods in this category include

206

TracIn [75], which tracks how individual examples influence the “learning” (i.e., updating the internal parameters of the model). Intuitively, the most influential training examples may contain representative learned features. Measuring importance of different image regions. Finally, another set of techniques rely on masking parts of the input image, retraining a model, and quantifying how much the performance decreases when evaluated on similarly masked images (Fig. 3f). Anatomical regions that most impact the model’s performance can be interpreted to be the most important. A prominent disadvantage of this method is that it involves extensive computation to retrain the model using the training set, and rerun the model on the entire validation set. A related technique (sometimes called OCCLUSION [67]) masks images only on the validation set. However, the results from this latter test-time masking technique should be interpreted with caution, because the model was not previously trained for input containing such masked regions and the model may not produce sensible predictions. Thus, both false positive and false negative conclusions are possible (i.e., the model relies on a feature that was not detected by this technique, and this technique highlights a feature that is not required for the model).

Case Studies: Imaging Applications in Ophthalmology In the following sections, we describe selected case studies with interesting explainability analysis or the use of explainability methods directly as an intervention. Clinicians Assessment of Explainability Methods in Ophthalmic Diagnosis While attribution methods have been compared for models trained on standard machine learning datasets, they have not been extensively compared for use on medical images; this study aimed to determine which interpretable methods were preferred by clinicians. Singh et al. [76] trained a model to categorize horizontal foveal

C.-Y. Deng et al.

cross-sectional OCT images as one of four classes: normal, drusen, diabetic macular edema, and choroidal neovascularization. They selected 13 common methods used to explain DL models and used each method to create saliency map outputs for 20 retinal images. They then tasked 14 clinicians with rating the saliency map outputs. To compare the saliency methods, ratings were normalized for each clinician (by subtracting the mean and “rescaling between 0 and 5”). They found that the Deep Taylor method [77] was rated the highest among these methods, though the ratings between clinicians were only modestly correlated. In a qualitative survey, 11 of the 14 raters (79%) preferred having an explainable machine learning model for assistance. Factors that appeared to influence the ratings reportedly included whether the entire abnormality was highlighted instead of just the edges or only a portion of the abnormality, and whether there was distracting “noise” in the saliency maps. The strength of this study lies in the systematic comparison across methods. Some limitations include the possibility that the normalization process confounded the analysis (for example, raters may have found all methods satisfactory, but the normalization enlarged the difference between their ratings); the small sample size (n = 20 images) and the lack of comparison of popular techniques such as GradCam. Explainability methods: Deep Taylor and 12 other methods. Guidance in AI-assisted Detection of Diabetic Retinopathy DR is a common complication of diabetes. While significant work has focused on developing and validating algorithms for detecting DR and grading DR severity [1–3], data were lacking regarding the ability of assistive technologies (i.e., augmented intelligence [78]) to improve DR screening. Sayres and Taly et al. [4] developed a tool to assist readers in DR grading and conducted a study to compare an “unassisted” control arm with two interventional (assisted) arms: (1) “grades only”: a histogram illustrating the relative confidence in each of five DR grades and (2) “grades plus heatmap”: both the histogram

Beyond Predictions: Explainability and Learning from Machine Learning

207

Fig. 4 The three conditions tests were (left) unassisted, (middle) histograms indicating the confidence of AI predictions for each DR grade, and (right) the histograms

and a heatmap highlighting regions that most influence the prediction. Figure provided by authors of [4], used with authors’ permission

and an explanatory IntGrad [58] heatmap highlighting regions that most strongly influences the model’s prediction. While the histogram helped communicate the model’s confidence beyond a single prediction (e.g., “DR present” or “moderate DR”), the heatmap tended to highlight regions containing DR-associated regions. In the study, the readers were randomized to the three arms (unassisted control and two types of assistive explanation methods) with crossovers between arms and asked to grade DR severity (Fig. 4). They found that model assistance increased accuracy compared to “unassisted”. Looking more closely at the different types of assistance, “grades plus heatmaps” improved accuracy for images with DR compared to unassisted, but reduced accuracy for images without DR. Both “grades only” and “grades plus heatmap” increased the sensitivity of readers for any level of DR without meaningfully impacting specificity. The authors hypothesized that seeing confidence histograms for each DR grade may help focus the reader’s attention to borderline or uncertain cases to help avoid missing pathologies. A strength of the study lies in the different forms of assistance evaluated. A limitation of the heatmap-based assistance is that the model was trained to predict DR severity scores instead of localization/segmentation tasks, thus these posthoc explanatory heatmaps may not necessarily highlight lesions accurately. In particular, the authors observed that the heatmaps generated for model-predicted “no-DR” cases did not appear to be useful to readers and indeed increased the number of false positive reads by readers. Further work is needed to better understand mitigation

strategies for “negative assistance” where model “assistance” may have the opposite effect. Explainability methods: model confidence histograms, IntGrad. Understanding Cardiovascular Risk Factors Poplin and Varadarajan et al. [30] showed that several cardiovascular risk factors, such as age, self-reported sex, smoking status and systolic blood pressure, could be accurately identified from retinal fundus photograph using deep learning. Two of these predictions were particularly surprising in terms of accuracy: age could be detected within 3.26 years on average and sex with an area under the receiver operating characteristic curve (AUC) of 0.97. Though younger patients are known to have a characteristic opalescent sheen in the retinal nerve fiber layer that fades with age [79], age was not known to be characterizable to within a few years from a fundus photograph; similarly, accurate sex determination from the fundus was not known to be possible. To understand why this was possible, the authors trained a separate model using soft attention [49] and used these attention maps to determine the “importance” of each region in the image in shaping the model output. To understand what the attention map was highlighting, the authors presented attention maps for 100 fundus photographs to 3 ophthalmologists (1 attention map per task), while blinding the ophthalmologists to the actual prediction task to avoid biasing their interpretation. Age was most associated with attention in the vessels, and sex with both vessels and the optic disk. The strength of this explainability work was in this blinded

208

review by ophthalmologists for a relatively large number of images, instead of qualitative statements for a small number of images. Some limitations included the fact that the attention maps were based on a separate, smaller and simpler model instead of the (presumably) more performant Inception-v3 [80] model—thus providing an interpretation of the source of predictive information—but not of the original predictions. Some predictions (e.g., body-mass index and diastolic blood pressure) were also associated with “nonspecific features”, meriting further investigation. Explainability methods: soft attention with blinded heatmap evaluation. Detecting OCT-derived DME Status from Fundus Photography DR screening programs typically rely on detecting surrogate markers such as hard exudates on fundus photography to make diabetic macular edema (DME)-associated referral decisions. However as many as 87% of cases may then fail to meet the retinal thickening or fluid presence criteria for treatment [81]. Varadarajan, Bavishi,

Fig. 5 The authors retrained models using images with circular areas of increasing size around the fovea or optic disk visible. On evaluation with similarly masked images, the models with access to the fovea increased in performance quickly, indicating that the region around the fovea was important for accurate prediction of OCT-derived DME status from fundus photographs. Figure reproduced from [82] (license: CC BY 4.0)

C.-Y. Deng et al.

and Ruamviboonsuk et al. [82] showed that DME presence defined based on 3D-OCT scans could be predicted by deep learning from 2D fundus photographs, with false positive rates half that of human graders using hard exudates as a surrogate for DME. As such, this technology has the potential to help identify patients for referrals due to DME and reduce false positive referrals (i.e., patients who seem to have DME based on their fundus photograph but do not show signs of DME on OCT). To understand how this prediction was possible, the authors first applied an ablation-based approach by first collecting annotations of the locations of the fovea and optic disk, then creating cropped images with only a certain radius around the fovea (or disk) visible. Finally, they trained and evaluated models on these images (e.g., a model that only sees 25% of the disk diameter away from the fovea; a second model that only sees up to 50% of the disk diameter away from the fovea, etc.). The authors found that the performance of a model increases more rapidly if it sees more context around the fovea than if it sees more context around the optic

Beyond Predictions: Explainability and Learning from Machine Learning

disk (Fig. 5). In fact, the model that saw about 1 disk diameter away from the fovea approaches the performance of a model that sees the complete fundus photograph. They concluded that most of the information content was present near the fovea. The strength of this work lies in precise and targeted region segmentation via manual labeling, enabling insights based on anatomic locations (which may vary depending on physiologic variations, the imaging procedure and patient positioning). Limitations of this approach include the need to retrain a different model for each input setup (based on amount of visible area), which can be computationally prohibitive for many researchers. Further hyperparameter tuning of the models would have exacerbated the computational needs, though it is unclear if the conclusions will differ meaningfully if the models were tuned for each visible area setup. Interestingly, Narayanaswamy and Venugopalan et al. [83] followed up with additional explainability work to study if the precise morphological features (if present) could be extracted. They applied an approach called CycleGAN (where GAN stands for generative adversarial network, a method which generates images instead of interpreting images). CycleGAN is a deep learning approach to “transform” an image of one category to another category; in this case, it was developed to transform a non-DME image into a DME image or a DME image into a nonDME image. The authors then repeatedly applied this CycleGAN to non-DME images (to make them more “DME-like”) and observed that it was adding hard exudates to the fundus, and making the fovea lighter in appearance. The reverse was true for the CycleGAN model making DME images to make them less “DME-like”: removing hard exudates and darkening the fovea. Using this information, the authors further showed that hand-engineering features based solely on the pixel intensity around the fovea, together with hard exudate presence, approached the accuracy of the original model (that was not developed using any predefined features). One subtle but crucial aspect of this work is that the CycleGAN method specifically explains what the model has learnt; it is the subsequent hand-engineered

209

feature model that “confirms” the information is sufficient to develop an accurate DME classification model. The strength of this overall approach of using a CycleGAN was to help accentuate changes (via repeated applications) that would otherwise be too subtle to detect. The limitation is that GAN-based models can often be tricky to train; this method may not always work. Explainability methods: ablation/occlusion, CycleGAN Detecting Anemia from Fundus Photographs Mitani et al. [34] trained a deep learning model that can quantify hemoglobin concentration with a mean absolute error of 0.73 mg/dL and detect anemia with an AUC of 0.88. Though signs such as conjunctival pallor have been known to be indicative of severe anemia, it had not been previously shown that the retinal fundus could help detect anemia accurately. To understand how this may be possible, the authors conducted extensive ablation experiments by occluding different parts of the fundus photo during training, training a new model, and testing again on similarly occluded images. The occlusion strategies included: occluding just a horizontal central band of different heights (thus partially or completely blocking out the optic disk and fovea); occluding everything except a central band (thus showing only a part of or the entire optic disk and fovea); occluding circular disks of different sizes (thus incrementally occluding areas starting with the fovea) and occluding the periphery of the image in concentric circles (thus incrementally occluding the borders of the fundus image) (Fig. 6). From these experiments, the authors observed that the fastest drop in performance was when the central band was blocked, though not when the central circular area was blocked, suggesting that the optic disk region was the most important. In the same study, additional qualitative saliency analysis via GradCam [56], SmoothGrad [59] and Guided Backprop [57] all indicated that model predictions were most sensitive to the blood vessels near the optic disk. Finally, the authors applied a blurring perturbation in a similar way to the occlusion analysis (i.e., by training and evaluating models using

210

C.-Y. Deng et al.

Fig. 6 Different types of masking and changes used in [34]. a showing only the central horizontal region to reveal structures such as the optic disk and fovea; b masking the central horizontal region to remove the optic disk and fovea; c showing only the central circular

region to reveal structures such as the fovea; d masking the central circular region to remove structures such as the fovea; and e applying a “blur” to the images to remove sharp “edges” such as finer blood vessels and the optic disk borders

images blurred to different extents), and found that blur appeared to substantially impact the model performance, with performance plateauing near an AUC of 0.6 when the distinction between vessels and the optic disk was no longer visible. The strengths of this explainability analyses lie in how extensive they were, adopting multiple strategies to investigate the regions of the retinal fundus anatomy that contained information about anemia and hemoglobin. The limitations of the analysis were that the authors identified anatomies that appeared to be relevant, but did not confirm hypotheses via targeted engineering of features. Explainability methods: ablation/occlusion, GradCam, SmoothGrad, Guided Backprop

states in the US. Though diabetes is known to be associated with conjunctival vessel changes [85], such changes had not previously been shown to assist in detecting poor sugar control or DR. To understand how these surprising findings were possible, the authors first adopted similar ablation strategies to the previously mentioned anemia paper, finding that performance dropped substantially faster when the center of the image was occluded (pupil/iris region), compared to a comparable area of occlusion at the periphery of the image. The authors further applied multiple saliency techniques (GradCAM [56], IntGrad [58] and Guided Backprop [57]) to evaluate which parts of the images had the most impact on the model predictions. Of note, instead of evaluating single images, the authors generated saliency maps for 100 images, “stacked” and averaged them, and visualized the averaged result. The results from the different techniques appeared generally consistent. They concluded that the pupil, iris, corneoscleral junction, and conjunctival vessels appeared to be most important for extracting information about HbA1c and DR from external eye photographs. Another notable aspect of this explainability approach was the use of a positive control (in this case cataract), which can be clearly visible from external eye photographs. All previously

Detecting Signs of Diabetes from External Eye Photographs Babenko and Mitani et al. [84] trained a deep learning model to predict systemic parameters and diabetic retinal disease from external eye photographs, capturing the eyelids and anterior segment. Their results were reportedly significantly better than “baseline models” incorporating metadata (self-reported variables: age, selfreported sex, race/ethnicity, and years with diabetes) and generalized to patients presenting for DR screening at primary care clinics across 18

Beyond Predictions: Explainability and Learning from Machine Learning

211

Fig. 7 Areas in the baseline fundus photograph that were highlighted by IntGrad developed microaneurysms in the follow-up visit. Figure adapted from [29] (license: CC BY 4.0)

mentioned techniques accurately indicated the importance of the pupillary area in the cataract prediction, improving confidence in the other conclusions. The strengths of this work lie in the multiple complementary explainability approaches used, the averaging of the saliency maps across many images to produce a quantitative (as opposed to potentially cherry-picked qualitative analysis of a handful of images) and the use of the positive control. The main limitation is again the inability to clearly identify the specific features responsible for the prediction despite narrowing down to the most relevant anatomic regions. Explainability methods: ablation/occlusion, GradCAM, IntGrad, Guided Backprop Detecting Locations that Eventually Develop Diabetic Retinopathy Lesions During DR screening, many patients turn out not to have DR after their fundus is examined or their fundus photograph is graded. Thus, instead of annual screening, many patients may only need screening every other year or more, which can save resources on the part of the healthcare system as well as reduce the burden of travel and potentially missing work to attend screening. Bora et al. [29] developed a deep learning model to predict, in patients who do not have DR at an index screening, whether they will develop DR within 2 years. They adopted the IntGrad [58], Guided Backprop [57] and BlurIG [60] saliency methods, and found a few patterns of behavior.

First, in patients with images already showing subtle microaneurysms at the index visit but were potentially erroneously assigned “no DR”, the saliency maps correctly highlighted these lesions. These cases were a good positive control. Second and remarkably, in some cases, the authors observed that the highlighted regions in the saliency maps did not contain any discernible abnormality. However, on the follow-up visit, microaneurysms were visible at the location highlighted by the saliency map, suggesting that the model is potentially able to discern subtle abnormalities that are not visible to the human eye (Fig. 7). While this surprising finding is a strength of the explainability work, the main limitation is that it remains unknown if this observation is consistent for all saliency maps, and whether these saliency maps could be then used to specifically guide grading for the risk of subtle lesions appearing in a future visit (i.e., prognostication). Explainability methods: IntGrad, Guided Backprop, BlurIG

Case Studies: Applications Outside Ophthalmology Identifying Confounders for COVID-19 Chest Radiography Models DeGrave and Janizek et al. [86] investigated to what extent deep learning models could learn spurious confounders (“shortcuts”) instead of

212

C.-Y. Deng et al.

true discriminative features to detect COVID-19 in chest radiographs. One of the motivations of the work was to better contextualize the large number of studies resulting from the availability of datasets containing exclusively COVIDpositive images, and their use in combination with negative controls from other datasets. The authors show that models trained using such combined datasets demonstrated poor external generalization (i.e., to new datasets). Saliency analysis via expected gradients [63] revealed that the model predictions were associated with changes in spurious features such as laterality markers and shoulder positioning relative to the image border. The authors further showed that a CycleGAN [87] model similarly “translated” images from positive to negative (and vice versa) by modifying the laterality marker. The strengths of this study lie in the multiple complementary explainability techniques used to show the confounding factors learned by the model. One main limitation of this study is the lack of technical solutions for the generalization problem more generally; the authors showed that systematically cropping images to remove laterality markers did not appear to improve generalization performance, thus suggesting that other, unknown spurious correlations may contribute to the issue. On the other hand, the authors showed that a model trained on a dataset containing COVIDpositive and COVID-negative images from the same institution (i.e., presumably without the aforementioned confounders) successfully generalized, underscoring the importance of a properly constructed dataset for use in deep learning, where models can learn to utilize spurious confounders to make predictions. Explainability methods: ablation/occlusion, expected gradients, CycleGAN. Identifying Novel Prognostic Features Colorectal Cancer Pathology Slides

in

There has been substantial interest in applying deep learning to applications in cancer pathology, where histopathology forms the cornerstone of disease diagnosis and prognostication, and therefore informing treatment decisions. Often, the pathologists are responsible for grading

and/or staging the tumor by interpreting the tissue morphology and extent of spread. Wulczyn et al. [88] developed a weakly supervised deep learning model to prognosticate colorectal cancer cases, whereby the model was tasked with learning prognostic features from large, gigapixel images (i.e., each image is 109 pixels hundreds to thousands of times bigger than the 106–107 pixels in typical smartphone photographs). By using a second deep learning model to cluster cropped image patches (which are individually only hundreds of pixels across), the authors found that one feature in particular (tumor-adipose feature or “TAF”) was strongly identified by the model as prognostic, and that the amount of TAF per case was itself prognostic. Further, two pathologists and three non-pathologists could reliably identify this feature when given 200 image patches (half of which contained TAF), with accuracies of 87% or higher, and inter-pathologist concordances of 94%. The strength of this aspect of the work was the successful extraction of a novel, previously unknown prognostic feature that could be identified readily by both experts and non-experts. A limitation is that further work is needed to better understand the biological significance of TAF and how best pathologists could learn to score for and prognosticate using this feature. Explainability methods: using second model for explainability (clustering based). Helping Doctors Learn to Predict All-cause Mortality from ECG Raghunath et al. [89] developed a model to predict 1-year all-cause mortality from resting 12-lead electrocardiogram (ECG) traces, attaining an AUC of 0.85 even within ECGs interpreted as “normal” by physicians. The predicted events appeared to include both cardiac and noncardiac causes of death, suggesting that cardiacspecific features were not completely responsible for the model’s predictive capability. To examine how this was possible, the authors assembled a set of 100 pairs of age- and sex-matched true positives and true negative cases (i.e., cases correctly predicted to experience all-cause death within 1 year and cases correctly predicted not to die within 1 year) along with patient age, sex,

Beyond Predictions: Explainability and Learning from Machine Learning

and nine computed ECG measurements. Next, 10 cardiologists were tasked with identifying which ECG of each pair would experience death within 1 year (note: this exactly matches the statistical interpretation of AUC, albeit excluding erroneous predictions: false positives and false negatives). The cardiologists achieved accuracies in the range 61–78%, but were not informed of their results. Next, the cardiologists were asked to study a second set of 100 pairs of ECGs, and to repeat the experiment with the first set of ECGs. On this second attempt, the cardiologists' accuracies improved to 60–93%, with 8 of 10 cardiologists performing better. The features they reported learning to look for were: higher heart rates, quality of ECG baseline, and slight left atrial enlargement. Guided GradCAM [56] was also used to visualize predictive features for a small set of 3 ECGs, though it was not clear if the 10 cardiologists had access to this information, or any other explainability methods for their second, “learning set” of ECGs. The strength of this study was that the cardiologists learned from the examples provided, and appeared to improve in their ability to perform the task the model was attempting. The limitation of this aspect of the study is that the explainability method did not appear to play a role in the cardiologists’ learning, so it is possible that additional advancements utilizing this or other explainability methods could improve the results. The authors’ preprint manuscript [90] also appeared to show a similar cardiologist study with more pairs of ECGs involved (401 and 240 instead of 100 and 100) but fewer readers (3 instead of 10), which did not show an obvious learning effect. The reasons for the difference are unclear, but future work in this area could help shed light on ways to improve the ability of people to learn from these models. Explainability methods: Guided GradCam.

213

lesions or other manifestations of the disease in question. The case studies above, however, have leveraged explainability techniques to (1) understand anatomical correlates of novel predictions, (2) directly influence clinical decision-making in retrospective image review, (3) potentially identify locations of future disease, (4) identify potential model confounders, and (5) derive novel prognostic features. These successful studies had several features in common. First, many studies leveraged multiple explainability techniques in concert to ensure that findings generalized across different methods (which each have their strengths and weaknesses). A common approach was to leverage different categories of complementary methods such as saliency-based techniques to highlight important image regions and ablationbased techniques to mask specific regions [34, 84]. Second, several studies used blinding to ensure that the experts reviewing each saliency map did not know the model’s prediction task (or final prediction), minimizing bias [30]. Third, some studies applied quantitative methods by averaging many saliency maps, quantifying results of expert reviews of saliency maps [84]. Fourth, several studies used a control (whether a positive or negative control) to help validate the explainability techniques within the context of the specific images and prediction tasks, thus improving the confidence that explainabilityderived observations were reliable [29, 84]. Lastly, several studies developed a second model to “magnify” the distinguishing differences learned by the model [83, 86]. These secondary models were important because the changes between categories were otherwise subtle and difficult to understand [83, 86].

Perspective and Future Directions Summary of Learnings from Case Studies A common use of explainability methods is qualitative sanity checks that deep learning models for disease detection highlight known

Despite the successful applications of explainability in the above studies, substantial future work remains. For one, many of these saliency techniques are imperfect: they may not necessarily highlight all features learned by the model (e.g., the saliency technique may highlight only a

214

subset of the DR lesions in a fundus photograph [29]); they may not precisely delineate the anatomical extent of the feature (e.g., the model may highlight only parts of the edges instead of the entire lesion [76]) and they may highlight spurious features that were neither sufficient nor necessary for the model (e.g., removing the feature does not affect the model’s performance [86]). While we expect explainability techniques to continue to improve, readers should bear in mind that many explainability techniques are built on heuristics and intuition of how the associated machine learning method works, as opposed to strong underlying theories. Thus, flawed observations derived from explainability techniques may be due to issues with either the original prediction model or the explainability technique [91]. Continued improvements in these techniques will be important for future work. In addition to improving existing explainability methods, it is important to develop new techniques to describe why certain parts of images or features were identified as being important. First, using words is a natural way for people to communicate concepts. For example, “This image was predicted as DR instead of no DR because this microaneurysm is too red to be a dust speck”. Though such text modeling techniques (called “image captioning” [92]) exist and have indeed been used in studies to generate clinical reports [93, 94], they are rarely used for explainability purposes because of the technical difficulty of the machine learning task, and the difficulty in collecting datasets containing examples of such “explanation statements”. Future advances in this area may enable these capabilities. Second, while machine learning has been used for novel predictions (e.g., anemia, kidney disease, or cardiovascular risk from fundus photographs), it is extraordinarily difficult to understand and for people to subsequently learn to recognize the specific predictive features. We look forward to such work in “learning from machine learning” and the resultant creation of new medical knowledge. Finally, while explainability methods are improving, a subtle but important area is the post-processing, visualization, and interpretation of the outputs of these

C.-Y. Deng et al.

techniques. For example, saliency-based techniques often produce a grid of numbers (“saliency map”) indicating the relative “importance” of each region of a given input image. Though this saliency map is often visualized as a “heatmap”, the appearance of this heatmap can vary dramatically based on post-processing choices: whether and how to normalize the numerical values, selection of the color scheme (i.e., what color indicates what value, which has implications for visual perception [95, 96]) and whether and how to threshold the heatmap to remove “noise”. The saliency map can also be displayed as a contour plot to outline or thresholded to highlight the most salient regions while not occluding the original image during visualization [97, 98]. These visualization choices can affect the ease of interpretation and warrant further study in future work.

Conclusion Explainability of machine learning models is important for both users and researchers to understand both individual predictions and the model generally. While many machine learning models have some inherent explainability, such as using risk scores to weigh several risk factors to arrive at an overall risk estimate, segmenting an OCT scan into layers or detecting the location of abnormalities, this is not true for all machine learning methods. The explainability techniques for understanding how these methods work are improving with time, though even now we can extract useful insights from these techniques regarding which anatomical locations a model is most influenced by. We encourage researchers using explainability techniques or reading a paper using such techniques to understand the nuances behind the technique to ensure conclusions drawn are commensurate with any limitations of the technique. Future advances in machine learning and associated explainability techniques will undoubtedly further improve our understanding and trust in these models, and help propel efforts toward improving patient care and generating novel biomedical insights.

Beyond Predictions: Explainability and Learning from Machine Learning Acknowledgements Our sincere appreciation goes to Boris Babenko, Avinash Varadarajan, and Dale Webster for manuscript review and helpful discussions.

References 1. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10. 2. Ting DSW, Cheung CY-L, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318:2211–23. 3. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018;1:39. 4. Sayres R, Taly A, Rahimy E, et al. Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology. 2019;126:552–64. 5. Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology. 2017;124:962–9. 6. Gulshan V, Rajan RP, Widner K, et al. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol. 2019;137:987–93. 7. Bhaskaranand M, Ramachandra C, Bhat S, Cuadros J, Nittala MG, Sadda SR, Solanki K. The value of automated diabetic retinopathy screening with the eyeart system: a study of more than 100,000 consecutive encounters from people with diabetes. Diabetes Technol Ther. 2019;21:635–43. 8. Lee CS, Baughman DM, Lee AY. Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration. Ophthalmol Retina. 2017;1:322–7. 9. Peng Y, Dharssi S, Chen Q, Keenan TD, Agrón E, Wong WT, Chew EY, Lu Z. DeepSeeNet: a deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs. Ophthalmology. 2019;126:565–75. 10. Grassmann F, Mengelkamp J, Brandl C, Harsch S, Zimmermann ME, Linkohr B, Peters A, Heid IM, Palm C, Weber BHF. A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography. Ophthalmology. 2018;125:1410–20. 11. Burlina PM, Joshi N, Pacheco KD, Freund DE, Kong J, Bressler NM. Use of deep learning for detailed severity characterization and estimation of 5year risk among patients with age-related macular degeneration. JAMA Ophthalmol. 2018;136:1359–66.

215

12. Asaoka R, Murata H, Iwase A, Araie M. Detecting preperimetric glaucoma with standard automated perimetry using a deep learning classifier. Ophthalmology. 2016;123:1974–80. 13. Medeiros FA, Jammal AA, Thompson AC. From machine to machine: an OCT-trained deep learning algorithm for objective quantification of glaucomatous damage in fundus photographs. Ophthalmology. 2019;126:513–21. 14. Phene S, Dunn RC, Hammel N, et al. Deep Learning and Glaucoma Specialists: The Relative Importance of Optic Disc Features to Predict Glaucoma Referral in Fundus Photographs. Ophthalmology. 2019;126:1627–39. 15. Thompson AC, Jammal AA, Berchuck SI, Mariottoni EB, Medeiros FA. Assessment of a segmentation-free deep learning algorithm for diagnosing glaucoma from optical coherence tomography scans. JAMA Ophthalmol. 2020;138:333–9. 16. Hood DC, De Moraes CG. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology. 2018;125:1207–8. 17. Redd TK, Campbell JP, Brown JM, et al. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. Br J Ophthalmol. 2018. https://doi.org/10.1136/ bjophthalmol-2018-313156. 18. Brown JM, Campbell JP, Beers A, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136:803–10. 19. Milea D, Najjar RP, Zhubo J, et al. Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med. 2020;382:1687–95. 20. Fang L, Cunefare D, Wang C, Guymer RH, Li S, Farsiu S. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed Opt Express. 2017;8:2732–44. 21. De Fauw J, Ledsam JR, Romera-Paredes B, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med. 2018;24:1342–50. 22. Vermeer KA, van der Schoot J, Lemij HG, de Boer JF. Automated segmentation by pixel classification of retinal layers in ophthalmic OCT images. Biomed Opt Express. 2011;2:1743–56. 23. Schlegl T, Waldstein SM, Bogunovic H, Endstraßer F, Sadeghipour A, Philip A-M, Podkowinski D, Gerendas BS, Langs G, Schmidt-Erfurth U. Fully automated detection and quantification of macular fluid in oct using deep learning. Ophthalmology. 2018;125:549–58. 24. Lu D, Heisler M, Lee S, Ding GW, Navajas E, Sarunic MV, Beg MF. Deep-learning based multiclass retinal fluid segmentation and detection in optical coherence tomography images using a fully convolutional neural network. Med Image Anal. 2019;54:100–10.

216 25. Yan Q, Weeks DE, Xin H, Swaroop A, Chew EY, Huang H, Ding Y, Chen W. Deep-learning-based prediction of late age-related macular degeneration progression. Nat Mach Intell. 2020;2:141–50. 26. Babenko B, Balasubramanian S, Blumer KE, Corrado GS, Peng L, Webster DR, Hammel N, Varadarajan AV. Predicting Progression of Agerelated Macular Degeneration from Fundus Images using Deep Learning. 2019. arXiv [cs.CV]. 27. Yim J, Chopra R, Spitz T, et al. Predicting conversion to wet age-related macular degeneration using deep learning. Nat Med. 2020;26:892–9. 28. Arcadu F, Benmansour F, Maunz A, Willis J, Haskova Z, Prunotto M. Deep learning algorithm predicts diabetic retinopathy progression in individual patients. NPJ Digit Med. 2019;2:92. 29. Bora A, Balasubramanian S, Babenko B, et al. Predicting the risk of developing diabetic retinopathy using deep learning. Lancet Digit Health. 2020. https://doi.org/10.1016/S2589-7500(20)30250-8. 30. Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, Peng L, Webster DR. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2:158–64. 31. Rim TH, Lee CJ, Tham Y-C, et al. Deep-learningbased cardiovascular risk stratification using coronary artery calcium scores predicted from retinal photographs. Lancet Digit Health. 2021;3:e306–16. 32. Cheung CY, Xu D, Cheng C-Y, et al. A deeplearning system for the assessment of cardiovascular disease risk via the measurement of retinal-vessel calibre. Nat Biomed Eng. 2021;5:498–508. 33. Son J, Shin JY, Chun EJ, Jung K-H, Park KH, Park SJ. Predicting high coronary artery calcium score from retinal fundus images with deep learning algorithms. Transl Vis Sci Technol. 2020;9:28. 34. Mitani A, Huang A, Venugopalan S, Corrado GS, Peng L, Webster DR, Hammel N, Liu Y, Varadarajan AV. Detection of anaemia from retinal fundus images via deep learning. Nat Biomed Eng. 2020;4:18–27. 35. Sabanayagam C, Xu D, Ting DSW, et al. A deep learning algorithm to detect chronic kidney disease from retinal photographs in community-based populations. Lancet Digit Health. 2020;2:e295–302. 36. Rim TH, Lee G, Kim Y, et al. Prediction of systemic biomarkers from retinal photographs: development and validation of deep-learning algorithms. Lancet Digit Health. 2020;2:e526–36. 37. Zhang K, Liu X, Xu J, et al. Deep-learning models for the detection and incidence prediction of chronic kidney disease and type 2 diabetes from retinal fundus images. Nat Biomed Eng. 2021;5:533–45. 38. Horton MB, Brady CJ, Cavallerano J, et al. Practice guidelines for ocular telehealth-diabetic retinopathy, 3rd ed. Telemed J E Health. 2020;26:495–543. 39. Selbst AD, Powles J. Meaningful information and the right to explanation. Int Data Priv Law. 2017;7:233–42.

C.-Y. Deng et al. 40. U.S. Food and Drug Administration. Executive summary for the patient engagement advisory committee meeting artificial intelligence (AI) and machine learning (ML) in medical devices. 2020. https://www. fda.gov/media/142998/download. Accessed 1 Jul 2021. 41. Lapuschkin S, Wäldchen S, Binder A, Montavon G, Samek W, Müller K-R. Unmasking clever hans predictors and assessing what machines really learn. 2019. arXiv [cs.AI] 42. Singh A, Sengupta S, Lakshminarayanan V. Explainable deep learning models in medical image analysis. J Imaging. 2020;6:52. 43. Adadi A, Berrada M. Peeking Inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access. 2018;6:52138–60. 44. Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine learning-based prediction models in healthcare. WIREs Data Min Knowl Discov. 2020. https://doi.org/10.1002/widm. 1379. 45. Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable AI: a review of machine learning interpretability methods. Entropy. 2020. https://doi. org/10.3390/e23010018. 46. Belle V, Papantonis I. (2020) Principles and practice of explainable machine learning. arXiv [cs.LG]. 47. Barredo Arrieta A, Díaz-Rodríguez N, Del Ser J, et al. Explainable artificial intelligence (xai): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion. 2020;58:82–115. 48. Ocular Hypertension Treatment Study Group, European Glaucoma Prevention Study Group, Gordon MO, et al. Validated prediction model for the development of primary open-angle glaucoma in individuals with ocular hypertension. Ophthalmology. 2007;114:10–19. 49. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhutdinov R, Zemel R, Bengio Y. Show, attend and tell: neural image caption generation with visual attention. 2015. arXiv [cs.LG]. 50. Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. 2015. arXiv [stat.ML]. 51. Frosst N, Hinton G. Distilling a neural network into a soft decision tree. 2017.arXiv [cs.LG]. 52. Arya V, Bellamy RKE, Chen P-Y, et al. One explanation does not fit all: a toolkit and taxonomy of AI explainability techniques. 2019.arXiv [cs.AI] 53. Ribeiro MT, Singh S, Guestrin C. “Why should i trust you?”: Explaining the predictions of any classifier. 2016. arXiv [cs.LG] 54. Lundberg S, Lee S-I. A unified approach to interpreting model predictions. 2017. arXiv [cs.AI]. 55. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. 2015. arXiv [cs.CV] 56. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual explanations from deep networks via gradient-based localization.

Beyond Predictions: Explainability and Learning from Machine Learning

57.

58. 59.

60. 61.

62.

63.

64.

65. 66.

67. 68.

69.

70.

71.

72. 73.

74.

In: 2017 IEEE international conference on computer vision (ICCV). 2017. p. 618–26. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: the all convolutional net. 2014. arXiv [cs.LG]. Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks. 2017. arXiv [cs.LG] Smilkov D, Thorat N, Kim B, Viégas F, Wattenberg M. SmoothGrad: removing noise by adding noise. 2017. arXiv [cs.LG]. Xu S, Venugopalan S, Sundararajan M. Attribution in Scale and Space. 2020. arXiv [cs.CV]. Kapishnikov A, Bolukbasi T, Viégas F, Terry M. XRAI: Better Attributions Through Regions. 2019. arXiv [cs.CV]. Yang H-L, Kim JJ, Kim JH, Kang YK, Park DH, Park HS, Kim HK, Kim M-S. Weakly supervised lesion localization for age-related macular degeneration detection using optical coherence tomography images. PLoS ONE. 2019;14: e0215076. Erion G, Janizek JD, Sturmfels P, Lundberg SM, Lee S-I. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nat Mach Intell 2021:1–12. Sturmfels P, Lundberg S, Lee S-I. Visualizing the impact of feature attribution baselines. Distill. 2020. https://doi.org/10.23915/distill.00022. Goyal Y, Wu Z, Ernst J, Batra D, Parikh D, Lee S. Counterfactual visual explanations. 2019. arXiv [cs.LG] Olah C, Mordvintsev A, Schubert L. Feature visualization. Distill. 2017. https://doi.org/10.23915/ distill.00007. Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. 2013. arXiv [cs.CV]. Inceptionism: Going Deeper into Neural Networks. In: Google AI Blog. http://ai.googleblog.com/ 2015/06/inceptionism-going-deeper-into-neural.html. Accessed 15 Jun 2021. DeepDream—a code example for visualizing Neural Networks. In: Google AI Blog. http://ai. googleblog.com/2015/07/deepdream-code-examplefor-visualizing.html. Accessed 15 Jun 2021. Hinton G, Roweis S. Stochastic neighbor embedding. In: Proceedings of the 15th International Conference on Neural Information Processing Systems. Cambridge, MA, USA:MIT Press; 2002. p 857–64. Wattenberg M, Viégas F, Johnson I. How to use tSNE effectively. Distill. 2016. https://doi.org/10. 23915/distill.00002. van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579–605. Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, Sayres R. Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV). 2017. arXiv [stat.ML]. Ghorbani A, Wexler J, Zou J, Kim B. Towards automatic concept-based explanations. 2019. arXiv [stat.ML].

217

75. Pruthi G, Liu F, Sundararajan M, Kale S. Estimating training data influence by tracing gradient descent. 2020. arXiv [cs.LG]. 76. Singh A, Jothi Balaji J, Rasheed MA, Jayakumar V, Raman R, Lakshminarayanan V. Evaluation of explainable deep learning methods for ophthalmic diagnosis. Clin Ophthalmol. 2021;15:2573–81. 77. Montavon G, Bach S, Binder A, Samek W, Müller KR. Explaining nonlinear classification decisions with deep Taylor decomposition. 2015. arXiv [cs.LG]. 78. American Medical Association. Augmented intelligence in health care policy report. 2018. https:// www.ama-assn.org/system/files/2019-01/augmentedintelligence-policy-report.pdf. Accessed 30 Jun 2020 79. Vislisel J. Normal fundus—child. In: University of Iowa Health Care Ophthalmology and Visual Sciences EyeRounds.org. https://webeye.ophth.uiowa.edu/ eyeforum/atlas/pages/normal-fundus-child/index.htm. Accessed 21 Jun 2021 80. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. 2015. arXiv [cs.CV]. 81. Wong RL, Tsang CW, Wong DS, McGhee S, Lam CH, Lian J, Lee JW, Lai JS, Chong V, Wong IY. Are we making good use of our public resources? The false-positive rate of screening by fundus photography for diabetic macular oedema. Hong Kong Med J. 2017;23:356–64. 82. Varadarajan AV, Bavishi P, Ruamviboonsuk P, et al. Predicting optical coherence tomography-derived diabetic macular edema grades from fundus photographs using deep learning. Nat Commun. 2020;11:130. 83. Narayanaswamy A, Venugopalan S, Webster DR, Peng L, Corrado GS, Ruamviboonsuk P, Bavishi P, Brenner M, Nelson PC, Varadarajan AV. Scientific discovery by generating counterfactuals using image translation. In: Medical image computing and computer assisted intervention—MICCAI 2020. Springer International Publishing; 2020. p. 273–83. 84. Babenko B, Mitani A, Traynis I, et al. Detection of signs of disease in external photographs of the eyes via deep learning. Nat. Biomed Eng. 2022;6:1370– 83. 85. Banaee T, Pourreza H, Doosti H, Abrishami M, Ehsaei A, Basiry M, Pourreza R. Distribution of different sized ocular surface vessels in diabetics and normal individuals. J Ophthalmic Vis Res. 2017;12:361–7. 86. DeGrave AJ, Janizek JD, Lee S-I. AI for radiographic COVID-19 detection selects shortcuts over signal. Nat Mach Intell. 2021:1–10. 87. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired imageto-image translation using cycle-consistent adversarial networks. 2017. arXiv [cs.CV]. 88. Wulczyn E, Steiner DF, Moran M, et al. Interpretable survival prediction for colorectal cancer using deep learning. NPJ Digit Med 2021;4:1–13.

218 89. Raghunath S, Ulloa Cerna AE, Jing L, et al. Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network. Nat Med. 2020;26:886–91. 90. Raghunath S, Ulloa Cerna AE, Jing L, et al. Deep neural networks can predict mortality from 12-lead electrocardiogram voltage data. 2019. arXiv [q-bio.QM]. 91. Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. 2018. arXiv [cs.CV]. 92. Hossain MDZ, Sohel F, Shiratuddin MF, Laga H. A comprehensive survey of deep learning for image captioning. ACM Comput Surv. 2019;51:1–36. 93. Monshi MMA, Poon J, Chung V. Deep learning in generating radiology reports: a survey. Artif Intell Med. 2020;106: 101878.

C.-Y. Deng et al. 94. Huang J-H, Yang C-HH, Liu F, et al. DeepOpht: medical report generation for retinal images via deep models and visual explanation. 2020. arXiv [cs.CV]. 95. Borland D, Taylor MR 2nd. Rainbow color map (still) considered harmful. IEEE Comput Graph Appl. 2007;27:14–7. 96. Rogowitz BE, Treinish LA, Bryson S. How Not to Lie with Visualization. Comput Phys. 1996;10:268–73. 97. Nabulsi Z, Sellergren A, Jamshy S, et al. Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to unseen diseases. 2020. arXiv [eess.IV]. 98. Liu Y, Kohlberger T, Norouzi M, Dahl GE, Smith JL, Mohtashamian A, Olson N, Peng LH, Hipp JD, Stumpe MC. Artificial intelligence-based breast cancer nodal metastasis detection: insights into the black box for pathologists. Arch Pathol Lab Med. 2019;143:859–68.

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging Teleophthalmology and Digital Health: A Practical Guide to Applications Bjorn Kaijun Betzler, Tyler Hyungtaek Rim, Carol Y. Cheung, Tien Yin Wong, and Ching-Yu Cheng Abstract

Artificial Intelligence (AI) has diverse applications in modern health care. Deep learning (DL) systems in ophthalmology have been used to predict and classify several ocular pathologies. In recent years, researchers expanded the use of DL to predict systemic diseases and biomarkers from retinal images. In this relatively new field, we review the reasons why the eye is well suited for such applications and address practical considerations in building AI systems with ocular

images. This chapter will discuss applications of ocular imaging in the prediction of demographic parameters, body composition factors and diseases of the cardiovascular, neurological, metabolic, endocrine, renal and hepatobiliary systems. We examine the range of systemic factors studied from ocular imaging in current literature and discuss areas of future research, while acknowledging the limitations that AI systems in ocular imaging present. Keywords




B. K. Betzler Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore B. K. Betzler
T. H. Rim
T. Y. Wong (&)
C.-Y. Cheng (&) Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road Level 6, Singapore 169856, Singapore e-mail: [email protected] C.-Y. Cheng e-mail: [email protected] T. H. Rim
T. Y. Wong
C.-Y. Cheng Ophthalmology and Visual Sciences Academic Clinical Program (Eye ACP), Duke-NUS Medical School, Singapore, Singapore C. Y. Cheung Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China








Artificial intelligence Deep learning Neural network Imaging Eye Prediction Fundus photography Optical coherence tomography










Introduction In the 1960s, David Hubel and Torsten Wiesel explored the biological architecture of the visual cortex [1]. They noted that high-level neurons in the cerebrum react to the cumulative outputs of low-level neurons. As the visual signal makes its way through consecutive brain modules, complex patterns can be detected in any area of the visual field, via combinations of simpler, low-level patterns. Artificial Intelligence (AI) engineers were inspired by this architecture, which when implemented to deep learning (DL), gradually evolved into what we now call convolutional

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_16

219

220

B. K. Betzler et al.

neural networks (CNNs). The field of computerized visual perception and image recognition, which CNNs are adept at [2, 3], has widespread applications today. In modern-day health care, robust AI algorithms have thrived in image-centric specialties such as ophthalmology [4–6], radiology [7, 8], dermatology [9], and pathology [10, 11]. In ophthalmology, multiple AI papers have been published describing applications in ophthalmic diseases, including diabetic retinopathy [12–14], glaucoma [14], age-related macular degeneration [14–16, 17], and refractive error [18]. This chapter, however, aims to describe applications of AI in predicting systemic disease, based on ocular imaging techniques. Natural anatomy is a major reason why ocular imaging lends itself well to AI methods to predict systemic disease. For instance, the retinal vasculature can be visualized in a direct, noninvasive manner. Because the retina and other end organs, including the kidneys and brain, share similar anatomical and physiological properties, the retinal vessels are indirect

representation of the systemic microvasculature [19–21]. Furthermore, the retina is an extension of the central nervous system (CNS), and optic nerve fibers are effectively CNS axons—many neurodegenerative conditions that involve the brain and spinal cord have ocular manifestations [22, 23]. Also, the external eye is a primary area where jaundice and anemia manifest. The eye thus offers a uniquely accessible window to study systemic conditions The eye is an organ with many available imaging modalities—fundus photography (FP), optical coherence tomography (OCT), OCTAngiography (OCT-A), fluorescein angiography, ultrasound biomicroscopy, and anterior segment photographs; this list is non-exhaustive. By far, the most used ocular images in the development of DL algorithms in ophthalmology is fundus photographs, followed by OCT scans (Table 1 and Fig. 1). This corresponds to realworld utility as well—given that DL algorithms thrive on huge datasets, commonly employed modalities can provide more data to a computer algorithm.

Table 1 Summary of literature on ocular imaging applications Author, year

Area of application

Imaging methods

Learning model

Training set

Validation/test set

Aslam 2020 [24]

Diabetic status

OCT-A

Random forest

152 eyes 152 patients Manchester, UK

Leave-one-out cross validation

Babenko 2020 [25]

HbA1c

External eye images

CNN (Inceptionv3)

EyePACS (CA) 290,642 images 126,066 patients 4.0% Black, 6.3% Asian/Pacific Islander, 9.1% White, 77.2% Hispanic, 1.3% Native American, 2.1% Other California, USA

Validation set EyePACS (CA) 41,928 images 19,766 patients

External test sets EyePACS (18 nonCA) 27,415 images 27,415 patients EyePACS (18 nonCA) 5058 images 5058 patients Veterans Affairs (Georgia) 10,402 images 10,402 patients

(continued)

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

221

Table 1 (continued) Author, year

Area of application

Imaging methods

Learning model

Training set

Validation/test set

Benson 2020 [26]

Diabetic peripheral neuropathy

Fundus photographs

CNN (VGG16)

386 images from 96 control patients 200 images from 46 patients with neuropathy USA

62 images from 24 control patients 50 images from 12 patients with neuropathy

Cavaliere 2019 [27]

Multiple sclerosis

SS-OCT

SVM

48 control patients 48 patients with multiple sclerosis Spain

Leave-one-out cross validation

Chang 2020 [28]

Carotid artery atherosclerosis

Fundus photographs

CNN

15,408 images 6,597 patients South Korea

Validation set 1526 images 647 patients

Chen 2016 [29]

Hemoglobin (anemia)

Images of palpebral conjunctiva

SVM Artificial Neural Network

50 images China

50 images

Cheung 2020 [30]

Retinal vessel caliber

Fundus photographs

CNN

SEED study 5309 images 5309 patients

10 external datasets 5636 images 5636 patients

Dai 2020 [31]

Hypertension

Fundus photographs

CNN

2012 images 2012 patients Shenyang, Liaoning, China

5-fold cross validation

Gerrits 2020 [32]a

Age Gender Smoking status Systolic BP Diastolic BP HbA1c BMI Relative fat mass Testosterone

Fundus photographs

CNN (MobileNetV2)

Qatar biobank 7200 images 1800 patients 86% Qatari, 14% mix of 25 other countries

Validation set 2400 images 600 patients

Test set 2400 images 600 patients

Kang 2020 [33]

eGFR

Fundus photographs

CNN (VGG-19)

20,787 images 4970 patients Taiwan

Validation set 2189 images 621 patients

Test set 2730 images 621 patients

Kim 2020 [34]

Age GENDER

Fundus photographs

CNN (ResNet-152)

SBRIA 216,866 images 84,526 patients South Korea

Validation set 2436 images 2397 patients

Test set 24,366 images 20,823 patients

Mitani 2020 [35]b

Hemoglobin (anemia) Hematocrit RBC

Fundus photographs

CNN (Inceptionv4)

UK biobank 80,006 images 40,041 patients 1.2% Black, 3.3% Asian, 91.4% White

Validation set 11,457 images 5,734 patients

Test set 22,742 images 11,388 patients

Test set 1520 images 654 patients

(continued)

222

B. K. Betzler et al.

Table 1 (continued) Author, year

Area of application

Imaging methods

Learning model

Training set

Validation/test set

Nunes 2019 [36]

Alzheimer’s disease Parkinson’s disease

SD-OCT

SVM

20 patients with Alzheimer’s disease 28 patients with Parkinson’s disease 27 healthy controls Portugal

10-fold cross validation

Pérez del Palomar 2019 [37]

Multiple sclerosis

SS-OCT

Random forest with adaboost

180 healthy controls 80 patients with Multiple Sclerosis Spain

10-fold cross validation

Poplin 2018 [38]

Age Gender Smoking status Systolic BP Diastolic BP HbA1c BMI Major adverse cardiovascular events

Fundus photographs

CNN (InceptionV3)

UK Biobank 96,082 images 48,101 patients 1.2% Black, 3.4% Asian/PI, 90.6% White EyePACS 1,682,938 images 236,234 patients 4.9% Black, 5.5% Asian/PI, 7.7% White, 58.1% Hispanic, 1.2% Native American

Test Set UK Biobank 24,008 images 12,026 patients EyePACS 1958 images 999 patients

Rim 2020 [17]c

Age Gender Body muscle mass Height Weight Creatinine Diastolic BP Systolic BP Hematocrit Hemoglobin

Fundus photographs

CNN (VGG16)

Severance main hospital 86,994 images 27,516 patients South Korea

Internal test set 21,698 images 6,879 patients

External test sets Severance Gangnam hospital 9,324 images 4,343 patients Beijing eye study 4,324 images 1,060 patients SEED study 63,3275 images 7726 patients UK biobank 50,732 images 25,366 patients

(continued)

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

223

Table 1 (continued) Author, year

Area of application

Imaging methods

Learning model

Training set

Validation/test set

Rim 2021 [81]

Coronary Artery Calcification RetiCACd

Fundus Photographs

CNN (EfficientNet)

36,034 images 15,911 patients South Korea

Internal test set 8930 images 3965 patients

External test set 1 18,920 images 8707 patients External test set 2 1054 images 527 patients

Sabanayagam 2020 [40]

Chronic kidney disease

Fundus photographs

SEED 10,376 images 5188 patients South Korean

Internal test set SEED 2594 images 1297 patients

External test sets SP2 7470 images 3735 patients Beijing eye study 3076 images 1538 patients

Samant 2018 [41]

Diabetes

Infrared Iris images

Random forest

180 diabetic patients 158 non-diabetic controls India

–

Son 2020 [42]

Coronary artery calcification

Fundus photographs

CNN (InceptionV3)

44,184 images 20,130 patients South Korea

Five-fold cross validation

Vaghefi 2019 [43]

Smoking status

Fundus photographs

CNN

Auckland diabetic eye Screening database Total cohort 165,104 images 81,711 patients

Training set—60% Validation set—20% Test set—20%

Xiao 2021 [44]

Hepatobiliary diseases Liver cancer Liver cirrhosis Chronic viral hepatitis NAFLD Cholelithiasis Hepatic Cyst

External eye (Slit Lamp) Images Fundus photographs

CNN (ResNet-101)

2481 slit-lamp images from 1252 patients 1989 fundus images from 1138 patients Guangzhou, China

1069 slit-lamp images from 537 patients 800 fundus images from 468 patients

(continued)

224

B. K. Betzler et al.

Table 1 (continued) Author, year

Area of application

Imaging methods

Learning model

Training set

Validation/test set

Zhang 2020 [45]

Hypertension FPG TG Age Gender alcohol status Smoking status BMI waist-hip Ratio Hematocrit Total bilirubin Direct bilirubin

Fundus photographs

CNN (Inceptionv3)

1,222 images 625 patients Training set—80% Henan, China

Validation set—10% Test set—10%

BMI, body mass index; BP, blood pressure; CA, California; CAC, coronary artery calcium; CNN, convolutional neural network; FPG, fasting plasma glucose; HbA1c, hemoglobin A1C; HCT, hematocrit; NAFLD, non-alcoholic fatty liver disease; OCT-A, optical coherence tomography angiography; RetiCAC, deep learning retinal coronary artery calcium; SD-OCT, spectral domain optical coherence tomography; SS-OCT, swept source optical coherence tomography; SBRIA; Seoul National University Bundang Hospital Retinal Image Archive; SEED, Singapore Epidemiology of Eye Diseases; SP2, Singapore Prospective Study Program; SVM, support vector machine; TG, triglyceride a Gerrits et al. reported results on a total of 17 cardiometabolic risk factors. Only 9 parameters deemed “predictable” are shown b Mitani et al. reported prediction results of 31 complete blood count components, detailed results can be found in their supplementary material c Rim et al. reported results on a total of 47 systemic biomarkers. Only 10 parameters deemed “predictable” are shown d RetiCAC score defined as the probability of the presence of CAC based on retinal fundus photographs

Fig. 1 Overview of predictable disease or biomarkers from ocular imaging

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

In this chapter, we will explore practical considerations in building AI systems with ocular images. We will discuss how ocular imaging is used in the prediction of demographic parameters; body composition factors; and diseases of the cardiovascular, neurological, metabolic, endocrine, renal, and hepatobiliary systems.

Building AI Systems with Ocular Images Machine learning systems can be classified according to the amount and type of supervision received during training [46]. There are four major categories—supervised learning, unsupervised learning, semisupervised learning, and reinforcement learning. Most algorithms built for healthcare applications are supervised learning models—the desired solutions, or labels, are provided as inputs alongside the training examples. Convolutional neural networks (CNNs) are one important supervised learning algorithm. In unsupervised learning, the training data is unlabeled. The system tries to learn without a teacher by spotting trends or delineating groups in the dataset. Anomaly detection is an example of an unsupervised learning task. Given that this chapter focuses on ocular images, further elaboration on CNNs is appropriate. A CNN consists of single input and output layers with multiple hidden layers. Hidden layers transform the input image by applying a set of mathematical weights, called filters (or convolution kernels) that act as feature detectors. A layer of neurons using the same filter produces a feature map, which highlights the areas within the image that activates the filter the most. Multiple feature maps are stacked, making a single convolutional layer capable of detecting multiple features anywhere in the image. During training, the CNN will learn the most useful filters for its task, and higher level layers learn to combine them into complex patterns. Regarding input images, the development of high-quality AI models requires meaningful data at a sufficient scale, which can be difficult to acquire. Khan et al. [47] conducted a global

225

review of publicly available datasets for ophthalmological imaging. Ninety-four open-access datasets were identified, of which the top three imaging modalities were fundus photography (54/94, 57%), OCT or OCT-A (18/94, 19%), and external eye photographs (7/94, 7%). The three largest datasets were contributed by Kermany et al. [6] (OCT), EyePACS (fundus photographs) [38], and MRL Eye [48] (external eye photographs). CNNs are commonly implemented in several popular software frameworks. Python-based frameworks such as TensorFlow [49] and PyTorch [50] have gained more popularity among the deep learning community. High-level interfaces such as Keras [39] have also made it much easier to develop DL systems, by simplifying the existing networks architectures and pretrained weights. Rather than starting from a completely blank neural network for ocular images, it is common for researchers to perform transfer learning. Transfer learning is the process of reusing pre-existing architectures developed for similar applications and refining the weights and filters for a different target domain. A feedforward approach is employed to fix the weights in lower layers (already optimized to recognize simpler structures in most images—eye or noneye) and retraining the weights of higher layers with backpropagation. This allows the model to recognize complex features of ocular images, with fewer training examples, less time, and less computational power. For instance, Rim et al. [17] adopted the VGG16 architecture, popular for everyday image classification, for the prediction of systemic biomarkers from retinal fundus photographs. Common architectures include AlexNet [2], ResNet [51], VGGNet [52], Inception V1 to V4 [53], and SENet [54]—all of which are examples or adaptations of CNNs. To evaluate the diagnostic performance of an AI system, a reliable reference standard (“ground truth”) is crucial. For imaging in ophthalmology, the reference standards are usually human assessments from ophthalmologists, reading center graders, trained technicians, or optometrists, although gold-standard machine-measured parameters are used too. When studying outcome

226

B. K. Betzler et al.

metrics, it is important to evaluate the design of a DL algorithm against the reference standards. For example, an algorithm trained on single field fundus photography will underperform against a reference standard of wide-field or seven-field fundus photography. In addition, being aware of the origin, demographics and characteristics of training and test datasets, and performance on external validation sets, is crucial. After creation of a robust AI system, it is important for algorithms to explain their results. Given the nature of neural networks as black-box decision systems [55], model explanation techniques are important to support patients’ and clinicians’ understanding of the algorithm’s results. The generation of heatmaps overlaid on input images are commonly employed by research groups to highlight abnormal areas on

ocular images [6, 40, 56, 57]. Popular techniques include Smooth Integrated Gradients [58], GradCAM [59], and Guided-backprop [60].

Prediction of Demographic and Lifestyle Parameters Demographic factors predicted by existing studies include age, gender, smoking status, and alcohol status [34, 38, 45–43] (Table 2). While studies reported robust predictions of age and gender on fundus photographs, Gerrits et al. [32] additionally questioned if their algorithm was indirectly predicting age or gender during their performance on other target biomarkers. For instance, considerable differences in model performance were found between females and males

Table 2 Predicting demographic parameters from ocular imaging Parameter

Performance

Author, year

Age

MAE: 2.78 years (95% CI, 2.55–3.05) R2: 0.89 (95% CI, 0.86–0.92)f

Gerrits 2020 [32]

UK biobank MAE: 3.26 years (95% CI, 3.22–3.31) R2: 0.74 (95% CI, 0.73–0.75)

EyePACS-2 K MAE: 3.42 years (95% CI, 3.23–3.61) R2: 0.82 (95% CI, 0.79–0.84)

Poplin 2018 [38]

Overall: MAE: 3.06 years (95% CI, 3.03–3.09) R2: 0.92 (95% CI, 0.92–0.93) Subgroup with HTN: MAE: 3.46 years (95% CI, 3.44–3.49) R2: 0.74 (95% CI, 0.75–0.76)

Subgroup with DM: MAE: 3.55 years (95% CI, 3.50– 3.60) R2: 0.75 (95% CI, 0.74–0.75) Subgroup of smokers: MAE: 2.65 years (95% CI, 2.75– 2.77) R2: 0.86 (95% CI, 0.86–0.86)

Kim 2020 [34]

Rim 2020 [17]

Internal test set MAE: 2.43 years (95% CI, 2.39–2.47) R2: 0.83 (95% CI, 0.82–0.84) External test sets Severance Gangnam hospital MAE: 3.38 years (95% CI, 3.30– 3.46) R2: 0.61 (95% CI, 0.59–0.64) Beijing eye study MAE: 3.78 years (95% CI, 3.63– 3.93) R2: 0.36 (95% CI, 0.31–0.41) Predicted age >55 Acc: 0.748 AUC: 0.850

SEED study MAE: 3.77 years (95% CI, 3.71–3.82) R2: 0.63 (95% CI, 0.62–0.64) UK biobank MAE: 4.50 years (95% CI, 4.46–4.55) R2: 0.51 (95% CI, 0.50–0.52)

Zhang 2020 [45]

(continued)

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

227

Table 2 (continued) Parameter

Performance

Gender

UK Biobank AUC: 0.97 (95% CI, 0.966–0.971)

EyePACS-2 K AUC: 0.97 (95% CI, 0.96–0.98)

Poplin 2018 [38]

Author, year

Overall: AUC: 0.969 (95% CI, 0.968– 0.971) Subgroup with HTN: AUC: 0.961 (95% CI, 0.959– 0.962)

Subgroup with DM: AUC: 0.955 (95% CI, 0.953–0.958) Subgroup of smokers: AUC: 0.978 (95% CI, 0.976–0.979)

Kim 2020 [34]

Internal test set AUC: 0.96 (95% CI, 0.96–0.96) Acc: 0.91 (95% CI, 0.90–0.91) Sen: 0.93 (95% CI, 0.91–0.94) Spec: 0.82 (95% CI, 0.80–0.85) External test sets Severance Gangnam hospital AUC: 0.90 (95% CI, 0.89–0.91) Acc: 0.82 (95% CI, 0.81–0.83) Sen: 0.90 (95% CI, 0.88–0.92) Spec: 0.80 (95% CI, 0.79–0.81) Beijing eye study AUC: 0.91 (95% CI, 0.89–0.93) Acc: 0.85 (95% CI, 0.82–0.87) Sen: 0.83 (95% CI, 0.77–0.89) Spec: 0.89 (95% CI, 0.84–0.93)

Smoking

Alcohol status

Rim 2020 [17]

SEED AUC: 0.90 (95% CI, 0.89–0.91) Acc: 0.83 (95% CI, 0.82–0.84) Sen: 0.70 (95% CI, 0.64–0.80) Spec: 0.84 (95% CI, 0.83–0.86) UK biobank AUC: 0.80 (95% CI, 0.79–0.80) Acc: 0.70 (95% CI, 0.69–0.70) Sen: 0.70 (95% CI, 0.67–0.76) Spec: 0.72 (95% CI, 0.71–0.73)

AUC: 0.97 (95% CI, 0.96–0.98) Acc: 0.93 (95% CI, 0.91–0.95)

Gerrits 2020 [32]

AUC: 0.704 Acc: 0.624

Zhang 2020 [45]

UK biobank AUC: 0.71 (95% CI, 0.70–0.73)

Poplin 2018 [38]

AUC: 0.86 Acc: 0.889 Spec: 0.939 Sen: 0.626

Vaghefi 2019 [43]

AUC: 0.78 (95% CI, 0.74–0.82) Acc: 0.81 (95% CI, 0.78–0.84)

Gerrits 2020 [32]

AUC: 0.794 Acc: 0.732

Zhang 2020 [45]

Predicting alcohol drinker AUC: 0.948 Acc: 0.863

Zhang 2020 [45]

Acc, accuracy; AUC, area under the receiver operating curve; CI, confidence interval; MAE, mean absolute error; SEED, Singapore Epidemiology of Eye Diseases; sensitivity; Spec, specificity

228

for testosterone and relative fat mass. However, the performance of gender prediction in agestratified subgroups, and vice versa, was similar, suggesting that the features used during age and gender prediction are largely independent [32]. Using saliency maps, Poplin et al. [38] reported that the blood vessels were highlighted in the models trained to predict age and smoking, while models trained to predict gender highlighted the optic disk, vessels, and fovea, with some signal distributed throughout the peripheral retina as well. Regarding smoking and alcohol status, despite current models showing considerable prediction performance [32, 38, 43, 45], one must note that the “ground truths” for these parameters are ultimately self-reported by patients. Hence, the performances of these algorithms would be limited by information bias and patients’ truthfulness when stating their smoking frequency and alcohol intake. We also note that using AI to predict age and gender inherently has poor clinical utility; however, these were two of the earliest parameters to be predicted from fundus photographs by neural networks due to their distinct, binary nature.

Prediction of Body Composition Factors Body composition factors predicted by existing studies include body mass index (BMI), body muscle mass, height, weight, relative fat mass, and waist-hip ratio (WHR) [38, 61–63] (Table 3). While BMI is a parameter of interest due to its well-established associations with all-cause [64] and cause-specific mortality [62], performance of current algorithms in BMI prediction is generally poor with low R2 values (R2: 0.13–0.17). Model generalizability for BMI and other body composition factors across ethnically distinct datasets was poor as well. Rim et al. [62] found that DL algorithms for prediction of height, bodyweight, BMI (and other non-body composition factors), which were trained on a South Korean dataset, showed limited generalizability in the UK Biobank dataset (majority White ethnicity) (R2  0.09). Proportional bias was observed,

B. K. Betzler et al.

where predicted values in the lower range were overestimated and those in the higher range were underestimated. Heatmaps showed that the optic disk area was mainly used to predict measures of body composition. The prediction of body muscle mass is noteworthy, as it is potentially a more reliable biomarker than BMI for cardiometabolic risk and nutritional status [62]. If future DL algorithms are able to demonstrate improved prediction results and generalizability, this could have clinical utility is screening for sarcopenia.

Prediction of Cardiovascular Disease and Its Risk Factors Cardiovascular disease (CVD) and biomarkers predicted by existing studies include systolic and diastolic blood pressure (BP), hypertension, retinal vessel caliber, coronary artery calcification, and carotid artery atherosclerosis (Table 4) [38, 45–62, 28–31]. All studies in this subset used fundus photography-based DL models; this was expected as fundus photography can directly capture many retinal features associated with increased cardiovascular risk, including vessel caliber, tortuosity, and bifurcations [64, 62]. In addition to predicting existing cardiovascular risk factors (age, gender, smoking, BMI, BP), Poplin et al. [38] constructed models to predict future onset of major adverse cardiovascular events within 5 years. The AUC of 0.70 using fundus photographs was comparable to the AUC of 0.72 using the composite European Systematic Coronary Risk Evaluation [60] (SCORE). It was acknowledged that hybrid models where fundus photography is augmented with clinical parameters were able to yield slightly better predictions [38]. With regard to BP, predictions from fundus photographs have been suggested to be more reflective of accumulated damage over time [62], resembling how HbA1c levels are reflective of blood glucose levels over months. However, model performance for systolic and diastolic BP prediction in current literature can be improved, with R2 values ranging from 0.16 to 0.40. In addition to predicting risk factors from fundus photographs, Cheung et al. [30]

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

229

Table 3 Predicting body composition factors from ocular imaging Parameter

Performance

Author, year

BMI

UK biobank MAE: 3.29 kg/m2 (95% CI, 3.24–3.34) R2: 0.13 (95% CI, 0.11–0.14)

Poplin 2018 [38]

MAE: 4.31 kg/m2 (95% CI, 4.02–4.63) R2: 0.13 (95% CI, 0.06–0.19)

Gerrits 2020 [32]

Classifying BMI  24.0 kg/m2 AUC: 0.731 Acc: 0.712

Zhang 2020 [45]

Internal test set MAE: 2.15 kg/m2 (95% CI, 2.12–2.19) R2: 0.17 (95% CI, 0.16–0.18)

Rim 2020 [17]

Body muscle mass

Internal test set MAE: 5.11 kg (95% CI, 5.04–5.19) R2: 0.52 (95% CI, 0.51–0.53) External test set Severance Gangnam hospital MAE: 6.09 kg (95% CI, 5.96–6.23) R2: 0.33 (95% CI, 0.30–0.35)

Rim 2020 [17]

Height

Internal test set MAE: 5.20 cm (95% CI, 5.13–5.28) R2: 0.42 (95% CI, 0.40–0.43)

Rim 2020 [17]

External test sets Severance Gangnam hospital MAE: 5.93 cm (95% CI, 5.80– 6.05) R2: 0.28 (95% CI, 0.25–0.30) Beijing eye study MAE: 5.48 cm (95% CI, 5.23– 5.73) R2: 0.23 (95% CI, 0.18–0.27) Weight

SEED study MAE: 6.21 cm (95% CI, 6.10–6.31) R2: 0.25 (95% CI, 0.24–0.27) UK biobank MAE: 7.09 cm (95% CI, 7.02–7.15) R2: 0.08 (95% CI, 0.06–0.09)

Rim 2020 [17]

Internal test set MAE: 7.69 kg (95% CI, 7.57–7.81) R2: 0.36 (95% CI, 0.34–0.37) External test sets Severance Gangnam hospital MAE: 9.63 cm (95% CI, 9.42– 9.84) R2: 0.19 (95% CI, 0.16–0.22) Beijing eye study MAE: 8.28 cm (95% CI, 7.89– 8.68) R2: 0.17 (95% CI, 0.11–0.22)

SEED study MAE: 9.69 cm (95% CI, 9.51–9.86) R2: 0.11 (95% CI, 0.10–0.13) UK biobank MAE: 11.81 cm (95% CI, 11.69– 11.92) R2: 0.04 (95% CI, 0.03–0.05)

Relative fat mass

MAE: 5.68 units (95% CI, 5.32–6.04) R2: 0.43 (95% CI, 0.37–0.48)

Gerrits 2020 [32]

Waist-hip ratio

Predicting WHR Male < 0.9, Female < 0.85 AUC: 0.704 Acc: 0.646

Zhang 2020 [45]

Acc, accuracy; AUC, area under the receiver operating curve; CI, confidence interval; MAE, mean absolute error; SEED, Singapore Epidemiology of Eye Diseases; sensitivity; Spec, specificity; WHT, waist-hip ratio

230

B. K. Betzler et al.

Table 4 Predicting cardiovascular disease and its risk factors from ocular imaging Parameter

Performance

Author, year

Systolic BP

UK biobank MAE: 11.35 mmHg (95% CI, 11.2–11.5) R2: 0.36 (95% CI, 0.35–0.37)

Poplin 2018 [38]

Internal test set MAE: 9.29 mmHg (95% CI, 9.16–9.43) R2: 0.31 (95% CI, 0.29–0.32)

Rim 2020 [17]

External test sets Severance Gangnam hospital MAE: 10.55 mmHg (95% CI, 10.3–10.8) R2: 0.17 (95% CI, 0.15– 0.20) Beijing eye study MAE: 13.2 mmHg (95% CI, 12.6–13.8) R2: 0.19 (95% CI, 0.15– 0.24)

Diastolic BP

MAE: 8.96 mmHg (95% CI, 8.32–9.58) R2: 0.40 (95% CI, 0.35–0.46)

Gerrits 2020 [32]

UK biobank MAE: 6.42 mmHg (95% CI, 6.33–6.52) R2: 0.32 (95% CI, 0.30–0.32)

Poplin 2018 [38]

Internal test set MAE: 7.20 mmHg (95% CI, 7.09–7.30) R2: 0.35 (95% CI, 0.33–0.36)

Rim 2020 [17]

External test sets Severance Gangnam hospital MAE: 7.59 mmHg (95% CI, 7.26–7.91) R2: 0.21 (95% CI, 0.18–0.24) Beijing eye study MAE: 8.09 mmHg (95% CI, 7.72–8.47) R2: 0.23 (95% CI, 0.17–0.28)

Hypertension

5-year MACE

SEED study MAE: 14.0 mmHg (95% CI, 13.7–14.2) R2: 0.21 (95% CI,0.19–0.22) UK biobank MAE: 13.6 mmHg (95% CI, 13.4–13.7) R2: 0.20 (95% CI, 0.19–0.21)

SEED study MAE: 7.14 mmHg (95% CI, 7.02–7.26) R2: 0.27 (95% CI, 0.25– 0.29) UK biobank MAE: 7.67 mmHg (95% CI, 7.59–7.74) R2: 0.16 (95% CI, 0.15– 0.17)

MAE: 6.84 mmHg (95% CI, 6.40–7.29) R2: 0.24 (95% CI, 0.18–0.30)

Gerrits 2020 [32]

Systolic BP > 140 mmHg or Diastolic BP > 90 mmHg AUC: 0.651 Acc: 0.609 Spec: 0.515 Sen: 0.705

Dai 2020 [31]

Systolic BP > 140 mmHg or Diastolic BP > 90 mmHg AUC: 0.766 Acc: 0.688

Zhang 2020 [45]

Performance of various input variables Algorithm only AUC: 0.70 (95% CI, 0.65–0.74) Age + systolic BP + BMI + gender + current smoker AUC: 0.72 (95% CI, 0.68–0.76)

Poplin 2018 [38]

(continued)

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

231

Table 4 (continued) Parameter

Performance

Author, year

Algorithm + age + systolic BP + BMI + gender + current smoker AUC: 0.73 (95% CI, 0.69–0.77) SCORE AUC: 0.72 (95% CI, 0.67–0.76) Algorithm + Score AUC: 0.72 (95% CI, 0.67–0.76) Retinal vessel calibre

SIVA-DLS showed high agreement with validated human measurements in 10 external datasets, with ICCs ranging from 0.82–0.95 Multivariable linear regression analysis Associations of CRAE with age, gender, mean arterial BP, BMI and total cholesterol, and associations of CRVE with gender, MABP, BMI, HbA1c and current smoking were largely identical (similar RN2 values) between SIVA-DLS and SIVA-human

Cheung 2020 [30]

Coronary artery calcification

Predicting CAC score > 100 AUC: 0.832 (95%CI, 0.802–0.863) Results of other CAC thresholds at 0, 200, 300, 400 units not described

Son 2020 [42]

Using RetiCAC to predict presence of CAC Internal Test Set AUC: 0.731 (95%CI, 0.712–0.751) External Test Set 1 AUC: 0.742 (95%CI, 0.732–0.753) External Test Set 2 AUC: 0.729 (95%CI, 0.685–0.773)

Rim 2021 [64]

AUC: 0.713 Acc: 0.583 Sen: 0.891 Spec: 0.404 F1: 0.611

Chang 2020 [28]

Carotid artery atherosclerosis

Acc, accuracy; AUC, area under the receiver operating curve; BMI, body mass index; BP, blood pressure; CAC, coronary artery calcification; CI, confidence interval; ICC, intraclass correlation coefficients; CRAE, central retinal artery equivalent; CRVE, central retinal vein equivalent; ICC, intraclass correlation coefficient; MACE, major adverse cardiovascular events; MABP, mean arterial blood pressure; MAE, mean absolute error; RetiCAC, deep learning retinal coronary artery calcium; SEED, Singapore Epidemiology of Eye Diseases; sensitivity; SIVA-DLS, Singapore I vessel assessment deep learning system; Spec, specificity

developed a DL to estimate retinal vessel caliber, specific retinal vessel features (such as retinal arteriolar caliber narrowing or venular caliber widening) related to CVD risk, and showed that the estimation of retinal vessel caliber measurements from fundus photographs can be fully automated, with high agreement between human and DL measurements, and went on to quantify the correlations between specific retinal vessel features and CVD risk factors. Another predicted parameter was coronary artery calcium (CAC), a

pre-clinical marker of atherosclerosis [58]. Son et al. [42] predicted abnormal CAC scores at various thresholds, producing an AUC of 0.832 when the threshold was set at >100 units. Furthermore, Rim et al. [64] derived a deep learningbased CAC score predicted from retinal photographs (RetiCAC) and used this new RetiCAC score for cardiovascular risk stratification. Based on RetiCAC, a new three-tier cardiovascular disease risk stratification system was proposed, which showed comparable performance as

232

B. K. Betzler et al.

current CT scan in predicting future CVD events [31]. Therefore, this study suggests that retinal photography may be adopted as a more costeffective method as compared with cardiac CT and non-radiation imaging modality for cardiovascular risk stratification in low-resource settings.

Prediction of Hematological Parameters Hematological parameters predicted by existing studies include hemoglobin concentration, red blood cell (RBC) count, and hematocrit [17, 29, 35, 45] (Table 5). Physical examination of the

Table 5 Predicting hematological parameters from ocular imaging Parameter

Performance

Author, year

Hemoglobin (anemia)

Predicting any anemia < 11 g/dL

Chen 2016 [29]

SVM Model Sen: 0.78 Spec: 0.83

CNN Model Sen: 0.75 Spec: 0.83

Predicting any anemia 9% External Test Sets EyePACS (18 non-California states) AUC: 0.70 (95% CI, 0.69–0.71) EyePACS (18 other non-California states) AUC: 0.73 (95% CI, 0.72–0.75) Veterans Affairs (Georgia) AUC: 0.70 (95%CI, 0.68–0.71)

Babenko 2020 [25]

TG > 1.71

AUC: 0.703 Acc: 0.667

Zhang 2020 [45]

Testosterone

MAE: 3.76 nmol/L (95% CI, 3.36–4.18) R2: 0.54 (95% CI, 0.48–0.60)

Gerrits 2020 [32]

Acc, accuracy; AUC, area under the receiver operating curve; CI, confidence interval; FPG, fasting plasma glucose; HbA1c, Hemoglobin A1c; MAE, mean absolute error; sensitivity; Spec, specificity; TG, triglycerides

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

improved due to low specificity rates. OCT-Abased outcome measures that were used to predict diabetes included ischemic areas around the foveal avascular zone (FAZ), FAZ circularity, mean capillary intensity, and mean vessel intensity [24]. Testosterone levels were predictable from fundus photographs, but Gerrits et al. [32] discovered in further analysis that the model was indirectly predicting gender. Model performance was affected when trained solely on male and female subgroups, implying that visual characteristics on fundus photography that are important for gender prediction are used for testosterone levels.

235

Prediction of Renal Disease and Biomarkers Renal parameters predicted by existing studies include chronic kidney disease (CKD), estimated glomerular filtration rate (eGFR), and Creatinine levels (Table 8). All three DL algorithms developed by Sabanayagam et al. [40] (fundus photography based, risk factor based, hybrid) showed good performance in internal testing (AUC > 0.9), and external testing (AUC of 0.733 on Singapore Prospective Study Program (SP2) dataset, AUC of 0.835 on Beijing Eye Study dataset). This suggests that fundus

Table 8 Predicting renal disease and biomarkers from ocular imaging Parameter

Performance

Author, year

CKD

Internal test set Seed AUC: 0.911 (95% CI, 0.886–0.936) Sen: 0.83 Spec: 0.83

Sabanayagam 2020 [40]

External test sets SP2 AUC: 0.733 (95% CI, 0.696– 0.770) Sen: 0.70 Spec: 0.70

Beijing Eye Study AUC: 0.835 (95% CI, 0.767– 0.903) Sen: 0.75 Spec: 0.75

eGFR

Predict eGFR < 90 mL/min/1.73 m2 AUC: 0.81 Acc: 0.73 Sen: 0.83 Spec: 0.62

Kang 2020 [33]

Creatinine

Internal test set MAE: 0.11 mg/dL (95% CI, 0.11–0.11) R2: 0.38 (95% CI, 0.37–0.40)

Rim 2020 [17]

External test sets Severance Gangnam hospital MAE: 0.12 mg/dL (95% CI, 0.12– 0.12) R2: 0.26 (95% CI, 0.24–0.28) Beijing eye study MAE: 0.11 mg/dL (95% CI, 0.10– 0.11) R2: 0.12 (95% CI, 0.06–0.18)

SEED study MAE: 0.17 mg/dL (95% CI, 0.16– 0.18) R2: 0.06 (95% CI, 0.04–0.09) UK biobank MAE: 0.15 mg/dL (95% CI, 0.15– 0.16) R2: 0.01 (95% CI, 0.001–0.02)

Acc, accuracy; AUC, Area under the receiver operating curve; CI, confidence interval CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; MAE, mean absolute error; SEED, Singapore Epidemiology of Eye Diseases; Sen, sensitivity; Spec, specificity; SP2, Singapore Prospective Study Program

236

photography alone has a similar predictive power to risk factor information (age, sex, ethnicity, diabetes, hypertension status), when used as inputs for CKD risk assessment. In addition, performance of the fundus photographs-only model in patient subgroups with diabetes and hypertension was comparable to the entire cohort, supporting the clinical utility of fundus photography-based algorithms as an alternative screening tool for CKD. Prior to 2020, only one DL model, based on kidney ultrasonography, was described for CKD screening by Kuo et al. [62]. This achieved an AUC of 0.904 with 0.921 specificity but 0.607 sensitivity and lacked external validation. Kang et al. [33] sought to predict early renal impairment from fundus photography, defined as eGFR < 90 ml/min/ 1.73m2, but observed poor specificity. They noted false positives in patients with retinal scarring, subretinal fluid, or optic disk swelling. Hence, clinical utility might be limited in patients with concomitant ocular pathologies that cause these presentations on fundus photography, such as ocular infection or inflammation. Features used to identify CKD or predict eGFR are unclear—saliency maps [40, 33] have highlighted changes related to retinal vasculature (dilatation of venules, rarefaction of vessels) and abnormal lesions characteristic of retinopathy (hemorrhages and exudations). Finally, we note that OCT-based algorithms to predict renal disease have not been explored in current literature. OCT, unlike fundus photography, allows imaging of the choroidal vasculature, and choroidal thinning has been associated with lower eGFR and higher microalbuminuria independent of age and other vascular risk factors [62, 64]. Whether these OCT-based metrics reflect renal microvascular damage better than standard tools should be tested in future studies.

Prediction of Hepatobiliary Disease and Biomarkers Hepatobiliary disease and biomarkers predicted by existing studies include total and direct bilirubin levels, liver cancer, cirrhosis, chronic

B. K. Betzler et al.

viral hepatitis, non-alcoholic fatty liver disease (NAFLD), cholelithiasis, and hepatic cysts (Table 9) [45, 44]. Rim et al. [17] had earlier tried unsuccessfully to predict alanine aminotransferase (ALT) and aspartate aminotransferase (AST) as continuous variables (R2  0.10) from fundus photography. Xiao et al. [44] showed that model performance on external eye/slit-lamp images was better than fundus photographs in liver cancer, cirrhosis, and chronic viral hepatitis. While excessive bilirubin accumulation causing yellowing of the sclera and conjunctiva is a common presentation in compromised liver function, retinal changes caused by these conditions that allow for prediction via fundus photography-based DL models are poorly understood. Xiao et al. [44] speculated that imperceptible retinal changes may be attributable to hyperammonemia, hypoalbuminemia, and decreased oestrogen inactivation. Elevated portal venous pressure secondary to cirrhosis or splenomegaly can remodel retinal vascular beds [54], while anemia secondary to splenic sequestration can be detected on fundus photography. Visualization techniques showed that in addition to the conjunctiva and sclera, iris morphology and color contained important predictive features [44].

Current Challenges and Areas of Future Research This chapter summarizes recent developments in predicting systemic disease from ocular images. While AI technology has performed well in research studies, there remain several challenges to be appreciated as AI becomes more integral to medical practice. Firstly, barriers of access to ophthalmic imaging datasets can be reduced— these include issues of cost, time, usability, and quality [47]. Labeling processes for publicly available datasets are often poorly defined; assurance of labeling accuracy is paramount because the standards used for labeling of ground truths have implications on any AI model trained on the dataset. Secondly, using ocular imaging to predict systemic disease would require collaborative efforts across departments. This might pose

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

237

Table 9 Predicting hepatobiliary disease and biomarkers from ocular imaging Parameter

Performance

Author, year

Total bilirubin

Predicting Total Bilirubin 3.4–17.1 lmol/L AUC: 0.764 Acc: 0.700

Zhang 2020 [45]

Direct bilirubin

Predicting Direct Bilirubin 0–3.4 lmol/L AUC: 0.703 Acc: 0.650

Zhang 2020 [45]

Hepatobiliary diseases (general)

Slit lamp model AUC: 0.74 (95% CI, 0.71– 0.76) Sen: 0.64 (95% CI, 0.60– 0.68) Spec: 0.73 (95% CI, 0.69– 0.76) F1: 0.68

Fundus model AUC: 0.68 (95% CI, 0.65– 0.71) Sen: 0.68 (95% CI, 0.63– 0.72) Spec: 0.62 (95% CI, 0.57– 0.66) F1: 0.65

Xiao 2021 [44]

Liver cancer

Slit lamp model AUC: 0.93 (95% CI, 0.91– 0.94) Sen: 0.89 (95% CI, 0.79– 0.99) Spec: 0.89 (95% CI, 0.87– 0.91) F1: 0.89

Fundus model AUC: 0.84 (95% CI, 0.81– 0.86) Sen: 0.73 (95% CI, 0.56– 0.90) Spec: 0.80 (95% CI, 0.77– 0.83) F1: 0.76

Xiao 2021 [44]

Liver cirrhosis

Slit lamp model AUC: 0.90 (95% CI, 0.88– 0.91) Sen: 0.78 (95% CI, 0.66– 0.90) Spec: 0.91 (95% CI, 0.89– 0.92) F1: 0.84

Fundus model AUC: 0.83 (95% CI, 0.81– 0.86) Sen: 0.82 (95% CI, 0.70– 0.95) Spec: 0.64 (95% CI, 0.60– 0.67) F1: 0.72

Xiao 2021 [44]

Chronic viral hepatitis

Slit lamp model AUC: 0.69 (95% CI, 0.66– 0.71) Sen: 0.55 (95% CI, 0.45– 0.65) Spec: 0.78 (95% CI, 0.76– 0.81) F1: 0.65

Fundus model AUC: 0.62 (95% CI, 0.58– 0.65) Sen: 0.59 (95% CI, 0.49– 0.70) Spec: 0.63 (95% CI, 0.60– 0.67) F1: 0.61

Xiao 2021 [44]

NAFLD

Slit lamp model AUC: 0.63 (95% CI, 0.60– 0.66) Sen: 0.69 (95% CI, 0.64– 0.74) Spec: 0.53 (95% CI, 0.50– 0.57) F1: 0.60

Fundus model AUC: 0.70 (95% CI, 0.67– 0.73) Sen: 0.62 (95% CI, 0.55– 0.69) Spec: 0.74 (95% CI, 0.71– 0.78) F1: 0.68

Xiao 2021 [44]

Cholelithiasis

Slit lamp model AUC: 0.58 (95% CI, 0.55– 0.61) Sen: 0.57 (95% CI, 0.46– 0.68)

Fundus model AUC: 0.68 (95% CI, 0.65– 0.71) Sen: 0.68 (95% CI, 0.57– 0.80)

Xiao 2021 [44]

(continued)

238

B. K. Betzler et al.

Table 9 (continued) Parameter

Hepatic cyst

Performance

Author, year

Spec: 0.58 (95% CI, 0.55– 0.61) F1: 0.57

Spec: 0.62 (95% CI, 0.59– 0.66) F1: 0.65

Slit lamp model AUC: 0.66 (95% CI, 0.63– 0.68) Sen: 0.68 (95% CI, 0.58– 0.79) Spec: 0.57 (95% CI, 0.54– 0.60) F1: 0.62

Fundus model AUC: 0.69 (95% CI, 0.65– 0.72) Sen: 0.57 (95% CI, 0.43– 0.71) Spec: 0.72 (95% CI, 0.69– 0.75) F1: 0.64

Xiao 2021 [44]

Acc, accuracy; AUC, Area under the receiver operating curve; CI, confidence interval; NAFLD, non-alcoholic fatty liver disease; Sen, sensitivity; Spec, specificity

difficulties as systemic parameters are not always required for management in ophthalmic clinics (for instance, eGFR or bilirubin levels), and vice versa. Hence, input images and target variables may need to be collected separately and deliberately [56]. Thirdly, high-quality ocular images may be difficult to acquire in patients with small pupils. Such patients may require pupil dilation with topical pharmaceuticals. Fourthly, it may sometimes be necessary to acquire datasets from different local and international centers for training or external validation purposes. State privacy and data regulatory rules need to be respected, the process of which is timeconsuming and cost-incurring. Finally, the potential for bias or error must be respected. Algorithmic outcomes reflect the data used to train them; they can only be as reliable (but also as neutral) as the data they are based on [58]. Projection of biases inherent in the training sets by AI systems is a concern for medical ethics [60], and ensuring generalizability across different geographical and ethnic groups is essential to avoid inadvertent, subtle discrimination in healthcare delivery [62]. Regarding areas of future research, the field of ocular imaging has untapped potential in predicting other systemic parameters. Several studies mentioned in this chapter attempted predictions of biomarkers in addition to those reported, albeit with varying (and often poorer) results [45–62]. For instance, Rim et al. [17] performed analysis on 47 biomarkers in total,

although only 10 were eventually deemed “predictable” (Table 1). In particular, the fields of predicting neurological and hepatobiliary disease from ocular imaging are relatively nascent. Currently, applications of DL in neurological disease are more prevalent in conventional neuroimaging methods like MRI. Much of the ongoing work bridging neurological disease and retinal imaging involves OCT [62] (Table 5), although vascular features on fundus photography have shown meaningful associations with cognitive decline. The models developed by Xiao et al. [44] were the first to establish qualitative associations between ocular features, liver cancer, and cirrhosis, and current literature will benefit from future studies which reaffirm their findings. Aside from fundus photography and OCT, the potential of slit-lamp/external eye imaging in predictive AI models is gaining recognition. Thyroid eye disease can manifest with specific ocular signs such as lid retraction, conjunctival injection, chemosis, periorbital edema, and proptosis [62]. Elevated cholesterol levels and atherosclerosis are associated with xanthelasma formation [62]. Obstructive sleep apnoea is associated with floppy eyelid syndrome [62], while ocular manifestations of gout and several autoimmune/connective tissue diseases include uveitis, dry eye syndrome, and cataracts, all of which can be visualized externally [62]. Considering the widespread availability of slit-lamp imaging and fundus photography in ophthalmic clinical practice, AI systems built on

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging

two different ocular imaging methods would provide alternatives and improve adaptability. Ultimately, major benefits of ocular-imagingbased AI systems may lie in screening programs in primary care settings or in communities. Today, teleophthalmology screening is increasingly performed in non-specialist settings, including primary care and retail stores [64], without an ophthalmologist present. For example, AI-based teleophthalmology screening programs of diabetic retinopathy have been adopted in the United States [52], United Kingdom [54], Singapore [56], India [58], and Zambia [60]. In principle, addition of various predicting models for systemic biomarkers to current software could enable low-cost, non-invasive screening for multiple diseases in the general population.

Conclusions The potential of ocular AI systems for clinical adoption has been established, and further efforts are underway to explore other systemic risk factors and biomarkers that could be predicted from the eye. To date, the clinical employment of ocular AI models for systemic disease is limited, but it has great capacity to improve the efficiency of clinical operations in coming decades, if the technology is carefully designed, operated, and monitored under the supervision of clinicians. Prospective studies are needed to evaluate realworld reliability, efficacy, and cost-effectiveness, and to gain acceptance from various stakeholders. Collaborative efforts are needed to ensure the best medical technology available is incorporated into practice for the benefit of patients.

References 1. Hubel DH, Wiesel TN. Receptive fields and functional architecture of monkey striate cortex. J Physiol. 1968;195(1):215–43. 2. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25:1097–105.

239

3. Zeiler MD, Fergus R, editors. Visualizing and understanding convolutional networks. European conference on computer vision. Springer;2014. 4. Schmidt-Erfurth U, Sadeghipour A, Gerendas BS, Waldstein SM, Bogunovic H. Artificial intelligence in retina. Prog Retin Eye Res. 2018;67:1–29. 5. Ting DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103(2):167–75. 6. Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, et al. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell. 2018;172(5):1122–31 e9. 7. Saba L, Biswas M, Kuppili V, Cuadrado Godia E, Suri HS, Edla DR, et al. The present and future of deep learning in radiology. Eur J Radiol. 2019;114:14–24. 8. McBee MP, Awan OA, Colucci AT, Ghobadi CW, Kadom N, Kansagra AP, et al. Deep learning in radiology. Acad Radiol. 2018;25(11):1472–80. 9. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W. Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol Ther (Heidelb). 2020;10(3):365–86. 10. Jiang Y, Yang M, Wang S, Li X, Sun Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun (Lond). 2020;40 (4):154–66. 11. Wang S, Yang DM, Rong R, Zhan X, Xiao G. Pathology image analysis using segmentation deep learning algorithms. Am J Pathol. 2019;189 (9):1686–98. 12. Abramoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, et al. Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci. 2016;57(13):5200–6. 13. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–10. 14. Ting DSW, Cheung CY, Lim G, Tan GSW, Quang ND, Gan A, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211–23. 15. Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, Bressler NM. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol. 2017;135(11):1170–6. 16. Grassmann F, Mengelkamp J, Brandl C, Harsch S, Zimmermann ME, Linkohr B, et al. A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular

240

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

B. K. Betzler et al. degeneration from color fundus photography. Ophthalmology. 2018;125(9):1410–20. Rim TH, Lee G, Kim Y, Tham YC, Lee CJ, Baik SJ, et al. Prediction of systemic biomarkers from retinal photographs: development and validation of deeplearning algorithms. Lancet Digit Health. 2020;2(10): e526–36. Varadarajan AV, Poplin R, Blumer K, Angermueller C, Ledsam J, Chopra R, et al. Deep learning for predicting refractive error from retinal fundus images. Invest Ophthalmol Vis Sci. 2018;59(7):2861–8. Cheung CY, Ikram MK, Klein R, Wong TY. The clinical implications of recent studies on the structure and function of the retinal microvasculature in diabetes. Diabetologia. 2015;58(5):871–85. Cheung CY, Ikram MK, Sabanayagam C, Wong TY. Retinal microvasculature as a model to study the manifestations of hypertension. Hypertension. 2012;60(5):1094–103. Courtie E, Veenith T, Logan A, Denniston AK, Blanch RJ. Retinal blood flow in critical illness and systemic disease: a review. Ann Intensive Care. 2020;10(1):152. London A, Benhar I, Schwartz M. The retina as a window to the brain-from eye research to CNS disorders. Nat Rev Neurol. 2013;9(1):44–53. Chiquita S, Rodrigues-Neves AC, Baptista FI, Carecho R, Moreira PI, Castelo-Branco M, et al. The retina as a window or mirror of the brain changes detected in Alzheimer’s disease: critical aspects to unravel. Mol Neurobiol. 2019;56(8):5416–35. Aslam TM, Hoyle DC, Puri V, Bento G. Differentiation of diabetic status using statistical and machine learning techniques on optical coherence tomography angiography images. Transl Vis Sci Technol. 2020;9 (4):2. Babenko B, Mitani A, Traynis I, Kitade N, Singh P, Maa A, et al. Detecting hidden signs of diabetes in external eye photographs. arXiv preprint arXiv:201111732. 2020. Benson J, Estrada T, Burge M, Soliz P. Diabetic peripheral neuropathy risk assessment using digital fundus photographs and machine learning. Annu Int Conf IEEE Eng Med Biol Soc. 2020;2020:1988–91. Cavaliere C, Vilades E, Alonso-Rodríguez MC, Rodrigo MJ, Pablo LE, Miguel JM, et al. Computer-aided diagnosis of multiple sclerosis using a support vector machine and optical coherence tomography features. Sensors (Basel). 2019;19(23). Chang J, Ko A, Park SM, Choi S, Kim K, Kim SM, et al. Association of cardiovascular mortality and deep learning-funduscopic atherosclerosis score derived from retinal fundus images. Am J Ophthalmol. 2020;217:121–30. Chen YM, Miaou SG, Bian H. Examining palpebral conjunctiva for anemia assessment with image processing methods. Comput Methods Programs Biomed. 2016;137:125–35. Cheung CY, Xu D, Cheng CY, Sabanayagam C, Tham YC, Yu M, et al. A deep-learning system for

31.

32.

33.

34.

35.

36.

37.

38.

39. 40.

41.

42.

43.

the assessment of cardiovascular disease risk via the measurement of retinal-vessel calibre. Nat Biomed Eng. 2020. Dai H, Alsalhe TA, Chalghaf N, Riccò M, Bragazzi NL, Wu J. The global burden of disease attributable to high body mass index in 195 countries and territories, 1990–2017: an analysis of the global burden of disease study. PLoS Med. 2020;17(7): e1003198. Gerrits N, Elen B, Craenendonck TV, Triantafyllidou D, Petropoulos IN, Malik RA, et al. Age and sex affect deep learning prediction of cardiometabolic risk factors from retinal images. Sci Rep. 2020;10 (1):9432. Kang EY, Hsieh YT, Li CH, Huang YJ, Kuo CF, Kang JH, et al. Deep learning-based detection of early renal function impairment using retinal fundus images: model development and validation. JMIR Med Inform. 2020;8(11): e23472. Kim YD, Noh KJ, Byun SJ, Lee S, Kim T, Sunwoo L, et al. Effects of hypertension, diabetes, and smoking on age and sex prediction from retinal fundus images. Sci Rep. 2020;10(1):4623. Mitani A, Huang A, Venugopalan S, Corrado GS, Peng L, Webster DR, et al. Detection of anaemia from retinal fundus images via deep learning. Nat Biomed Eng. 2020;4(1):18–27. Nunes A, Silva G, Duque C, Januário C, Santana I, Ambrósio AF, et al. Retinal texture biomarkers may help to discriminate between Alzheimer’s, Parkinson’s, and healthy controls. PLoS ONE. 2019;14(6): e0218826. Pérez Del Palomar A, Cegoñino J, Montolío A, Orduna E, Vilades E, Sebastián B, et al. Swept source optical coherence tomography to early detect multiple sclerosis disease. The use of machine learning techniques. PLoS One. 2019;14(5):e0216410. Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2 (3):158–64. Gulli A, Pal S. Deep learning with Keras. Packt Publishing Ltd; 2017. Sabanayagam C, Xu D, Ting DSW, Nusinovici S, Banu R, Hamzah H, et al. A deep learning algorithm to detect chronic kidney disease from retinal photographs in community-based populations. Lancet Digit Health. 2020;2(6):e295–302. Samant P, Agarwal R. Machine learning techniques for medical diagnosis of diabetes using iris images. Comput Methods Programs Biomed. 2018;157:121–8. Son J, Shin JY, Chun EJ, Jung K-H, Park KH, Park SJ. Predicting high coronary artery calcium score from retinal fundus images with deep learning algorithms. Transl Vis Sci Technol. 2020;9(2):28. Vaghefi E, Yang S, Hill S, Humphrey G, Walker N, Squirrell D. Detection of smoking status from retinal images; a Convolutional Neural Network study. Sci Rep. 2019;9(1):7180.

Artificial Intelligence in Predicting Systemic Disease from Ocular Imaging 44. Xiao W, Huang X, Wang JH, Lin DR, Zhu Y, Chen C, et al. Screening and identifying hepatobiliary diseases through deep learning using ocular images: a prospective, multicentre study. Lancet Digit Health. 2021;3(2):e88–97. 45. Zhang L, Yuan M, An Z, Zhao X, Wu H, Li H, et al. Prediction of hypertension, hyperglycemia and dyslipidemia from retinal fundus photographs via deep learning: a cross-sectional study of chronic diseases in central China. PLoS ONE. 2020;15(5): e0233166. 46. Géron A. Hands-on machine learning with Scikitlearn, keras, and tensorflow: concepts, tools, and techniques to build intelligent systems. O'Reilly Media;2019. 47. Khan SM, Liu X, Nath S, Korot E, Faes L, Wagner SK, et al. A global review of publicly available datasets for ophthalmological imaging: barriers to access, usability, and generalisability. Lancet Digit Health. 2021;3(1):e51–66. 48. Fusek R. MRL eye dataset. Accessed 28 May 2019. 49. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, et al., editors. Tensorflow: a system for largescale machine learning. In: 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16). 2016. 50. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. Pytorch: an imperative style, highperformance deep learning library. arXiv preprint arXiv:191201703. 2019. 51. He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. 52. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. 2014. 53. Owsley C, McGwin G Jr, Lee DJ, Lam BL, Friedman DS, Gower EW, et al. Diabetes eye screening in urban settings serving minority populations: detection of diabetic retinopathy and other ocular findings using telemedicine. JAMA Ophthalmol. 2015;133 (2):174–81. 54. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al., editors. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. 55. Scanlon PH. The english national screening programme for diabetic retinopathy 2003–2016. Acta Diabetol. 2017;54(6):515–25. 56. Hu J, Shen L, Sun G, editors. Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. 57. Xie Y, Nguyen QD, Hamzah H, Lim G, Bellemo V, Gunasekeran DV, et al. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health. 2020;2(5): e240–9. 58. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. A survey of methods for

59.

60.

61.

62. 63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

241

explaining black box models. ACM Comput Surv (CSUR). 2018;51(5):1–42. Das T, Pappuru RR. Telemedicine in diabetic retinopathy: access to rural India. Indian J Ophthalmol. 2016;64(1):84–6. Smilkov D, Thorat N, Kim B, Viégas F, Wattenberg M. SmoothGrad: removing noise by adding noise. 2017. Bellemo V, Lim ZW, Lim G, Nguyen QD, Xie Y, Yip MYT, et al. Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. Lancet Digit Health. 2019;1(1):e35–44. Iwakiri Y. Pathophysiology of portal hypertension. Clin Liver Dis. 2014;18(2):281–91. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vision. 2019;128(2):336–59. Xu J, Xue K, Zhang K. Current status and future trends of clinical diagnoses via image-based deep learning. Theranostics. 2019;9(25):7556–65. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: the all convolutional net. 2014. Morley J, Machado CCV, Burr C, Cowls J, Joshi I, Taddeo M, et al. The ethics of AI in health care: a mapping review. Soc Sci Med. 2020;260: 113172. Char DS, Shah NH, Magnus D. Implementing machine learning in health care—addressing ethical challenges. N Engl J Med. 2018;378(11):981–3. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366 (6464):447–53. Sharon Y, Schlesinger N. Beyond joints: a review of ocular abnormalities in gout and hyperuricemia. Curr Rheumatol Rep. 2016;18(6):37. Abramoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018;1:39. Detrano R, Guerci AD, Carr JJ, Bild DE, Burke G, Folsom AR, et al. Coronary calcium as a predictor of coronary events in four racial or ethnic groups. N Engl J Med. 2008;358(13):1336–45. Cheung CY, Ikram MK, Chen C, Wong TY. Imaging retina to study dementia and stroke. Prog Retin Eye Res. 2017;57:89–107. Chua J, Chin CWL, Hong J, Chee ML, Le TT, Ting DSW, et al. Impact of hypertension on retinal capillary microvasculature using optical coherence tomographic angiography. J Hypertens. 2019;37 (3):572–80. Cheung CY, Ong YT, Ikram MK, Ong SY, Li X, Hilal S, et al. Microvascular network alterations in the retina of patients with Alzheimer’s disease. Alzheimers Dement. 2014;10(2):135–42. Mulè G, Vadalà M, La Blasca T, Gaetani R, Virone G, Guarneri M, et al. Association between

242

76.

77.

78.

79.

80.

81.

82.

B. K. Betzler et al. early-stage chronic kidney disease and reduced choroidal thickness in essential hypertensive patients. Hypertens Res. 2019;42(7):990–1000. Frost S, Kanagasingam Y, Sohrabi H, Vignarajan J, Bourgeat P, Salvado O, et al. Retinal vascular biomarkers for early detection and monitoring of Alzheimer’s disease. Transl Psychiatry. 2013;3(2): e233. Santos M, Hofmann RJ. Ocular manifestations of obstructive sleep apnea. J Clin Sleep Med. 2017;13 (11):1345–8. Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, Menotti A, De Backer G, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987–1003. Bhaskaran K, Dos-Santos-Silva I, Leon DA, Douglas IJ, Smeeth L. Association of BMI with overall and cause-specific mortality: a population-based cohort study of 3
6 million adults in the UK. Lancet Diabetes Endocrinol. 2018;6(12):944–53. Wagner SK, Fu DJ, Faes L, Liu X, Huemer J, Khalid H, et al. Insights into systemic disease through retinal imaging-based oculomics. Transl Vis Sci Technol. 2020;9(2):6. Rim TH, Lee CJ, Tham YC, Cheung N, Yu M, Lee G, et al. Deep-learning-based cardiovascular risk stratification using coronary artery calcium scores predicted from retinal photographs. Lancet Digit Health. 2021;3(5):e306–16. Petzold A, Balcer LJ, Calabresi PA, Costello F, Frohman TC, Frohman EM, et al. Retinal layer segmentation in multiple sclerosis: a systematic review and meta-analysis. Lancet Neurol. 2017;16 (10):797–812.

83. Kuo CC, Chang CM, Liu KT, Lin WK, Chiang HY, Chung CW, et al. Automation of the kidney function prediction and classification through ultrasoundbased kidney imaging using deep learning. NPJ Digit Med. 2019;2:29. 84. Chang HC, Sung CW, Lin MH. Serum lipids and risk of atherosclerosis in xanthelasma palpebrarum: a systematic review and meta-analysis. J Am Acad Dermatol. 2020;82(3):596–605. 85. Flegal KM, Kit BK, Orpana H, Graubard BI. Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA. 2013;309(1):71–82. 86. Wong TY, Kamineni A, Klein R, Sharrett AR, Klein BE, Siscovick DS, et al. Quantitative retinal venular caliber and risk of cardiovascular disease in older persons: the cardiovascular health study. Arch Intern Med. 2006;166(21):2388–94. 87. Seidelmann SB, Claggett B, Bravo PE, Gupta A, Farhad H, Klein BE, et al. Retinal vessel calibers in predicting long-term cardiovascular outcomes: the atherosclerosis risk in communities study. Circulation. 2016;134(18):1328–38. 88. Keane PA, Sadda SR. Retinal imaging in the twentyfirst century: state of the art and future directions. Ophthalmology. 2014;121(12):2489–500. 89. Cuadros J, Bresnick G. EyePACS: an adaptable telemedicine system for diabetic retinopathy screening. J Diabetes Sci Technol. 2009;3(3):509–16. 90. Dutton JJ. Anatomic considerations in thyroid eye disease. Ophthalmic Plast Reconstr Surg. 2018;34(4S Suppl 1):S7–12.

Natural Language Processing (NLP) in AI J. K. Wang, S. K. Wang, E. B. Lee, and R. T. Chang

Abstract

Natural language processing (NLP) is a growing field of artificial intelligence (AI) that combines machine learning and linguistics to enable computers to understand and generate human language. Applications of NLP range from voice assistants like Apple’s Siri and Amazon’s Alexa to text summarization, machine translation, and spam filtering. NLP is particularly challenging given the complexity and hierarchical nature of human lan at the most basic level, individual words can take guage, which can include subtle meanings. Fortunately, rapidly improving computing power, new tools and avenues of mass data collection, and recent improvements in NLP algorithms (large language models) have all made it possible to train computers to understand human language more efficiently and more accurately.

Introduction Natural language processing (NLP) is a growing field of artificial intelligence (AI) that combines machine learning and linguistics to enable com-

J. K. Wang
S. K. Wang
E. B. Lee
R. T. Chang (&) Ophthalmology Department, Stanford University, Stanford, USA e-mail: [email protected]

puters to understand and generate human language. Applications of NLP range from voice assistants like Apple’s Siri and Amazon’s Alexa to text summarization, machine translation, and spam filtering. NLP is particularly challenging given the complexity and hierarchical nature of human language, which can include subtle meanings. Even at the most basic level, individual words can take on multiple meanings. Moving up in complexity, human language can convey the same semantic meaning using different combinations of words. At the highest level, contextual relationships become critical in understanding how multiple sentences work together to express complex messages. Ultimately, successful implementation of NLP requires understanding both syntax (relationship among elements of text) and semantics (meaning being conveyed). Fortunately, rapidly improving computing power, new tools and avenues of mass data collection, and recent improvements in NLP algorithms (large language models) have all made it possible to train computers to understand human language more efficiently and more accurately.

Overview of Methods The foundation of NLP is converting human language into a format amenable to the logic and mathematics of computer algorithms. The earliest NLP algorithms were based on complex sets of hand-written rules. Decision trees, for instance,

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_17

243

244

were used to classify words within a sentence into their parts of speech [1]. Eventually, statistical models (also loosely referred to as machine learning models) came into favor, applying probabilistic decision-making to NLP tasks by leveraging large corpora of annotated text. A canonical example is the application of the Naïve Bayes model to spam filtering. Particular words have distinct probabilities of occurring in spam versus legitimate emails (e.g. “Viagra” is much more likely to appear in spam versus not). By learning from many examples of hand-labeled emails (spam versus legitimate), the algorithm can estimate the probability that a given word occurs in spam versus legitimate email. Finally, probabilities for individual words can then be aggregated to estimate the likelihood that a given email (a collection of words) is spam or not. Most recently, deep learning methods, first popularized for computer vision, have now been optimized for NLP syntax and text ingestion. The field has made rapid progress in foreign language translation and speech to text. Deep learning (DL), a subset of machine learning, uses complex statistical models known as neural networks to achieve high performance across a variety of classification or pattern recognition tasks. DL has been made possible over the past decade due to advancements in graphical processing unit (GPU) computing—the ubiquitous video card, and an ever-increasing abundance of digital data stored in the cloud, accessible anywhere and everywhere via the Internet (5G speeds). Deep learning methods are known to make a trade-off between increased accuracy and performance for decreased interpretability (insight into why the model makes certain decisions or predictions). While a lengthier discussion of deep learning is beyond the scope of this review, two main topics of deep learning in the context of NLP are worth mentioning. First, deep learning NLP methods are rooted in the concept of word embeddings, which are computer representations of individual words in the format of multi-dimensional real number vectors. Intuitively, word embeddings encode the meaning of words such that words that are closer in the embedding space are expected to be similar in meaning. Second, deep

J. K. Wang et al.

learning advancements for any application, NLP or otherwise, involve optimizing the architecture of neural network algorithms. Commonly used neural network architectures for NLP include recurrent neural networks (RNN), long shot-term memory (LSTM) networks, and transformer networks. The major breakthrough deep architecture is bidirectional encoder representation transformer network (BERT), designed by Google Research, which has made major strides in question answering and language inference [2].

NLP and Health Care Applications of NLP in health care range from speech recognition for virtual scribing to natural language understanding to summarize biomedical documents to word generation to provide pointof-care clinical decision support. While new techniques in NLP are often domain-agnostic, healthcare-specific NLP advancements are occurring in both academia and industry. In 2019, researchers at Korea University designed BioBERT, a neural network architecture that repurposed Google’s BERT model for the specific tasks of understanding biomedical literature, including biomedical named entity recognition, biomedical relation extraction, and biomedical question answering [3]. In industry, dozens of NLPpowered startups have sought to tackle clinical challenges including AI-assisted medical scribing (DeepScribe, Suki, Robin Healthcare) and revenue cycle management (AKASA). The United States Food and Drug Administration (FDA) evaluates and approves AI-based clinical technologies under the category of “software as a medical device” (SaMD). FDA-approved AI algorithms to date are almost entirely based on computer vision techniques that analyze image data (e.g. radiology, pathology, cardiology, and ophthalmology) to automate physician workflows or even make diagnostic or prognostic predictions [4]. Indeed, medical imaging data is oftentimes cleaner and more structured than medical text (a bit more objective as long as image acquisition is standardized), and thus computer vision algorithms are more mature than NLP algorithms (which have to deal with multiple

Natural Language Processing (NLP) in AI

meanings and subjective human behavior). Despite these limitations, NLP has the potential to be a fast follower of computer vision in achieving clinical adoption. Very recent breakthroughs in generative pre-trained transformer networks with reinforcement learning (Chat-GPT3 and 4) and diffusion models (DALL-E) released by OpenAI have taken the world by storm in the areas of text and image creation.

NLP Applications in Ophthalmology (1) Documentation assistance A key area in which NLP can contribute to ophthalmology is in assisting with documentation. This could range from context-driven autocomplete suggestions while writing EHR notes to fully automated virtual scribing of patient encounters. Given that ophthalmologists have higher patient volumes than most other specialties, many ophthalmologists currently employ scribes who can help ease their clerical burden and allow them to focus on the patient in front of them [5]. Although not yet widely implemented, automated scribing technologies using NLP such as those developed by DeepScribe, Autoscribe, Tali AI, Suki AI, and Nuance DAX (acquired Saykara), could potentially scale these services. ScribeAmerica already leverages human scribes virtually to help train NLP algorithms by annotating recorded doctor-patient visits. By ambiently listening to many visits, interpreting the content discussed, and providing the semantic output, at scale, these technologies may one day be able to replace the services normally performed by inperson scribes. Ultimately, this documentation assistance could free up more time for patient care. In addition to helping generate notes, NLP could be used to summarize existing clinical notes and test results or even aggregate information in the published literature. Complex patients frequently have an overwhelming amount of clinical data and harnessing that data efficiently for precision medicine can be challenging. A recent attempt to use NLP to extract descriptors of

245

microbial keratitis from the EHR was published by Woodward MA et al., using 400 encounters of 100 patients, with an internal validation of 100 and external validation of 59 [6]. The gold standard descriptors around ulcer centrality, depth, and corneal thinning ranged 15–80% documentation. So, while the NLP algorithm had up to 80% sensitivity, the accuracy of this approach has been hindered by inconsistent charting and lack of standardization. This can be very disease-specific and thus initial NLP algorithms likely will be looking for associations currently not noticed by clinicians. A classic association discovered by human pattern recognition was tamsulosin and floppy iris syndrome, but there may be many others found through NLP EHR chart review using unsupervised learning techniques. (2) Clinical knowledge synthesis One challenge in not only ophthalmology but also all medical specialties is the exponential growth of published medical knowledge [7]. Due to the sheer volume of new journal articles each day, it is not realistic for ophthalmologists to consistently “keep up with the literature” and integrate the latest data into their practices. Conferences and continuing medical education (CME) programs attempt to address this by highlighting the most pertinent updates, but many ophthalmologists still practice based on the instruction they received in residency and fellowship, even decades later. NLP may offer another resource to help providers stay up to date if it can be used to synthesize new clinical knowledge. While a human cannot process all the new papers in a field, a computer might be able to and distill them down to more digestible bits of information. Early precursors to this idea include Semantic Scholar, an NLP-based search engine which provides a one-sentence summary of journal articles [8]. By compressing the literature into a format that can be quickly interpreted, Semantic Scholar and similar tools may allow ophthalmologists to sort through new papers more efficiently and identify “just in time” information pertinent to their practice.

246

(3) Clinical decision support (CDS) Clinical decision support refers to tools that aid in clinical decision-making by providing recommendations based on individual patient data [9]. For situations in which these data are captured in the EHR as text such as clinical notes, NLP may be a powerful means to automate this process. For instance, when following a set of guidelines, NLP could be used to extract relevant details from a physician’s assessment and plan and thus be able to use that information to suggest management options. Interpretation of management plans with NLP may also be helpful in detecting and preventing medical errors, from incorrect orders to unintentional lapses in care. One example of this latter application is Ferrum Health, a company that uses NLP to read unstructured medical notes and identify patients needing to return for follow-up tests. Similar to how prescribing two interacting drugs in an EHR might trigger a warning, the recognition of certain phrases in a given context by NLP could be used to notify ophthalmologists to double-check their work. In designing such a system, the potential advantages of improved care and fewer errors will need to be balanced with the risk of alert fatigue. However, if implemented well, a system with NLP-compatible clinical decision support may ultimately be better for patients. (4) Image interpretation with words Ophthalmology is a highly visual specialty with increasing reliance on multimodal imaging. As such, the integration of deep learning into ophthalmology has been primarily characterized by computer vision as mentioned earlier. While images such as fundus photos or optical coherence tomography (OCT) scans of the retina can convey rich amounts of information, adequately describing them in words remains a challenge. Indeed, to document their exam findings, many ophthalmologists prefer a rough schematic illustrating what they saw over a free-text description. Considering the wealth of computer vision data already available in the field, another potential application for NLP may be to convert ophthalmic

J. K. Wang et al.

images into readily interpretable words. Although computer vision can identify image features of diagnostic value and use them to influence clinical management, the language of computer vision is not one that patients or even many providers can easily comprehend. The pairing of computer vision and NLP in ophthalmology may help address this problem by allowing for the generation of accurate and automated descriptions from images of the eye. This would standardize the documentation and interpretation of ophthalmic imaging data and potentially make the management of certain eye diseases more objective. One example of data labeling software is Centaur Labs. If used in conjunction with telemedicine, automated interpretation of images with explainable documentation might further enable eye screening and gain trust among users. This combination could be especially impactful in low-resource environments, possibly allowing patients with these conditions to undergo most of their monitoring in primary eyecare settings. Challenges for Ophthalmology

NLP

Adoption

in

(1) Interoperability Despite these promising applications, the adoption of NLP technologies into clinical practice faces several significant challenges. Chief among these is the issue of interoperability, or the broad applicability of NLP algorithms among different software, closed computing systems, and practice settings. Factors contributing to the challenge of overcoming interoperability issues can be generally divided into inadequate representation in the training data and unaddressed biases in text. (1:1)

Limited Training Data

For NLP algorithms to return accurate and useful outputs in new clinical settings, they must be trained to recognize and process the distinct documentation idiosyncrasies present in any practice across a nation, or the world. While national registries such as the American Academy of Ophthalmology’s IRIS (Intelligent Research In Sight) Registry have been designed

Natural Language Processing (NLP) in AI

to provide multi-site data, these resources are still in the early stages of ingestion, data mapping, harmonization of data, and organization into useful real-world evidence [10]. Participants in these registries must represent multiple geographic populations to capture the diverse demographic characteristics of the general population. Since these large-scale studies often overrepresent well-resourced practices, special efforts must also be made to include poor, rural, or isolated clinical environments. Documentation of rare or complex medical cases must also be appropriately represented in the training data so NLP algorithms can accurately process these notes when they are present. (1:2)

Bias

Interoperability is also affected by the biases intrinsic to individual ophthalmologists, regional practices, and even databases. Like all humans, ophthalmologists are influenced by cultural biases from their community, education biases based on their training background, implicit biases present in all individuals, and external driving factors such as limited time, malpractice risk, and financial gain. These biases can be subtly expressed in provider documentation due to the textual nature of notes either consciously or unconsciously. The purpose and design of documentation can also introduce bias. While EHRs may be designed to optimize clinical practice at the point of care, administrative claims documentation is designed to facilitate insurance reimbursement. These systems will organize, collect, and report data in the formats best suited for their purpose. Taking into account these factors is essential to recognizing how bias manifests in documentation, and must be considered when using all this data to train NLP algorithms for specific use cases. (1:3)

Leading Largest USA EHR Dataset: The IRIS Registry Limitations

The IRIS Registry is a national clinical repository started by the AAO but is now administered by Verana Health. As of 2020, approximately 18,000 physicians in ophthalmology practices

247

have contracted with the registry, which has compiled the EHRs of 60 million patients across the United States, representing 349 million visits [11]. Data from certain structured fields are automatically extracted, and free texts from notes are also available. Because the IRIS registry was a raw data ingestion for MIPS quality score reporting only, there were a lot of limitations to the database, and matching information across 70 different EHR systems all with different structured and unstructured fields is still a huge challenge given the diversity of documentation across 18,000 physicians. Imaging data, often stored on a separate PACS system, is not included currently in the IRIS registry, which is unfortunate since images can be a more objective outcome metric endpoint. Thus, NLP systems are still a bespoke process for specific tasks processing clinical free text and generating a structured output [12]. Standards such as a common data model for EHR export would greatly improve the data curation and pre-processing steps before training an NLP algorithm. (1:4)

Data Sharing

One of the major challenges in creating large databases for better generalizability of algorithms trained on the data is the issue of data sharing by hospital systems. The academic and business potential of NLP algorithms inevitably invites competition, and advancements in algorithms or data collection may be carefully protected by individual groups. Also real world data may be collected differently among sites, which could embed human bias into the training data. In the US, there is a strong disincentive to share when the penalties for losing protected health information (PHI) are steep. While the drive of competition is a powerful motivator for innovation, it can also act as a barrier that disincentives collaboration. One way researchers can mitigate this challenge is the utilization of decentralized machine learning techniques, such as federated learning. In federated learning, machine learning algorithms are trained on multiple servers that contain local datasets, without copying them. In essence, the algorithms can be trained across

248

J. K. Wang et al.

institutions to build robust models (as long as they use the same ground truth labeling definitions) without needing to share the data itself, thereby avoiding issues with data privacy or data access rights. This and other privacy-preserving encryption or decentralization efforts such as blockchain will be essential for developing and disseminating the diverse, representative datasets required to train clinically useful NLP algorithms. (2) Regulation

syntax, semantics, composition, and diction, and is currently not well enough represented in any benchmarking tool to accurately measure how an NLP algorithm trained on ophthalmology data might perform. Before NLP algorithms can be objectively evaluated against one another and approved for clinical deployment, established benchmarking tools must be designed to ensure performance is adequately measurable and comparable across multiple algorithms. (2:2)

As DL and NLP algorithms improve, clinical implementation will face the challenge of regulation. When used for medical purposes without being part of a hardware medical device, these algorithms are classified as “Software as Medical Devices (SaMD)” [13]. The International Medical Device Regulators Forum (IMDRF) is an international group of volunteers that organized a working group, chaired by the FDA, to guide innovation and access to safe and effective SaMD technologies. Careful adherence to recommendations and regulations set by the IMDRF will be required for SaMDs, including NLP algorithms, before being integrated into clinical practice. The US FDA focuses on safety and risk as an emphasis, along with specific definitions for clinical evaluation [14, 15]. (2:1)

Benchmarking

When a new technology system is introduced, its effectiveness is tested by measuring its performance on a known workload and comparing it to the performance of systems tested on the same workload. This process, known as benchmarking, is still in active development regarding the assessment of NLP algorithms [16]. While the human ability to understand language is adaptable and applicable in novel settings, most NLP algorithms above the single-word level are designed for specific tasks and struggle to accurately process novel data. The General Language Understanding Evaluation (GLUE) benchmark tool was developed to address this limitation by offering a wide collection of data and tasks for NLP algorithms to train on [17]. However, the language used in ophthalmology is unique in its

Updates and Maintenance

A unique quality of SaMD technology is the ability for constant and rapid iteration or modification. Unlike errors or failures with hardware, software can be readily updated without massive logistical recalls or manufacturing requirements. This rapid and wide-scale ability to patch or repair SaMDs is both an advantage and a unique burden for researchers and developers that hope to implement their algorithms in clinical settings. Updates and maintenance can be for adaptive, perfective, corrective, or preventative purposes. Adaptive changes are made to accommodate changes in the environment, whether technological, legal, or cultural. Perfective changes aim to improve software performance and meet increased demands in clinical or research settings. Corrective and preventative changes resolve and protect against errors in software performance, with the latter aimed at addressing latent faults before they become operational faults. All of these changes require thorough testing and approval by regulatory bodies such as the FDA before implementation. SaMDs also require more frequent changes to accommodate the rapid pace of technological advancement and growth in new data availability. The malleability of SaMD technology therefore offers a unique advantage and responsibility that researchers and clinicians must embrace if they wish to implement NLP algorithms in real-world clinical settings. (3) Socio-technical Environment Similar to regulatory pressures from legal and administrative bodies, DL and NLP algorithms face challenges in implementation from social,

Natural Language Processing (NLP) in AI

cultural, and technical perspectives. Historically, innovations in medicine have met skepticism and resistance from established medical institutions. Physicians who have decades-long practices may also resist the implementation of technologies based on DL and NLP algorithms, especially if they inform the medical decision-making process. Much concern has been raised about the “black-box” nature of DL methods such as convoluted neural networks, as the exact methodology of how these algorithms arrive at their outputs is unclear. Addressing these concerns is necessary to assuage not only clinicians but also certain patients who distrust medical or technological institutions. If socially and culturally accepted, another challenge facing the integration of NLP algorithms is the technical knowledge required to implement, interpret, and understand the outputs from these algorithms. Most modern medical school curriculums only address elementary clinical statistics, and few, if any, delve into the basics of computer science. Understanding the statistical significance and limitations of algorithm outputs, as well as their clinical implications, will require additional study or resources. More and more physicians need better statistics and data science backgrounds to interpret the results of big data, which is beginning to drive medical decisions. In summary, NLP is just beginning to enter ophthalmology. As millions of EHR data points, especially free text in exam findings, plus the assessment and plan, are aggregated into a single registry to track disease, NLP will be the key algorithm to find commonalities and enhance physician workflow and patient care [18]. This is a rapidly evolving area of AI with transformative potential.

249

References 1. Magerman DM. Statistical decision-tree models for parsing, ACL-95, p. 66–53, ACL, 1995. 2. https://arxiv.org/abs/1810.04805. 3. https://arxiv.org/pdf/1901.08746.pdf. 4. https://models.acrdsi.org/. 5. Fitch RH, et al. Induced microglia and auditory temporal processing in rates: a model for language impairment? Cereb Cortex. 1994;4(3):260–70. 6. Woodward MA, et al. Development and validation of a natural language processing algorithm to extract descriptors of microbial keratitis from the electronic health record. Cornea. 2021. https://doi.org/10.1097/ ICO.0000000000002755. Online ahead of print. 7. Durack DT. The weight of medical knowledge. N Engl J Med. 1978;298(14):773–5. 8. https://arxiv.org/abs/2004.15011. 9. Fushman D, et al. What can natural language processing do for clinical decision support? J Biomed Inform. 2009;42(5):760–772. 10. Park DW, et al. The American academy of ophthalmology’s IRIS® registry (Intelligent research in sight clinical data): a look back and a look to the future. Ophthalmology. 2019;124(11):1572–4. 11. https://www.aao.org/iris-registry/data-analysis/knightstemplar-pediatric-ophthalmology. 12. Kreimeyer K, et al. Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J Biomed Inform. 2017;73:14–29. 13. http://www.imdrf.org/docs/imdrf/final/technical/ imdrf-tech-131209-samd-key-definitions-140901.pdf. 14. http://www.imdrf.org/docs/imdrf/final/technical/ imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf. 15. https://www.fda.gov/media/100714/download. 16. https://arxiv.org/pdf/2007.04792.pdf. 17. https://arxiv.org/pdf/1804.07461.pdf. 18. https://www.modernretina.com/view/natural-language-processing-helps-ophthalmologists-access-dataimproves-data-curation.

Global Experiences

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America SP-ROP (Panamerican Society of ROP) Alejandro Vazquez de Kartzow, Pedro J. Acevedo , Gabriela Saidman , Vanina Schbib , Claudia Zuluaga , Guillermo Monteoliva , Marcelo Carrascal , Adrian Salvatelli , Susana Patiño, Juan Marmol, Juan Lavista Ferres, and Maria Ana Martinez Castellanos Abstract

In the 1990s, ROP was identified as the main cause of blindness in children in Latin America. From that moment on, regional efforts began to prevent, detect and treat ROP. Today, there are numerous barriers to providing adequate ROP care, and one of the biggest problems in ROP programs is the difficulty in accessing and referring preterm A. V. de Kartzow Hospital Clínico Viña del Mar, Limache 1741, Viña del Mar, Valparaíso, Chile A. V. de Kartzow
P. J. Acevedo Centro Ilarco Torre A Consultorio 401, Transversal 60 (Av Suba) No. 115–58, Bogotá, Colombia G. Saidman Red ROP Provincia de Buenos Aires, Hospital Evita Pueblo, 136 #2905 Provincia de Buenos Aires, 1884 Calle Buenos Aires, Berazategui, Argentina V. Schbib Red ROP Provincia de Buenos Aires, Hospital Interzonal de Agudos Esp. en Pediatría “Sor María Ludovica” Dirección, Calle 14 No 1631–La Plata (1900), Provincia de Buenos Aires, Buenos Aires, Argentina C. Zuluaga Clinica Imbanaco Grupo Quiron Salud Cali, Carrera 38A #5A109cons 502 Torre B CP760042, Cali, Colombia

babies, as well as the limited number of human resources available for the timely detection, treatment, and monitoring of ROP. For this reason, the importance of telemedicine arises as a diagnostic tool that will allow us to overcome these obstacles. Telemedicine records, stores and shares retinal images and epidemiological data remotely among specialists, and is an emerging

G. Monteoliva (&) Red ROP Provincia de Buenos Aires, Hospital Interzonal General de Agudos “General San Martín” Dirección, Calle 1 esq. 70–La Plata (1900) Provincia de Buenos Aires, Buenos Aires, Argentina e-mail: [email protected] M. Carrascal Buenos Aires 450, Neuquén, Argentina A. Salvatelli Universidad Nacional de Entre Rios, Oro Verde, Entre Ríos, Ruta Prov, Argentina S. Patiño Río Magdalena No, Col. Tizapán, 167 Int. 9, San Ángel, Del. Álvaro Obregón C.P. 01090, México J. Marmol
J. L. Ferres
M. A. M. Castellanos Asociación Para Evitar La Ceguera en México, Hospital “Luis Sánchez Bulnes” I.A.P. Vicente García Torres No, San Lucas Coyoacán C.P, 46, 04030 Ciudad de México, México

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_18

253

254

technology in ophthalmology. This would have a potential utility to exceed the diagnostic goals in the care of ROP. Part 1: Needs and Cost-efficiency of ROP Smartphone Networks in Latin America

Introduction Retinopathy of Prematurity (ROP) is a dynamic neuro vasoproliferative disease of the immature postnatal retina, incompletely vascularized, that affects premature babies. The disease is characterized by a proliferation of abnormal fibrovascular tissue on the edge of the vascular and avascular retina. Its physiopathology is complex and its etiology is multifactorial. It affects exclusively premature newborns, especially those equal to or under 1,500 g of weight at the time of birth and/or equal to or under 32 weeks of gestational age; nonetheless, it can manifest on

Fig. 1 Visual impairment associated with premature infants and estimates of retinopathy of prematurity at the regional level: Half of 63,000 estimated blind children

A. V. de Kartzow et al.

older prematures, denominated as unusual cases exposed to risk factors to develop ROP [1, 2]. The causes of childhood blindness vary from region to region, but half of the 63,000 children who are blind from retinopathy of prematurity (ROP) worldwide [3, 4] live in Latin America (Fig. 1). Blindness due to ROP is reduced by avoiding preterm birth and improving standards in neonatal care. Currently, ROP is considered a quality parameter of perinatal and neonatal care [1, 5]. In the 1990s, ROP was identified as the main cause of blindness in children in Latin America [6–9]. From that moment on, regional efforts began to prevent, detect and treat ROP [10]. The second step was to identify the magnitude of the problem through different epidemiological studies that showed its incidence and its substantial variation between countries, Neonatal Intensive Care Units (NICU) even within the same city, which showed that the incidence was directly related to the application of neonatal standards

from ROP live in Latin America. Several strategies, including Telehealth programs are required

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

and especially respiratory care in each of the NICUs [9]. Since 2005, and with the purpose of knowing the state of the art and the situation of ROP in Latin America, four regional ROP workshops have been held, organized and supported by PAHO/WHO [11, 12]. Prevention strategies in Latin America have used some of these multidisciplinary health approaches [13–18]: • Prevention of preterm birth • Improvements in neonatal care (neonatologists and nurses) • Detection and treatment (ophthalmologists and anesthesiologists) • Vision rehabilitation (low vision and occupational therapists) • Social workers from the NICU, for adequate follow-up and ensuring the control of premature babies. Neonatologists, midwives and nurses today play a key role in preventing blindness caused by ROP. Electronic Medical Records and Digital Health projects may improve these strategies, through healthcare timely interventions in timelines. At the regional level, an alliance was formed between the Pan American Health Organization (PAHO), the International Agency for the Prevention of Blindness (IAPB) and other international non-governmental organizations to support groups of national professionals as part of the Vision 2020 program. In August 2013, the Panamerican Society of Premature Retinopathy (SP ROP) was created, a collaborative ophthalmo-neonatal and nursing group, whose main objective is to contribute to improving the comprehensive quality of health care and life of the premature newborn and their families in the Pan-American population, promoting work and seeking solutions in a framework of regional collaboration, stimulating and supporting actions, together with the exchange of scientific knowledge in different congresses, meetings and electronic media. In 2015, the creation of the WhatsApp group of SP ROP stands out, a Pan-American network, in which about 256 ophthalmologists, neonatologists

255

and nurses participate, with the purpose of exchanging knowledge, conducting teaching and education, providing support in clinical cases and in their therapeutic decisions, promoting scientific discussion, and sharing multimedia resources (photographs and videos), documents (articles and books on the specialty) and links of interest [19]. On the other hand, important and relevant publications can be found on the website of the Panamerican Society of Retinopathy of Prematurity (SP ROP), www.sprop.org. Regional ROP guidelines recommend collecting essential minimum data at the national level, about the number of neonatal units and the number (%) of units with ROP services. Likewise, minimum essential data at the level of each neonatal unit, about the number of preterm infants who are eligible for screening, the number (%) examined, the number (%) with ROP of any stage, the number (%) with indications for treatment, the number (%) treated and the birth weight and gestational age of the babies who needed treatment, to construct the following indicators: (a) Detection coverage, (b) Proportion of preterm newborns with any stage ROP and (c) Treatment coverage for ROP. It is necessary to introduce national policies to ensure the care of preterm infants and the financing of services for neonatal care and screening in countries where it is not yet established. Adherence to clinical guidelines by all countries will help improve the quality and effectiveness of care, provide guidelines for good performance, reduce the possibility of negative effects on patients and help health professionals make informed decisions. Today, there are numerous barriers to providing adequate ROP care, and one of the biggest problems in ROP programs is the difficulty in accessing and referring preterm babies, as well as the limited amount of human resources available for the timely detection, treatment and monitoring of ROP [20, 21]. For this reason, the importance of telemedicine arises as a diagnostic tool that will allow us to overcome these obstacles. Telemedicine records, stores and shares retinal images and epidemiological data remotely among specialists, and is an emerging technology in

256

ophthalmology. This would have a potential utility to exceed the diagnostic goals in the care of ROP [20, 22–24]. Telemedicine promises to be a very good strategy to reach places where there are no trained ophthalmologists to screen [25]. Capturing images for photo documentation in ROP with a smartphone (Retphone) is a low-cost resource, accessible to all, massive, dynamic and highly efficient, which has quickly proven its importance in clinical practice [26, 27]. There is a practical theoretical course-workshop created, and promoted by SP ROP, to acquire the necessary skills and competencies for this purpose. It seeks to develop teamwork, joining forces and creating a solid network of agile and practical collaboration. The idea is that we can all speak the same language in terms of ROP, in order to equitably reduce the differences and gaps existing in the current indicators. The goal is to reduce the leading preventable cause of childhood blindness in Latin America and for every premature newborn who develops a serious stage to be timely and adequately detected and treated, obtaining better anatomical, functional and visual results, when patients are treated at the most appropriate time. The Panamerican Society for Retinopathy of Prematurity (SP ROP) encourages the integration of all telemedicine initiatives, and in this chapter we will summarize some concepts. SP ROP is working on some smartphone ROP Telemedicine projects and regional proposals, focused on the technological advances, including artificial intelligence (AI) tools, using international e-health standards, for integration to personal health records, working in all levels of prevention in the mentioned timelines, with useful integrated datasets, including validated Smartphone ROP images. In this chapter, we will mention some of these projects.

New International ROP Classification: ICROP 3 The International Classification of Retinopathy of Prematurity is a consensus statement that creates a standard nomenclature for classification of

A. V. de Kartzow et al.

retinopathy of prematurity (ROP). It was initially published in 1984 [28], expanded in 1987 and revisited in 2005 [29]. The recently published third revision, the International Classification of Retinopathy of Prematurity, Third Edition (ICROP 3) [30], includes challenges due to technological, imaging and therapy advances such as: 1. Concerns about subjectivity in critical elements of disease classification. 2. Innovations in ophthalmic imaging. 3. Novel pharmacologic therapies (e.g., antivascular endothelial growth factor agents) with unique regression and reactivation features after treatment compared with ablative therapies. 4. Recognition that patterns of ROP in some regions of the world do not fit neatly into the current classification system. Objectives: • Standard disease classification is essential for advancing in clinical research. • Improvement in neonnatal care, anti-VEGF therapy; imaging: evolution in the understanding of ROP pathophysiology and clinical management. • ICROP 3 is updating ROP classification in response to advances through a combination of evidence-based literature and expert consensus. • Goal: improve quality and standardization of ROP care worldwide, and provide a foundation for improving clinical care and research in the future. ICROP 3: Subjectivity The ICROP 3 Committee found evidence of International variation in the diagnosis of treatment requiring ROP, and an improved need for standardization of the diagnosis of Treatment Requiring ROP. Measures might include improved training in the grading of ROP, using an international approach, and further development of ROP image analysis software [31, 32].

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

257

ICROP 3: Anti-VEGF Treatment and Retinal Imaging • Plus disease: it is essential to define the indication for treatment. The decision-making algorithm used in SP ROP includes Plus disease [33, 34]. • Zone I: in some cases it may be part of minimal data, for decision-making in ROP evaluation. • Plus and pre plus will be measured by vascular thickness in Zone I and not by quadrants. Image analysis software tools including AI will help to perform a precise diagnosis [32]. • Flat-appearing extraretinal neovascularization can occur in eyes with Zone I or Posterior II Zone disease, in the absence of an obvious ridge: classified as Stage 3-Zone I, atypical morphology (net of flat neovascular proliferation) [29]. • Early diagnosis of a ROP is very important. Any registration method with images can be very useful, including smartphone images. • Gold standard laser treatment does not have the same effectiveness in cases of severe ROP in posterior retinal areas. • Documentation for diagnosis, parents or family shared decision-making and informed consent, and follow-up is mandatory. • Consider the medico-legal aspects of this documentation. • Intravitreal Bevacizumab as the first line of therapy: more discussion and research works are needed for evidence-based decisionmaking. Relevance of documentation with images needed for this evidence [35–37]. Recognition that “plus disease” and “pre plus” reflect a continuous spectrum of vascular abnormality [33] ROP experts have different cut points of vascular abnormality required for plus and pre plus disease. The continuous spectrum of vascular severity score may reflect more accurately the behavior of expert ROP clinicians (Fig. 2). AI tools may help in this approach [31, 32, 38, 39].

Fig. 2 Continuous Spectrum of Vascular Severity. From Normal to Plus Disease. Information Systems including AI tools can help to detect with precision the changes in this spectrum, between comparative images in the timeline of prematurity vessel development

Fig. 3 Zone I: 2 radius papila-fovea. Zone II divided into Posterior Zone II (2 disk diameters anterior to Zone I) and Anterior Zone II (from Posterior Zone II to equator)

Intermediate Posterior Zone II (Fig. 3) • From a therapeutic point of view, the Posterior Zone II of ICROP 3 behaves clinically as Zone I [40]. • A ROP may occur beyond the posterior retina in large and heavy children in other parts of the world with limited access to proper care and prevention of severe ROP.

258

A. V. de Kartzow et al.

Fig. 4 A ROP in “Zone I secondary to notch”: Plus disease, shunts and hemorrhages

Zone nomenclature for “Notch” (Fig. 4) • Take the “notch” into account to determine the most posterior zone in making treatment decisions. • ROP evaluation through Telemedicine Networks using images, including Smartphone photos, could perform Type 2 versus Type 1 diagnosis, decision-making for follow-up and time for treatment. Retinographies can be obtained in the follow-up of diagnostic or

treatment (regression-reactivation) assessments Telemedicine. • In Stage 4A ROP (macula on, Fig. 5, left) time frame for an effective treatment is short. ROP images and Telemedicine can help a lot for Referral Warranted ROP surgical treatments in these cases. Early Vitrectomy: fixing initial detachments is the best opportunity of preserving good vision, and that makes Stage 4A a therapeutic window that cannot be missed (Fig. 5 left).

Fig. 5 Left: ROP Stage 4A image performed with Retcam. Right: ROP Stage 5A (visible optic disk) performed with Optos. When Retinal Detachment ROP

Stages are detected, they must be immediately referred for early treatment in ROP Networks. The Vitreoretinal Pediatric Surgeons perform these image studies

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

259

Regression features: 1. 2. 3. 4. 5. 6.

Fig. 6 Reactivation (above) and Regression (below) vascular patterns must be documented

Stage 5 ROP: subcategorization of ROP 5: • Stage 5A: optic disk is visible (funnel open) (Fig. 5, right) • Stage 5B: Retrolental vascular tissue (closed funnel) • Stage 5C: changes in the anterior segment. In patients with retinal detachment, surgical treatment is better than natural history. Keeping the retina viable opens the door for future therapies to improve vision. Regression and reactivation patterns: • Regression: (Fig. 6). Definition of ROP regression and its sequelae, whether spontaneous or after laser or antivascular endothelial growth factor treatment. Regression can be complete or incomplete. Location and extent of the peripheral avascular retina (PAR) should be documented. Regression • Spontaneous • Post treatment: – After intravitreal injection anti-VGEF (1–3 days) – After laser (10–14 days).

Decrease in plus (Fig. 6) Vascularization toward the periphery Involvement of the tunica vasculosa lentis Better pupillary dilation Clearer media Resolution of intraretinal hemorrhages.

Reactivation: (Fig. 6) may include new ROP lesions and vascular changes. When reactivation of ROP stages occurs, the modifier reactivated (e.g., “reactivated Stage 2”) is recommended. After treatment with anti-VEGF, extended follow-up must be performed. If follow-up is not possible, laser to Zone II or III is advised. Telemedicine with images and integration of information and communication in digital health projects can help this long-term follow-up: • New demarcation line with Stage 3 ROP and Plus disease (Fig. 6) • May not progress in the normal sequence of the acute phase. • Long-term sequelae: – Importance of transition between eyecare teams sharing information. – Images shared for early detection. – Referral to higher complexity hospitals for diagnostic studies, decision-making, patient information and treatment, if required. – ROP is a lifetime disease and mandatory follow-up is required. Image documentation in this timeline can help to compare the retinal status of these premature babies during all their lives. – Even premature babies who don’t have ROP can show long-term sequelae. – Higher risk of retinal detachment in patients with a history of premature birth [41]. – Difficult cases: attached posterior hyaloid, multiple breaks. – PAR: Persistent Avascular Retina (Fig. 7). – Ophthalmologists and Adult Retina Experts should be aware of Adult ROP

260

A. V. de Kartzow et al.

study data collection, and likely affect the recommended timing and disease severity at treatment. The use of digital retinal images and recent technological tools like AI may improve accuracy in diagnosis, telemedicine and referral and treat warranted ROP. The SP ROP LATAM diagnosis, treatment and follow-up decision-making

Fig. 7 Adult ROP. 15-year-old male patient, never treated. PAR: persistent avascular zone. (Courtesy Maria Ana Martinez Castellanos, MD) In the timeline of retina follow-up screening it is useful to compare with previous images and data

features and search for personal health data and images of previously premature patients. ROP long-term sequelae features [42]: 1. In peripheral or posterior retina [43] 2. May be with or without treatment 3. It is described by location by zone and extension in quadrants (nasal). 4. Features: a. rhegmatogenous detachments b. poor development of the macular area c. irregularities in the perifoveal vasculature.

Conclusions of ICROP 3 The new classification ICROP 3 should improve the quality and standardization of ROP care around the world and provide a stronger foundation for future research and clinical care. The ICROP 3 remains a tool guiding ophthalmoscopic documentation during a physical exam and will take time to learn and record accurately. The recommended changes will gradually be incorporated into diagnosis and

Regarding the PAAO WHO ROP Clinical Guide recommendations, monitoring for ROP screening should be carried out considering Stage, Zone and Plus/No Plus disease), according to the following scheme (Fig. 8). Introducing the New ICROP 3 and the continuous spectrum of vascular changes, we could think about some useful tools to build networks in Latin America, in the future, with validated Smartphone images as data. Prevention of severe Type 1 ROP should be considered as a timeline in the personal health records of premature infants, with failures in timely interventions that could prevent more severe stages of ROP (Fig. 9). Smartphone teleophthalmology and ubiquity Ubiquity means that something exists everywhere at the same time. To achieve ubiquity in health information systems, strategies are thought to achieve accessibility and quality of the system. It is related to smartphones and the explosion of information, and how this has impacted our daily lives, allowing multidisciplinary prematurity healthcare teams’ integration with patient's family who become involved in decision-making [44]. The telephone becomes an inseparable appendage of people, incorporated into everyday tasks. In addition, telephones are photographic and video cameras that are carried everywhere and allow calls and the use of messaging and email systems. These cameras are constantly progressing in quality, providing potential benefits for use in telemedicine projects with retinal imaging. In addition, programming tools have emerged for cell phones that equate them with computers. There are multiple applications for the care of

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

261

Fig. 8 Panamerican Society of ROP, SP ROP, ROP decision-making for diagnosis, treatment and follow-up algorithm

Fig. 9 Timeline of health history of premature infants. Prevention and follow-up interventions using validated data and images). The goal should be “normal vascularization” and “type 2 + regression” in the timeline until the

most peripheral vascularization is complete. Then, lifetime retinal examinations should be performed (PAR, Peripheral avascular Retina, and sequelae)

specific diseases. However, these applications basically repeat the errors of the health systems of 20 years ago, since their work disintegrated, with non-standardized and disaggregated information, which contributes even more to the fragmentation of personal care. They are not part of a system nor are they conceived with a ubiquitous health system approach [45].

The image consultations must be integrated into the electronic medical record (EHR). Without this fundamental component, which allows a holistic view of the patient, you only have access to more information when you need it, with a sophisticated phone, with no more than audio and video. The Food and Drug Administration already promotes the development of medical

262

A. V. de Kartzow et al.

Table 1 Ubiquity components. Networks workflows use different tools of these components, depending on the maturity of the information system of the organization U-health component

Features

People

Change in the healthcare paradigm and the empowerment of patients, the way in which they are involved in caring for their health changes, in turn, impacts the health team

Devices

It is necessary to analyze the data cycle from the patient's perspective, with information integrated and protected by different computer mechanisms, protecting sensitive health information

Systems

They are made up of patient platforms to access their information and interact with the health system, specifically patient portals. From these applications, it is possible to work on prevention together with patients, on integration

Interaction

Through the devices with the health team systems, represented by telemedicine 18. Health informatics projects require a high-cost investment

applications and smartphone devices that improve health care and offer valuable information to consumers and health professionals, assuming its role of public responsibility, of regulating these developments and controlling those that pose a risk to the health of consumers [46]. Ubiquitous health or u-health According to Plazzotta et al., it is the persistence of the health system (regardless of the device used to access it), the information system and the technology that processes this information. In uhealth, it is necessary to have four components: people, devices, systems and interactions (Table 1) [44]. Ubiquity criteria A ubiquitous system must be • • • • •

reliable scalable interoperable usable adaptive.

It has three objectives for the underlying health system: 1. Reduce the time or gap from when the image is captured until it is processed and, eventually, it serves for decision-making.

2. Reduce costs, either for the health team, or through the institutions; reducing blindness from ROP. 3. Reduce medical errors. The digital medical record makes available evidence, algorithms, recommendations and alarms to assist ROP Networks. Smartphone images in ROP: technique When we record images of the fundus of a patient, there are several objectives that we can have: to attach a photograph in the clinical history for its registration and subsequent monitoring over time, teaching in real time to the patient, family members, medical or paramedical personnel in training, pre- and post-operative comparison, as well as medical information networks (telemedicine, telecare, tele interconsultation, etc.) [44, 47]. The use of smartphones as a tool for fundus imaging in patients has been going on for several years. The first attachments used in smartphones were the Peek (Peek Vision Ltd, 2013) and the D-EYE Retina (D-EYE Srl, 2014). Since then, different implements or accessories have been developed for taking fundus images, with 3D printing plastic supports for lenses and telephones being the most frequently observed in recent years. Within the group of ophthalmologists of

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

263

Fig. 10 Use of smartphones for taking images of premature newborns. Above: smartphone regular method. Below, “hands-free” method

the Panamerican Society of Premature Retinopathy (SP ROP), techniques have been developed for taking images of the fundus with unsupported smartphones, the Keeler MIO® (2020) being the most recent development for taking images with the “hands-free” device (Fig. 10). Before explaining the technique for taking images, it is important for the reader to understand that, like everything else in the practice of medicine, this process requires a process of learning, pulse, patience, perseverance and practice (called 4-P SYSTEM by its acronym in Spanish), especially if we want to achieve fundus images in pediatric patients. Equipment preparation Practically, any smartphone on the market is useful for taking images [48]. The main recommendation when choosing the phone is that the light source and the lens on the back of the phone are as close as possible so as to reproduce the coaxial effect observed in ophthalmology. As in clinical practice, we recommend the use of 28 or 30 diopter lenses for image registration in pediatric patients and for adults or larger images with

the 20 diopter lens. Clean the lens surfaces before you start recording. Image registration On the smartphone, go to the camera function and select the video mode. Activate the light source or flash of the camera so that it is permanently on (Fig. 11) and adjust the zoom on the screen to 50%. You now have your equipment ready to register. With the patient under pharmacological mydriasis, with or without the use of blepharostat, proceed to observe the red reflection of the fundus on the phone screen located 50–60 cm from the patient’s face and place the 20, 28 or 30 lens diopters in front of the eye as it does in indirect ophthalmoscopy to visualize the details of the fundus. Start filming and completely fill the lens surface with the fundus image using a “BIOM or Zoom type” maneuver, moving the phone closer or further away from the lens surface (Fig. 12). It is common to observe glare or reflections on the surface of the lens that interferes with the image; to avoid these reflections, gently rock the lens while shooting. Finally, stop shooting.

264

A. V. de Kartzow et al.

Fig. 11 Smartphone configuration. Video mode and light source ignition

Image edition Once the filming is achieved, we will proceed to the editing and extraction of the photos or videos that we want to save. By opening the recorded video, we have the possibility to review the reel of the recording and evaluate frame by frame. Once you have found the image you want to extract or record, proceed to take a screen capture or screenshot (Figs. 13 and 14); the image will automatically be recorded in your phone’s photo album. With the images selected and extracted from the video, you can improve the quality of the photo to highlight details and optimize the visualization of the lesions. All smartphones have a photo editing application (Fig. 15) where you can modify the brightness, contrast, lighting, clarity, etc. You can also find applications or APPS for photo editing on digital platforms, but these are not normally required.

Smartphone Hands-Free Indirect Funduscopy: ROP Images Barriers to ROP screening and difficulties with subsequent evaluation and management include poor access to care, lack of physicians trained in

ROP and issues with objective documentation. Digital retinal imaging can help address these barriers and improve our knowledge of the pathophysiology of the disease. Multiple advancements in technology have produced numerous imaging modalities for ROP (Table 2). The standard method for retinal imaging in premature babies is by using a contact fundus camera (Ret Cam). The use of digital imaging using the RetCam was shown to have high sensitivity and specificity when compared with the gold standard, indirect ophthalmoscopy, in terms of ROP diagnosis. However, potential limitations of the RetCam include relatively low resolution and capturing dark fundi and mainly its high cost. Digital retinal imaging is an important factor in telemedicine and allows remote interpretation of images. RetCamTelemedicine programs have been established in many countries (EE UU, India, Chile and Mexico). Due to the high cost of the RetCam, in lowincome countries, the use of the smartphone was incorporated to capture retinal images. High-quality cameras and high-resolution video features of smartphones can also be utilized for retina imaging. Lord et al., in 2010, published the

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

265

Fig. 12 Taking images with a smartphone. Red reflex, phone/lens location, initial focus and filling of the lens with image using BIOM or Zoom technique

266

A. V. de Kartzow et al.

Fig.13 Review of the filming and selection of the image to be photographed. Video paused to choose the best frames

Fig.14 Screen Capture or Screenshot of selected retinal image

first report on the use of smartphones for retinal imaging. Dr Shizuo Mukai in 2013 described in detail a relatively simple technique of fundus photography in human and rabbit eyes using a smartphone. After this initial breakthrough, there

has been much advancement in the field of smartphone-based retinal imaging [49–52]. The evolution of the high-resolution smartphone camera offers the potential to revolutionize traditional fundus photography. By

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

267

Fig. 15 Retinal image edition using the smartphone app Table 2 Fundus cameras and smartphone-based systems: view angle, mydriasis requirement, contact versus no contact systems, fluorescein angiography system availability and price. Comparative table Camera system

View angle

Mydriasis

Contact

FA

Price

Optos

200°

No

No

Yes

High

Icon

100°

Yes

Yes

Yes

High

Retcam

130°

Yes

Yes

Retcsm 3, yes

High

3Nethra neo

120°

Yes

Yes

No

Middle

Volk prestige

50°

Required

No

No

Middle– Low

Volk vista

55°

Required

No

No

Low

Smartphone

From 50° (2D lens) to 70–90° (40D lens)

Yes

No

No

Low

replacing a binocular indirect ophthalmoscope with a smartphone [53], many ophthalmologists are innovating a new field of funduscopy. Not only is the technique inexpensive and relatively easy to learn, but the expansion of mobile networks into all sectors of everyday life also is creating unique opportunities for telemedicine, resident training and clinical care around the world 3. Smartphone-based fundus photography provides a cost-effective and widely available method of capturing narrow-field color fundus images in premature babies. Modern smartphone cameras have an optical system and a

coaxial inbuilt light source that can be used to capture retinal images. The simplest method of smartphone-based color photography involves coupling a condensing lens to a smartphone camera functioning as an indirect ophthalmoscope 5. Smartphone fundus photography using a condensing lens is based on the same principle as indirect ophthalmoscopy. The viewer’s eye is replaced by the display screen of a smartphone. The light source of the indirect ophthalmoscope is replaced by the flashlight of a smartphone camera.

268

Technique and positioning After pupil dilatation, the smartphone camera application is opened. The camera mode is changed to video. The flashlight is turned on. Condensing lens is used to focus the fundus, while an image of the focused fundus becomes visible on the smartphone display screen. The examiner holds the smartphone in one hand, while the lens is held in another hand near the patient’s eye. Many times with this technique’s help is required to avoid the movements of the baby (Fig. 16). Scleral depression is not possible with this technique given both hands are employed. Different devices were developed to support mobile phones, which made it possible for the examiner to have one hand free (Table 1) [54]. The main concept is to use the smartphone screen to perform the exam rather than a binocular indirect ophthalmoscope (Fig. 17). Regarding this concept and thinking about the evaluation of premature babies, a SP ROP Team from Argentina developed a novel device for hands-free digital indirect funduscopy through a smartphone [26]. The headband-mounted holder for a smartphone gives to the observer the possibility to obtain a steady and neat view of the ocular fundus through a smartphone and different types

Fig. 16 Handheld technique for ROP Smartphone images, requiring collaboration from NICU’s team

A. V. de Kartzow et al.

Fig. 17 Visualization on smartphone screen, similar to the Binocular Indirect Ophthalmoscopy

of condensing lenses (+20 D, +28 D or a +40 D), with different degrees of field of view and magnification. It proves to be very comfortable and safe for digital indirect fundoscopy, giving the examiner the possibility to freely use one of her/his hands for scleral indentation and/or globe rotation during examination (Figs. 18 and 19). With these maneuvers, the far periphery of the fundus can be nicely observed and the globe is kept steady for better video capturing. Goodquality fundus pictures from video clips may be obtained at any moment during the examination. Also, during examination the fundus image may be observed simultaneously on a TV screen (Fig. 20). Although smartphone fundoscopy can be achieved with multiple techniques, this prototype of a headband-mounted holder proved to be a very simple to use, comfortable and a useful

Fig. 18 The hands-free helmet method leaves both hands free allowing eyeglobe rotation in the evaluation and imaging of premature infants

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

269

Advantages of hands-free helmet device: • Learning curve. Similar to the conventional method of indirect ophthalmoscopy • Leaves both hands free • Adaptable to any type of smartphone • Cost-efficiency • Smartphone telemedicine networks (Diagnosis, treatment and follow-up) • Decision-making [55]. What’s next:

Fig. 19 Ophthalmologist performing screening and image capture in NICU

• Incorporation of apps with international standards such as DICOM, HL7/FHIR and SNOMED CT. • Validation and standardization process for smartphone data integration into electronic personal record systems (Security and Confidentiality). • Artificial Intelligence tools. • Telementoring and streaming tools through highspeed Internet (including Smartphone 5.0). Teaching/Training

Fig. 20 After delivery from NICU, ophthalmologist performs retina screening and images through hands-free device. Smartphone allows streaming images to a monitor, to explain in teaching and parental information

device for hands-free digital indirect funduscopy through a smartphone in different clinical situations, and may have a high impact in clinical practice, especially in settings without the availability of sophisticated fundus imaging technology for telemedicine and teaching 1. Since the Digital Indirect Ophthalmology Hands-Free method is similar to the BIO procedure, an accessible learning curve is the operator takes advantage of the movements of rhis head in the examination, and by using a smartphone as a recording element they make it an economical solution.

One of the main activities in the ROP Networks building is training screening and Smartphone retinal image registration. With the hands-free device, practice starts with simulation eyes (Fig. 21): First of all trying to find the red reflex; then making small tilting movements with the lens until a clear retinal image is obtained, filling the lens. Different lenses can be used (28, 20, 40, super 66, panretinal, etc.) adapting the working distance (lens distance from the eye, hands-free device arms length) depending on the lens power used. Once the video is obtained, the edition is performed, to obtain screenshots, as previously explained. Validation of Smartphone images for Telemedicine in Networks: preliminary results The purpose of this validation research is to evaluate the diagnostic accuracy of Binocular

270

A. V. de Kartzow et al.

(Fig. 1D) (3 pediatric ophthalmologists, 2 pediatric retinal experts) evaluated videos and pictures. • Unit of analysis: clinical funduscopy and smartphone images of the retina evaluation by telemedicine. BIO screening was performed as usual. For smartphone imaging, the imagers used the helmet device for hands-free digital indirect fundoscopy through a smartphone (Fig. 22). • The SP ROP (Panamerican Society of Retinopathy of Prematurity) algorithm for decision-making for Referral and Treatment Warranted ROP was followed and a standard consensus combined by both Teams was agreed, to compare ophthalmoscopy and telemedicine results, using percentage agreement.

Fig. 21 Training with simulation eyes is the first important step in the learning curve for Smartphone images in ROP and infants. This is very helpful to train the working distance, lens magnification and the autofocus of the Smartphone used. Recommended to start and when smartphone model is changed

Indirect Ophthalmoscopy (BIO) versus Smartphone Telemedicine in ROP babies [56]. • A cross-sectional study was designed for a comparative evaluation between clinical and telemedical diagnosis accuracy, performed in eyes with ROP, evaluating in different centers of Buenos Aires (Argentina), from March to December 2020, analyzing the first 100 results from a validation protocol of 400 total cases. • Two Latin American Regional Teams were built. Team 1: (Fig. 1A) Clinical BIO Screening (4 pediatric ophthalmologists), who performed clinical evaluation and imaging (Fig. 1C). Team 2 (Fig. 1B): Tele-ophthalmologists Team

Results of this preliminary study: for all the first 100 eyes registered, the average gestational age was 30.9 weeks; and the average birth weight was 1517.1 g. Regarding the SP ROP algorithm that evaluates NO PLUS (n:80)/PLUS (n:20), corresponding to consensual “Posterior Vascular Changes”, we found a preliminary accuracy of NO PLUS: 85.4% (n:24) and PRE PLUS: 58.3% (n:15). For ROP Stage 3: 84.3% (n:8); Type 1 ROP: 83.7% (n:20); TYPE 2 ROP: 81.2% (n:37). Additional significant image data found: Hemorrhages: 88.1% (n:57); vitreoretinal traction: 84.2% (n:57) (Fig. 23) [57, 58]. As a conclusion of the study, high accuracy was found using the SP ROP algorithm and smartphone images in ROP telemedicine triage. A more complete data analysis is needed for the final conclusions of this regional study. The complete study research includes 15 variables including hemorrhages, and the quality of retinal videos and photos. ROP and High-Risk Prematurity Follow-Up Networks One of the most relevant issues in the prolonged process of ophthalmological control of preterm newborns is the loss of follow-up. They are usually due to two fundamental aspects.

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

271

Fig. 22 Workflow of Telemedicine Validation Protocol. A BIO clinical evaluation and data entry. B Digital Indirect Ophthalmology Hands-Free Smartphone Retinal image recording and edition. Data Entry. C Images

uploaded to research database. D (Courtesy of MAILOR, IMO): Telemedicine reading center evaluation and data entry

For monitoring in the short and medium terms, we must take into account the small number of specialists in the diagnosis and treatment of ROP, who also tend to focus on large cities. This forces premature newborns to move large areas for treatment. After this, they return to localities devoid of the trained specialist to detect reactivations, regressions and/or long-term sequelae in a timely manner, and experts who recognize them must refer these cases through networks to higher complexity ophthalmologic centers (Fig. 24). It is understood then the relevance of being able to train a work team in taking images so that

the ROP Networks are not only cost-effective for the Health system but also result in an obvious improvement in the quality of care for the Neonate and his family. The second important aspect to consider in network workflows are those patients who do not continue long-term follow-up, due to geographical, economic or educational barriers. For this reason, it is important to establish alerts and schedule alarms, in the computerized registration system to be able to locate them and reincorporate them into the control process. Understanding that functional blindness is as important as structural blindness, early diagnosis and a multidisciplinary

272

A. V. de Kartzow et al.

Fig. 23 Preliminary results of Smartphone images in ROP Telemedicine Validation (100 first cases). Left: Type 1 ROP Accuracy 83.7%; Center: Type 2 ROP Accuracy:

81.2%; Right: Stage 3 ROP Accuracy: 84.3%. Complete Study Analysis is needed, including images and technique quality, for standardization of this method

approach in this critical period of vision and neurological development becomes a mandatory requirement in prematurity networks. We should incorporate in this process visual stimulators, teachers who are specialists in Braille. We must support teachers who carry out the due school accompaniment. The useful Information in networks, including imaging data, should be shared in these networks, integrating the family into all the healthcare processes. The incorporation of smartphone images on these networks is not only essential to help parents become more involved in the pathology of their children but is also an irreplaceable tool for the tele-education of colleagues, thus favoring the functioning of the ROP Network and the sense of belonging to an interdisciplinary work team that leads to a better quality of life for these children, promoting a significant reduction in long-term morbidity, which results in a benefit for the entire community. In Table 3, we can see the usefulness of retinal images in the follow-up of prematurity multidisciplinary approaches. Pediatric ophthalmology subnetwork teams can share images and

information about comorbidities with different healthcare teams through electronic health records, integrated with Personal Health Portals from the patient and family (Table 3).

Prematurity and Pediatric Ophthalmology Subnetworks (Comorbidities) • • • • • •

ROP Perinatal infections (Fig. 25, above images) Strabismus Neuroophthalmology Tumors (Fig. 25, below images) Genetic (Fig. 25, right image).

Teamwork • • • • •

Prenatal Perinatal NICU Prevention High Risk Prematurity Follow-Up (short term) Neurodevelopment

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

273

Fig. 24 APEC, México, first pediatric retina unit in Latin America

Table 3 Importance of documentation with smartphone retinal images in networks. Many data from prematurity and comorbidities can be shared through interoperable systems. Electronic Health Records and Personal Health Portal of patient and family allows integration of information, and better decision-making in health care Data source

Retinal images: clinical cases

Prematurity and comorbidities

Perinatal infections, genetic disorders, vascular-associated signs

Neurodevelopment

Vascular retinal and disk signs

Immature Retina evolution screening

Immature vessels, retinal zone

Retina with vascular alterations monitoring

Shunts, tortuosity, flat vascularization, etc.

ROP monitoring

ICROP 3: normal, pre plus, plus; new zones, stage nomenclatures

ROP treatment

Images recording; decision-making

ROP treatment evolution

Plus and stage; regression; reactivation; long-term anti-VEGF follow-up

Medium-term follow-up: school age and prematurity

Retinal images associated with ophthalmological findings: e.g., myopia and strabismus

Long-term follow-up: premature retina and ROP sequelae

Adult ROP: PAR (Peripheral Avascular Retina); peripheral signs: lattice, etc.

274

A. V. de Kartzow et al.

Fig. 25 Smartphone retinal images in newborn are useful data to share in Prematurity Follow-Up Networks. Multidisciplinary approaches sharing useful data along the lifetime timeline of Personal Electronic Health Records can help in decision-making and health care

• • • • •

Long-Term Follow-Up Tertiary Prevention Psicopedagogy Transitions Legal/Ethics.

All teams should undergo teaching programs, fellowships, research and evaluations for useful data validation. Part 2: International E-health Standards: Artificial Intelligence

PAHO-WHO Standards for Telemedicine ROP Projects in Latin America HL7, the International PAHO-WHO standard, is a document approved by consensus that provides rules, guidelines and characteristics for common use, in order to obtain an optimal level of results in a given context. HL7 is a non-profit organization that provides a framework and standards for the integration and retrieval of electronic information associated with health in all care processes, as well as tools to develop projects. These standards are

proposed by voluntary working groups supervised by a committee. In 2013, they were released free of charge to the international community 21, 22. E-health Standards classification (Table 4). A resumed Classification of e-health Standards. The Panamerican Society of ROP, SP ROP, is focusing the proposal and projects on some of the international standards [59–61]. ROP Smartphone Proposals

Networks:

SP

ROP

Use of Digital Health Standards: 1. DICOM and ROP Smartphone validated images 2. SNOMED CT and ICROP 3. 1. DICOM Standard and Smartphone ROP images DICOM standard (Digital Imaging and Communications in Medicine) is the globally recognized standard for the exchange of medical images, designed for handling, visualization, storage, printing and transmission. It includes the

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

275

Table 4 Resumed Classification of e-health Standards. SP ROP proposals and projects are based on these international standards Terminology standards

• They provide a common language for the description of symptoms, diagnoses and treatments, allowing the interpretation and subsequent use of the information • An example is Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT), an international terminology standard, compatible with HL7 and to which each country adheres • This standard allows a term to be assigned to the medical act. The terms of this standard are proposed by groups of experts in each área

Messaging standards

• Definitions of structure, content and requirements of the data • An example is the Digital Imaging and Communications in Medicine (DICOM) imaging standard, used by diagnostic imaging services. In ophthalmology, we recommend using DICOM-compatible equipment • In relation to smartphone images in the ROP, they still require validation toward international standards and adaptation to the DICOM standard • As we all know, images and videos “weigh” too many bytes to transfer; for this reason, messaging services are used, which run parallel to clinical information (Picture Archiving and Communication Systems)

Platform standards

• The new HL7 standard, called Fast Health Interoperability Resources (FHIR), integrates messaging, documentation and services through smartphones into a single holistic vision • This new HL7 FHIR standard and multiple programming systems applied to smartphones that include images (Application Programming Interface) • This will allow achieving useful interaction in tele-ROP networks between smartphones and computers, something unthinkable a few years ago

definition of a file format and a network communication protocol. The needs and motivations that justify the criteria for the design of a low-cost and distributed system with application into telemedicine are described. Finally, we also describe a procedure that is currently used by a Latin American group of experts to acquire this type of image. This represents the starting point for the definition of the system. This would allow access to expert knowledge from remote locations for diagnosis and treatment, consultation and training of new professionals. A ROP Network from Argentina [62] has developed a smartphone registration system that consists of a helmet that can dock a Smartphone to capture fundus exams on video. This mounting system leaves the hands free for the ophthalmologist to hold the child and an exam loupe at the same time (Fig. 26). Once the images are obtained through the Smartphone, the professionals analyze by visual inspection and manually the frames of interest

and in the cases they consider necessary, they reconstruct a panoramic retinal fundus image manually as shown in Figs. 27 and 28. On the other hand, to share images and videos among professionals, traditional messaging applications and platforms are currently used in immature networks, which makes them susceptible to degradation by the information compression methods used by these applications and vulnerable to their confidentiality. This proposal is the starting point of a Research and Development Project of the National University of Entre Ríos, in collaboration with members of the SP ROP network, and the preliminary analysis of the problem is presented. The architecture is proposed for an information system support in the ROP, the technology to be used for the standardization of medical images and a study on an initial set of images and videos obtained by the Digital Indirect Ophthalmology acquisition method, in order to determine its applicability in the proposed information system.

276

A. V. de Kartzow et al.

Fig. 26 Helmet, smartphone holder (iPhone) and Volk 28 D magnifying glass, making Video ROP

Fig. 27 A ROP, with manual trimming and reconstruction

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

277

Fig. 28 Immature Retina in Anterior Zone II, with manual trimming and reconstruction

Proposed System Architecture • Improvements in its clinical application both at the level of data acquisition, processing and use. • Regarding the acquisition, it will be proposed to establish a protocol that includes the characteristics of the hardware to be used for the recording of images. • Validation work to define these standards is being carried out by the SP ROP Smartphone Telemedicine Team. • Regarding data processing, digital image processing techniques will be applied to improve its diagnostic quality [63–67]. • Standardization and normalization of images. • Storage and transport format for this type of data with a high degree of interoperability. • This ROP diagnostic information (continuum severity of vascular changes, ROP Stage, Fig. 29) can form a large database of clinical cases not only with diagnosis but also with data that can be linked to the maternal medical history, demographic data, nutritional data, the nutritional status of the baby and its weight gain, treatments on the premature infant, etc.

• This database will contain large volumes of information. The architecture of the information system was defined according to the scheme of Fig. 30. • A comprehensive telemedical system is proposed, based on APPs that capture, preprocess and standardize the recorded image sequences, considered as an image provider of a PACS (ModalityType), adopting the DICOM standard [68]. • For the implementation of this system, three stages were devised: 1. Establishing a medical imaging modality with the smartphone by developing an APP that records the Digital Indirect Ophthalmology videos and those in accordance with the DICOM standard. It must interoperate with a PACS server using HL7 to obtain the worklists or load the patient's attributes manually [69]. 2. Stage of processing and standardization of the images. This stage involves two steps: the first is to improve and protocolize the imaging in such a way as to be able to

278

A. V. de Kartzow et al.

Fig. 29 Image diagnostic Data from ROP: superior: continuum severity of vascular changes; below: ROP stages with no retinal detachment, considered by SP ROP algorithm

compare successive studies in the same patient, add the image processing modality to the PACS in such a way that from the acquired video not only the area of interest is obtained, but also that it may have the sufficient quality to apply algorithms for its characterization and diagnosis [70]. 3. Development of automatic learning algorithms for the delineation, registration and automatic diagnosis that allow automating some of the tasks in the ROP screening. • DICOM enables the integration of scanners, servers, workstations, printers and network hardware from multiple vendors within an image storage and communication system. Different machines, servers and workstations

have a DICOM conformance statement (conformance statements) that clearly states the DICOM classes they support. • DICOM has been widely adopted by hospitals and is making inroads into small clinics and doctor’s offices. It complies with three main premises, interoperability, security and sustaining the quality of data and images. The development of the TEST version for Android is presented (Fig. 31), also data requested from the patient manually or through Worklist (HL7) (Fig. 32). After completing the data and pressing the “Capture Image” button, the capture application of the smartphone itself opens, allowing you to select a single image or videos.

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America Fig. 30 Methodological summary of work with their respective standard communication protocols. Green dotted line: First stage of work, Smartphone application. Yellow upper box: Second stage of work

Fig. 31 Android App “Test” configuration menu with the PACS server

279

280

A. V. de Kartzow et al.

Fig. 32 Android App “Test” patient data entry menu and capture image button

• Patient data and video images converted to image series are integrated into DICOM files adding as “Modality Type” to the ODI Smartphone along with the capture data. The guidelines are followed as indicated by WG09 (DicomWorkingGroup 09): Ophthalmology, American Academy of Ophthalmology (AAO) (Fig. 33).

• Analyzing each processed and cropped frame, 58% of the frames have a digital resolution of 2 Mpixels. • Conclusions specifically in a new instrument for digital diagnosis in ophthalmology, respecting international standards. In this design, the objectives of giving added value to the diagnosis have been pursued, facilitating

Fig. 33 Processing of the frames with information of interest of ROP; clipping and mask for collation of useful pixels

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

its use and ensuring diagnostic quality, portability and interoperability. • These objectives open a great workspace for a third stage that would meet the requirements of SP ROP, these being the application of intelligent systems such as big data, machine learning and convolutional neural network, and in the analysis of the applications of the first stage. • Finally, it has emerged from the analysis of the histograms that it is necessary to normalize the images, in their mean and deviation, in order to facilitate the application of automatic techniques with the least possible variability. • Likewise, the present development in progress can be applied to other medical areas of primary care, timely referral and counter-referral systems, e.g., dermatology, dentistry and veterinary areas, adding the use of mobile video shaping to the DICOM standard that will have many potential uses in multidisciplinary prevention tasks, integrating information from healthcare teams, for better decision-making.

2. SNOMED CT and ICROP 3 Some useful concepts: • Health information: It is made up of thousands of ideas modeled in many ways. They include episodes from the lives of people, families and communities. It describes the perceptions of health team members, patients, politics, anatomical, physiological, psychological and cultural interconnections, and the stories that the people say about their lives and health. Health sciences are social sciences, and that complexity is reflexed in the domain of data information. If health records were limited to a single person or group of people, that complexity would not be a problem. That complexity arises when the information needs to be used for interventions in a whole population. There emerges the need of using statistical models.

281

Statistics: It is a branch of mathematics that relies on simplified models of reality to help us make decisions. Statistics is a tool that helps us to simplify complex domains, and to make big decisions. Classifications: They are tools of statistics. They are simplifications of reality. One of the objectives of classifications is to reduce the dispersion of a set of elements through their categorization into smaller groups: e.g., Classification CIE 10 reduces an enormous universe of ideas of health information into approximately 13,000 categories. This drafting is very useful for some purposes, and also has many unwanted side effects, like loss of meaning and accuracy, being the consequence of the loss of information that is precious for health care. Classification definitions: • Systems that arrange or organize like or related entities. • They are intended for the classification of clinical conditions and procedures to support statistical data analysis across the healthcare system. • They are designed to support other applications in health care, including reimbursement, public health reporting, quality or case care assessment, education, research and performance monitoring. This is the case of ICROP 3. • The only information that provides is a single “father-son” (monohierarchy) relationship between ideas. • They require entities to be classified in only one category. In health care, we are constantly facing the problem that one idea must be classified into several categories. Terminologies provide a common reference point for the comparison and aggregation of data about the entire healthcare process, recorded by multiple individuals, systems or institutions. Terminologies: • Are controlled healthcare vocabularies that encompass diseases, clinical findings, procedures and health outcomes.

282

• Can be used by physicians, nurses, allied health professionals and researchers. • Provide clinical content and expressivity for clinical documentation and reporting. • Allow modeling complex models through polyhierarchical relationships and through nonhierarchical relationships called Attributes. Thus when we store data in our information systems, if it belongs to a terminology, we not only have the data but also its terminological relationships. Data belonging to a terminology becomes a way of compressing an enormous amount of inferred information. The terminologies represent the complexity of a domain, and they can describe exactly what is happening to the patient. If we have to choose between Classification and Terminologies, it is not a matter of one or the other, it is a matter of using both. We can map terminologies with the elements of classifications. The benefits of using both tools: there are maps that connect both tools.

A. V. de Kartzow et al.

4.

5.

6. 7.

8.

Proposal: Terminology Extension of the ICROP 3 Classification It warrants:

9.

1. Accessibility: records will contribute to semantic interoperability across a wide range of clinical applications between healthcare providers in different clinical settings, and they can improve the capabilities of health 10. information exchange. e.g., ROP Smartphone Networks. 2. Semantic Interoperability: it can be defined as ensuring the precise meaning of exchanged information that is understandable by other systems or applications not initially 11. developed for this purpose. 3. Accuracy: clinical representations are automatically encoded using a variety of coding applications. Terminologies can capture all codes regardless of context. Incorrect data

resulting from human errors are unlikely. In contrast to classifications coding systems, in which human judgment is an important element of the coding process. Comprehensiveness: terminologies have better coverage of the clinical ideas. New concepts can be built from existing concepts. That contributes to the extensibility of the domain. We do not have to wait for the terminology to be adapted or updated. We just extend the terminology. Consistency: concepts of terminology are consistent among different users and across all clinical applications. Currency: terminologies can be updated continuously according to the needs. Ganularity: terminologies are more specific than classifications. In contrast, less common diseases in classifications are grouped together in “catchall” categories. For example, Adult ROP, which can lead to loss of information. Definition: because of its logical structure, terminology makes more sense and is easier for clinicians to understand. Contrasting with classifications that can be impeded by coding conventions and sometimes clinically irrelevant details needed for reimbursement of healthcare services. Relevancy: A clinical terminology could be more useful in clinical applications, information retrieval, and research. In contrast, classification systems are intended for the classification of clinical conditions and procedures in other applications. Precision: We can represent any aspect of the specific domain very precisely. Unlike classifications, terminologies do not allow the presence of ideas like “Unspecified” and “other specified” that can affect the ability of systems to collect data related to certain conditions, procedures, etc. Timeliness: Terminologies are designed to be used at the point of care by clinicians, while classification codes are usually assigned by professional coders after the patient's episode of care is complete.

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

Proposal: To build an extension of the SNOMED CT [61] terminology to model the complete sub-domain of ROP, including all its aspects. The ROP domain is composed of multiple highly intertwined sub-domains (Fig. 34). Those names constitute a complex of clinically blended ideas that will be recorded in the Electronic Health Records (EHR). The richness of free text can be kept with ideas built with terminology, while at the same time avoiding the problems we all know about. The entire ROP domain is composed of model aspects, each one contributing in an important way to the construction of the clinical idea, e.g., choosing the right treatment at the right time. ROP uses special zoning of the eye anatomy. These zones are not available in the anatomic structure in the terminology. However, we can extend the Anatomical Structure (Fig. 35). Several synonyms can be used in terminologies. We can create definitions in several languages. We can add more information to every concept, e.g., definition of Zone II Posterior: from Zone I (defined as 2 radio papilla-fovea) adding one disk diameter of Zone extension. We can

Fig. 34 Schematic model of the ROP domain, showing all aspects. Many models, as required in different networks, can be modeled and updated, via SNOMED CT Terminology Standard

283

build many definitions, save the changes and finish the process. Something that was not present can be simply created. Terminologies can create an extension of any aspects of sub-domains that we may need, like Extension of Disorders (ROP Zone, Stage, Vascular status, etc.). Definition of Concepts by Attributes (Fig. 36): relation of concepts not by hierarchical relationships but through attributes. Just like a Classification, we can ask for: • List of all the patients that have ROP of bilateral eye in Zone II Stage 1 Pre Plus • List of patients that have a ROP at postmenstrual age, lesser than 33 weeks • List of all the patients that have ROP • List of all the patients that have Type 1 ROP, etc. Once we finish constructing the extension, we have a Classification model or a classifier. When you use terminologies in Electronic Medical Records, every time that you use pediatric ophthalmology, ROP, the conception models, a lot of new concepts spare with precise definitions. Thus, we can obtain tons of new information.

284

A. V. de Kartzow et al.

Fig. 35 SNOMED CT Terminology Standard: Extension of Anatomical Structures. The New ICROP 3 anatomical definitions can be modeled for integration to EHR

Fig. 36 SNOMED CT Terminology Standard: Definition Concepts by Atribute. Different concepts and ideas in ROP Evaluation, sharing useful information through interoperability, in networks

The major benefit of using terminology is that electronic health records are not simply records but become meaningful records. Meaningful Health Records allow inferences to be made about the health and needs of the patient and the entire population (Fig. 37):

• The biggest problem in the construction of algorithms has nothing to do with technologies or techniques. It has to do with understanding the concepts to be modeled. • The Primary objective of terminologies is to ensure that the meaning (semantics) of ideas is conveyed undisturbed, without change.

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

285

Fig. 37 SNOMED CT Terminology, ICROP 3 and SP ROP Decision-Making Algorithm: meaningful records model. The New Classification of ROP can be modeled and integrated into EHR with more complexity

The SP ROP Algorithm must interchange at least 3 variables: • Stage of the Disorder • Presence/Absence of Plus Disease • Anatomical Zone affected. With Terminologies, we can see the meaning of these concepts. The algorithm uses one single concept. The complexity of terminologies avoids potential errors and bias in decision-making in every case of ROP evaluated by the algorithm constructed by Clinical Terminologies, integrated into the Electronic Medical Records. In the case of the use of Smartphones, current technologies allow the use of electronic health records through portable devices, adding value to the examination and use of information, with better decision-making in ROP, especially in Networks and Prevention of blindness Projects.

Artificial Intelligence and ROP Smartphone Images in Latin America SP ROP Mexican Team/Microsoft Azure Project Interest in Smartphone funduscopy and artificial intelligence as a tool to help in diagnostic, followup and decision-making is growing worldwide. Regarding the Icrop 3 new concept of continuum spectrum of vascular severity, assistive and autonomous diagnostic software for ROP 8. Analytic software could provide a risk estimate of adverse outcomes at each evaluation based on a multi-point scale of vascular severity, zone and stage over time, as well as incorporating additional factors that influence the clinical course. It is conceivable that a computer could examine images of the retina using previous machine learning training to categorize Zone I vascular severity in a more objective manner than a clinician can [71, 72].

286

With the support of Microsoft Philanthropist, the Association to Prevent Blindness in Mexico, Hospital “Luis Sánchez Bulnes” I.A.P. (APEC), Business Data Evolution and their partner organizations have developed an algorithm based on Microsoft’s AI to detect blindness in premature babies by capturing an image of the fundus with a mobile phone. APEC worked with Business Data Evolution and various Latin American organizations to develop a project that uses Azure’s Artificial Intelligence (AI) to determine Retinopathy of Prematurity (ROP) in time and, consequently, prevent blindness in premature babies. Artificial Intelligence for timely diagnostics. In Mexico, 12 children a day lose their vision due to ROP. One of the main causes of blindness in children is the lack of resources for an early diagnosis of the condition, especially since such detection must be made in the first few weeks of a newborn’s life. Unfortunately, in Mexico and in many Latin American countries, citizens lack access to a routine ophthalmic exam, much less treatment. The goal is to ensure that the disease is diagnosed in time so that it can be cured: • Purpose to democratize Artificial Intelligence techniques for ROP diagnosis allowing a quick and effective treatment for critical cases. • Interdisciplinary team of RoP experts, Artificial Intelligence and Technology. International collaboration from Mexico, Colombia and Argentina. • Software was donated by Microsoft Philanthropist Tech for a social impact program. • Deep learning networks are capable to learn to discriminate between healthy or sick retinas from an image dataset previously classified by an ROP expert team. • We used Transfer Learning on a pre-trained convolutional deep neural network (InceptionV3) for automatic feature engineering adding a couple of layers, one fully connected and a final logistic (softmax activation function) for binary classification (health, sick). • We trained two models: one with photographs taken by cellphone and another one with images from a Pictor Camera. We collected

A. V. de Kartzow et al.

108 cellphone images for no ROP (healthy) and 239 with any ROP level (sick). For Pictor Camera, we worked with 284 no ROP images and 83 with any ROP level. • A Cloud Platform as well as Artificial Intelligence tools in Azure were provided by Microsoft Philanthropist for modeling and implementation purposes. A random split of 80% training and 20% validation was made for final modeling and testing. The dependent variable was binary: healthy or sick. After 7 Epochs, the model achieved 84% accuracy on validation data for cellphone images and 72% for Pictor Camera with 5 Epochs. Baseline accuracy by human experts is around 60%. Finally, LIME image explainer was applied for opening the neural network black box and obtain a visual explanation of the reasons that the model considered to assign a probability of no ROP versus any ROP level to an image. In recent years, Artificial Intelligence experienced a high improvement and accessibility for modeling. Azure platform allows to democratize technology for healthcare practitioners and train new models using previously defined architectures without needing a big number of images. The final implementation of this model, as an app installed in the cellphone, allows a quick, accurate and low-cost screening in geographic areas where access is difficult for an ROP expert and professional equipment for evaluation. A worldwide web service will be available for uploading an image and obtaining, in real time, the probability of ROP, as well as a second image showing the affected retina area that is identified by the model as sick. This work results just pretend to assign a pre-diagnosis based on AI and prioritize cases for the final decision by an ROP expert. The final model and web service could be useful as a training platform for students interested in ROP specialization. An algorithm based on Microsoft Azure The main component of the solution being developed by APEC, through Microsoft Cloud, is the deep neural network algorithm based on Machine Learning, previously trained for ROP

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

diagnosis, using Transfer Learning in a previously trained deep convolutional neural system. Along the way, the project collected around 3,000 photographs, and specialists helped us sort them into sick and healthy eyes. Then, Azure’s Artificial Intelligence uses the collected images to train the algorithm to recognize common patterns among diseased eyes, as compared to healthy eyes (Fig. 38). The process includes a series of steps to evaluate the algorithm’s reliability indexes: the first step involved collecting the photographs, the second, training the algorithm, and the third step focuses on validating it with never-before-seen photos, as well as verifying its accuracy indexes, which are above 85%, compared to 60% when completed by humans (Fig. 39) [32]. Through Microsoft Azure’s solutions, such as Blob Storage, Cognitive Services and virtual machines, the algorithm works as an emulator of the human eye. Those neural networks have learned to distinguish colors, lines and edges (Fig. 40). In addition, a global web service was installed in the algorithm’s neural network to upload the photos and be able to obtain the probability of the presence of ROP in real time (Fig. 41).

287

A mobile app to detect cases The second component of the solution that is being developed by APEC and Business Data Evolution, also on Microsoft Azure, is a mobile phone app. The app will allow specialists to take a picture of a baby’s eyes from their cell phones. The image will be automatically sent to the web service and processed with APEC’s AI algorithm (Figs. 42 and 43). The technical objective of the app is to make it possible for an ophthalmologist to obtain the detail of the baby’s retina with his or her own mobile phone using an Indirect Ophthalmoscopy lens. Then, by using the algorithm, the app lets the user know, in a matter of seconds, whether or not the child suffers from ROP. Vessels status [73], regarding the Continuous Spectrum of Vascular Severity, can be evaluated (Fig. 44). The Mexican Project with Regional collaboration will promote this AI tool for other countries to implement the use of AI solutions to save children’s vision in the region and around the world. The SP ROP Decision-Making Algorithm can be integrated into the case evaluation (Fig. 45).

Fig. 38 Microsoft Azure Philanthropist and APEC Mexico AI Project: workflow of images evaluation

288

A. V. de Kartzow et al.

Fig. 39 Microsoft Azure Philanthropist and APEC Mexico AI project: Azure plattform deep learning processes

Fig. 40 Microsoft Azure Philanthropist and APEC Mexico AI project: cognitive services. The algorithm works as an emulator of the human eye

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

289

Fig. 41 Microsoft Azure Philanthropist and APEC Mexico AI project: healthy versus sick premature eyes

Fig. 42 Smartphone image acquisition, delivered through an App to Azure’s AI Cloud for evaluation

Conclusion Smartphone Networks for better decision-making in ROP severe cases, diagnostic, treatment and follow-up can be a cost-efficient way to integrate useful information in Latin American settings. Working in Timeline of Prematurity, integrating validated Smartphone ROP Retinal images with Electronic Personal Health Records, through e-health standards can be an important tool, to make better decisions with less errors.

The SP ROP proposals and research projects support the scope of working in the premature newborns’ lifetime timeline of their Personal Electronic Health Records. In the scheme below, we synthesize the tools, projects and ideas, integrating them with the international PHO WHO e-health standards. With this information and validated images, more accurate evaluations will be made in the future on health care of prematurity and ROP Networks, sharing useful information through the timelines (Fig. 46).

290

A. V. de Kartzow et al.

Fig. 43 Microsoft Azure Philantropics and APEC Mexico AI Project: Automatic best retinal picture extraction

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

291

Fig. 44 Left: ICROP 3 continuous spectrum of vascular severity. Right: AI automated vessels image extraction for AI evaluation

Fig. 45 Deep learning process: once automatic image extraction is performed from the videos, and vessels demarcation the AI Platform is trained through deep convolutional neural networks, learning to detect normal from sick vessels or vascular changes. The ICROP 3 continuous spectrum of vascular severity can be detected, as well as regression and reactivation vascular patterns. *The AI algorithm and the mobile app developed as part

of the described initiative was a joint effort by the following organizations: Clínica Oftalmológica Peñaranda (Colombia), Red RoP de la Provincia de Buenos Aires (Argentina), Centro Integral de Salud Visual Daponte (Argentina), Hospital Italiano de Buenos Aires (Argentina), Clínica de Alta Especialidad Visual (Mexico), Association to Avoid Blindness (Mexico), Business Data Evolution (Mexico)

292

A. V. de Kartzow et al.

Fig. 46 SP ROP proposals and tools to work in early detection of ROP. This ICROP 3 classification-based scheme shows the integration of all the projects, for a future addition of data to electronic health and personal

records in premature patients’ timeline. Data will be shared until adult life, to make better decisions in prevention and health care

Acknowledgements Paulina Ramirez Neria MD, Brenda Peña MD, Celia Sanchez MD, Julieta Pereyra MD, Lisseth Chinchilla MD, Tomás Ortiz Basso MD, Bruno Boietti MD, Carlos Peñaranda MD, Carlos Valdes Lara MD, Alex Sanchez MD, Nicolas Crim MD, Andrés Kytchethal MD, Rodrigo Torres, MD, Marina Brussa MD, Juan Carlos Silva MD, Silvina Gonzalez MD, Ana Speranza MD, Giselle Ricur MD, Andrea Zin MD ,Julio Urrets Zavalia PhD MD, Domenico Lepore MD, Graham Quinn , MD, MSCE.

4. Lawn JE, Vazquez T, Fielder A, Gilbert C. Website [Internet]. Preterm-associated visual impairment and estimatesof retinopathy of prematurity at regional and global levels for 2010. Pediatr Res. 2013;74(suppl 1):35–49. 10. 1038/pr.2013.205 JAMA Network Open. 2021;4(5):e219245. https://doi.org/10.1001/ jamanetworkopen.2021.9245 (Reprinted) May 5, 2021 2/2. https://jamanetwork.com/ on 06/03/2021. 5. Vázquez A, Standen J. Retinopatía de la prematuridad. Incidencia en 16 años de seguimiento. Hospital público, Viña del Mar (1996–2012). 2013; Oct, 20–25 [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/ c602dc58-06fa-0acd-9e7d-be3654491033. 6. Gilbert C, Rahi J, Eckstein M, O’Sullivan J, Foster A. Retinopathy of prematurity in middleincome countries [Internet]. Lancet. 1997;350:12–4. https://doi.org/10.1016/s0140-6736(97)01107-0. 7. Carrion JZ, Fortes Filho JB, Tartarella MB, Zin A, Jornada ID Jr. Prevalence of retinopathy of prematurity in Latin America. Clin Ophthalmol. 2011;1 (5):1687–95. 8. Website. 9. Zin AA, Moreira MEL, Bunce C, Darlow BA, Gilbert CE. Retinopathy of Prematurity in 7 Neonatal Units in Rio de Janeiro: screening criteria and workload implications [Internet]. Pediatrics. 2010; 126:e410–7. https://doi.org/10.1542/peds.2010-0090. 10. Arnesen L, Durán P, Silva J, Brumana L. A multicountry, cross-sectional observational study of retinopathy of prematurity in Latin America and the

References 1. Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84(2):77–82. 2. Chan-Ling T, Gole GA, Quinn GE, Adamson SJ, Darlow BA. Pathophysiology, screening and treatment of ROP: a multi-disciplinary perspective. Prog Retin Eye Res. 2018;62:77–119. 3. Bourne RRA, Flaxman SR, Braithwaite T, Cicinelli MV, Das A, Jonas JB, et al. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis [Internet]. Lancet Glob Health. 2017; 5:e888–97. https://doi.org/10.1016/s2214-109x(17)30293-0.

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

Caribbean. Rev Panam Salud Publica. 2016;39 (6):322–9. Website [Internet]. Available from: Website [Internet]. Available from: Organización Panamericana de la Salud. Plan de Acción para la Prevención de la Ceguera y de las deficiencias Visuales 2009–2013. 49o Consejo Directivo de OPS. 61 a Sesión del Comité Regional de OMS para las Américas; 28 septiembre –2 octubre. 2009. Washington: OPS, 2009 http://www.paho.org/hq/dmdocuments/2009/ CD49-19-s.pdf?ua=1. Website [Internet]. Available from: Website [Internet]. Available from: Organización Panamericana de la Salud. Plan de Acción para la Prevención de la Ceguera y de las deficiencias Visuales.2014–2019. 53o Consejo Directivo de OPS. 66 a Sesión del Comité Regional de OMS para las Américas; 29 Septiembre – 3 Octubre. 2014. Washington: OPS, 2014. https:// www.paho.org/hq/index.php?option=com_content& view=article&id=9774:2014-53rd-directing-council &Itemid=40507&lang=es. Website [Internet]. Available from: Organización Panamericana de la Salud. Guía de práctica clínica para el manejo de la retinopatía de la prematuridad. Washington D.C: USA, OPS, 2017. http://iris.paho. org/xmlui/handle/123456789/34948. Hariharan L, Gilbert CE, Quinn GE, Barg FK, Lomuto C, Quiroga A, et al. Reducing Blindness from Retinopathy of Prematurity (ROP) in Argentina Through Collaboration, Advocacy and Policy Implementation [Internet]. Health Policy Plan. 2018;33:654– 65. https://doi.org/10.1093/heapol/czy004. Website [Internet]. Available from: Alda E, Lomuto CC, Benítez AM, Bouzas L, Brussa M, Cattaino A, et al. Results of the national program for the prevention of blindness in childhood by retinopathy of prematurity in Argentina (2004–2016). Arch Argent Pediatr. 2018;116(6):386–393. https://doi.org/10.5546/ aap.2018.eng.386 https://www.sap.org.ar/docs/ publicaciones/archivosarg/2018/v116n6a05e.pdf. Resultados del Programa Nacional de Prevención de la Ceguera en la Infancia por Retinopatía del Prematuro en Argentina (2004–2016) [Internet]. Archivos Argentinos de Pediatria. 2018;116. https:// doi.org/10.5546/aap.2018.386. Retinopatía del prematuro [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/188ab9ed-daa30749-a887-76d8989be119. Orozco: Retinopatía del prematuro en Latinoamérica [Internet]. [cited 2021 Aug 29]. https://paperpile.com/ app/p/319458ad-66c7-0cac-b09a-c47ca341d5f7. Networking by WhatsApp: smartphones and their use for Retinopathy of Prematurity (ROP) in Argentina and Latin America. Vol.61, PB00112. [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/ 73f66b26-4abd-0756-8fe4-ea540ae1dc80. Gilbert C. characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development:

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

293

implications for screening programs [Internet]. Pediatrics. 2005;115:e518–25. https://doi.org/10.1542/ peds.2004-1180. Ossandón D, Zanolli M, Stevenson R, Agurto R, Ortiz P, Dotan G. A national telemedicine network for retinopathy of prematurity screening [Internet]. J Am Assoc Pediatr Ophthalmol Strabismus. 2018;22:124–7. https://doi.org/10.1016/j.jaapos. 2017.11.005. Richter GM, Williams SL, Starren J, Flynn JT, Chiang MF. Telemedicine for retinopathy of prematurity diagnosis: evaluation and challenges [Internet]. Surv Ophthalmol. 2009; 54:671–85. https://doi.org/ 10.1016/j.survophthal.2009.02.020. Quinn GE. Need for telemedicine in retinopathy of prematurity in middle-income countries—reply [Internet]. JAMA Ophthalmol. 2015; 133:361. https://doi.org/10.1001/jamaophthalmol.2014.4921. Jackson KM, Scott KE, Graff Zivin J, Bateman DA, Flynn JT, Keenan JD, et al. Cost-utility analysis of telemedicine and ophthalmoscopy for retinopathy of prematurity management. Arch Ophthalmol. 2008;126(4):493–9. Patel SN, Martinez-Castellanos MA, BerronesMedina D, Swan R, Ryan MC, Jonas KE, et al. Assessment of a tele-education system to enhance retinopathy of prematurity training by international ophthalmologists-in-training in Mexico [Internet]. Ophthalmology. 2017; 124:953–61. https://doi.org/ 10.1016/j.ophtha.2017.02.014. Website [Internet]. Available from: Saidman G, Monteoliva G, Urrets-Zavalia JA. 2 1 Ophthalmology section, department of Neonatology, Evita Pueblo Hospital, South Zone ROP net, Berazategui, Buenos Aires, Argentina. 2 Department of Ophthalmology, University Clinic Reina Fabiola, Universidad Catolica de Cordoba, Argentina. Headbandmounted holder for hands- free indirect fundoscopy through a smartphone [Internet]. 2016. https://pdfs. semantics-cholar.org/1a92/b3d8f7e2e36bf99d2cf45e 6d2009da0164a9.pdf. Binocular Ophthalmoscopy versus Smartphone images for Retinopathy of Prematurity (ROP) Telemedicine in Latinamerica: a preliminary report of diagnostic … [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/b8c23172-4eff00a9-85b9-498d4804fed5. Prematurity TCFTC of R of, The Committee for the Classification of Retinopathy of Prematurity. An International Classification of Retinopathy of Prematurity [Internet]. Arch Ophthalmol. 1984; 102:1130– 4. https://doi.org/10.1001/archopht.1984.010400309 08011. International Committee for the Classification of Retinopathy of Prematurity. The international classification of retinopathy of prematurity revisited. Arch Ophthalmol. 2005;3(7):991–9. Chiang MF, Quinn GE, Fielder AR, Ostmo SR, Paul Chan RV, Berrocal A, et al. International

294

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

A. V. de Kartzow et al. classification of retinopathy of prematurity, Third Edition. Ophthalmology [Internet]. 2021. https://doi. org/10.1016/j.ophtha.2021.05.031. Wang J, Ji J, Zhang M, Lin J-W, Zhang G, Gong W, et al. Automated explainable multidimensional deep learning platform of retinal images for retinopathy of prematurity screening. JAMA Netw Open. 2021;4 (5): e218758. Quinn GE. A step forward in using artificial intelligence to identify serious retinopathy of prematuritya start with a long road ahead. JAMA Netw Open. 2021;4(5): e219245. Campbell JP, Kalpathy-Cramer J, Erdogmus D, Tian P, Kedarisetti D, Moleta C, et al. Plus disease in retinopathy of prematurity: a continuous spectrum of vascular abnormality as a basis of diagnostic variability. Ophthalmology. 2016;123(11):2338–44. Campbell JP, Peter Campbell J, Ataer-Cansizoglu E, Bolon-Canedo V, Bozkurt A, Erdogmus D, et al. Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis [Internet]. JAMA Ophthalmol. 2016;134:651. https://doi.org/10.1001/jamaophthalmol.2016.0611. Dorta P, Kychenthal A. Treatment of type 1 retinopathy of prematurity with intravitreal bevacizumab (AVASTIN) [Internet]. Retina. 2010; 30: S24–31. https://doi.org/10.1097/iae. 0b013e3181ca1457. Padhi TR, Besirli CG. Re: Lepore et al. Intravitreal bevacizumab versus laser treatment in type 1 retinopathy of prematurity: report on fluorescein angiographic findings (Ophthalmology 2014;121: 2212–9) [Internet]. Ophthalmology. 2015; 122:e47– 8. https://doi.org/10.1016/j.ophtha.2015.02.005. Yetik H. Re: Lepore et al. Intravitreal bevacizumab versus laser treatment in type 1 retinopathy of prematurity: report on fluorescein angiographic findings (Ophthalmology 2014;121:2212–9) [Internet]. Ophthalmology. 2015; 122:e49. https://doi.org/10. 1016/j.ophtha.2014.11.031. Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RVP, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136(7):803–10. Choi RY, Brown JM, Kalpathy-Cramer J, Chan RVP, Ostmo S, Chiang MF, et al. Variability in plus disease identified using a deep learning-based retinopathy of prematurity severity scale. Ophthalmol Retina. 2020;4(10):1016–21. Revised Indications for the Treatment of Retinopathy of Prematurity [Internet]. Arch Ophthalmol. 2003;121:1684. https://doi.org/10.1001/archopht. 121.12.1684. Hamad AE, Moinuddin O, Blair MP, Schechet SA, Shapiro MJ, Quiram PA, et al. Late-onset retinal findings and complications in untreated retinopathy of prematurity. Ophthalmol Retina. 2020;4(6):602–12.

42. Cernichiaro-Espinosa LA, Williams BK Jr, MartínezCastellanos MA, Negron CI, Berrocal AM. Peripheral vascular abnormalities seen by ultra-widefield fluorescein angiography in adults with history of prematurity: report of three cases. Ophthalmic Surg Lasers Imaging Retina. 2018;49(4):278–83. 43. Chang E, Rao P. Adult retinopathy of prematurity: treatment implications, long term sequelae, and management. Curr Opin Ophthalmol. 2021;32 (5):489–93. 44. Informática en salud orientada a la comunidad. Edición de Kindle [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/207f1871-646c-0aa9-83 05-ef07f7b898aa. 45. Development and Evaluation of Reference Standards for Image-based Telemedicine Diagnosis and Clinical Research Studies in Ophthalmology. 2014;2014: 1902–1910 [Internet]. [cited 2021 Aug 29]. https:// paperpile.com/app/p/a6b23ee8-ec20-0b07-9573-130 4c1eddc4a. 46. Enforcement policy for remote ophthalmic assessment and monitoring devices during the coronavirus disease 2019 (COVID-19) Public health emergency guidance for industry and food and drug administration staff April 2020 [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/44352851-ee83-00c6-8d 99-51b93d3d9db3. 47. Shahbaz R, Salducci M. Law and order of modern ophthalmology: teleophthalmology, smartphones legal and ethics. Eur J Ophthalmol. 2021;31(1):13–21. 48. How to Take Retinal Images with a Smartphone [Internet]. [cited 2021 Aug 29]. https://paperpile.com/ app/p/0461a95d-34d1-0241-9428-ac08068110fc. 49. Haddock LJ, Kim DY, Mukai S. Simple, Inexpensive technique for high-quality smartphone fundus photography in human and animal eyes [Internet]. J Ophthalmol. 2013;2013:1–5. https://doi.org/10.1155/ 2013/518479. 50. Iqbal U. Smartphone fundus photography: a narrative review [Internet]. Int J Retin Vitr. 2021:7. https://doi. org/10.1186/s40942-021-00313-9. 51. Khanamiri HN, Nakatsuka A, El-Annan J. Smartphone fundus photography [Internet]. J Vis Exp. 2017. https://doi.org/10.3791/55958. 52. Website [Internet]. Fung THM, Kuet M-L, Patel, CK, et al. Retinal imaging in infants. Surv Ophthalmol. https://doi.org/10.1016/j.survophthal.2021.01.011. 53. How to Take Retinal Images with a Smartphone [Internet]. [cited 2021 Aug 29]. https://paperpile.com/ app/p/0461a95d-34d1-0241-9428-ac08068110fc. 54. Fundus Cameras and Smartphone—based Systems, adapted from “Imaging in Retinopathy of Prematurity”. Valikodath N, Cole E, [...], Chan RVP [Internet]. [cited 2021 Sep 1]. https://paperpile.com/app/p/ fb40393e-6f7f-0e11-bbde-e18085fa5499. 55. Networking by WhatsApp: smartphones and their use for Retinopathy of Prematurity (ROP) in Argentina and Latin America. Vol.61, PB00112. [Internet].

Smartphone Telemedicine Networks for Retinopathy of Prematurity (ROP) in Latin America

56.

57. 58. 59.

60.

61.

62. 63.

64.

[cited 2021 Aug 29]. https://paperpile.com/app/p/ 73f66b26-4abd-0756-8fe4-ea540ae1dc80. Binocular Ophthalmoscopy versus Smartphone images for Retinopathy of Prematurity (ROP) Telemedicine in Latinamerica: a preliminary report of diagnostic … [Internet]. [cited 2021 Aug 29]. https:// paperpile.com/app/p/b8c23172-4eff-00a9-85b9-498d 4804fed5. Website. Website. Organización Panamericana de la Salud. Conversaciones sobre eSalud. Washington: Organización Panamericana de la Salud; 2014 [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/27c0441f51db-0658-88ff-9fa4b7b70b83. Principios de interoperabilidad en salud y estándares. Edición de Kindle [Internet]. [cited 2021 Aug 29]. https://paperpile.com/app/p/c112d120-d775-0fd0-ab 30-917cc0221057. Benson T, Grieve G. SNOMED CT [Internet]. Principles of Health Interoperability. 2021;293–324. https://doi.org/10.1007/978-3-030-56883-2_16. Website [Internet]. Grupo ROP Argentina. http:// gruporopargentina.blogspot.com/. “Implementación y aplicación de algoritmos Retinex al preprocesamiento de imágenes de retinografía color” [Internet]. [cited 2021 Sep 1]. https:// paperpile.com/app/p/a277100b-f70a-012c-8a78-e305 44c73da6. Análisis e implementación de metodologías de segmentación automática de vasos sanguíneos en imágenes digitales de fondo ocular en color Escuela y Workshop Argentino de Ciencia de las Imágenes [Internet]. [cited 2021 Sep 1]. https://paperpile.com/ app/p/39d40db7-3da9-06f0-8f39-1fd5fdb4887a.

295

65. Clasificador de Retinopatías Diabéticas en Imágenes de Fondo ocular Utilizando la Pendiente Espectral Multifractal [Internet]. [cited 2021 Sep 1]. https:// paperpile.com/app/p/620d6a10-15d1-07af-9b56-91f6 fbe3a3e7. 66. Comparación de dos métodos de Registración Aplicados a Imágenes Retinográficas—[Internet]. [cited 2021 Sep 1]. https://paperpile.com/app/p/aebbcded6403-02f7-8aa6-9850c0696707. 67. Monitoring of Changes in Fundus Image, [Internet]. [cited 2021 Sep 1]. https://paperpile.com/app/p/ 4be9f7b5-dd7b-080d-b791-114c8025ca2d. 68. Website [Internet]. Google FIREBASE. https:// firebase.google.com/?hl=es-419. 69. Website [Internet]. Pacs Server Orthank. https:// www.orthanc-server.com/download.php, Open source. [13] Pacs server DCM4CHEE, https://www. dcm4che.org, Open source. 70. Website [Internet]. Bootstrap v4.3.1,https:// getbootstrap.com, Open source. 71. Website [Internet]. A Revision of the International Classification of Retinopathy of Prematurity Michael X. Repka, MD, MBA Published:July 30, 2021. https://doi.org/10.1016/j.ophtha.2021.07.014 PlumX Metrics. 72. Quinn. A step forward in using artificial intelligence to identify serious retinopathy of prematurity—a start with a long road ahead, Quinn GE. JAMA Netw Open. 2021. 73. Moleta C, Campbell JP, Kalpathy-Cramer J, Chan RVP, Ostmo S, Jonas K, et al. Plus disease in retinopathy of prematurity: diagnostic trends in 2016 Versus 2007. Am J Ophthalmol. 2017;176:70–6.

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later Giselle Ricur , Roger Zaldivar , Roberto Zaldivar , and Raul Guillermo Marino

Abstract

Keywords

It has been more than 20 years since Instituto Zaldivar initiated its journey in Teleophthalmology. Our experience has always been based on synchronous eyecare models of delivery with the use of video conferencing technology to link the eye doctors with their patients at different remote sites. The situation created by the COVID-19 pandemic, with its initial lockdown period during the first wave (March 2020), boosted our teleophthalmology program into a hybrid initiative in just 45 days. This changed not only our patient’s experience but also the Instituto Zaldivar’s business model, as well as the physician’s buy-in of new operations. The new engagement rules under this novel scenario allowed us to create a nationwide network model which we coined “Zaldivar Virtual World”.

Telemedicine Teleophthalmology Cataract and Refractive Surgery Video consultation Hybrid model of care Synchronous remote examination

G. Ricur (&) Medical Education and Development Manager, Instituto Zaldivar, Mendoza, Argentina e-mail: [email protected] R. Zaldivar
R. Zaldivar Instituto Zaldivar, Mendoza, Argentina R. G. Marino Facultad de Ciencias Exactas y Naturales, UNCUyo, Mendoza, Argentina Secretaría de Investigación, Internacionales Y Posgrado, UNCUyo, Mendoza, Argentina














Introduction Teleophthalmology is definitely not a new term. Published papers, that can be readily accessed by simple MESH term search strategies, can be found as early as 1994 [1–3]. They define its first use, applications, and future trends. Although with time the original applications were oriented mostly for screening purposes, and specifically for posterior segment conditions such as diabetic retinopathy and optic nerve cupping in cases of chronic glaucoma, the external and anterior segment was also targeted due to its user-friendly approach. During the following years since its beginning, teleophthalmology applications have flourished and now encompass not only screening initiatives but also diagnosis and treatment orientation, tele mentoring as well as continuing education programs for both eyecare professionals and the general population in hopes of preventing vision-threatening conditions. Thus, over time, teleophthalmology—also known as ocular telehealth—has expanded to other areas of eye care because it can efficiently extend

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_19

297

298

patients’ access to care and improve the eyecare system’s productivity [4–6]. Nonetheless, over the past year and a half, we have witnessed one of the most paradigmatic shifts in the history of patient care, as the COVID19 pandemic challenged the continuum of care. With many patients and physicians confined to their homes, the global medical community turned to information and communication technologies (ICT) as a solution or tool to bridge the gaps created due to lockdown measures or the fear of risking getting sick with COVID. Currently, many countries in the Northern Hemisphere are overcoming the pandemic situation, and the Southern Hemisphere is still under the effects of the second wave. This reality has created a new normal in eye care, one in which telemedicine is being implemented both at eyecare facilities and in the homes of physicians and patients. In December of 1997, WHO offered its first working definition of Health Telematics as “a composite term for health-related activities, services and systems, carried out over a distance by means of information and communications technologies, for the purposes of global health promotion, disease control and health care, as well as education, management, and research for health” [7]. Later, in 2011, Bashshur et al. described telemedicine as “a modality of care that challenges the traditional sine qua non dependence on physical presence and contact between providers and patients for medical/healthcare delivery” [8]. And finally in March of 2020, HIMSS published a new perspective, where “digital health connects and empowers people and populations to manage health and wellness, augmented by accessible and supportive provider teams working within flexible, integrated, interoperable and digitallyenabled care environments that strategically leverage digital tools, technologies and services to transform care delivery” [9]. As we can see, the common denominator has always been the use of ICTs to support healthcare services, but with time the paradigm changed, focusing more on transforming strategically the healthcare models based on the patients’ and population’s needs.

G. Ricur et al.

It has been more than 20 years since we first started our journey in Teleophthalmology; we embrace HIMSS’ current view where ICTs facilitate digitally enabled care environments allowing healthcare providers to take appropriate care of our patients, changing their user experience regardless of location. The situation created by the COVID-19 pandemic, with its initial lockdown period during the first wave (March 2020), finally boosted our teleophthalmology program into a hybrid initiative in just 45 days. This changed not only our patient's experience but also the Instituto Zaldivar’s business model, as well as the physician’s buy-in of new operations. The new engagement rules under our scenario allowed us to create a nationwide network model which we coined “Zaldivar Virtual World”.

Our Evolution During These Past 20 Years As published before in the first edition of Springer’s h book “Teleophthalmology”, our experience has always been based on synchronous eyecare models of delivery with the use of video conferencing technology in order to link the eye doctors with their patients at different remote sites [10].

Sites While Instituto Zaldivar is still considered an ambulatory surgery center specializing mainly in cataract and refractive surgery, it has expanded both its outreach and specialty care over the years. Currently, as depicted in Fig. 1, it has five main locations, two in the province of Mendoza, Argentina, two in the nation’s capital city of Buenos Aires (approximately 1,000 km away from Mendoza), and one in the city of Montevideo, Uruguay. All five are connected virtually with 10 remote sites spread out through Argentina, creating the Zaldivar Virtual World Network (ZVW) with practitioners in different

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later Fig. 1 Remote site map: the following map shows the main locations of the different virtual sites. The orangecolored icons belong to Instituto Zaldivar’s main locations, while the red icons show the location of the ZVW remote sites

299

300

specialty areas, all supported by unified visual communication technology which also has evolved over the past 2 decades.

Telecommunication and Information Network In the past, we relied on a telemedicine program designed in 1997 to enhance our outreach. Initially, we used it to facilitate patient and physician education through our websites and to offer knowledge transfer by broadcasting live surgeries via satellite to scientific meetings and expositions. At that point in time, the audio and video feed for the virtual meetings were transmitted either by microwave or satellite links (Nahuel1A of Nahuelsat S.A.) with 972 MHz of Ku Band bandwidth; similar at the time to a T1 bandwidth capability, but with a slightly higher latency. Land-based telecommunication networks were all copper-based, with modem dialup connections which in fact were quite decent for website navigation and still image transfer (128 kbps). Push-to-talk communicators together with the traditional fax machines and the novel use of emails were used as backup communication channels between Mendoza and Buenos Aires to exchange clinical information. During late 2001 and early 2002, a severe economic, political, and social crisis hit Argentina and the country defaulted, producing a severe impact on patient flow and business. Almost amazingly coincidentally during that time, “Operation Lindbergh” (the world’s first remotely assisted telesurgery) had been accomplished, marking the introduction of video technology to the telemedical field. So amidst the national crisis, we decided to follow up with our patients remotely in real time. We leveraged our experience in analog video conferencing and opened our first virtual clinic in Buenos Aires, equipped with what was then considered state-of-the-art digital communication technology. Three ISDN lines were installed (384 kbps) and our first digital video conferencing solutions were acquired, a Tandberg Solution (Reston Virginia, US) with its IP-capable H323 video conferencing

G. Ricur et al.

codecs, and later on we transitioned toward the new Polycom units (San José, California, US) with multipoint capabilities, IP content sharing, and increased sound experience with stereo technology. At the time, we only had an end-toend VPN network in place between Mendoza’s headquarters and the original remote site in Buenos Aires. Unified Communications and Telepresence Our next step or upgrade came 10 years later, in 2011, with the arrival of the concept of unified visual communication solutions over IP with telepresence technology. That year, we started to design a new model of care (MOC) which we were finally able to implement at the end of 2012. At this point, the VPN was structured on a MPLS service that linked all 3 new sites with an Active Directory linked to all the resources including the EMR. As defined by Pleasant, Blair (2008) unified communications (UC) is “a business and marketing concept describing the integration of enterprise communication services such as instant messaging (chat), presence information, voice (including IP telephony), mobility features (including extension mobility and single number reach), audio, web and video conferencing, fixed-mobile convergence (FMC), desktop sharing, data sharing (including web connected electronic interactive whiteboards), call control and speech recognition with non-real-time communication services such as unified messaging (integrated voicemail, e-mail, SMS and fax)”. UC is not necessarily a single product, but a set of products that provides a consistent unified user interface and user experience across multiple devices and media types. When video conferencing is the primary component, it is characterized as a unified visual communication solution. Telepresence can be defined as a range of products designed to link two physically separated rooms so they resemble a single conference room, regardless of location. It's an innovative, new technology that creates a unique in-person experience between people, places, and events in their work over the Network. This allows for

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

301

Fig. 2 Graphic, demonstrating the timeline of the evolution of the use of ICTs applied to our visual healthcare model

optimized environmental conditioning to provide the best audio and video and overall user experience, replicating a life size in-person experience. Cloud-Based Video Conferencing Solutions Finally in 2018, we decided to partner with Google and migrated all our institutional workspace and applications to the G Suite Enterprise solution, which had a major impact on our Telemedicine program. Initially, the use of video conferencing through the Google Meet application was designated mainly for corporate meetings. But in March 2020, the pandemic accelerated its use and our medical consultations moved from the CISCO-based video conferencing system to “Google Meet”, since the patients were no longer walking into the clinics that hosted the Telepresence Suites. By using Google´s WebRTC API, we were able to add realtime communication capabilities, thus supporting video, voice, and generic data that could be sent between the different sites, allowing for screen sharing of the data and imagery while performing the video call over the web browser. In summary, the following graphics show the timeline and how we have evolved over the years, based on technological innovations and the evolution of our video conferencing portfolio that have allowed us to improve our services (Fig. 2).

Teleconsultation Examining Rooms and Lanes Design Ophthalmic examining rooms or lanes design also evolved over the years with the never-

ending adoption of new ophthalmic diagnostic tools and solutions. The digital transformation has cut deeply into all the processes, including the design and decor of the telemedicine suites in order to make room for the new hardware, while still optimizing the circulation space required to maximize time spent seeing a patient. In early 2000, the exam rooms hosted the usual ophthalmic diagnostic equipment. The slit lamps were originally adapted with 1 CCD video camera; the video images were displayed on color CRT (cathode-ray tube) monitors, either on television sets or computer monitors (Fig. 3) and the novel IP Tandberg VTC solution was used. While ergonomic factors were considered, in relation to the proximity of the monitors to both the physician’s and the patient’s view, in no way were they projected or scaled up to provide a detailed and close-up view that could induce an immersive feeling. Nonetheless, our communication and etiquette protocols enhanced the experience, and our patient satisfaction surveys at the time revealed that they in fact did feel comfortable, could hear and see well their providers, and were in general very satisfied with the experience [10]. Over the course of the following years, new cameras, and hardware and software solutions were commercially available, and based on the scope of our program, we purchased those that targeted a more user-friendly environment (Fig. 4). In 2011, we designed our first telepresence suites both in Mendoza and Buenos Aires (Fig. 5a,b). The new telemedicine suites were custombuilt to comply with the products and room layout requirements and standards that CISCO (San Francisco, US) required in order to ensure

302

G. Ricur et al.

Fig. 3 Photograph, courtesy of Instituto Zaldivar Archive, that documents the use of 1CCD video cameras adapted to conventional slit lamps

Fig. 4 Photographs, courtesy of Instituto Zaldivar Archive

Fig. 5 The Instituto Zaldivar virtual telemedicine clinics in Buenos Aires (a) and Mendoza (b), equipped with TelePresence solutions. Photographs courtesy of Instituto Zaldivar Archive

an immersive experience. By the end of 2012, these two suites were finally equipped with 3000 TelePresence Systems each. All the remaining examining rooms and lanes received either the E20 VoIP phone series or the EX90 all-in-one communicating solution with a highdefinition resolution video that enabled telepresence encounters, with a 6 Mbps point-topoint bandwidth. These units had an added value, since they had incorporated a document camera

that allowed for easy, instant document or diagnostic imagery sharing between the connected sites. As the years passed, the legacy units were replaced with their updated versions, and in 2018 we merged our telepresence processes with cloud-based solutions. We now host both the original TelePresence suites (Cisco) with new virtual eye lanes equipped with Google Corporate video conferencing solutions (G SUITE

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

303

Project Team and Protocols

Fig. 6 Updates to the clinic’s telemedicine program included cloud-based video conferencing solutions at all 15 locations, so physicians can perform in-person and virtual patient examinations, regardless of the examining office’s or lane’s design. Photograph courtesy of the author: Giselle Ricur, MD, MBA, MS, FATA

ENTERPRISE) at all 15 locations. We completely transformed our workflow processes into hybrid eyecare models where the physicians regularly attend both in-person and virtual patient visits, with either technologies, depending on the day of the week. This can be done in any office, anytime, and the process is quite user-friendly (Fig. 6).

As reported in the past, since the initiation of the teleophthalmology program, the project has always been supported by the work of an interdisciplinary team. Physicians, ophthalmologists, ophthalmic technicians, IT specialists (analysts, developers, and programmers), community managers, and audio–video broadcasting experts have worked side-by-side to improve the workflow designs and support the implementation of the program and the capacity building of the Institute’s personnel. Protocols involving patient engagement, consent forms, real-time medical attention, and follow-up, as well as satisfaction metrics and other key performance indicators, have been constantly reviewed and updated as needed. This was a critical step, especially in the midst of the pandemic scenario where in-house solutions were designed and implemented to accommodate 100% virtual consultations. This implied not only synchronizing patients’ and providers’ agendas but also testing their IT devices, bandwidth capabilities, and digital literacy as well as redesigning the reimbursement coding and billing processes, and last of all rethinking the patient-training logistics in the use of the new telemedicine platform.

Methods For the purpose of the chapter, although we retrospectively analyzed all the virtual consultations performed at Instituto Zaldivar since the year 2001, we focused our analysis on two data sets: all the virtual consultations performed with telepresence (2012–2021), and specifically those occurring during the first wave of the COVID pandemic: March 2020–April 2021. OriginPro 9 SR2 software was used to analyze the data, and then exported to Microsoft Excel (Microsoft Office Professional Plus 2016) for the purpose of designing the graphics used. All patients signed consent forms when booking their teleconsultations, and our workflow processes passed both ISO 9001 audits and reviews from our IRB.

VEX: The Next Normal in Virtual Workflows Regardless of the technology used, telemedicine has been an integral part of our patient care model for more than 20 years. As stated above, we redesigned our workflow and mastered the virtual eye examination experience (VEX) based on our work through the years, and it has really sustained the organization during the COVID-19 crisis. The VEX experience is a patient-centered model of eye care focusing on both synchronous and asynchronous examinations of the anterior and posterior segments, with the support of visual communication solutions. When social distancing was mandated by the Argentinean

304

G. Ricur et al.

government on March 19, we decided to leverage our VEX experience by rescheduling all previously programmed in-person appointments as teleconsultations. Thus, we created a new virtual workflow by reconfiguring patient appointment templates in the practice management system, designing user-friendly guides and video tutorials, as well as writing new COVID-19 telemedicine-specific consent forms. The guides and consent forms were emailed to patients with an invitation to meet online. Links and images as examples can be found in the Annexus section. All staff members involved in the new workflows were trained virtually through webinars (since lockdown measures were in place), customized to match each individual process or needs (e.g., administrative staff, technicians, and physicians), and a general communication strategy, including both internal and external communication on our website and social media channels, was implemented. Our pre-COVID virtual workflow was used as the basis for the new VEX workflow which added the possibility of a complete 100% virtual pathway, as described in the following two graphics.

Pre-COVID Virtual Workflow

Fig. 7 Pre-COVID virtual workflow: the patient books an online appointment, and then walks in for mandatory testing performed by technicians. Once screening is uploaded, they overview the case remotely with the physician, who, in turn, views and speaks to the patient

over Telepresence solutions, and performs not only the remote slit lamp examination of the eye but also the case management and discussion with the patient and their family members

See Fig. 7.

Covid Virtual Workflow See Fig. 8. It is of note that patients were divided into 3 pathways, depending on the nature of the consultation. One group of patients was triaged online only. These were managed remotely using video conferencing exclusively. This avoided having them undertake any travel to the institute. The second group of patients was seen in person after their virtual triage, due to the necessity for further testing. Depending on their condition and test results, they were seen either in person or remotely by the specialist in real time with the aid of a digital slit lamp, or asynchronously once back at home with a teleconsultation that included replaying the videotape of their slit-lamp examination and screening tests. The last group of patients required mandatory in-person office visits due to the urgent nature of their conditions. This workflow followed one of two pathways: the attending physician was present in the

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

305

Fig. 8 COVID-19 Virtual Workflow: After booking online the appointment and having completed the online triage, the patient walks in for mandatory testing performed by technicians; the physician views (a, b) and speaks to the patient over video conferencing

cloud-based solutions, whether at home (c) or within one of the facilities (d, e), in order to maintain medical distance. Pictures in Fig. 4, courtesy of Lucio L. Arias, MD

examination or surgical room, or the specialist was at home or in another remote setting within one of the facilities in order to avoid the risk of infection.

arrive (due to restrictive importation regulatory issues at the time, which retained the telepresence equipment in customs for almost a year), we were able to examine our first patient by the end of 2012. From then on, CISCO telepresence video conferencing became the primary standard, and 9384 patients were examined virtually since. During the first year of the pandemic outbreak, virtual consultations increased by 390%, and it is of note that they were all performed using the cloud-based enterprise-grade video conferencing solution developed by Google. We had already adopted the latter technology in 2018, and it benefitted us immeasurably due to the restrictive lockdown measures in place during most part of the year 2020 (Argentina is one of the countries that had the longest quarantine period in the world). The following graphic shows the spike in 2020 of the virtual eye examinations performed synchronously over our network from December 2012 to April 2021.

Results and Discussion Population Assisted Remotely Since 2001, more than 36,000 patients have consulted online the Instituto Zaldivar. They have been assisted remotely, either by online web consultations or hybrid consults by means of video conferencing their eye examination in real time. The great tipping point came with the change in the eyecare delivery model in 2011. That year, we designed the telepresence model, in Mendoza and Buenos Aires. After almost two years of negotiations, room building construction, and waiting for the CISCO hardware to

306

G. Ricur et al.

VIRTUAL EYE EXAMS (VEX) DEC 2012 - APRIL 2021

Fig. 9 This graph depicts the increase in the use of virtual consults during the COVID-19 outbreak. It is noted that the proposed cut-off date for the purpose of this

Virtual Eye Exams (VEX) Dec 2012– April 2021 See Fig. 9.

Patient Demographic Information Age The patients’ ages ranged from newborn babies to a 98-year-old patient. The distribution of the age groups varied from year to year, but, in general terms, the patients that consulted the most represented the 60–70-year-old age group, followed in second place by the 50–60-year-old group, and in third place by those 30–40-year-old patients. During the pandemic, the group that consulted the most were patients aged 30–40, followed by those 60–70, and then 20–30 years old. See Figs. 10 and 11.

Gender Regarding gender, in total, 5203 and 4078 patients were registered as female and male, respectively. Regardless of the year, female

chapter (April 2021) causes the abrupt decrease in the plotted line, which can be easily observed

patients slightly outweighed male patients, in the use of teleophthalmology models of care, although the difference was not statistically significant if analyzed between 2012 and 2021. Statistically, when analyzing the standard deviation (SD) and the mean values of the number of men and women for the period 2010–2021, it is observed that the samples present less than 1 SD of difference. Nonetheless, during the first wave of the SARS Cov-2 outbreak and initial lockdown in 2020, women clearly chose virtual attention as their MOC, over men (in absolute values; see dots in orange color in the following graph). Based on similar reports, females tend to be the ones making the household decisions regarding their families, including researching online for medical advice and booking medical appointments for their family members [11, 12]. See Fig. 12.

Origin Historically, our patient population has mainly represented those living in Mendoza and the metropolitan area of Buenos Aires. While patients coming from other provinces have been a constant minority, when summed up with those traveling from Buenos Aires to Mendoza for

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

307

PATIENT AGE DISTRIBUTION DEC 2012 - APRIL 2021

Fig. 10 This graph depicts the age distribution of the patients that accessed the use of virtual consults during the past ten years

Fig. 11 This graph depicts the age distribution of the patients that accessed the use of virtual consults during the year 2020

308

G. Ricur et al.

PATIENT GENDER DEC 2012 - APRIL 2021

postoperative follow-up, regular annual controls, and other specialty consultations such as COVID triage with our clinicians who are cardiologists, Cornea and External eye diseases, for pediatric conditions and orthoptic therapies, as well as for remote monitoring of retinal conditions such as macular degeneration and diabetic retinopathy (Fig. 13).

Image Quality and Transmission

Fig. 12 This graph shows the gender distribution of the patients that accessed the use of virtual consults during the period 2012–2021, compared to 2020

their surgeries, it has shown an important impact on Mendoza’s health tourism. Pre-pandemic, over 55,000 patients would travel to Mendoza in search of health tourism [13]. During the pandemic lockdown, all domestic travel was banned, and therefore the number of patients opting for virtual consults rose, and so did the origin stats. The other variable affecting the usual origin rate is the creation of the ZVW network during the first wave of the COVID outbreak. Adding 10 new locations, where patients could be assisted with mandatory ophthalmic imaging prior to their virtual consultation, also accounted for the increase in new cities on the map.

Specialty Eye Care In regards to the specialities most consulted, being a Cataract and Refractive ambulatory surgical center, this area of expertise has clearly represented Instituto Zaldivar´s core business model. Nonetheless, during the COVID outbreak, not only did the web consults regarding the different specialities grow, but the VEX consults also reflected the same pattern. In summary, patients requested not only virtual eye care inquiries or first-time visits for vision-correction conditions, but also

As mentioned before, the quality of the imagery transmitted between the sites has varied significantly over the last 20 years, due to the innovations in both video technology per se, and the larger bandwidth, thanks to the new telecommunication network designs and infrastructure available currently. Telepresence solutions clearly changed the experience by offering H.264 video codecs for high quality at lower bitrates, with native 1080p high-definition cameras and encoding/decoding processes. When one adds this with a low-latency architecture and a low bandwidth utilization, together with a wideband advanced audio coding with low delay (AAC LD) and a multichannel spatial audio with echo cancelation and interference filters to eliminate feedback from mobile devices, a truly immersive experience is gained. This is truly enhanced with the proper room design and decor. Recently, 4 K video technology has surpassed the quality of 1080p high-definition cameras, but the fact that the network designed for the outbreak was open and decentralized impacted the quality of service. It was dependent not only on the smartphone’s or the device’s video capability used by both the physicians and patients but also on the domestic bandwidth available. The downside resulted in an estimated drop from an SLA of 97.4–68% (Fig. 14). Nonetheless, we were able to sustain the program, and the increase in its use was close to 400% during the 2Q of 2020, with a stable decrease as lockdown restrictions were modified over the months. All no-readable or suspicious images were registered and reported, and those patients were referred for an in-person evaluation.

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

309

Fig. 13 This graph shows the distribution of specialty care accessed during the period 2012–2021, compared to the year 2020 which is highlighted in red

Fig. 14 This table compares the characteristics of the different types of networks available before and during the pandemic period

Image Acquisition and Reception Innovation also came about not only with the new video cameras but also with the new slitlamp adapters for the cameras as well as with the smartphone holders. The anterior segment of the eye has particularly benefited, since it has a quite user-friendly user approach and is relatively quick to evaluate, when compared to the need for additional lenses, more time-consuming and overcoming the glare artifacts that posterior segment slit-lamp mediated assessments imply. From the original 1 CCD video cameras to the 3 CCD technology, from the use of small portable security video cameras to the new

smartphone and Google Pro cameras, slit-lamp adapters have been innovated and redesigned to allow for the constant technological evolution. While the authors prefer adapters that conserve binocular vision and ergonomic positioning, many new models are constantly being marketed, allowing for the use of not only smartphone cameras but tablets and other hand-held devices with cameras as well (Fig. 15).

Virtual Doctor-Patient Relationship Patient satisfaction has always been our main driver, and the core of our “360° Zaldivar Experience” model of eye care. In fact, we were

310

G. Ricur et al.

• Not sure as to recommend or not the VEX Experience: 1% (6) • Would not recommend the VEX Experience: 1% (7)

Sustainability

Fig. 15 New slit-lamp adapters convert conventional models into digital. In this particular case, the MARCO iONSM imaging system has an intra-optics beam splitter/ camera adapter that conserves binocular vision, has a steady mount for the new iPhones, and is a very ergonomic device with an intuitive display, facilitating its adoption. Photograph courtesy of Dr. Giselle Ricur

one of the first eye care institutions to certify total quality assurance, under the ISO 9001 standards, back in 1998. The need to reach out to our patients under a complete 100% virtual model during the first wave led us to increase the awareness of how to enhance the quality of each encounter. During the 2020 outbreak, patient satisfaction surveys were automatically sent by email upon having completed a VEX encounter during the lockdown. Of a total of 3704 virtual consults performed, only 528 (14.25%) patients replied. When asked if they were willing to recommend the VEX Experience to a family member or friend, the answers were. • Totally recommend the VEX Experience: 75% (397) • Most probably recommend the VEX Experience: 16% (85) • Would recommend depending on the case or nature of the eye condition: 6% (33)

Downsizing the costs of maintaining large and complex equipments, such as the CISCO telepresence units, being able to rapidly adapt any clinical—or even surgical—scenario with webbased video conferencing solutions, and leveraging our telecommunication network made favorable impact on the total cost of our ocular telehealth program. Additionally, the opening of the ZVW network expanded our patient outreach, increasing our rate of new admissions and surgeries. Finally, although the COVID outbreak itself caused an almost complete halt of all domestic and international travel, undoubtedly the adoption of telemedicine services helped decrease the usual carbon footprint. The epidemic allowed us to optimize the patient’s worklife balance by decreasing the number of unnecessary trips to the doctor’s office, saving time and money. Last but not the least, it allowed us to optimize the use of the Institute’s resources while concentrating more on better serving its patients.

Our Legacy: Teleophthalmology Training Programs In response to positive feedback about our experience, many colleagues turned to us for aid and advice in their own telemedicine efforts. Therefore, we designed a training program to remotely train peers in how to implement teleophthalmology programs of their own. We have also participated in webinars and personal online training sessions in lieu of hosting or traveling to the usual telemedicine workshops held in the regular ophthalmic meetings and conferences year-round.

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later

311

Fig. 16 Patient portal, using google workplace

Conclusions The technological innovation of telemedicine has offered us many advantages over the past 20 years. It efficiently linked our experts and optimized diagnostic capacity, the quality of patient care, and the entire medical process. Under the context of the current pandemic, we have kept patients’ eye care and monitoring at a distance through the use of high-definition visual communication systems. This essentially transformed our regular consultation process into an immersive experience, regardless of geographic location. Safety being one of the most important concerns, the VEX experience design allowed us to maintain social and medical distance while preserving the availability of our ophthalmologists to our patients, and at the same time maintaining connectivity and a sense of presence among all those involved. It also helps to keep non-urgent cases out of the already overloaded hospitals. We have already noticed the costeffectiveness of telemedicine at the operational level, and we have sustained the profitability of our business by linking all the Instituto Zaldivar offices with the ZVW network and keeping them virtually open for business during the mandated quarantine. Thus, implementing telemedicine has

truly enhanced our patient outreach. Through live consultations, both synchronously and asynchronously, and in a safe and secure manner, we have optimized our time and efforts in managing workload while sustaining our practice over the years, through the different crises or sanitary emergencies Argentina has faced since 2001. This new model of eye care—the VEX experience—offers a safe and efficient means of being able to stay close to your patients. In our opinion, telemedicine has the potential to become the new normal in many places, even after the pandemic has receded. Acknowledgements Luis Alberto Arcuri CIO Instituto Zaldivar, Mendoza Argentina [email protected]. Federico Roberto Scaiola Software Developer Instituto Zaldivar, Mendoza Argentina federico. [email protected]. Eduardo J. Rosales Former CTO Instituto Zaldivar, Mendoza Argentina eduardo. [email protected].

Annexus The following figures show the guides and consent forms either accessed through the patient portal or emailed directly to them (Figs. 16, 17, 18). Link Tutorial: https://iz-videos.s3.amazonaws. com/guia-test-visual.mp4

312

Fig. 17 Patient consent form for virtual consults, using google forms

G. Ricur et al.

Cataract and Refractive Surgery: Teleophthalmology’s Challenge in Argentina, 20 Years Later Fig. 18 Email link for performing the virtual consults and screening tests

313

314

References 1. De Sutter E, De Molder R, Gabriel P, Maelfeyt J, Mattheus R, Ghesquiere J. Tele-ophthalmology. The T.I.M.E. project. A Tele-Medicine project in the region Nord-Pas de Calais and Zuid-WestVlaanderen. Bull Soc Belge Ophtalmol. 1994;252: 37–42. 2. Ogle JW, Cohen KL. External ocular hyperemia: a quantifiable indicator of spacecraft air quality. Aviat Space Environ Med. 1996;67(5):423–8. 3. Flowers CW Jr, Baker RS, Khanna S, Ali B, March GA, Scott C, Murrillo S. Teleophthalmology: rationale, current issues, future directions. Telemed J. 1997;Spring 3(1):43–52. 4. Rathi S, Tsui E, Mehta N, Zahid S, Schuman JS. The current state of teleophthalmology in the United States. Ophthalmology. 2017;124(12):1729–34. 5. Sim DA, Mitry D, Alexander P, et al. The evolution of teleophthalmology beyond diabetic retinopathy screening programs in the United Kingdom. J Diabetes Sci Technol. 2016;10(2):308–17. 6. Labiris G, Panagiotopoulou E, Kozobolis VP. A systematic review of teleophthalmological studies in Europe. Int J Ophthalmol. 2018;11(2):314–25.

G. Ricur et al. 7. WHO Group Consultation on Health Telematics Report. http://apps.who.int/iris/bitstream/handle/10665/ 63857/WHO_DGO_98.1.pdf;jsessionid=99323BEA 0EBF2CC9BE0D5138C844AB3B?sequence=1 8. Bashshur R, Shannon G, Krupinski E, Grigsby J. The taxonomy of telemedicine. Telemed J E Health. 2011;17(6):484–94. 9. https://www.himss.org/news/himss-defines-digitalhealth-global-healthcare-industry 10. Ricur G, Zaldivar R, Batiz MG. Cataract and refractive surgery post-operative care: teleophthalmology’s challenge in Argentina. In: Yogesan K, Kumar S, Goldschmidt L, Cuadros J, editors. Teleo phthalmology. Berlin, Heidelberg: Springer;2006. 11. Whaley CM, Pera MF, Cantor J, et al. Changes in health services use among commercially insured US populations during the COVID-19 pandemic. JAMA Netw Open. 2020;3(11):e2024984–202498. 12. Lam K, Lu AD, Shi Y, Covinsky KE. Assessing telemedicine unreadiness among older adults in the United States during the COVID-19 pandemic. JAMA Intern Med. 2020;180(10):1389–91. 13. https://ecocuyo.com/el-turismo-de-salud-mueve-55000-visitantes-cada-ano-en-mendoza/

Teleophthalmology in Brazil Alexandre Chater Taleb

Over the last two decades, telemedicine has been facing several challenges. While technology and connectivity are evolving rapidly, many legal, ethical and social issues have raised and are still unsolved. When we look at countries that are continentally wide and populationally large such as Brazil, some of these challenges seem overwhelming. So do the opportunities. Brazil is the fifth largest country in the world and house of 220 million inhabitants. Although the number of ophthalmologists in Brazil (21,361 as per 2021 census) [1] is considered saturated by WHO parameters (adequate is 1 ophthalmologist per 17,000 people), there are still many underserved areas that do not have current eyecare daily (Fig. 1). It may take up to five months (average time of 81 days) to book an ophthalmologist appointment in the public health system (SUS) and 1,689 cities (out of 5,570) have current working ophthalmologists [2]. Although these 1,689 cities represent 30% of all cities in Brazil, they account for 79, 5% of the Brazilian population. Despite the advocacy efforts of the Brazilian Council of Ophthalmology to increase access, still, there is an outrageous number of citizens without proper and regular eye care.

A. C. Taleb (&) Universidade Federal de Goiás, Goiania, Brazil e-mail: [email protected]; [email protected]

Telemedicine, is, in my personal definition, a “new way of medical care”. As an ophthalmologist I can, many times, manage my patients clinically, sometimes surgically and, now, sometimes virtually. This virtual strategy of care is an option for both public and private eyecare sectors, especially in a large country such as Brazil. Brazilian legislation on telemedicine had been two steps behind the world until May 2022, when the Brazilian Federal Medical Council [3] published its resolution 2.314/2022 that defines and regulates telemedicine as a medical service mediated by communication technology, stating ethical rules and procedures that allow doctors to perform teleconsultation, telediagnosis, telesurgery, teletriage and telemonitoring. During the COVID-19 pandemic, for the first time in Brazil, a bill was approved to legalize telehealth practice, although temporarily. In December 2022, the new Telehealth bill [4] (14.510/22) was approved, providing legal basis for online care and establishing principles, rules and responsibilities to those involved in Telemedicine and Telehealth care in Brazil. Teleophthalmology use in Brazil began as early as 1998, among university-based ophthalmology clinics [5–7]. At that time, the feasibility of acquiring digital images and comparing them to live examinations was the first concern. Storeand-forward systems of a specialized second opinion and teletriage by general practitioners were, also, concerns studied then.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_20

315

316

A. C. Taleb

Fig. 1 Number of ophthalmologists per state, in Brazil (2019)—from a total of 20,455 ophthalmologists. Conselho Brasileiro de Oftalmologia. Ottaiano et al. [2]

The first commercial full-time teleophthalmology service in Brazil was established in 2002, between Brasilia and Goiânia, providing second opinions though videoconference for Retina and Vitreous diseases, Neurophthalmology, Glaucoma and Uveitis [8]. From then on, many different settings for teleophthalmology have been established all over Brazil, some of which are worth mentioning. The Federal University of Goiás (UFG) was one of the nine founding institutions of the cTelessaúde Brasil Program, a national telehealth

program run by the Brazilian Ministry of Health designed to provide tele-education and teleassistance to Primary Care Health Units in rural areas and small cities. UFG became responsible for the National Telehealth Blindness Prevention Program, serving as an educational center and reading center for fundus photography sent from all over Brazil. Aiming to diagnose the four main causes of blindness —cataract, glaucoma, diabetic retinopathy and AMD—it started in 2008 and is still ongoing [9].

Teleophthalmology in Brazil

317

Fig. 2 Ophthalmologist control deck at TeleOftalmo project

The Federal University of Rio Grande do Sul (UFRGS) deployed the TeleOftalmo Project, a teleophthalmology remote visual assessment service, including refraction, autotonometry, biomicroscopy and fundus images to provide eye care to cities without regular care. Their experience started in 2017 and saw over 30,000 patients (by the end of 2021) [10, 11] and report a 70% rate of resolutivity. 3 out of 10 patients had to be referred for presential ophthalmology consultation (Fig. 2). UFMG—Federal University of Minas Gerais —runs the largest teleEKG public program in Brazil. They have also implemented a teleophthalmology service for diabetic retinopathy screening and counseling that has been reported to reduce referrals by over 70% since 2012 [12]. The Paulista School of Medicine has been working in several teleophthalmology services. One of them was a national multicenter type 1 Diabetes Study that included patients from 2014 to 2018 to validate diabetic retinopathy screening [13]. They also use teleophthalmology strategy in indigenous populations [14]. The COVID-19 pandemic brought a new scenario to the teleophthalmology arena. During the first months, specialized eyecare clinics and hospitals were closed, and patients had no other

alternative than to be seen online. It created the opportunity for teleorientation, as did the Brazilian Council of Ophthalmologist, which put together over 80 voluntary ophthalmologists that donated their time and knowledge to do video consultations in order to help patients that were under treatment and had nowhere to go or no eye doctor to talk to [15]. Some of the Brazilian ophthalmologists, myself included, have even been practicing teleophthalmology on a daily basis, seen patients that are either in an office or at home (Fig. 3). There are private Brazilian companies that offer service of artificial intelligence to enhance reports and Fundus images and OCTs reading [16]. Augmented and virtual reality are been used to improve surgical procedures and to perform remote visual field assessment. Although it may be more expensive to buy telemedicine ready equipments, or to adapt existing ones to a digital interface, the ability to broaden the reach of one's practice and offer proper eye care to those who might not get it otherwise, is worth every effort to implement a teleophthalmology practice. It is our strong belief that teleophthalmology can no longer be considered a promise. It is here to stay, to be used as an eyecare strategy, suitable for screening, follow-up visits, pre-and post-

318

A. C. Taleb

Fig. 3 Patient with cataract examined through the teleophthalmology suite by virtual Doctor Ltd. (Alexandre Taleb)

operative care and even first consultations. As medical devices evolve, we will be soon monitoring intraocular pressure and visual acuity from patients’ homes in real time and be able to use all data collected to predict future treatments and outcomes.

References 1. Beniz Neto J, Umbelino CC. Censo 2021–Conselho Brasileiro de Oftalmologia. Available from https:// cbo.net.br/2020/admin/docs_upload/034327Censocbo 2021.pdf 2. Ottaiano JAA, Ávila MP, Umbelino CC, Taleb AC. As Condições de Saúde Ocular no Brasil–2019. Conselho Brasileiro de Oftalmologia. Available from https://www.cbo.com.br/novo/publicacoes/condicoes_ saude_ocular_brasil2019.pdf 3. Conselho Federal de Medicina. 2022. Available from https://sistemas.cfm.org.br/normas/visualizar/ resolucoes/BR/2022/2314 4. Brasil. 2022. Available from http://www.planalto.gov. br/ccivil_03/_ato2019-2022/2022/lei/L14510.htm 5. Taleb AC, Böhm GM, Avila M, Wen CL. The efficacy of telemedicine for ophthalmology triage by a general practitioner. J Telemed Telecare. 2005; 11 (1_suppl):83-5. Doi: https://doi.org/10.1258/13576 33054461958. Available from https://doi.org/10. 1258/1357633054461958.

6. Taleb AC, Böhm GM, Avila M, Wen CL. Full Time teleophthalmology service in Brazil. J Telemed Telecare. 2004; 10(1_suppl):S33. Doi:https://doi.org/ 10.1258/1357633054461958. Available from https:// www.liebertpub.com/doi/abs/, https://doi.org/10.1089/ 153056204323057022 7. Pennella AD, Schor P, Burnier MN. Description and test of a digital image system and a virtual enviromment to be used in a remote second referral option in ophthalmology. Investig Ophthalmol & Vis Sci. 2005; 46(13):2756. Available from https://iovs. arvojournals.org/article.aspx?articleid=2402197 8. Finamor LPS, et al. Teleophthalmology as an auxiliary approach for the diagnosis of infectious and inflammatory ocular diseases: evaluation of an asynchronous method of consultation. Revista da Associação Médica Brasileira [online]. 2005; 51 (5):279-84. Available from: https://doi.org/10.1590/ S0104-42302005000500020 9. Santos AF, Souza C, Alves HJ, Santos SF. Telessaúde. Um instrumento de Suporte Assistencial e Educação Pernanente. Ed UFMG; 2006. 10. Lutz de Araujo A, Moreira TDC, VarvakiRados DR, Gross PB, Molina-Bastos CG, Katz N et al. The use of telemedicine to support Brazilian primary care physicians in managing eye conditions: the teleOftalmo project. PLoS ONE. 2020; 15(4):e0231034. https://doi.org/10.1371/journal.pone.0231034. 11. de Araujo AL, Rados DRV, Szortyka AD, Falavigna M, Moreira TC, Hauser L, Gross PB, Lorentz AL, Maturro L, Cabral F, Costa ALFA, Martins TGDS, da Silva RS, Schor P, Harzheim E, Gonçalves MR, Umpierre RN. Ophthalmic image

Teleophthalmology in Brazil acquired by ophthalmologists and by allied health personnel as part of a telemedicine strategy: a comparative study of image quality. Eye (Lond). 2021;35(5):1398–404. https://doi.org/10.1038/ s41433-020-1035-5. 12. Alkmim MB, Figueira RM, Marcolino MS, et al. Improving patient access to specialized health care: the telehealth network of minas gerais, Brazil. Bull World Health Organ. 2012;90(5):373–8. https://doi. org/10.2471/BLT.11.099408. 13. Malerbi FK, Morales PH, Farah ME, Drummond KR, Mattos TC, Pinheiro AA, et al. Brazilian type 1 diabetes study group. Comparison between binocular indirect ophthalmoscopy and digital retinography for

319 diabetic retinopathy screening: the multicenter Brazilian type 1 diabetes study. Diabetol Metab Syndr. 2015;7:116. 14. Malerbi, FK Fabbro, ALD Vieira-Filho JPB, Franco LJ. The feasibility of smartphone based retinal photography for diabetic retinopathy screening among Brazilian Xavante Indians. Diabetes Res Clin Pract. 2020; 168: 108380. 15. Conselho Brasileiro de Oftalmologia. Brasil que Enxerga. Available from https://www.vejabem.org/ noticia/brasil-que-enxerga-teleorientacao-gratuita-emsaude-ocular1588618248. 16. RedCheck IA. Avaliable from https://redcheck.com. br/site

Veteran Affairs (VA) Ocular Telehealth Programs April Maa, Timothy Elcyzyn, Robert Morris, and Leonard Goldschmidt

Abstract

This chapter discusses the Veterans’ Healthcare Administration (VHA) national asynchronous ocular telehealth programs. An overview of the Veteran Affairs (VA) organization and the national structure for telehealth is provided, followed by a discussion about diabetic teleretinal screening, the first VA eye telehealth endeavor. Best practice elements for an eye telehealth program are highlighted. TeleEye Screening and Technology-based Eye Care Services (TECS), two of the newer VA

ocular telehealth programs arising from the legacy diabetic teleretinal program, are discussed. Information about set up, equipment, patient referrals, data collection, imager and reader training, quality assurance and monitoring are detailed for each of the programs. The future of ocular telehealth within the VA concludes the chapter. Please note that the views expressed in this chapter are those of the individual authors, and in no way represent the views of the Veterans’ Health Administration or the United States federal government. Keywords

A. Maa (&) Veterans’ Integrated Service Network (VISN) 7, Clinical Resource Hub, Duluth, Georgia e-mail: [email protected] Department of Ophthalmology, Emory University School of Medicine, Atlanta, Georgia T. Elcyzyn Veterans’ Integrated Service Network (VISN) 4, Clinical Resource Hub, Pittsburgh, USA R. Morris Salisbury VA Health Care System, Salisbury, NC, USA College of Optometry, The Ohio State University, Columbus, OH, USA L. Goldschmidt VA Palo Alto Healthcare System, Palo Alto, CA, USA Stanford University Medical School, Stanford, CA, USA







Ocular telehealth Tele-ophthalmology Tele-optometry Telemedicine Eye Diabetic teleretinal screening Macular degeneration Glaucoma Tele-glaucoma Tele-macula VA VHA Veteran affairs Mobile health Asynchronous Store-and-forward


































Introduction Veterans Health Administration (VHA) Organizational Structure The Veterans’ Health Administration (VHA) is the largest integrated healthcare system in the United States, serving approximately nine million enrolled Veterans each year. Geographically stretching from the east coast of the United States

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_21

321

322

to Guam, the system comprises 1272 healthcare facilities, including 170 VA Medical Centers and 1102 outpatient clinics [1]. The healthcare system is divided into 5 levels of complexity [2]: 1a, 1b, 1c, 2, and 3. The largest 1a facilities are located in urban areas and provide the highest volume and greatest range of tertiary care, contain the most complex intensive care units, and offer significant teaching and research activities. Smaller level 3 facilities are located in rural areas and have the lowest volume and complexity and generally offer primary care, mental health services, and more limited specialty care. The mission of the Veterans’ Affairs (VA) is: “To fulfill President Lincoln’s promise; ‘To care for him who shall have borne the battle, and for his widow, and his orphan’, by serving and honoring the men and women who are America’s Veterans.” To achieve this mission, the enterprise is structured with a Central Office responsible for policy and overarching administrative needs such as human resources and information technology. VHA Central Office is subdivided into large program offices, e.g., “National Surgery Office”

A. Maa et al.

and “Patient Care Services”. These national offices provide strategic direction and policies for the front-line care facilities. Across the U.S., VA Healthcare Systems are divided into 18 Veterans’ Integrated Service Networks (VISNs), which are regional systems that work together to meet the health care needs of Veterans within the network. Figure 1 illustrates the 18 VISNs in the VA healthcare enterprise, to which healthcare systems belong. Each healthcare system, in turn, has satellite outpatient clinics designed to provide outpatient primary care services closer to where Veterans live. The majority are classified as Community Based Outpatient Clinics (CBOCs) and in addition to primary care services may also include mental health, optometry, radiology, podiatry, preventive care, immunizations and vaccinations, social work, and women’s health services. Figure 2 is a schematic illustrating the organization of one of the VISNs within VHA, VISN 7, its associated healthcare systems, and one system’s CBOCs. Note that the diagram is not comprehensive.

Fig. 1 Geographic map illustrating the 18 VISNs across the United States that comprise the VA healthcare network. Credit VA intranet website, facility locator and leadership directory, used with permission from VA national

Veteran Affairs (VA) Ocular Telehealth Programs

323

VISN 7

Carl Vinson VA Medical Center (Dublin GA)

North Arcadia CBOC

Blairsville CBOC

Charlie Norwood VA Healthcare System (Augusta GA)

Rome CBOC

Atlanta VA Healthcare System

Pickens CBOC

Tuscaloosaa VA Medical Center (Tuscaloosa AL)

West Cobb CBOC

Ralph A Johnson VA Healthcare System (Charleston SC)

NE Cobb CBOC

Columbia VA Healthcare System (Columbia SC)

Carrollton CBOC

Central Alabama VA Healthcare System (Montgomery AL)

Stockbridge CBOC

Fig. 2 Illustrative schematic of one VISN and its associated healthcare systems and CBOCs for a representative healthcare system (Atlanta). Credit April Maa, MD

VA Eye Care Working in partnership under the office of Specialty Care Services, approximately 1000 optometrists and 800 ophthalmologists [3] currently serve the eye care needs of Veterans across the enterprise. By collaborating to provide a full spectrum of primary, secondary, and tertiary eye care services, a total of 4.2 million encounters were provided in fiscal year (FY) 2019 (October 1, 2018 to September 30, 2019). These services included comprehensive examinations, periodic medical examinations, problem-focused visits, low vision and blind rehabilitation services, surgical procedures, and follow-up care, as well as preventative care services using telehealth modalities. Eye care demand is ever-growing in VA. This is related to factors such as the high prevalence of eye disease in the older Veteran population, which is accompanied by a frequent utilization of services, as well as the growing demand for spectacles needed for correction of refractive errors and presbyopia. Given this mounting demand and despite VA’s efforts to keep up by increasing the number of outpatient clinics, it is not uncommon for many VA clinics to have significant access challenges for eye care. One solution the VA has adopted to improve Veterans’ access is to refer patients to local non-VA hospitals and clinics, or “Community Care”, to be used as a supplement to VA care. Using Community Care as a supplement to VA care, while beneficial to Veterans in terms of access, has its pitfalls, including reduced care coordination, additional cost to VA, and lack of sub-

specialists in medically underserved areas. It is not infrequent that community care providers have significant access issues as well, such as long wait times for appointments and extensive travel distances from a Veteran’s home. These shortcomings do little to improve the access burden the community care initiative is often meant to address. As an alternative solution, VA has begun to address these eye care access issues with the use of telehealth.

VA Telehealth The VA has long been the leader in the use of telehealth/telemedicine for the delivery of care for its Veterans. The VA has a robust telehealth practice in many areas of medicine and is led strategically by the VHA Office of Connected Care (OCC). Within OCC, there are divisions that handle quality, training, technology, and implementation/marketing. There are also local telehealth champions at both the VISN and healthcare system (commonly referred to as facility) level, the VISN Telehealth Coordinator and the local Facility Telehealth Coordinator (FTC). Through this robust infrastructure, the VA supports several telehealth programs across the enterprise, providing 5.6 million episodes of telehealth care to 1.6 million individual Veterans in FY 20 [1]. With the onset of the novel coronavirus-19 (COVID-19) pandemic, VA was one of the few large healthcare systems who could increase and implement telemedicine practice expeditiously in response to the unexpected yet timely need. The VA’s successful use

324

A. Maa et al.

Fig. 3 Schematic illustrating the organizational structure of the VA Office of connected care. Note Individual names may not be current. Credit VA Office of connected care, used with permission

of telehealth during the public health emergency was largely facilitated by the excellent infrastructure of telemedicine technologies that existed pre-pandemic. Figure 3 illustrates the VHA OCC organizational structure. There are many reasons why telemedicine has thrived in the VA even before the COVID-19 pandemic, as well as several features of the VA healthcare system that is conducive to the practice of telehealth that is not readily duplicated outside of the VA. First, the VA has a longstanding, enterprise-wide electronic medical record (EMR), known as Computerized Patient Record System (CPRS), at every VA facility across the country. Regardless of where in the system the Veteran is treated, their inpatient records, outpatient records, images, lab results, upcoming appointments are stored in CPRS and easily accessible from anywhere in the VA. Images are also transferable and viewable from any hospital across the system through VistA Imaging, which functions as the VA’s internal Picture Archiving and Communications System (PACS) and is generally vendor neutral. VistA Imaging can accept many different types of images or documents from various sources, either manually scanned or transferred automatically using Digital Information and

Communication in Medicine (DICOM). Second, the VA has a widespread information technology (IT) infrastructure that allows for secure transfer of both information and images from one location to another. Third, VA providers are authorized to practice across state lines without being licensed in each individual state, as long as they are practicing within their scope and credentialed and privileged at a VA facility. This “anywhere to anywhere care” is facilitated by federal supremacy rules and by the VA Mission Act that applies to all providers within the VA system. Finally, in a capitated system, providers earn incentives but generally are not compensated based on the amount of relative value units (RVU) they generate, so when telehealth was not billable to insurance, VA providers were still motivated to use this care modality as the system is driven by access needs and cost-efficient care.

VHA Office of Connected Care Eye Telehealth Structure Current to the date of publication of this textbook, there are 3 national ocular telehealth programs: TeleEye Screening (formerly Diabetic Teleretinal Imaging), Technology-based Eye

Veteran Affairs (VA) Ocular Telehealth Programs

325

Care Services (TECS), and Tele-Low Vision. The initial aspects of this chapter will focus on the first asynchronous ocular telehealth program, Diabetic Teleretinal Imaging. A detailed discussion is dedicated to this legacy program, because it forms the basis for the other two newer eye telehealth programs, TeleEye Screening and TECS. Furthermore, the OCC structure that supports Diabetic Teleretinal Imaging continues to change and support the evolving needs of TeleEye Screening, TECS, and Tele-Low Vision.

fundus cameras located at the hospitals and CBOCs, trained staff to operate them, and business processes to address technology issues and monitor quality in the program. In recent years, the program has expanded to include screening Veterans at risk for glaucoma and macular degeneration and providing glasses; taking the lessons of the original TRI program (discussed in greater depth in the Tele-Eye Screening and TECS sections) and applying them in new and innovative ways to help the Veteran population.

National Ocular Telehealth Programs in VHA

TRI—History

I. Diabetic Teleretinal Imaging (TRI)

Introduction The most common and well-established ocular telehealth program in the world is Diabetic Teleretinal Imaging, so it is no surprise that it was the first major ocular telehealth screening program the VA launched. Diabetic retinopathy is the leading cause of blindness in those under age sixty in the United States [4]. Veterans have a higher rate of diabetes than the general population (20% versus 8%) [3], and those who were exposed to Agent Orange, a herbicide used in the Vietnam War time period, are at greater risk to develop diabetes. Veterans may also live far from the eye clinics where optometry or ophthalmology is present, as some VA medical centers have large catchment areas and it may not be feasible for Veterans to regularly travel to see an optometrist or ophthalmologist in-person. To improve access to timely diabetic eye exams, the VA made a relatively early attempt (compared to other healthcare entities) to address diabetic eye screening with alternative methods. In an effort to reduce the burden of vision loss on its Veteran patients, VA conceived and implemented a wide-ranging telehealth program to detect diabetic retinopathy in its patients. TRI, operating nationwide since approximately 2007, currently supports a network of non-mydriatic

The Diabetic Teleretinal Imaging program was originally born out of a desire to reduce the number of patients presenting with sight threatening diabetic retinopathy and requiring blind rehabilitation. The program was modeled after the Joslin Vision Network, part of the Joslin Diabetes Center, and was conceived by ophthalmologists and optometrists who noted that many of the diabetic patients in blind rehabilitation centers or with advanced diabetic eye disease had eluded the existing screening mechanisms then present in the organization. Many patients lived far from medical care, especially specialty eye care, and had incurred end-stage diabetic eye complications because the disease often did not manifest until advanced symptoms were evident for the patient. Now part of the larger VHA organization’s efforts in delivering health care remotely, the Diabetic Teleretinal Imaging program serves as a model for how any organization can change their focus to prevention of chronic disease complications, rather than simply “wait” for patients to present for treatment of end stage complications. While this priority of prevention of complications is now considered elemental to health care, in the early 2000s when the program was being planned, it was relatively radical for its time and actually faced a lack of support from the ophthalmology and optometry professional organizations. By focusing however, on patient wellbeing and clinical outcomes, the program set the technical, administrative, and programmatic

326

standards upon which the best telemedicine programs are built. Within this section, the authors wish to highlight the current thinking on which successful ocular telehealth programs should be constructed. The below sub-sections also emphasize how the OCC’s different departments are critical to starting, maintaining, and sustaining successful telehealth programs.

TRI—Clinical Processes 1. Overall set up Since primary care is the fundamental entry and healthcare service point for Veterans, fundus cameras were placed in the Veteran’s primary medical care home, including remote CBOCs. The CBOCs are distant from the main hospital and close to where the Veterans live. In addition, Veterans have the added convenience of receiving an eye screening when they are visiting with primary care, offering “same day service” or “walk ins” that would capture the patient when they were already on site. Non-mydriatic images are captured by Telehealth Clinical Technicians (TCTs) who receive special training. These images are stored in the computer system to be interpreted by readers (VA credentialed eye providers) at a different time point. The reader provides a report back to the primary care provider with results and recommendations for follow-up care, usually within a 3–7-day timeframe. 2 Patient selection/referral into the program One of the challenges with ocular telehealth programs can be patient buy-in and utilization. Buy-in for TRI is generally well established within the VA system. However, if the TRI program relied exclusively on the referrals of primary care providers, it is possible that the in the course of a busy clinic day, the provider may inadvertently forget to refer the patient for screening or not realize that the patient is “due”

A. Maa et al.

for their diabetic eye exam. One of the most powerful tools the VA possesses to facilitate telehealth is the VA EMR, CPRS. Built into CPRS are items called “clinical reminders” that function as electronic patient “flags”. These are regularly used by primary care to track preventative items the patient is due for, including diabetic eye exam, colonoscopy, and diabetic foot exam, to name just a few. These clinical reminders have background “logic” that can be customized at a local facility level. In general, for TRI, if the patient has a diagnosis of diabetes, and there is no “record” of an eye exam (i.e., no Common Procedural Terminology (CPT) code of 99451, 99250, 92004, and 92014) within the last 2 years, the patient will flag as “due”. Therefore, when the patient is seen by primary care, the due diabetic eye reminder will activate, alerting primary care that the patient is due for a diabetic eye exam, and they can be sent directly over to the TRI clinic for fundus photos. The clinical reminders also allow for past due and future due reports to be run. The TCT can pull these reports, call the patients ahead of time, and schedule their appointments for eye screening to be done on the same day as another VA appointment. 3 Fundus protocol and dilation As mentioned above, with 20% of Veterans cared for in the VA having diabetes, there is an outsized disease burden that is not present outside the VA. Therefore, when initially implementing TRI, clinicians undertook pilot programs to first determine which fundus camera views had the highest sensitivity and specificity of detecting diabetic retinopathy [5–7]. Published literature confirmed that a standard three 45degree fundus images of each eye proved to have the best reliability for detecting disease while still balancing the confines of a clinic schedule. While the majority of images are acquired from undilated eyes, protocols exist for facilities to include dilation, which some sites adopt. An external photo of each eye is also taken. Figure 4 illustrates the complete protocol.

Veteran Affairs (VA) Ocular Telehealth Programs

327

Fig. 4 VA diabetic teleretinal photographic protocol. Credit VHA Office of Connected Care, used with permission

4 Information capture and image interpretation The Office of Information Technology (OIT) and National Biomedical Engineers (Biomed) were present at the outset of TRI program development, working collaboratively with the eye and primary care clinicians, to facilitate the

implementation of the TRI program. There was a strong desire (and it was logistically sensible) to utilize both CPRS and VistA Imaging, operating throughout the organization since 1999, as the OIT/Biomed framework to transfer clinical information, store, and read the images.

328

Once the primary care provider is flagged by the CPRS reminder that the patient is due for diabetic eye screening, they can enter an order or a consult into CPRS for TRI screening. This order/consult is received by the TCT, who then schedules the patient (or receives the patient as a walk-in) and takes the images. Clinical information, such as duration and type of diabetes as well as most recent hemoglobin A1C values, are entered into CPRS using a standardized clinical note template by the TCT. The template has fields called “health factors” that capture discrete data elements that enter the VA’s vast informatics backbone, the Corporate Data Warehouse (CDW). The importance of these health factors will become evident later in the chapter. The TCT then enters a second CPRS photo consult that populates the fundus camera’s DICOM modality worklist. The primary care provider is listed as the responsible provider on this encounter ensuring they receive the results of the TRI study when completed. The DICOM worklist process ensures that the correct patient name is associated with the images and prevents images from being transferred to the wrong patient medical record. Images are transferred via DICOM into VistA Imaging for future viewing and permanent archiving. During implementation, Biomed/OIT designed a reading worklist for TRI, a software application called “TeleReader” that displays a chronological list of images. Providers reading images can select the study and then it will “lock” to prevent another reader from accessing the same study. Once the reader reviews the clinical information and images, they enter an interpretation utilizing a second CPRS template, link the interpretation to the photo consult or order which sends the results back to the primary care provider for review. The completed study disappears from the TeleReader queue. TeleReader is also designed to be used by other asynchronous telehealth programs, for example, tele-dermatology. Because of the VA’s vast information network, readers can be local (at the same facility) or remote (at a different facility or “hub”). Interpretations provided by the TRI readers are considered consultative, in that the

A. Maa et al.

liability and responsibility to act upon the recommendations fall to the primary care provider or whomever ordered the TRI screening. 5 Training of imagers Training of the TCTs is accomplished by a national infrastructure established and supported by OCC. The method of training has changed moderately over the years since TRI inception, but comprises (1) Electronic standardized presentations (video and interactive online sessions) that cover basic topics such as consult entry, template process, etc., and (2) hands-on, inperson training with a certified preceptor to assess competency in scheduling, consult management, and imaging. These two components follow each other in close succession (e.g., next day), and after the training is complete, the preceptor generally reviews a few studies of the new TCT to provide feedback. The TCT is then “certified” and must complete a minimum number of TRI studies and meet a target image readable rate each year to maintain competence. The preceptor is an individual who has at least 1 year of experience in TRI imaging and has completed additional training to be recognized as a certified TRI trainer. Local FTCs are the facility champions for telehealth and they are primarily responsible for assisting in the training of the TCTs, who may be licensed practical nurses (LPNs), medical assistants, or health technicians. The TCTs are often cross-trained to do a variety of different telehealth modalities so they serve as a “jack of all trades” and support multiple different telehealth programs. 6 Reader training for TRI The VA only utilizes credentialed and privileged VA eye providers for image interpretation. These providers are typically also credentialed and privileged for in-person eye care and whom have also completed focused telehealth training on diabetic retinopathy and the TeleReader software. Most of the reader training occurs electronically online, with videos and standardized

Veteran Affairs (VA) Ocular Telehealth Programs

courses that discuss relevant topics such as how to use the TeleReader application, how to use the image enhancement and viewing tools in VistA Imaging, and how to complete the CPRS note.

TRI—Administrative Processes 1. Quality assurance, monitoring, and improvement processes OCC has established a set of “rules” called “Conditions of Participation” for all the telehealth programs across the nation. These are rules and best practices that each telehealth program is expected to meet in order to ensure high quality and safe telehealth patient care. Some examples of TRI conditions of participation include required use of TeleReader software, image readable rates for TCTs, and protected time for reading TRI studies. The national OCC quality team also conducts a comprehensive review of all telehealth programs within each VISN every 2 years that evaluates the safety and effectiveness of all their telehealth programs, as well as their level of compliance with the conditions of participation. A report is provided to each VISN, with encouragement and acknowledgement for items being done well and suggestions for improvement. The quality team prepares a summary report from each VISN telehealth review which is presented to a peer review panel who provides further recommendations for improvement. These recommendations are reviewed by the VHA telehealth executive team for decisions on how to handle the concern and identify action items for reporting improvement. If a site is innovating on a new telehealth program, this is also their opportunity to share their experience and innovation with national telehealth. National telehealth does not “dictate” nor “police” what local telehealth programs are conducted, provided each program meets the Conditions of Participation. However, OCC does monitor the regulatory compliance and quality of care, provided through both established national telehealth programs and

329

local innovations, through the network of VISN and facility telehealth coordinators. OCC also maintains a quality dashboard, which gathers information from the CDW and displays an interface that allows users (VISN level or local FTC, OCC quality team, clinical champion) to actively monitor a specific program, their facility’s telehealth programs, their VISN’s telehealth programs, or even the whole VHA system. The dashboard currently includes the following information for ocular telehealth programs: percent of eye telehealth studies interpreted within 7 days, patient satisfaction results regarding receiving results in a timely manner, percent of VA clinics where eye services are offered (in person or via telehealth), image unreadable rates, and Healthcare Effectiveness Data and Information Set (HEDIS) diabetic retinal exam performance measure (monthly chart review reporting the percent of Veterans with diabetes who have a timely retinal evaluation). In addition to the information above, OCC sets national benchmarks for the TRI program, e.g., percent of images read within 7 days of being taken (>92%), image ungradable rate of 20% or less. As referenced above, the CPRS templates utilized during the clinical care encounter have linked health factors, which are discrete data elements that enter into the VA’s CDW. These health factors are used to monitor quality. For example, image gradeability is commonly incorporated into the TCTs performance appraisal. An image is considered gradable if at least 3 disk diameters of the retina are clearly visible in each field, in addition to a gradable macula and optic nerve. The CPRS TRI reader template has a field that addresses image clarity/readability and the reader selects “yes” or “no”. If no, the reader needs to select why the image is not gradable—whether it be from media opacity, vitreous hemorrhage, small pupils, or imager factors (e.g., wrong field taken). These selections are linked to health factors, and the dashboard displays percent of ungradable images, with the reasons, by pulling the health factors from the CDW. Results are provided to the facilities and concerns from the field, or items

330

discovered on quality site review, are shared with a quality peer review team for recommendations. Patient satisfaction is measured by a third party who sends email surveys to patients who have completed a telehealth appointment. Patients may choose to fill out a survey via email, and the results are compiled for national, with an ability to “drill down” to specific CBOC location, facility location, or specific programs. One can also evaluate at a bigger picture level, for example, all the synchronous telehealth program results across a particular healthcare system or VISN. 2. Field support The OCC’s robust administrative and technology support is yet another way the office supports front-line telehealth providers. This infrastructure is important for the implementation of any new telehealth program, including new eye programs besides TRI. One way to disseminate information about eye telehealth programs are the monthly national “Community of Practice” calls. There is one focused specifically on eye care and is attended by many VISN and facility telehealth coordinators, eye telehealth providers, preceptors, and TCTs. This meeting is led by the two Tele-Eye Care Co-Leads, the OCC Asynchronous Telehealth Lead, and national trainers, to answer questions from the field and to provide regular updates. This forum provides a space for discussion of problems, questions, and support for implementation. Many times this forum has allowed national representatives to be aware of problems sites are facing and to be proactive about those issues. The Tele-Eye Care Workgroup members and the Tele-Eye Care Co-Leads can then address these problems by utilizing the resources of OCC and the group’s expertise to reduce these barriers. 3 Technology support Equipment that is endorsed and vetted by the OCC technology division is supported at a high level by the telehealth office. Vetting telehealth equipment is a time-consuming process. For

A. Maa et al.

example, the fundus camera that is chosen by the VA to do TRI first had to be vetted by OIT and Biomed to ensure it was DICOM compliant, was compatible with VistA Imaging, and was approved to be connected to the VA network. Then the fundus camera was vetted for usability —since many of the TCTs taking fundus photographs are not specialized for eye care, ease of use and ease of training for this equipment is rigorously tested before being approved. Once the equipment is approved, however, OCC can maintain a government contract to purchase these cameras, which facilitates the ordering of the equipment for any site wishing to implement a new TRI site, and the OCC technology office maintains a 24-h hotline to troubleshoot technology failures. In addition, should a defect in the equipment arise, OCC technology office liaisons with the vendor, tracking the problem and its solution to completion for the entire enterprise.

Conclusion In summary, the VA was one of the early adopters of ocular telehealth, during a time when diabetic teleretinal screening elsewhere was in its infancy. The clinical and administrative processes that were developed for TRI implementation in the VA are comprehensive and robust, serving as an excellent blueprint for additional telehealth programs, both asynchronous and synchronous. These OCC processes continue today, supporting the newest ophthalmic telemedicine programs for glaucoma and macular degeneration, TeleEye Screening, Technology-based Eye Care Services (TECS), and Tele-Low Vision. II. TeleEye screening

Introduction and History The TeleEye Screening program represents an evolution of TRI that expands beyond the diabetic-only population to include targeted screening for Veterans “at-risk” for age-related macular degeneration (AMD) and glaucoma.

Veteran Affairs (VA) Ocular Telehealth Programs

Inclusion of these groups was necessary as the prevalence of these eye diseases increases in aging populations, a health concern in VA where over half the Veterans utilizing care are over 65 years old, many of whom have additional health factors that further increase risk of developing these conditions. Because many Veterans are unaware of their personal risk and symptoms are often absent early in the disease process, program efforts are focused on identifying and improving assessment rates of these high-risk individuals to prevent a potential vision-loss epidemic over the coming decades, which would not only be detrimental to the quality of life of many Veterans, but potentially over-burden VA eye care resources and elevate both direct and indirect care costs related to treatment and vision rehabilitation services. Since the goal of the program is to detect eye disease in early stages, at a time when the vast majority within VHA have reached the age of increasing prevalence, resources from TRI were leveraged in an expeditious and strategic fashion. This included leveraging the existing nationwide infrastructure of digital retinal imaging systems, established consult pathways, clinical scheduling processes, human capital of VA-certified retinal imagers and readers (licensed eye care providers), business processes, and health informatics techniques. The training and quality assurance programs that were developed and continually enhanced during the TRI era required minimal addition and adaptation to meet the TeleEye Screening Program needs. This resourcefulness provided an expansion of services that was both time-efficient and cost-effective for the organization, and neutralized many of the start-up barriers —such as cost involved with initial structural setup and clinical integration—that may hinder the ability of large healthcare systems to implement such complex yet vital programs focused on the prevention of vision loss within their populations. By continuing the legacy of TRI, the TeleEye Screening program serves as a pioneer in ocular healthcare by screening for the three most common causes of permanent vision loss within aging populations—diabetic retinopathy, macular degeneration, and glaucoma. The program is

331

truly representative of VA’s mission to be a leader in preventative care services and to meet the ever-changing needs of Veterans over time.

TeleEye Screening—Clinical Processes 1. Overall set up The foundational setup for TeleEye screening did not change in the evolution from diabetic TRI to the new, expanded program (see Diabetic Teleretinal Imaging section above). The program utilizes the same asynchronous format as did diabetic TRI, in which data and images are captured by a VA-certified Imager at the patient location and forwarded to a reader (teleconsultant) to be reviewed at a later time or at a remote location. After the reader provides review, assessment, and recommendation for the consultation, the primary care provider receives a CPRS view alert to the results and recommendations so that action can be taken for the Veteran’s follow-up care. As with diabetic TRI, Veterans maintain the convenience of obtaining the eye screening at their primary medical care home, whether a VHA hospital or CBOC, depending on where the Veteran resides. This opportunity for closer access is important, as it provides an earlier opportunity for assessment and earlier detection of eye disease. The program’s focus on removing distance as a barrier is especially important for those in highly rural areas who are known to go longer periods without eye care, and thus have a higher risk for eye disease and vision loss [8]. To reduce the number of visits a Veteran must make to VA for the eye screening visit, attempts are made to pair or “co-appoint” the TeleEye Screening service with an existing healthcare appointment in advance, or to provide walk-in or same-day service. This practice also serves to lessen the overall travel-pay costs incurred by VA. Some sites have begun to place greater emphasis on “dedicated” schedulers for the program, as an effort to match the increase in Veteran demand for the service with timely access and maximum clinic utilization.

332

2 Patient selection/referral into the program The TeleEye Screening program places heavy emphasis on a national clinical reminder that was specifically developed to standardize the service across the VA. The “Eye Care At Risk Screen” reminder is an all-in-one CPRS patient flag in that it identifies the appropriate “at-risk” candidates for diabetic retinopathy screening, macular degeneration, and glaucoma screening. The clinical reminder is embedded in the CPRS program and can be activated across all systems within a network at once or for individual clinics separately. The clinical reminder logic is complex and relies on multiple data factors being queried within the CDW for each individual. The candidacy criteria and taxonomies developed were informed by evidence-based medicine and are structured to be modifiable as newer research informs and further defines risks correlated with macular degeneration and glaucoma. Broadly speaking, the “at-risk” candidate cohorts have characteristics defined by age, ethnicity, personal health factors/conditions, and interval since the last ocular assessment. As such, the logic queries the patient demographic section of the EMR for age and ethnicity, diagnosis by International Classification of Disease (ICD) codes to identify medical conditions/diagnoses documented by providers, and unique clinical stop codes (i.e., numbers representing a visit to either optometry or ophthalmology) for past clinical visits. All defined criteria must be met for a Veteran to be identified as a member of the “atrisk” cohort and eligible for screening. As an example, a candidate that would meet the “at risk” for macular degeneration cohort would be over 60 years of age, of Caucasian ethnicity, with a history of cigarette smoking, and not known to have received a fundus assessment within the last two years. Likewise, a candidate that would meet the “at risk” for glaucoma cohort would be: over 40 years of age, of African ancestry, with a family history of glaucoma, and not known to have received a fundus assessment within the last two years. As mentioned, the logic also includes detection of the traditional TRI

A. Maa et al.

diabetic candidates that have long served as the foundation of the program and continue to represent the vast majority of the population screened in the VA. After a TeleEye screening is performed, the clinical reminder is “processed” for a period of time as determined by the results of the screening, thereby removing the “due” status. This change in status occurs automatically when the qualifying episode of care (e.g., TeleEye Screening, inperson eye clinic appointment) is recognized by the software logic. The reminder will become due again when it is time for the next clinically indicated episode of care. The clinical reminder also recognizes care that has been provided in-person by optometry and ophthalmology eye clinics, and upon recognition of the clinical visit and findings (per applied ICD codes), resets the reminder appropriately. Manual ad-hoc processing of the clinical reminder is also possible for Veterans who receive primary care service within VHA but eye care outside the network. The ability of the reminder logic to capture various episodes of care helps to ensure accurate accounting of patient care status both internal and external to VHA so that duplication of services can be avoided and that care occurs at appropriate intervals. Figure 5 illustrates the “At risk Eye” Reminder. Reporting capabilities have been developed for the “Eye Care At Risk Screen” clinical reminder so that the total number of “at-risk” Veterans within a healthcare system can be identified, as well as reporting down to the individual level with current status, last date of retinal assessment, and next date due. Future clinical appointments are also identified within the reporting so that an attempt to “co-appoint” the TeleEye Screening with another upcoming appointment can be made. Therefore, in addition to providing a mechanism to identify the appropriate “at-risk” Veterans in need of screening, the clinical reminder provides a mechanism for monitoring to ensure Veterans receive and return for care at appropriate intervals based on established quality care guidelines, with a reporting tool that helps with program accounting and planning for future care visits.

Veteran Affairs (VA) Ocular Telehealth Programs

333

Fig. 5 CPRS screenshot of “At Risk Eye” clinical reminder. Credit VHA CPRS, used with permission

3 TeleEye imaging protocol and intraocular pressure measurement TeleEye Screening maintained the validated disease-detection imaging standard used in diabetic TRI. This protocol requires the capture of three 45-degree fundus images (central, superiortemporal, and nasal fields) and an external image of each eye. See Fig. 4 in Diabetic TRI section. The capture of a quality central image, which provides visibility of both the macula and optic nerve head, is critical to ensure the consulting reader can identify indicators for both macular degeneration and glaucoma. Readers utilize image display software that allows manipulation of the images (i.e., zoom, brightness and contrast adjustment, color filters, side-by-side comparison, historical/sequential comparison) to greatly enhance details when evaluating image sets. Image quality is evaluated and documented by the reader for every patient encounter. As with

diabetic TRI, image quality rates are tracked for imagers individually and for each facility and VISN to ensure high-quality standards are maintained within the program. Although some local TRI programs chose to incorporate the measurement of intraocular pressure (IOP) into diabetic screening protocols, the advent of the TeleEye Screening program meant that IOP measurement became a foundational procedure and is required to be done on every patient. Critical in determining the risk of developing glaucoma in the glaucoma-risk cohort, IOP is also measured in the other “atrisk” cohorts (diabetic retinopathy, macular degeneration) to ensure all Veterans receiving TeleEye are screened for glaucoma. A lesson learned from the initial diabetic TRI program was that many patients had multiple eye conditions identified in the screening [9, 10], so it was not unusual for a single patient to have the presence of diabetic retinopathy, macular degeneration,

334

and glaucoma—all conditions previously unknown to the patient prior to screening. The TeleEye Screening program took note of this lesson and chose to apply a single protocol for retinal imaging, IOP measurement, and data collection that is agnostic to the specific cohort(s) for which the Veteran may have been identified for entry into the program. In addition to ensuring each Veteran is screened for multiple conditions at once, the use of a single protocol reduces complexity for program imagers and staff so that separate protocols are not required for each individual “at-risk” group. A specific rebound tonometer was selected and endorsed for use in the program due to ease of use and transport, not requiring topical anesthesia, and utilizing single-use sterile, disposable probes that improved process efficiency and reduced risk of disease transmission. Prior to selection, the device was found to have good correlation with Goldmann tonometry, considered the gold-standard for accurate intraocular pressure measurement [11]. 4 Information capture and image interpretation As with diabetic TRI, nationally standardized CPRS templates to gather patient information were developed for both imagers and readers. Modernized, these new “smart” templates contain software logic that modifies elements and content to be captured by user dependent upon the Veteran’s specific “at-risk” condition(s) and selections made. For example, if a Veteran is identified to be at risk for macular degeneration, the logic automatically modifies the imager template to include content to capture smoking status and family history of the condition. The ability to capture elements specific to the patient’s condition helps to streamline the process and determine the education and recommendations necessary to modify risk factors, such as smoking cessation classes or pharmaceutical cessation devices offered through the VA. Similarly, the reader template is responsive based on the selections made so that findings can be captured down to the level of detail necessary to appropriately guide the referral and

A. Maa et al.

recommendations for care. This creates a process that is fluid and efficient for the readers. The unique health factors captured within these “smart” templates can be used for reporting and to identify elements of interest, such as frequency of specific findings and rates of referrals (Fig. 6). 5 Imager training & patient education materials New training material and protocols were developed for TeleEye Screening. This included standardized online course modules focused on providing certified imagers with a fundamental knowledge and understanding of macular degeneration and glaucoma, features of the conditions as seen on retinal imaging, and the effect of these conditions on patient vision. Standardized consult-pathway and template training presentations were developed for program preceptors to deliver to their local TeleEye Screening teams, as well as official User Guides that outline the features and use of the “Eye Care At-Risk Screen” clinical reminder. An online electronic training module for providing patient education was developed to aid TCT imagers in their discussions with Veterans about diabetic retinopathy, macular degeneration, and glaucoma. The training focused on the elements that should be provided as part of routine patient education, such as helping Veterans understand why they are considered to be at risk, factors that can be modified in effort to promote self-management to reduce risks, and the importance of routine eye care for early detection of eye disease to help prevent vision loss. Additionally, a national VHA OCC TeleEye Screening patient education pamphlet was created for the purpose of bringing awareness to Veterans of these vision-threatening conditions and the importance for Veterans identified to be “at-risk” to receive ocular screening. Figure 7 A&B illustrates the TeleEye Screening brochure. For inclusion of tonometry and eye pressure measurement, a robust training module was created to provide certified imagers with a detailed instructional guide for use of the rebound tonometer endorsed for the program. Live virtual demonstration sessions and training events were

Veteran Affairs (VA) Ocular Telehealth Programs

335

Fig. 6 CPRS screenshot excerpts of the Tele-Eye screening reader note. Credit VHA CPRS, used with permission

held in joint by VHA and the vendor training team, followed by hands-on training by the local program preceptor and/or eye clinic staff. Additional training was provided for actions to take in event of urgent findings as related to tonometry measurements (i.e., if IOP measured is >30, first recheck then contact reader promptly). After training requirements are completed and a program preceptor endorses the imager’s proficiency, the imager is scheduled to demonstrate

practical competency to a VHA licensed eye care provider (or their designee). Only after receiving training and certification that verifies proficiency in both retinal image capture (as noted in the diabetic TRI section) and tonometry can an imager begin to participate in the TeleEye Screening program. Annual competency assessment ensures an imager maintains the knowledge and proficiency to ensure quality standards are maintained within the program.

336

Fig. 7 TeleEye screening brochure. Credit VHA Office of Connected Care, used with permission

A. Maa et al.

Veteran Affairs (VA) Ocular Telehealth Programs

6 Reader training Reader training follows the already established diabetic TRI pathway, with standardized online courses that provide instruction on consult pathways, template use, and VistA Imaging tools. Increased focus was made to standardize reading protocols for macular degeneration and glaucoma, to ensure reliability and consistency among readers to bolster the effectiveness of the screening program.

TeleEye—Administrative Processes Quality assurance, quality improvement, field support, and technology support all follow the established diabetic TRI OCC infrastructure.

Conclusion TeleEye Screening is the natural “next step” evolution of the VA’s successful diabetic TRI program. Implementation began in 2019, and more facilities are converting their diabetic TRI program into the more robust TeleEye Screening Program, which will allow the VA enterprise to detect more diseases in the at-risk Veteran population at an earlier stage to prevent vision loss.

III. Technology-based (TECS)

Eye

Care

Services

Introduction and History Another VA ocular telehealth program arising from the success of TRI, Technology-based Eye Care Services (TECS), began as a three-location pilot in March 2015 at the Atlanta VA Healthcare System. It was initially funded by the VA Office of Rural Health (ORH), with the goal to provide telehealth-based screening eye exam and prescription glasses for rural Veterans who lived distant from an in-person VA eye clinic. Many Veterans have refractive error and presbyopia,

337

especially given the age of the Veteran population utilizing VA for services, and therefore, provision of eyeglasses can significantly improve their quality of life. In addition, most Veterans have an earned benefit to receive eyeglasses from the VA at no cost, so this program aimed to improve access for rural Veterans to receive a screening eye exam and provision of an eyeglass prescription from their primary medical care home. Since the advent of TECS in 2015, this telehealth eye care delivery model has rapidly spread across the VA enterprise, with greater than 60 sites covering more than 20 different VA healthcare systems at the time of the writing of this book chapter. TECS is primarily spread through the support of the VA ORH, but in recent years has also been funded and established through grants from the OCC and the VISN Telehealth Clinical Resource Hubs (CRH) which has begun incorporating specialty care services in FY 20 and FY 21. Figure 8 shows the states where TECS sites are located, current to time of writing of this chapter (summer 2021).

TECS—Clinical Processes 1. Overall set up In TECS, a certified ophthalmology technician is stationed at the clinic to serve as the patient site imager. These technicians have specialized eye care training; the minimum certification requirement level is a Certified Ophthalmic Assistant (COA), obtained from International Joint Commission of Allied Health Professionals in Ophthalmology (iJCHAPO). In a space as small as 10  12 feet, the technician performs the components of a typical in-person eye work up, asking pertinent chief complaint, history of present illness, ocular, family, and social history questions, and ocular symptoms. After the history is gathered the technician performs a vision check, manifest refraction, pupil check, anterior chamber depth check with penlight, measures intraocular pressure (IOP), and then dilates the patient for fundus photographs. The fundus photography protocol is the same as the TRI protocol (see

338

A. Maa et al.

Fig. 8 Map of active and implementing TECS sites current as of summer 2021. Credit Daryl Johnson, PhD and April Maa, MD; used with permission

Fig. 4 above in diabetic TRI section). Information is entered into the CPRS medical record, images are transferred to VistA Imaging, and a remote eye provider reads the images, usually the same day it is done. Figure 9A and B illustrates a typical TECS clinic set up. The TECS equipment varies slightly from location to location, but the “standard” TECS

A

package includes a 3-instrument table upon which the fundus camera (which can be a nonmydriatic fundus camera—same as utilized by diabetic TRI or TeleEye Screening) is placed, along with an automated lensmeter. The other side of the 3-instrument table supports a phoropter, which is then aligned with a vision chart mounted to the wall. TECS uses the same

B

Fig. 9 A & B A typical TECS room set up. Credit Aaron Jerrells LPN, first published in Elsevier Ocular Telehealth Textbook, used with permission from Elsevier

Veteran Affairs (VA) Ocular Telehealth Programs

rebound tonometer as TeleEye Screening, and Fig. 9 shows an advanced TECS site which is capable of both ocular coherence tomography (OCT) and static perimetry testing (visual field machine is present). Demo eyeglass frames are displayed either by a wall mount or a floor mount. The TECS site also has standard eye lane equipment such as a finoff transilluminator, trial frames, trial lenses, and color plates. Similar to TRI and TeleEye Screening, TECS readers are licensed, credentialed, and privileged VA eye providers. However, different from the other eye telehealth programs, the TECS readers are online at the same time as the ophthalmic technicians, and typically monitor multiple TECS sites simultaneously through internal messaging (IM). TECS is a telemedicine program, meaning that unlike TRI and TeleEye Screening, the eye provider assumes ownership and liability of the patient’s care. TECS readers can prescribe eyeglasses and medications as indicated after review of the TECS study, and may choose to follow up the asynchronous visit with a synchronous phone call or even a video call to provide medical counseling and education to the patient, especially if there is disease present. 2 Patient selection/referral into the program Patients are referred to the TECS program in multiple ways—patients can self-refer, primary care can refer the patient, or patients can be referred from the diabetic eye exam or eye “atrisk” clinical reminder. TECS generally serves as one entry point for the Veteran to access VA eye care services. Schedulers, referring providers, the ophthalmic TECS technicians, and the patients are educated that if they have a known chronic eye disease (e.g., glaucoma) or an acute eye problem (e.g., sudden vision loss) that they should not utilize the TECS program. Information about TECS is disseminated through a national brochure, a wall poster, and a short presentation “lunch and learn” for the CBOCs and primary care providers once a TECS site becomes operational. The ophthalmic TECS technician often serves as the initial CBOC

339

champion of the program. A TECS screening exam satisfies the clinical reminders for either a diabetic, macular degeneration, or glaucoma screen, which prevents redundancy with TeleEye Screening or diabetic TRI programs. Figure 10A and B illustrates the TECS brochure for patients (Figs. 11 and 12). 3 TECS protocols The fundus imaging protocols are the same as diabetic TRI and TeleEye Screening, and include a external photograph of each eye. This way, if a diabetic enters the TECS program, the photo set is accurate to detect diabetic retinopathy. Almost all patients are dilated, which facilitates high-quality photos, a very low image unreadable rate (30 mmHg). This is usually achieved through internal messaging (IM) or by phone. While the reader will refer based on their assessment of the patient information and photographs, there are also builtin recommendations for referral such as 20/40 vision in either eye, moderate hypermetropia for gonioscopy, or history of flashes or floaters that have changed. While a TECS screening is certainly not intended to replace an in-person eye exam, the TECS protocol has been shown to be comparable to an in-person exam in terms of diagnosing common eye conditions [12]. Many other published papers demonstrate the accuracy of using diabetic photos to diagnose non-diabetic eye conditions [13, 14]. 4 Information capture and Image Interpretation TECS, similar to diabetic TRI and TeleEye Screening, has national standardized CPRS templates for both the ophthalmic technician and the reader. The CPRS note templates follow the layout of an in-person eye clinic note, and there

340

A. Maa et al.

A

B

Fig. 10 A & B The TECS brochure. Credit VHA Office of Connected Care, used with permission

Veteran Affairs (VA) Ocular Telehealth Programs

A

B

C

Fig. 11 A, B, & C Screenshot excerpts of the TECS ophthalmic technician CPRS note template

341

342

A. Maa et al.

A

B

Fig. 12 A & B Screenshot excerpts from the TECS reader CPRS note template

are linked health factors for almost every entry in the TECS template, which is critical for quality monitoring and research. The figure below illustrates part of the technician and reader template. Images are transferred via a consult entered in CPRS, which populates the modality worklist of the camera. This is the same process as diabetic TRI and TeleEye Screening. 5 Imager training & patient education materials Technicians receive training through the patient side facility. They already have the majority of

the skills to accomplish the work since they are, at minimum, COA certified. When the TECS technician is initially hired, he/she first does work ups in the eye clinic on in-person patients. This allows the eye providers and technician’s supervisor to observe the new TECS technician first hand, thereby building trust and establishing confidence with that technician’s skills. After, the eye clinic checks off the TECS eye technician’s competencies (e.g., manifest refraction, lensometry). Once the competencies are verified, the technician goes to the CBOC where the TECS National Program Coordinator, an IJCAHPO

Veteran Affairs (VA) Ocular Telehealth Programs

certified ophthalmology technician, trains the technician on the nuances of seeing patients through the TECS asynchronous process. This training can be conducted virtually or in-person, and typically focuses more on the details of how the clinic day flows, communication with the reading provider (usually via IM), using the TECS technician CPRS template, and specific points in the protocol where the technician should notify the provider (e.g., high IOP). The local TECS Site Director will review 10 random patient work ups by the new ophthalmic TECS technician, focusing on accuracy of history, refraction, and image quality. If any deficiencies are noted, the technician receives further training from the supporting eye clinic or the TECS National Program Coordinator. There are also multiple online resources for new TECS technicians, including a TECS demonstration video, that can be viewed at any point in the training process. TECS ophthalmic technicians are also required to view the rebound tonometry and fundus photography protocol video created by OCC, as the eye pressure measuring device and the fundus imaging protocol are the same for diabetic TRI, TeleEye Screening, and TECS. As part of their scope of practice obtained during iJCAHPO certification, ophthalmic technicians can educate patients about their disease process. The TECS brochure as well as diseasespecific handouts can be provided to patients for education about both the program and their specific disease process. 6 Reader training TECS has a detailed reader training program that follows both the diabetic TRI and current TeleEye Screening Programs. (See subsection on TRI training in section I: diabetic teleretinal). Since TECS readers are already VA credentialed and privileged eye providers, training is focused on processes—how to locate patient studies, how to utilize tools in the display software (VistA Imaging), e.g., changing contrast, and how to fill out the CPRS note templates. As part of training,

343

TECS also provides the readers a practice set of patients and then administers a “test” using real TECS patients to simulate TECS reading. The quiz is graded, and results provided to the local TECS Site Director before the reader is “certified” to read for TECS.

TECS—Administrative Processes 1. Quality assurance, monitoring, and improvement processes TECS also follows the same quality metrics as diabetic TRI and TeleEye Screening and is subject to the same Conditions of Participation. In addition, TECS has a detailed quality assurance and improvement program that supplements the OCC dashboard. TECS quality monitoring is primarily gathered by a third-party internal VA data team comprising of data statisticians, an epidemiologist, and a health economist. In addition to the basic demographics and rurality of the patients served, multiple metrics centered on quality of care, access, cost, and customer satisfaction are measured on a quarterly basis, summed annually, and reported to VA program stakeholders. Quality measures include image quality score, concordance of reads (peer review) within a reading group, and agreement in diagnoses between TECS referrals and in-person follow-up exams. Access is measured by calculating wait times, defined as the number of days between the time the patient calls for (or desires) an appointment to the date the patient is seen. Initially, cycle time through TECS in Atlanta was also assessed, showing that the telehealth appointment was shorter than a typical in-person eye clinic exam. Cost savings from the organizational and patient perspective is also calculated. A random sample of patients is selected from each CBOC location that has TECS and the difference in driving time calculated when the patient travels to the main hospital for care versus the CBOC for care. Patient satisfaction is measured by the OCC survey. Tables 1 and 2 show excerpts from the annual TECS report from FY 21, Quarter 3.

344

A. Maa et al.

Table 1 Excerpt of demographic data from TECS National Report, FY 21, 3rd Quarter

Number of patients

Urban

Rural

Highly rural

Total

3119

3115

145

6379

Age, mean ± SD

63.0 ± 13.0

66.8 ± 13.3

67.7 ± 13.3

65.0 ± 13.3

Female

54.7 ± 12.6

55.8 ± 12.9

52.0 ± 20.5

55.0 ± 12.8

Male

64.4 ± 12.5

67.6 ± 12.9

68.6 ±12.3

66.1 ± 12.8

Female

466

14.9%

210

6.7%

8

5.5%

684

10.7%

Male

2653

85.1%

2905

93.3%

137

94.5%

5695

89.3%

NHW

1213

38.9%

2387

76.6%

120

82.8%

3720

58.3%

NHB

1638

52.5%

399

12.8%

8

5.5%

2045

32.1%

Hispanic

64

2.1%

48

1.5%

1

0.7%

113

1.8%

Gender, n %

Race-ethnicity, n%

Asian

11

0.4%

5

0.2%

0

0.0%

16

0.3%

Native american

13

0.4%

12

0.4%

4

2.8%

29

0.5%

Pacific islander

18

0.6%

10

0.3%

1

0.7%

29

0.5%

Unknown

162

5.2%

254

8.2%

11

7.6%

427

6.7%

All patients, n % of total

276

8.8%

227

7.3%

7

4.8%

510

8.0%

Female, n % of females

69

14.8%

22

10.5%

2

25.0%

93

1.5%

Male, n % of males

207

7.8%

205

7.1%

5

3.6%

417

6.5%

All patients, n % of total

359

11.5%

124

4.0%

3

2.1%

486

7.6%

Female, n % of females

71

15.2%

18

8.6%

0

0.0%

89

1.4%

Male, n % of males

288

10.9%

106

3.6%

3

2.2%

397

6.2%

OEF/OIF

Homeless

Last VA eye exam^^, n %

2526

Less than 2 years

1781

70.5%

1117

2100 53.2%

48

47.1%

2946

62.3%

2–5 years

607

24.0%

776

37.0%

47

46.1%

1430

30.2%

6–10 years

86

3.4%

146

7.0%

4

3.9%

236

5.0%

Over 10 years

52

2.1%

61

2.9%

3

2.9%

116

2.5%

To monitor quality of care, especially when a new TECS site starts, local TECS Site Directors perform a random ten patient chart review for each new reader after their training is complete, and then the providers’ ongoing competency in providing telehealth eye care is monitored by the VA’s customary provider clinical review processes called OPPE, ongoing professional practice evaluation. OPPE is the same process that is used for in-person care so the telehealth care review is not different from in-person care review. As with TRI and TeleEye Screening,

102

4728

TECS can be read by local VA eye providers or by a remote hub. Nationally, groups of TECS readers also participate in peer review, where 5 TECS studies are randomly selected per month and assigned to another reader in that group to be reviewed. Peer review asks each reader whether they (1) agree with the findings and diagnosis and (2) whether they agree with the management. Major discrepancies are reviewed and addressed by the local TECS site director. Table 3 shows an excerpt of the image quality and peer review results.

Veteran Affairs (VA) Ocular Telehealth Programs

345

Table 2 Excerpt of wait time to TECS appointment, TECS national report, FY 21, 3rd quarter

N

Urban

Rural

Highly rural

Totala,b

3142

3130

145

6417

Wait times, n % 0 day

1305

41.5%

1614

51.6%

67

46.2%

2986

46.5%

1 day

147

4.7%

143

4.6%

5

3.4%

295

4.6%

2–7 days

320

10.2%

436

13.9%

15

10.3%

771

12.0%

8–14 days

224

7.1%

229

7.3%

18

12.4%

471

7.3%

15–30 days

351

11.2%

291

9.3%

24

16.6%

666

10.4%

>30 days

795

25.3%

417

13.3%

16

11.0%

1228

19.1%

Wait time is defined as time from desired date (or date of call to scheduler, if Veteran requested “next available”) to the date of TECS appointment a Missing RUCA (n = 0) b Missing wait times (n = 7)

Table 3 Excerpt of image quality and peer review results, National TECS Report, FY 21, 3rd Quarter Total

Atlanta

Nb

Disagreements, n (%)

Image quality scorea, mean± SD

Image quality score  4, n (%)

239

5

2.1

5.8 ± 0.7

15

44

1

2.3

5.9 ± 0.4

2

6.3 4.5

3

0

0.0

6.0 ± 0.0

0

0.0

Central alabama

20

1

5.0

5.8 ± 0.7

1

5.0

Central IOWA

8

0

0.0

6.0 ± 0.0

0

0.0

Chicago-hines

Augusta

10

0

0.0

6.0 ± 0.0

0

0.0

Clarksburg

4

0

0.0

4.5 ± 3.0

1

25.0

Dublin

2

0

0.0

6.0 ± 0.0

0

0.0

9

0

0.0

6.0 ± 0.0

0

0.0

45

2

4.4

5.7 ± 0.7

5

11.1

Minneapolis

1

0

0.0

6.0 ±

0

0.0

Montana

9

0

0.0

5.7 ± 0.7

1

11.1

Oklahoma city

1

0

0.0

6.0 ±

0

0.0

Kansas city Memphis

44

0

0.0

5.9 ± 0.7

2

4.5

South dakota

4

0

0.0

5.5 ± 1.0

1

25.0

Wilmington

35

1

2.9

5.9 ± 0.5

2

5.7

Omaha

a

Score out of a 6-point scale b Includes completed peer reviews between April 1st-June 30th, 2021 SD field per TECS clinic missing because peer review only available for one patient

2 Field support As a national ocular telehealth program, TECS is supported by the infrastructure and resources of OCC. OCC supports training and implementation. TECS can be implemented by a facility on their own, or if funding is obtained by VA ORH,

the ORH TECS Program Office supports implementation using a process called implementation facilitation. With implementation facilitation, there is active project management specifically tailored to the facility, geared towards addressing the specific implementing needs of the local VA. Many aspects of TECS implementation are

346

customizable in order to ultimately foster sustainability—the continuation of the program once funding is done. The end goal is that the TECS telehealth model is integrated into the way the clinic delivers eye care in the future. Moreover, as discussed previously, OCC has an excellent team to support implementation of all the ocular telehealth programs. Both VISN and facility telehealth coordinators can learn about TECS from the internal OCC website, where they can find detailed protocols, brochures, national templates, and other relevant materials about the program. These resources help TECS sites, whether new or established, be standardized across the country. TECS issues and information about TECS are also discussed on the national Eye Care Community of Practice Calls. 3 Technology support OCC supports some of the TECS equipment nationally as discussed above in prior sections. The facility’s local Biomed department may also provide support in the event of technology failure.

TECS—Conclusion TECS is a unique ocular telehealth program that aims to provide asynchronous telemedicine eye care to Veterans from their primary medical care home. TECS was built upon the excellent foundation of diabetic teleretinal screening, and continues to grow and succeed in providing highquality care under the direction and guidance of OCC. The TECS program is intimately connected with both primary care and specialty eye providers—and over time will allow the VA to have a strong telehealth eye care delivery infrastructure that allows for comprehensive population management while reducing Veteran healthcare disparities and improving Veteran access to eye care. IV. Tele-Low Vision The third ocular telehealth program run by the VA is Tele-Low Vision. Unlike the others, Tele-

A. Maa et al.

Low Vision is synchronous, and the patient usually presents first to the CBOC, with subsequent visits completed with direct video to the patient’s home, or back at the CBOC. This program is described in depth in the Chapter discussing low vision in the textbook.

Future Directions of Ocular Telehealth in the VA The future of ocular telehealth within the VA system is incredibly bright. OCC is a true partner with all the clinical stakeholders, supporting the advancement of more innovative ways to deliver eye care using telemedicine techniques. Future innovations include (1) more mobile units, (2) tele-follow-ups, (3) Others—anterior segment telehealth, remote refraction, home monitoring, and 4) big data research.

Mobile Units Certain VA healthcare systems have significant experience with mobile vans and are very familiar with utilizing mobile units to serve their rural and highly rural population. One example is the VA located in Cheyenne, Wyoming. They have placed the TECS program on a small van (Fig. 13) that is currently accompanying a larger primary care van, that travels to remote locations to serve Veterans.

Tele-Follow-Ups Furthermore, just as TECS built upon the success of TRI, many VA facilities are extending the TECS screening exam infrastructure to encompass tele-follow-ups, where patients with disease may alternate between a telehealth exam and an in-person clinic exam. For instance, a patient with ocular hypertension, stable on one medicine, may choose to have 1 or 2 visits at their CBOC through telehealth, potentially doing a visual field and/or optical coherence tomography (OCT) at that telehealth visit along with an IOP

Veteran Affairs (VA) Ocular Telehealth Programs

347

Fig. 13 The first TECS mobile van for eye care. Credit Cheyenne Wyoming VA TECS team, used with permission

check. Then, if stable, they may see their inperson eye provider yearly. If there is concern that the patient’s disease has progressed, the reader may send the patient back to the eye clinic for an in-person exam in a sooner timeframe. The literature provides several examples of international eye clinics utilizing this approach, which has been verified to be high quality and safe [15–17]. Patient selection is critically important in these care delivery models. Patients with unstable or severe disease are not candidates for this type of care. National TECS teleglaucoma and tele-macula protocols, written by the respective sub-specialists in the field, are currently being deployed in a few TECS clinics that are outfitted with visual field machines and/or OCT imaging capability.

Other Developments—Anterior Segment, Remote Refraction, Remote Monitoring Other innovations are centered on eye clinics developing more detailed anterior segment imaging protocols using synchronous slit lamp

video or slit lamp photographs, and the national Tele-Eye Care Workgroup is assessing remote refraction technology that is available on the market now and determining if it can be used within the VA system. Furthermore, with the development of home eye monitoring tools for AMD or eye pressure, there may be a time when these technologies are also integrated into the VA’s ocular telehealth programs.

Big Data and Research One of the greatest strengths of the VHA ocular telehealth programs, facilitated by CPRS, is the availability of data about each photo set and the metadata associated with each patient. The diabetic TRI, TeleEye Screening, and TECS templates are standardized across VHA. In all programs, information is entered into the relevant template during a regular clinical care encounter and the information entered creates a linked “health factor”. This health factor can be extracted through the VA’s database, CDW, which can then be queried using Structured Query Language (SQL) such that elements of the

348

eye exam or the entire clinical note can be “pulled” for every patient with a few key strokes. For instance, best manifest refracted vision is a health factor in the TECS template. This vision can be queried from the CDW and associated with other patient data such as patient medical history, lab values obtained nearest to the TECS visit, and ultimate patient visual outcome (if longitudinal data is available). These features about the ocular telehealth templates help facilitate research both within the VA and with outside collaborators regarding predictive analytics or artificial intelligence (AI). The contribution of VA data to research in AI has already led to important findings published in the literature [18–20] that will help improve the use of these innovations in the future for Veterans and for patients across the world.

Conclusion VHA, one of the largest unified healthcare systems in the country, has done telehealth successfully for decades. Ocular telehealth is thriving within the VA system, facilitated by the national infrastructure and organizational support that mitigates or removes the typical barriers that block telehealth. Other places can look to the VA as an example to glean best practices for a model of telehealth implementation and sustainability. The VA ocular telehealth programs allow for the enterprise to have a powerful tool to supplement traditional in-person care; allowing the system to have a vigorous population health management strategy that incorporates VA clinical practice guidelines, improves access, and reduces healthcare disparities across the eye care spectrum.

A. Maa et al.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

References 1. Affairs V. VHA Quick Facts 5/2021 ed2021. 2. Rural Veterans Health Care Atlas. Available from: https://www.ruralhealth.va.gov/docs/atlas/CHAPTER_ 02_RHRI_Pts_treated_at_VAMCs.pdf. 3. VA Pyramid Analytics Database [Internet]. 2021. 4. Congdon N, O’Colmain B, Klaver CC, Klein R, Munoz B, Friedman DS, et al. Causes and prevalence

17.

of visual impairment among adults in the United States. Arch Ophthalmol. 2004;122(4):477–85. Cavallerano AA, Cavallerano JD, Katalinic P, Tolson AM, Aiello LP, Aiello LM, et al. Use of joslin vision network digital-video nonmydriatic retinal imaging to assess diabetic retinopathy in a clinical program. Retina (Philadelphia, Pa). 2003;23(2):215–23. Cavallerano AA, Conlin PR. Teleretinal imaging to screen for diabetic retinopathy in the veterans health administration. J Diabetes Sci Technol. 2008;2 (1):33–9. Cavallerano JD, Aiello LP, Cavallerano AA, Katalinic P, Hock K, Kirby R, et al. Nonmydriatic digital imaging alternative for annual retinal examination in persons with previously documented no or mild diabetic retinopathy. Am J Ophthalmol. 2005;140 (4):667–73. Centers for Disease C, Prevention. Eye-care utilization among women aged > or = 40 years with eye diseases–19 states, 2006–2008. MMWR Morb Mortal Wkly Rep. 2010; 59(19):588–91. Maa AY, Patel S, Chasan JE, Delaune W, Lynch MG. Retrospective evaluation of a teleretinal screening program in detecting multiple nondiabetic eye diseases. Telemed J E Health. 2016. Chasan JE, Delaune B, Maa AY, Lynch MG. Effect of a teleretinal screening program on eye care use and resources. JAMA Ophthalmol. 2014;132(9):1045–51. Rodter TH, Knippschild S, Baulig C, Krummenauer F. Meta-analysis of the concordance of Icare ((R)) PRO-based rebound and Goldmann applanation tonometry in glaucoma patients. Eur J Ophthalmol. 2020;30(2):245–52. Maa AY MC, Lu X, Janjua R, Howell A, Hunt K, Giangiacomo A, Lynch MG. Diagnostic accuracy of TECS: part I of TECS compare trial. Ophthalmol. 2019. MG. MAPSDWCJL. Retrospective evaluation of teleretinal screening program in detect multiple non-diabetic eye diseases. Telemed E-Health J. 2016. In Press. Sperduto RD, Hiller R, Podgor MJ, Palmberg P, Ferris FL 3rd, Wentworth D. Comparability of ophthalmic diagnoses by clinical and reading center examiners in the visual acuity impairment survey pilot study. Am J Epidemiol. 1986;124(6):994–1003. Kassam F, Yogesan K, Sogbesan E, Pasquale LR, Damji KF. Teleglaucoma: improving access and efficiency for glaucoma care. Middle East Afr J Ophthalmol. 2013;20(2):142–9. Kiage D, Kherani IN, Gichuhi S, Damji KF, Nyenze M. The muranga teleophthalmology study: comparison of virtual (Teleglaucoma) with in-person clinical assessment to diagnose glaucoma. Middle East Afr J Ophthalmol. 2013;20(2):150–7. Owsley C, Rhodes LA, McGwin G Jr, Mennemeyer ST, Bregantini M, Patel N, et al. Eye care quality and accessibility improvement in the community (EQUALITY) for adults at risk for glaucoma: study rationale and design. Int J Equity Health. 2015;14(1):135.

Veteran Affairs (VA) Ocular Telehealth Programs 18. Lee AY, Yanagihara RT, Lee CS, Blazes M, Jung HC, Chee YE, et al. Multicenter, head-tohead, real-world validation study of seven automated artificial intelligence diabetic retinopathy screening systems. Diabetes Care. 2021;44(5):1168–75. 19. Phene S, Dunn RC, Hammel N, Liu Y, Krause J, Kitade N, et al. Deep learning and glaucoma

349 specialists: the relative importance of optic disc features to predict glaucoma referral in fundus photographs. Ophthalmol. 2019;126(12):1627–39. 20. Kuzmak P, Demosthenes C, Maa A. Exporting diabetic retinopathy images from VA VistA imaging for research. J Digit Imaging. 2019;32(5):832–40.

Retinal Screening of Patients with Diabetes in Primary Care Clinics Why Has Uptake of This Promising Idea Been So Low? Kanagasingam Yogesan, Andrew Wilcock, and Ateev Mehrotra

Abstract

Diabetic eye disease is the most common form of blindness in the industrialized world. Many cases could be prevented via screening. Unfortunately many patients with diabetes do not receive retinal screening because of the logistics in getting to an ophthalmology appointment. To address there has been a push to have retinopathy screening at the primary care physician practices through telemedicine. It is a compelling idea with data to support its effectiveness, but few primary care doctors use it. In 2015, based on publicly available records on providers who serve the Medicare fee-for-service population, only 255 non-ophthalmologist billed for more than ten teleretinal screening sessions. In this chapter, based on our interviews with primary care physicians in USA (approached 200 of them), we describe the underlying reasons and potential solutions. We found only few primary care providers (PCP) have implemented such screening into their practice. That number is unlikely to

K. Yogesan (&)
A. Wilcock
A. Mehrotra Department of Health Care Policy, Harvard Medical School, Harvard University, Cambridge, MA, USA e-mail: [email protected] K. Yogesan School of Medicine, University of Notre Dame, Fremantle, Australia

increase until camera manufacturers bring down the upfront costs of fundus cameras, logistical barriers are addressed, and new payment models are introduced to make teleretinal screening more attractive to PCPs. Over 30 million U.S. adults live with diabetes; roughly one-third of them suffer from diabetic retinopathy (DR) [1]. Left untreated, DR progresses into partial vision loss or permanent blindness [2]. Although treatments for DR are readily available, rates of detection are low. Only 50 to 60% of patients with diabetes receive the annual retinopathy screening by ophthalmologists or optometrists recommended by guidelines. Logistics and cost are key barriers to DR screening [3]. Many patients, particularly those in rural communities and on Medicaid, have trouble even finding an eye specialist in their area who will see them.

Promise of Teleretinal Screening for Diabetic Retinopathy Teleretinal screening at primary care practices is a potential solution to low rates of screening. Primary care physicians (PCP) already provide chronic disease management for patients with diabetes. Teleretinal screening could make it possible for patients with diabetes to have eye screening at their regular visits to their PCP instead of having to visit an eye specialist.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_22

351

352

Several companies sell the non-mydriatic fundus cameras needed for DR screening. Since relatively little training is required to use the equipment, medical assistants and nurses could capture retinal images and electronically send them to ophthalmologists or optometrists for evaluation. Only patients with moderate or severe DR would need to visit an eye specialist for management. Teleretinal screening addresses the logistical barrier of finding a nearby eye specialist, and in an era of increasing deductibles, may also save patients’ money. Growing evidence shows that it improves DR screening rates. A study on its use by five North Carolina clinics found that DR screening rose from 26 to 40% [3]. Another study found that using teleretinal screening in primary care increased DR screening by 16% in a large safety-net system in Los Angeles [4]. Already roughly 90% of patients with diabetes in the Veterans Health Administration system receive teleretinal screening [5]. To facilitate the use of teleretinal screening in primary care clinics, new billing codes have been developed and Medicare and many private health plans reimburse PCPs for capturing retinal images.

Few Primary Care Practices are Using Teleretinal Screening Despite this promise, there is still little use of teleretinal screening by PCPs outside of these large systems and case examples. Publicly available records on providers who serve the Medicare fee-for-service population show that only 255 non-ophthalmologist/optometrists billed for more than 10 teleretinal screening sessions (HCPCS code 92250 or 92227 and 92228) through 2015. Fundus camera manufacturers also report little interest and few sales to PCPs. Why has uptake of teleretinal screening been so low among PCPs? Conversations with PCPs, including those who offer teleretinal screening, indicate that a key barrier is misalignment between practice finances and how the technology is sold. The upfront cost of a fundus camera is sizeable (>$15,000). Although profitable practices can invest in the

K. Yogesan et al.

equipment, many practices are priced out of the market. Clinics more likely to struggle with DR screening adherence (e.g. rural clinics, safety-net providers) are also more likely to have thinner profit margins and tighter budgets. There are also logistical barriers. Adding this new service requires training and takes up the valuable time of medical assistants or nurses. A room must be set aside to house the equipment, a major issue in small, busy practices. In addition, PCP practices need to identify and contract with ophthalmologists or optometrists who will read the images remotely and then manage the reports they receive from the eye specialist and contact patients with those results. Another barrier is motivation. Clinic leaders recognize the logistical barriers for retinopathy screening and the clinical benefits of increased screening. However, they also recognize there are many other pressing clinical issues in their patient population such as how to address barriers to mental health services. It is therefore not surprising that most primary care practices decide that the upfront costs and logistics of teleretinal screening are not worth it and focus their energies in other clinical areas. One hope was that providing reimbursement for teleretinal screening would create a financial incentive for primary care practices to implement teleretinal screening. However, given the current reimbursement amounts and structure, many practices simply do not have enough eligible patients with diabetes to justify the added upfront expense. Interestingly, instead of seeing this as a way to increase revenue, practices who have implemented teleretinal screening are often motivated by the need to increase their performance on DR screening quality measures. Such quality measures have increased importance given they are used as a basis for reimbursement under new payment models such as Accountable Care Organization contracts.

How to Move Forward Several changes will likely increase interest among PCPs in teleretinal screening. Fundus camera manufacturers could address the high

Retinal Screening of Patients with Diabetes in Primary Care Clinics …

cost of entry by developing products that are specifically designed to meet the needs of primary care practices. As is typical in many industries [6], suppliers focus on selling cameras to high-end customers, i.e. ophthalmologists and optometrists who use the equipment every day. PCPs need simpler, portable and much lowerpriced cameras that may not offer the highest possible resolution or all the features of top-ofthe-line products. Portable fundus cameras might even be shared by multiple clinics, each of which lacks enough patients to justify an investment in their own equipment. Primary care practices also need a complete teleretinal screening package that some private companies have begun to offer. Ideally, the package solution would include the camera, the telemedicine software, staff training how to take images, as well as facilitating the reading of those images by optometrists or ophthalmologists. The growing use of artificial intelligence may also help with several logistical barriers. New artificial intelligent systems can automatically read retinal images with high rates of sensitivity and specificity [7]. If such systems are part of the package then the patient and PCP receive a diagnosis within minutes of image capture, eliminating the need for patient followup and increasing satisfaction among both providers and patients. The final key change is a shift in the financing model. Currently, a primary care practice must buy or lease a camera and the assumption is that primary care practices will recoup the costs of that investment by billing insurers when they obtain a retinal image. For example, Medicare pays $36 for taking retinal images (HCPCS 92250). Other financing models may make teleretinal screening more attractive to PCPs. For example, insurers might fully subsidize the package solution under the assumption the plan’s quality scores would improve through higher screening rates. Another option is to allow private companies that provide the package solution to bill insurers. The private companies then would “kick back” some of this reimbursement to primary care practices who

353

reliably generate a monthly volume of diabetic eye exams. Under such financing options, there is little upfront risk for primary care practices and only financial benefit.

Summary Teleretinal imaging in primary care practices is a promising way to improve DR screening adherence rates and prevent blindness. Ideally, patients with diabetes in the US would be screened using this technology. However, few PCPs have implemented such screening into their practice. That number is unlikely to increase until camera manufacturers bring down the upfront costs of fundus cameras, logistical barriers are addressed, and new payment models are introduced to make teleretinal screening more attractive to PCPs.

References 1. Zhang X, Saaddine J, Chou C, et al. Prevalence of Diabetic Retinopathy in the United States, 2005–2008. JAMA. 2010;304(6):649–56. 2. Hartnett ME, Key IJ, Loyacano NM, Horswell RL, DeSalvo KB. Perceived barriers to diabetic eye care qualitative study of patients and physicians. Arch Ophthalmol. 2005;123(3):387–91. 3. Jani PD, Forbes L, Choudhury A, Preisser JS, Viera AJ, Garg S. Evaluation of diabetic retinal screening and factors for ophthalmology referral in a telemedicine network. JAMA Ophthalmol. 2017;135 (7):706–14. 4. Daskivich LP, Vasquez C, Martinez C, Tseng C, Mangione CM. Implementation and evaluation of a large-scale teleretinal diabetic retinopathy screening program in the Los Angeles county department of health services. JAMA Intern Med. 2017;177(5): 642–9. 5. Chasan JE, Delaune B, Maa AY, Lynch MG. Effect of a teleretinal screening program on eye care use and resources. JAMA Ophthalmol. 2014;132(9):1045–51. 6. Christensen MC. The innovator’s dilemma: when new technologies cause great firms to fail. Boston, MA: Harvard Business Review Press; 2016. 7. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–10.

Tele-Ophthalmology for Diabetic Retinopathy in the UK Peter H. Scanlon

Abstract

Keywords

Aim: of the English NHS Diabetic Eye Screening Programme (DESP) is to reduce the risk of sight loss among people with diabetes. Methodology: 2-field mydriatic digital photography. Benefits: Reduction in Vision Impairment and Blindness. Current developments: The English NHS DESP will be introducing extension of the screening interval in low-risk groups and digital surveillance clinics with OCT as a second line of screening for those with screen positive maculopathy at primary screening. Further Developments: The NHS DESP is currently evaluating the safe and cost-effective use of Artificial Intelligence as a primary grading tool and newer camera technologies.

Diabetic retinopathy Screening Telemedicine Blindness Imaging Sight-threatening diabetic retinopathy

This chapter includes details on progress in the English NHS Diabetic Eye Screening Programme with the use of OCT as a second line of screening, extension of the screening interval in low-risk groups, artificial intelligence (AI) and newer camera technologies. P. H. Scanlon (&) Gloucestershire Hospitals NHS Foundation Trust, Sandford Rd, Cheltenham, Gloucestershire, England e-mail: [email protected]
















Introduction The English NHS Diabetic Eye Screening Programme [1] commenced in 2003 and has been a major contributor to diabetic retinopathy (DR) no longer being the leading cause of working age blindness and further reductions from 5.0% of new blindness certifications in 2012/13 to 3.5% in 2018/19 Public Health Outcomes (PHO) data [2]. This was recognised by the World Health Organisation in their 2019 World Report on Vision [3] in which concluded that ‘this provides compelling evidence that systematic diabetic retinopathy screening, coupled with timely treatment of sight-threatening disease, can reduce vision impairment and blindness’.

Methodology in England The National Screening Programme in England recommends the measurement of visual acuity with distance spectacles and a pinhole followed by mydriatic two-field digital photography. The recommended fields are a macular-centred field and a disc-centred field (Fig. 1).

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 K. Yogesan et al. (eds.), Digital Eye Care and Teleophthalmology, https://doi.org/10.1007/978-3-031-24052-2_23

355

356

(a)

P. H. Scanlon

(b)

Fig. 1 (a) Right macular centred field (b) Right disc centred field

Methodological Differences in Scotland, Wales and Northern Ireland Screening Programmes Wales has a very similar programme to England; Northern Ireland attempts two-field non-mydriatic photography in those less than 50 years. The screening programme in Scotland takes one 45° field centred on the fovea using staged mydriasis on all of its population, attempting non-mydriatic photography first and only dilating those in whom there are poor-quality photographs.

treatment at the appropriate stage of the disease process and prevent sight loss. If one was to wait until symptoms developed the disease would be much more advanced and treatment would not be so effective. In order to achieve this, every eye that is photographed in the NHS Diabetic Eye Screening Programme is graded for a Retinopathy (R) level and a Maculopathy (M) level. Retinopathy progresses with increasing ischaemia: Grade

English screening programme

Equivalence in the early treatment diabetic retinopathy study (ETDRS)

Risk of developing proliferative DR

R1

Background DR

Mild nonproliferative DR (NPDR)

6.2% within 1 year

R2

Preproliferative DR

Moderate to severe NPDR

11.3% within 1 year

R3

Proliferative DR

Proliferative DR

M0

No maculopathy

M1

Maculopathy

The Aim of the Programme is To reduce the risk of sight loss among people with diabetes by the early detection (and treatment if needed) of diabetic retinopathy as part of a systematic programme that meets national standards. Since the NHS Diabetic Eye Screening Programme commenced in 2003, all people with diabetes in England have been offered annual digital photographic screening. The purpose of this is to detect diabetic eye disease to initiate

No equivalence

Tele-Ophthalmology for Diabetic Retinopathy in the UK

Referrals from the English NHS DESP In the year 2017–18 in the English Screening Programme, 2,237,932 people with diabetes were screened and there were 139, 250 referrals:

357

The two largest groups for referral were 66,107 referrals with ungradable images and 52,227 with background DR and screen positive maculopathy.

English Screening programme grading levels in the worst eye

Current outcome

Numbers in 2017–18 from 2,237,932 screened

R0M0

Annual rescreen

1,574,795

No DR

No maculopathy

R1M0

Background DR

No maculopathy

Annual rescreen

523,887

R1 M1

Background DR

Maculopathy

Referral routine

52,227

R2M0

Pre-proliferative DR

No maculopathy

Referral routine

6,129

R2M1

Pre-proliferative DR

Maculopathy

Referral routine

8,669

R3M0

Proliferative DR

No maculopathy

Referral urgent

2,201

R3M1

Proliferative DR

Maculopathy

Referral urgent

3,917

U

Ungradable

Ungradable

Referral routine

66,107

Management of Patients with Ungradable Images We have demonstrated previously [4] that with ungradable images, the commonest predictor was older age and the commonest cause was obvious central cataract (57%) with 21% having early cataract. Small pupil size and other pathology (e.g. corneal scar or asteroid hyalosis) were other causes. There are a significant number who have their cataracts removed and then return to the screening programme with clear images at their next screen. However, management of the group who cannot be returned to the screening programme does present quite a significant challenge.

Management of Patients with Screen Positive Maculopathy and Background Diabetic Retinopathy (R1M1) The definition of maculopathy using twodimensional markers in the English NHS Diabetic Eye Screening Programme: (DESP)

1. Exudate within 1 disc diameter (DD) of the centre of the fovea 2. Circinate or group of exudates within the macula 3. Any microaneurysm or haemorrhage within 1DD of the centre of the fovea only if associated with a best VA of  6/12 (if no stereo). In a recent study [5] that we undertook of patients with R1M1 in the Gloucestershire Screening Programme, of 659 patients, 652 (99%) were initially assessed at a digital surveillance clinic with SD-OCT after their screening episode and 7 (1%) were assessed at HES. 13 (2%) patients had missing or incomplete data (e.g. absence of an OCT result). Of the remaining 639 patients assessed at the SD-OCT clinic for suspected R1M1 and with SD-OCT results, 18 (3%) received treatment with laser or VEGF injection. Hence the specificity for detection of maculopathy requiring treatment within 12 months using two-dimensional screening markers is low.

358

Improving the Specificity of Detection of Diabetic Macular Oedema Needing Treatment in Screen Positive Maculopathy patients Digital surveillance clinics were introduced into the NHS DESP pathway in 2012 and an overview of these pathways was published in 2017 [6]. In the latest service specification document (2019−20) no.22 for the NHS Diabetic Eye Screening Programme [7] it states that ‘The provider shall refer people with diabetes to digital surveillance clinics that, in the opinion of the Clinical Lead, need more frequent review and do not require referral to the HES’. The reliance on local protocols with these more frequent reviews has led to problems with variation in management of these patients. Introducing Optical Coherence Tomography into the digital surveillance clinics in the pathway of the NHS DESP gives us the opportunity to standardise these reviews and increasing the specificity of detection of diabetic macular oedema and diabetic macular oedema requiring treatment without dropping the high sensitivity of detection. Introducing this at the digital surveillance clinic means that only those who are screen positive for diabetic maculopathy with two-dimensional markers will be offered the OCT (three-dimensional) scans. The introduction of OCT into the digital surveillance pathway of the NHS Diabetic Eye Screening Programme has been shown to be cost-effective [5]. Several English regions have used OCT for a number of years (e.g. Birmingham and Gloucestershire) but on a National level, Scotland introduced OCT into their screening pathway in 2019 and England is currently going through the process of introducing it across all regions.

Extension of the Screening Intervals in Low-Risk Groups In 2016, the UK National Screening Committee reviewed [8] the evidence available on extension of screening intervals in low-risk groups. They

P. H. Scanlon

recommended a change from one-year to twoyear screening intervals for people at low risk of sight loss who they defined by two successive diabetic eye screening appointments with photographic grading of no DR. They recommended that the current annual screening interval should remain for all those with mild retinopathy detected in either eye. There had been a number of publications to support this evidence base [9– 12]. There has been a delay in England in implementing this extension in low-risk groups mainly because of the effect of the Covid-19 epidemic which put on-hold some of the planned changes in the programme. There may be some advantages in the future in adding in some additional risk factor data [13, 14] but the extra risk factor data is not currently routinely available in the English NHS DESP. Deep learning has also been used [15] in the prediction of developing diabetic retinopathy that could be used in the future to optimise the screening interval but this has not yet been considered for use in the English NHS DESP as our main focus is on developing the possibility of the use of artificial intelligence algorithms for grading in the English NHS DESP.

The Use Artificial Intelligence for Grading in UK Diabetic Eye Screening Programmes Automated analysis of images using Artificial Intelligence (AI) has been a hot topic for years, and is heralded as the only way forward for DESP. Automated Number Plate Recognition, facial recognition, gait analysis and multi-target acquisition tools are commonplace in the security and defence portfolios and progress is being made with self-driving cars. Concerns over data protection and compliance with the data protection laws are commonly expressed. There are three current AI systems: 1. Sequential process algorithms—Featurebased detection 2. Neural networks—Trained by marking up lots of true positive cases

Tele-Ophthalmology for Diabetic Retinopathy in the UK

359

3. Convolutional neural networks—Deep learning—Derived from many ‘what constitutes a true case’

• The auto grader had a sensitivity of 97% and specificity of 38% in the 2015 EQA round and a false negative rate of 0 to 0.6% during IQA.

Algorithms can be set to preferentially detect either of these levels:

The Scottish auto grader is based on the detection of microaneurysms in one macular centred field. The UK screening programmes inform patients who attend for screening that other eye conditions will not necessarily be detected and so they will need to attend for regular eye examinations to detect other eye diseases. In 2015, a Health Technology report was produced that evaluated a number of algorithms in detection of any retinopathy and referable retinopathy. Unfortunately, the algorithm used in Scotland at that time iGradingM only graded a macular field and was unable to grade the disc centred field (classifying this as ‘ungradable’) although I understand that it is now capable of grading this second field that we use in England. At the time the two algorithms that were considered to have performed well in the HTA evaluation were EyeArt and Retmarker. EyeArt had a sensitivity of 94.7% (95% CI 94.2 to 95.2%) for any retinopathy. For manual grades R0 and no maculopathy (M0), specificity was 20% (95% CI 19 to 21%). Sensitivity was 93.8% (95% CI 92.9 to 94.6%) for referable retinopathy and 99.6% (95% CI 97.0 to 99.9%) for proliferative retinopathy; Retmarker had a sensitivity of 73.0% (95% CI 72.0 to 74.0%) for any retinopathy, For manual grades R0 and no maculopathy (M0), specificity was 53% (95% CI 52 to 54%) for Sensitivity was 85.0% (95% CI 83.6 to 86.2%) for referable retinopathy and 97.9% (95% CI 94.9 to 99.1%) for proliferative retinopathy. A threshold analysis testing the highest ARIAS cost per patient before which ARIASs became more expensive per appropriate outcome than human grading, when used to replace the first level human grader, was Retmarker £3.82 and EyeArt £2.71 per patient. A study by Heydon (20) was the only study to evaluate an algorithm in England in a prospective

1. AI to detect referable level of DR 2. AI to detect the DR/No DR level A study [16] from Holland used an algorithm (IDx-DR) to detect referable diabetic retinopathy, recommending its use in primary care to decrease the demand on ophthalmologists. Of the included 1415 persons, 898 (63.5%) had images of sufficient quality for use of the algorithm. Referable diabetic retinopathy (RDR) was diagnosed in 22 persons (2.4%) using EURODIAB with a sensitivity of 91% (95% CI: 0.69–0.98) and specificity of 84% (95% CI: 0.81–0.86), and 73 persons (8.1%) using International Classification of DR (ICDR) with a sensitivity of 68% (95% CI: 0.56– 0.79) and specificity of 86% (95% CI: 0.84– 0.88). In the UK, screening programmes undertake the primary care assessments for referral to an ophthalmology department and only very few patients are referred with diabetic retinopathy by their optometrists. Hence, the cost-effective use of an algorithm for detection of DR or referable DR needs to take into account how it could be used in an effective and cost-effective manor within screening programmes. Of the UK screening programmes, Scotland has led the way in the use of automated analysis. After a period of software development and validation [17, 18], Scotland introduced automated grading into the Scottish National Diabetic Retinopathy Screening Programme in 2011. The only publication since that time on the impact of introducing automated grading is a poster [19] at Diabetes UK in 2017. In the abstract, it states • In the six-month period in 2015, 60,465 (58.1%) of the episodes (average 392/day) were passed on to the auto grader. Of these 30,183 (49.9%) were final graded as having no retinopathy by the auto grader.

360

group of patients. The sensitivity (95% CIs) of EyeArt was 95.7% (94.8 to 96.5%) for referable retinopathy (This comprises sensitivities of 98.3% (97.3 to 98.9%) for mild-to-moderate nonproliferative retinopathy with referable maculopathy, 100% (98.7, 100%) for moderate-tosevere non-proliferative retinopathy and 100% (97.9, 100%) for proliferative disease. EyeArt agreed with the human grade of no retinopathy (specificity) in 68% (67 to 69%), with a specificity of 54.0% (53.4 to 54.5%) when combined with non-referable retinopathy. More recently there are papers published using deep learning algorithms that show high sensitivity and specificity for the detection of DR [21]. However, there are many challenges in the clinical validation and deployment of these models in clinical practice. Many of the training datasets do not necessarily reflect the populations in screening programmes that the algorithm is going to be used in. An additional challenge for the deep learning algorithms is that, unlike sequential processing algorithms, we do not fully understand how a neural network reaches a particular decision, or which exact features it identifies to make those decisions. As Caroline Styles wrote in her article from a Scottish perspective when considering how these types of algorithms might be used in the screening programme in Scotland in the future ‘It will be new to us that we don’t understand or control exactly what artificial intelligence does.’ My own view is that there is good evidence from Scotland that an algorithm can be used safely at the retinopathy/no retinopathy level to reduce the number of normal images that the human grader needs to grade. This might reduce the amount of grading that is required in the English NHS DESP to 50% of the current level. My understanding is that further research is going to be conducted in England in the clinical setting of English screening programmes, looking at the consequences of any false positives and negatives, the cost-effectiveness of the system and the experience of health professionals and patients.

P. H. Scanlon

I feel that a phased implementation in one or a small number of programmes should be conducted alongside this research at the retinopathy/no retinopathy level.

Newer Camera Technologies for Use in Screening—Hand Held, Small Devices and Scanning Confocal Ophthalmoscopes I need to state at the beginning of this chapter that I am not a supporter of hand-held cameras for screening because I have never seen a study report adequate sensitivities and specificities of s hand-held camera for detection of sightthreatening diabetic retinopathy against a reference standard of seven-field stereo-photography. That does not mean that I am not a supporter of small devices of phone type size. My problem is the inherent movement of the operator and patient when photographing a patient’s eye with a hand-held device. This can be counteracted by using a fixed chinrest and fixing the camera itself which can be undertaken without it needing to be expensive. A small digital device that performed well against seven-field stereo-photography was the ‘Fundus on Phone’ device that performed well against seven-field stereo-photography in a study [22] that reported in 2015. 301 patients (602 eyes) with type 2 diabetes underwent standard seven-field digital fundus photography with both Carl Zeiss fundus camera and the ‘Fundus on Phone’ device at a tertiary care diabetes centre in South India. The sensitivity and specificity for detecting any DR by FOP were 92.7% (95% CI 87.8–96.1) and 98.4%(95% CI 94.3–99.8), respectively, and the kappa (ĸ) agreement was 0.90 (95% CI–0.85–0.95 p