Sensing Technology: Proceedings of ICST 2022 (Lecture Notes in Electrical Engineering, 886) 9783030988852, 9783030988869, 3030988856

Citation preview

Lecture Notes in Electrical Engineering 886

Nagender Kumar Suryadevara · Boby George · Krishanthi P. Jayasundera · Joyanta Kumar Roy · Subhas Chandra Mukhopadhyay   Editors

Sensing Technology Proceedings of ICST 2022

Lecture Notes in Electrical Engineering Volume 886

Series Editors Leopoldo Angrisani, Department of Electrical and Information Technologies Engineering, University of Napoli Federico II, Naples, Italy Marco Arteaga, Departament de Control y Robótica, Universidad Nacional Autónoma de México, Coyoacán, Mexico Bijaya Ketan Panigrahi, Electrical Engineering, Indian Institute of Technology Delhi, New Delhi, Delhi, India Samarjit Chakraborty, Fakultät für Elektrotechnik und Informationstechnik, TU München, Munich, Germany Jiming Chen, Zhejiang University, Hangzhou, Zhejiang, China Shanben Chen, Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai, China Tan Kay Chen, Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore Rüdiger Dillmann, Humanoids and Intelligent Systems Laboratory, Karlsruhe Institute for Technology, Karlsruhe, Germany Haibin Duan, Beijing University of Aeronautics and Astronautics, Beijing, China Gianluigi Ferrari, Università di Parma, Parma, Italy Manuel Ferre, Centre for Automation and Robotics CAR (UPM-CSIC), Universidad Politécnica de Madrid, Madrid, Spain Sandra Hirche, Department of Electrical Engineering and Information Science, Technische Universität München, Munich, Germany Faryar Jabbari, Department of Mechanical and Aerospace Engineering, University of California, Irvine, CA, USA Limin Jia, State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing, China Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Alaa Khamis, German University in Egypt El Tagamoa El Khames, New Cairo City, Egypt Torsten Kroeger, Stanford University, Stanford, CA, USA Yong Li, Hunan University, Changsha, Hunan, China Qilian Liang, Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX, USA Ferran Martín, Departament d’Enginyeria Electrònica, Universitat Autònoma de Barcelona, Bellaterra, Barcelona, Spain Tan Cher Ming, College of Engineering, Nanyang Technological University, Singapore, Singapore Wolfgang Minker, Institute of Information Technology, University of Ulm, Ulm, Germany Pradeep Misra, Department of Electrical Engineering, Wright State University, Dayton, OH, USA Sebastian Möller, Quality and Usability Laboratory, TU Berlin, Berlin, Germany Subhas Mukhopadhyay, School of Engineering & Advanced Technology, Massey University, Palmerston North, Manawatu-Wanganui, New Zealand Cun-Zheng Ning, Electrical Engineering, Arizona State University, Tempe, AZ, USA Toyoaki Nishida, Graduate School of Informatics, Kyoto University, Kyoto, Japan Luca Oneto, Dept. of Informatics, Bioengg., Robotics, University of Genova, Genova, Genova, Italy Federica Pascucci, Dipartimento di Ingegneria, Università degli Studi “Roma Tre”, Rome, Italy Yong Qin, State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing, China Gan Woon Seng, School of Electrical & Electronic Engineering, Nanyang Technological University, Singapore, Singapore Joachim Speidel, Institute of Telecommunications, Universität Stuttgart, Stuttgart, Germany Germano Veiga, Campus da FEUP, INESC Porto, Porto, Portugal Haitao Wu, Academy of Opto-electronics, Chinese Academy of Sciences, Beijing, China Walter Zamboni, DIEM - Università degli studi di Salerno, Fisciano, Salerno, Italy Junjie James Zhang, Charlotte, NC, USA

The book series Lecture Notes in Electrical Engineering (LNEE) publishes the latest developments in Electrical Engineering - quickly, informally and in high quality. While original research reported in proceedings and monographs has traditionally formed the core of LNEE, we also encourage authors to submit books devoted to supporting student education and professional training in the various fields and applications areas of electrical engineering. The series cover classical and emerging topics concerning: • • • • • • • • • • • •

Communication Engineering, Information Theory and Networks Electronics Engineering and Microelectronics Signal, Image and Speech Processing Wireless and Mobile Communication Circuits and Systems Energy Systems, Power Electronics and Electrical Machines Electro-optical Engineering Instrumentation Engineering Avionics Engineering Control Systems Internet-of-Things and Cybersecurity Biomedical Devices, MEMS and NEMS

For general information about this book series, comments or suggestions, please contact [email protected]. To submit a proposal or request further information, please contact the Publishing Editor in your country: China Jasmine Dou, Editor ([email protected]) India, Japan, Rest of Asia Swati Meherishi, Editorial Director ([email protected]) Southeast Asia, Australia, New Zealand Ramesh Nath Premnath, Editor ([email protected]) USA, Canada: Michael Luby, Senior Editor ([email protected]) All other Countries: Leontina Di Cecco, Senior Editor ([email protected]) ** This series is indexed by EI Compendex and Scopus databases. **

More information about this series at https://link.springer.com/bookseries/7818

Nagender Kumar Suryadevara · Boby George · Krishanthi P. Jayasundera · Joyanta Kumar Roy · Subhas Chandra Mukhopadhyay Editors

Sensing Technology Proceedings of ICST 2022

Editors Nagender Kumar Suryadevara School of Computer and Information Sciences University of Hyderabad Hyderabad, Telangana, India Krishanthi P. Jayasundera University of Technology Sydney Sydney, NSW, Australia

Boby George Department of Electrical Engineering Indian Institute of Technology Madras Chennai, Tamil Nadu, India Joyanta Kumar Roy Eureka Scientech Research Foundation Kolkata, West Bengal, India

Subhas Chandra Mukhopadhyay School of Engineering Macquarie University Sydney, NSW, Australia

ISSN 1876-1100 ISSN 1876-1119 (electronic) Lecture Notes in Electrical Engineering ISBN 978-3-030-98885-2 ISBN 978-3-030-98886-9 (eBook) https://doi.org/10.1007/978-3-030-98886-9 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

ICST 2022, the 14th International Conference on Sensing Technology, was held at the Indian Institute of Technology Madras, India during January 16–18, 2022. The proceedings of ICST 2022 are published by Springer Nature in the book series LNEE (Scopus Indexed). The ICST 2022 received submissions focusing cutting-edge research on various aspects of sensors, sensing techniques, associated systems and new applications of sensors. It is with pleasure to note that like in the previous years of ICST, we have researchers from five different continents presented their research outcomes. We thank the authors for their great contributions. We thank Indian Institute of Technology (IIT) Madras, India, and Macquarie University, Sydney, Australia for the kind support provided to organize and execute the conference. Special thanks to the continuing education cell (CEC), IIT Madras for their extended help in managing registration and associated tasks. The technical programme committee and the reviewers did an excellent job through their quick and quality review. The complete process of paper submission, review process, and final submission were managed by the academic conference management system, Microsoft-CMT, sponsored by Microsoft Research. Springer book series Lecture Notes in Electrical Engineering (LNEE) which publishes the latest original research reported in proceedings, Scopus and Compendex indexing, has accepted to publish the proceedings in the LNEE. We thank Springer for the kind support. Hyderabad, India Chennai, India Sydney, Australia Kolkata, India Sydney, Australia

Nagender Kumar Suryadevara Boby George Krishanthi P. Jayasundera Joyanta Kumar Roy Subhas Chandra Mukhopadhyay

v

Contents

Graph Signal Processing Based Product Inspection Using Polarization Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrew Gigie, Saurabh Sahu, A. Anil Kumar, Kriti Kumar, M. Girish Chandra, and Tapas Chakravarty Real-Time Object Detection and Tracking from Videos . . . . . . . . . . . . . . . Madhurima Ghosh, Aousnik Gupta, Mehdi Hossain, Adrish Bose, Dipak Das, and Manvendra Singh Chauhan Deep Learning based Detection of Foot Lift Event Using a Single Accelerometer for Accurate Firing of FES . . . . . . . . . . . . . . . . . . . . . . . . . . . Bijit Basumatary, Rajat Suvra Halder, and Ashish Sahani A NIRS Based Device for Identification of Acute Ischemic Stroke by Using a Novel Organic Dye in the Human Blood Serum . . . . . . . . . . . . Raktim Bhattacharya, Dalchand Ahirwar, Bidisha Biswas, Gaurav Bhutani, and Shubhajit Roy Chowdhury

1

11

31

43

The Development of a Portable IoT-Enabled Aqueous Sulphur Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brady Shearan, Fowzia Akhter, and S. C. Mukhopadhyay

55

FEA Analysis on the Optimum Placement of Sensor for Early Detection of Damage in Concrete Pavements . . . . . . . . . . . . . . . . . . . . . . . . . Sakura Mukhopadhyay and Mohsen Asadnia

69

Fusion of Radar Data Domains for Human Activity Recognition in Assisted Living . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Julien Le Kernec, Francesco Fioranelli, Olivier Romain, and Alexandre Bordat

87

vii

viii

Contents

Statistical Performance Analysis of Radar-Based Vital-Sign Processing Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Gabriel Beltrão, Mohammad Alaee-Kerahroodi, Udo Schroeder, Dimitri Tatarinov, and M. R. Bhavani Shankar An Overview of Vital Signs Monitoring Based on RADAR Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Shahrokh Hamidi, Safieddin Safavi Naeini, and George Shaker A C-Band Intermodulation Radar System for Target Motion Discrimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Ashish Mishra and Changzhi Li Lossy Mode Resonances Supported by Nanoparticle-Based Thin-Films . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Ignacio Vitoria, Carlos Coronel, Aritz Ozcariz, Carlos Ruiz Zamarreño, and Ignacio R. Matias Deep Learning-based Out-of-Distribution Detection and Recognition of Human Activities with IMU Sensors . . . . . . . . . . . . . . . 149 Niall Lyons, Avik Santra, and Ashutosh Pandey A Simple, Linear Circuit for Measurement of Sub-pF Range of Capacitances Using a Double Differential Measurement Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 M. S. Mithun, Abin Dany Mathew, and P. Hemanth Sankar Vibration Measurement as Feedback from a Pneumatic Knife . . . . . . . . . 177 Dmytro Romanov, Olga Korostynska, Luis Eduardo Cordova-Lopez, and Alex Mason Carbon Nanotubes-Doped Tin Oxide-Based Thin-Film Sensors to Detect Methane Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Aniket Chakraborthy and Anindya Nag Development of an Acetone Sensor Using rGO-ZnO Composite . . . . . . . . 203 Fowzia Akhter, H. R. Siddiquei, and S. C. Mukhopadhyay Design and Development of an IoT-Enabled Sensor Node for Agricultural and Modelling Applications . . . . . . . . . . . . . . . . . . . . . . . . . 217 Brady Shearan, Fowzia Akhter, and S. C. Mukhopadhyay Planar Capacitive Touch Sensors—A Comparative Study . . . . . . . . . . . . . 231 Pamula Sreekeerthi, Nitheesh M. Nair, Garikapati Nagasarvari, and Parasuraman Swaminathan Application of Photoacoustic Sensing in Depicting Viscosity Information of Lubrication Oil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Abhijeet Gorey, Arijit Sinharay, Chirabrata Bhaumik, Tapas Chakravarty, and Arpan Pal

Contents

ix

IoT Enabled PoC Medical Diagnostic MEMS-Based Sensor Device for Kidney Healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Sumedha Nitin Prabhu and Subhas Chandra Mukhopadhyay Addressing Adversarial Machine Learning Attacks in Smart Healthcare Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Arawinkumaar Selvakkumar, Shantanu Pal, and Zahra Jadidi CdS-SnO2 Nanocomposite Sensor for Room Temperature Detection of NO2 Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Ajay Kumar Sao, Jatinder Pal Singh, Babita Sharma, Sandeep Munjal, Anjali Sharma, Monika Tomar, and Arijit Chowdhuri Epileptic Seizure Detection Using Continuous Wavelet Transform and Deep Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 Rahul Shukla, Balendra Kumar, G. Gaurav, Gagandeep Singh, and Ashish Kumar Sahani Filament Supply Sensing and Control for FFF/FDM 3D Printing Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Valentin Mateev, Martin Ralchev, and Iliana Marinova Electric Arc Discharge Power Estimation by CNN Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Valentin Mateev, Martin Ralchev, and Iliana Marinova UV Laser-Induced Graphene Electrode for Supercapacitor and Electrochemical Sensing Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Kalpana Settu, Jang-Zern Tsai, Yu-Chi Cheng, and Yu-Min Du Methods Tested to Optimize the Performance of WEBGL Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 Brahim Youssouf Goukouni, Muhammad Aamir, Waqar Ali, Zaheer Ahmed Dayo, Waheed Ahmed Abro, Muhammad Ishfaq, and Guan Yurong A Simple, Drift Compensated Method for Estimation of Isometric Force Using Sonomyography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 Anne Tryphosa Kamatham, Meena Alzamani, Allison Dockum, Siddhartha Sikdar, and Biswarup Mukherjee A Linear Process Analysis and Sensor Applications of a Pilot Water Treatment Plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Waqas Ahmed Khan Afridi and Subhas Chandra Mukhopadhyay An Accurate Model of Breather for Moisture Estimation for Transformer Health Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 Priti Mishra, Shailesh Kumar, and Tarikul Islam

x

Contents

A Lightweight Security Scheme for Failure Detection in Microservices IoT-Edge Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 Ali Hassan Sodhro, Abdullah Lakhan, Sandeep Pirbhulal, Tor Morten Groenli, and Habtamu Abie Comparison of the Routing Algorithms Based on Average Location Error and Accuracy in WSN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 P. Sakthi Shunmuga Sundaram and K. Vijayan Application of Variational Mode Decomposition to FMCW Radar Interference Mitigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Thilina Balasooriya, Prateek Nallabolu, and Changzhi Li Design of a Microwave Planar Device for Humidity Detection . . . . . . . . . . 433 Durga Nand Mahaseth, Tarikul Islam, and Upendra Mittal A Compact Wideband Planar Monopole Antenna with Defected Ground Structure for Modern Radar Sensing Systems . . . . . . . . . . . . . . . . 443 Zaheer Ahmed Dayo, Muhammad Aamir, Shoaib Ahmed Dayo, Permanand Soothar, Imran A. Khoso, Zhihua Hu, and Guan Yurong

About the Editors

Dr. Nagender Kumar Suryadevara received his Ph.D. degree from the School of Engineering and Advanced Technology, Massey University, New Zealand, in 2014. He is an Associate Professor at School of Computer and Information Sciences, University of Hyderabad, India. He has authored/co-authored two books, edited two books and published over 50 papers in various international journals, conferences, and book chapters. He has supervised over 100 graduate and post graduate students. He has examined over 30 postgraduate theses. He has delivered 36 presentations including keynote, tutorial, and special lectures. His research interests include wireless sensor networks, the Internet of things, and time-series data mining. He is a senior member of IEEE. Boby George received the M.Tech. and Ph.D. degrees in Electrical Engineering from the Indian Institute of Technology (IIT) Madras, Chennai, India, in 2003 and 2007, respectively. He was a Postdoctoral Fellow with the Institute of Electrical Measurement and Measurement Signal Processing, Technical University of Graz, Graz, Austria from 2007 to 2010. He joined the faculty of the Department of Electrical Engineering, IIT Madras in 2010. Currently, he is working as a Professor there. His areas of interest include magnetic and electric field-based sensing approaches, sensor interface circuits/signal conditioning circuits, sensors and instrumentation for automotive and industrial applications. He has co-authored more than 75 IEEE

xi

xii

About the Editors

transactions/journals, and more than 100 in tier-1 conference proceedings. He is an Associate Editor for IEEE Sensors Journal, IEEE Transactions on Industrial Electronics, and IEEE Transactions on Instrumentation and Measurement. Krishanthi P. Jayasundera graduated from University of Peradeniya, Sri Lanka with honours degree in Chemistry. She received both Masters and Ph.D. in Chemistry from Kanazawa University, Japan. She worked as postdoctoral researcher nearly 14 years at Massey University involving various research projects focused on the chemical synthesis and analysis of architecturally interesting molecules which have biological, environmental, and/or medicinal significance. At present she is working as research fellow at university of technology Sydney. She specialized in organic synthesis, spectroscopic analysis, biosensor development for small molecule detection. She has published over 40 research articles in different international journals and conference proceedings. She has also edited conference proceedings and books. Prof. Joyanta Kumar Roy has been working in electronics and automation engineering since 1984 as Company Director, Consulting Engineering, Developer, Researcher, and Educationist. He graduated from the Department of Physics from the University of Calcutta, India, and received Master of Science in Physics in 1977. He started his career as an entrepreneur in the year 1984 and founded a small manufacturing enterprise named System Advance Technologies Pvt. Ltd., dealing with turnkey execution of SCADA, automation, and industrial instrumentation system. In 2004, he obtained Ph.D. (Technology) in Applied Physics from University of Calcutta, India, and executed a number of projects related to control, automation, and instrumentation in several engineering sectors. After a long association with industry, he started his academic career from 2005. He worked with many educational institutes as principal, dean, and professor. His research group developed low-cost non-contact liquid-level transmitter, temperature transmitter, pressure transmitter, vortex flow transmitter, mass flow metre, etc. He has contributed more than 150 scientific and technical publications in the

About the Editors

xiii

form of books, book chapters, journal papers, conference papers, manuals, and engineering design of industrial xii About the Editors project. He is a technical speaker and has given invited talks in a number of international and national-level conferences. He organized many technical events, national and international events and worked as TPC member. He is the Founder Chairman of Eureka Scientech Research Foundation. He is a senior member IEEE and chapter chair IEEE Circuits & Systems, Region-10, India, Kolkata Section, Chairman and EC member IET (UK) Kolkata Network, Fellow of IWWWA, and Fellow of IETE. Presently, he is working as Editor of S2IS and a regular reviewer of research articles. His present research interest includes development of smart measurement and control system for water production and distribution, multifunction sensor, IoT-based m-health, technology-assisted living, smart home and city. Subhas Chandra Mukhopadhyay holds a B.E.E. (gold medallist), M.E.E., Ph.D. (India) and Doctor of Engineering (Japan). He has over 31 years of teaching, industrial and research experience. Currently he is working as a Professor of Mechanical/Electronics Engineering, Macquarie University, Australia and is the Discipline Leader of the Mechatronics Engineering Degree Programme. His fields of interest include Smart Sensors and sensing technology, instrumentation techniques, wireless sensors and network (WSN), Internet of Things (IoT), Mechatronics, etc. He has supervised over 45 postgraduate students and over 150 Honours students. He has examined over 75 postgraduate theses. He has published over 450 papers in different international journals and conference proceedings, written ten books and fifty-two book chapters and edited eighteen conference proceedings. He has also edited thirtyfive books with Springer-Verlag and thirty-two journal special issues. He has organized over 20 international conferences as either General Chairs/co-chairs or Technical Programme Chair. He has delivered 402 presentations including keynote, invited, tutorial, and special lectures. He is a Fellow of IEEE (USA), a Fellow of IET (UK), a Fellow of IETE (India). He is a Topical Editor of IEEE

xiv

About the Editors

Sensors journal. He is also an associate editor of IEEE Transactions on Instrumentation and Measurements and IEEE Reviews in Biomedical Engineering (RBME). He is a Distinguished Lecturer of the IEEE Sensors Council from 2017 to 2022. He chairs the IEEE Sensors Council NSW chapter.

Graph Signal Processing Based Product Inspection Using Polarization Imaging Andrew Gigie, Saurabh Sahu, A. Anil Kumar, Kriti Kumar, M. Girish Chandra, and Tapas Chakravarty

Abstract Polarization imaging has the ability to reveal certain material characteristics. With the availability of integrated polarization camera systems, they are increasingly being deployed for inspection tasks. In this paper, we process the data from polarization imaging using the emerging concepts of graph signal processing to effectively detect and localize these defects. Graph total variation is exploited to quantify the spread of minuscule defects such as fractures or cuts and graph filter is employed to localize these defects. Experimental results indicate that the proposed approach is capable of detecting and localizing defects as small as 1 mm with good accuracy compared to conventional approach. Keywords Polarization imaging · Product inspection · Graph signal processing

A. Gigie · S. Sahu (B) · A. Anil Kumar · K. Kumar · M. Girish Chandra Embedded Devices and Intelligent Systems, TCS Research, Bangalore, India e-mail: [email protected] A. Gigie e-mail: [email protected] A. Anil Kumar e-mail: [email protected] K. Kumar e-mail: [email protected] M. Girish Chandra e-mail: [email protected] T. Chakravarty Embedded Devices and Intelligent Systems, TCS Research, Kolkata, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_1

1

2

A. Gigie et al.

1 Introduction Industrial inspection has become an indispensable part in the manufacturing sector. The increasing need for good quality products and reduction of manufacturing costs has given a tremendous push for industries to invest in this field [3, 8, 10, 19]. In this paper, we mainly focus on the problem of product inspection in manufacturing process line that deals with the detection of surface level structural faults like cracks, dents etc [11]. To minimize the disruption to the manufacturing process line and also to ensure quality, error-free and quicker inspection are the two key factors to be considered for such inspection tasks. Hence, experienced and highly skilled labors are usually employed for this important task. However, this puts a severe requirement on the need for skilled man power and hence require an efficient monitoring system which is both efficient and computationally lightweight. In such inspection environments, it is not only preferable but also necessary to employ non-invasive techniques. More often than not, conventional RGB cameras are usually the choice for non-invasive inspection, as they are easily available for a wide range of specifications [13]. It is well known that these cameras are intensity based cameras and hence may fail in situations where the defects are minuscule and the defect color resembles that of the product. Also, traditionally these defects are identified using the background subtraction technique by suitably modeling the background. This technique, although computationally lightweight, has shown to be less sensitive to minuscule defects and gets severely affected by dynamic environment aberrations such as change in lighting scenario etc [6]. In order to avoid these limitations of the RGB camera and the traditional processing, in this paper, we propose to employ polarization imaging [18] and use Graph Signal Processing (GSP) [9] for robust defect identification. Polarization imaging has been an active area of research and is shown to provide some insights about the material properties [14, 17]. Classically, polarization imaging was applied by mounting a polarization filter in front of the standard camera and capturing the same scene at different polarization angles [1] and hence was more suitable for a static scenario. Of-late, integrated polarization cameras i.e., cameras with fixed polarization filter angles from different manufactures like Lucid Vision, Matrix Vision etc, are available [15, 16]. These recent cameras have the capability of simultaneously providing images at multiple polarization filter angles, thus making it suitable for dynamic environments. Due to these recent advancements, the usage of polarization imaging is becoming attractive and is increasingly being deployed in inspection scenarios [5]. Further, whenever a defect occurs, the defective region is often irregular and hence we employ GSP for efficient processing. In this paper, from the polarization imaging, we extract only the Degree Of Polarization (DOP) parameter and use this for defect detection and localization. It is well known that DOP depends upon the refractive index of the material and the zenith angle of the surface normal [2]. Thus, aberrations like dents or mild fractures results in changes to the DOP profile. Here, we propose to first build a template graph using a good reference product and then use this graph for inspection in a two step process.

Graph Signal Processing Based Product Inspection …

3

In the first step, which we refer as Defect detection step, we project the DOP of the new product onto this graph and determine whether it is smooth i.e., we verify whether it is in agreement with the topology of the template graph. This is done by computing a scalar value referred to as graph smoothness or Graph Total Variation (GTV) [9]. Suppose if the GTV is high then it indicates the product is defective and then in the next step referred as Defect localization, we use appropriate filters built in the GSP domain to localize these defects. This proposed system comprising of polarization imaging with the GSP based two step processing is efficient and computationally lightweight. The efficacy of this system is demonstrated through results which indicate that this approach is very sensitive to minuscule defects and at the same time it is robust against environmental factors like illumination changes etc.

2 Preliminaries 2.1 Polarization Imaging Polarization imaging is based on the principle that when an unpolarized light strikes a material surface, it is observed that the reflected light is partially polarized [4]. The intensity of such reflected polarization light varies sinusoidally as a function of the polarizing angle, say θ . Suppose if s = [s0 (x, y), s1 (x, y), s2 (x, y)]T denotes the well known stokes vector [12] at a pixel location (x, y), then the intensity observed at a particular polarizing angle θ , Iθ (x, y), can be expressed as [7]: Iθ (x, y) =

1 (s0 (x, y) + s1 (x, y) cos 2θ + s2 (x, y) sin 2θ ). 2

(1)

Now, if multiple intensities (atleast three) at different polarizing angles are measured then by using the above expression the stokes parameters s0 (x, y), s1 (x, y) and s2 (x, y) can be calculated. In our experiments, we use the polarization camera provided by Lucid Vision [15] which can instantaneously capture four images at θ = 0◦ , 45◦ , 90◦ and 135◦ . Using these four intensities i.e., I0 (x, y), I45 (x, y), I90 (x, y) and I135 (x, y), the stokes parameters are computed as [7]: s0 (x, y) = max(I0 (x, y)+ I90 (x, y), I45 (x, y)+ I135 (x, y)) s1 (x, y) = I0 (x, y) − I90 (x, y) s2 (x, y) = I45 (x, y) − I135 (x, y).

(2)

From the above computed stokes parameters we can compute the following important quantities:

4

A. Gigie et al.

 ρ(x, y) =

I (x, y) = s0 (x, y) s12 (x, y) + s22 (x, y) s0 (x, y)

(3) (4)

where I (x, y) and ρ(x, y) denotes the unpolarized intensity (i.e., image observed by a standard camera without polarization filter) and the DOP at a pixel location (x, y) respectively [7]. The DOP also indicates the amount of polarization in the reflected wave and resides in the range ρ(x, y) ∈ [0, 1]. It is also shown that DOP in general depends upon the refractive index of the material, say n, and the zenith angle, say φ, of the surface normal [2]. The exact expressions depend upon the type of reflection (refer to [2] for more details). For example for a polarized specular reflection, ρ(x, y) can be expressed as [2]:  2 sin φ tan φ n 2 − sin2 φ . ρ(x, y) = n 2 − 2 sin2 φ + tan2 φ

(5)

From the above equations one can observe that the polarization imaging not only provides the intensity but also the DOP which depends upon the material surface property like refractive index and shape. Suppose if any defect like cracks or dents appear on the surface, it results in variation of the material surface property which gets reflected in the DOP profile. Thus, in our proposed approach we use only the DOP for efficient defect inspection.

2.2 Graph Signal Processing Let G ∈ (V, E) denote a connected, undirected and weighted graph comprising of N nodes whose nodes and edges are indexed by the set V = {1, 2, ..., N } and E = { p, q, w pq }, p, q ∈ V, where w pq denotes the edge weight between nodes p and q with w pp = 0. Now, a graph signal, g, can be defined as signals whose samples are indexed by the the nodes of the graph G or in other words, the signal samples can be visualized as sitting on the nodes of the graph [9]. For an image, the graph signal can be the pixel values sitting on the nodes of the graph where each pixel represented by a node. Associated with this graph G, we can define an important matrix, adjacency matrix W as [W] p,q = w pq which is square symmetric matrix of order N . Another important matrix associated with the graphs, referred to as graph Laplacian L is th defined as L = D − W where  N D is a diagonal matrix with the p (1 ≤ p ≤ N ) diagonal entry d( p, p) = q=1 [W] p,q . Further, the normalized Laplacian can be expressed as L = D−1/2 (D − W)D−1/2 . Because the Laplacian matrix is a symmetric matrix, L admits the factorization L = UΛU H . The eigen vector matrix U and the corresponding diagonal eigenvalue matrix Λ provides a notion of frequency and hence in the graph framework, the diago-

Graph Signal Processing Based Product Inspection …

5

nal elements of Λ and U H is referred to as the graph frequencies and the Graph Fourier Transform (GFT) matrix [9]. For any graph signal g, the corresponding forward GFT and the inverse GFT can be computed as g˜ = U H g and g = U˜g respectively. Now, the frequency content of the above graph spectrum can be appropriately modified by using the following: y˜ = h(Λ)˜g (6) where h(Λ) is a diagonal matrix whose diagonal entries corresponds to filter response for different graph frequencies. Akin, to classical signal processing by appropriately designing h(Λ), one can have different filter configurations like low pass, high pass etc, in the GSP domain and hence h(Λ) is usually referred to as graph filter response matrix [9]. Now using the above described polarizing imaging and the GSP framework, in the following section we describe an inspection approach for quick and efficient defect detection and localization.

3 Proposed Inspection Approach Figure 1 depicts the proposed inspection approach. The system mainly comprises of the following blocks; Template generation, Defect detection and Defect localization. In the template generation, a good template graph is constructed. While in the defect detection block, the manufactured product is screened by checking whether it agrees with the topology of the template graph, if any disagreement is found then it goes to the defect localization where the defects are localized and examined. In the rest of this section, these three blocks are described in detail.

3.1 Template Generation In this block, a template graph of the DOP image profile from a good reference product is constructed. Let ρ (temp) denote the DOP image profile of a template product. Suppose, if g p = ρ (temp) (x p , y p ) and gq = ρ (temp) (xq , yq ) denote the DOP values at two different pixel locations i.e., corresponding to the nodes p and q respectively, then the graph edge weight w pq is computed as follows:  w pq =

f (g p , gq ), if f (g p , gq ) <  f and dist( p, q) < d 0, otherwise

(7)

where f (g p , gq ) = exp(−  g p − gq 22 /σ 2 ), σ denotes a scaling factor and dist( p, q) denotes the euclidean distance between the location of the pixels corresponding to the nodes p and q. Notice from the above expression, the graph is

6

A. Gigie et al.

constructed such that, the less correlated edges below a threshold  f are pruned and the graph is restricted to a neighborhood graph defined by the region threshold d . This is done to reduce the density of the graph to keep the further computations light. In practice, these thresholds can be designed by keeping in mind the requisite quality levels. Along with this graph construction, the associated template graph Laplacian matrix, Lt and the corresponding GFT, UtH is also computed. It is important to note that this template graph construction is a one time process and is not required to be done frequently.

3.2 Defect Detection The DOP image profile for a test product to be inspected, ρ (test) is obtained by using the polarization imaging as described in the previous section. Now considering ρ (test) as a the graph signal, we project this signal onto the template graph and compute the Graph Total Variation (GTV) [9] expressed as: Δ = (ρ (test) )T Lt ρ (test) .

(8)

The GTV Δ is a scalar quantity and from the expression one can notice that if ρ (test) is in agreement with the graph topology Lt then Δ will have a smaller value; on the other hand, it will have a large value when it deviates from Lt . Hence, by observing this quantity and comparing against an appropriate threshold, say de f , as depicted in the Fig. 1, we can make a decision of whether the product is defective or defect-free. As shall be demonstrated in the following section, this GSP based metric is more sensitive to even small defects compared to the standard mean squared error like metric. Now, if Δ > de f then the product is considered to be defective and we proceed to the next step of localizing these defects which is described in the following section.

3.3 Defect Localization When Δ > de f , then as mentioned earlier the graph signal deviates from the template graph topology or in other words we can say the graph signal is not smooth with the template graph Laplacian L t . In the graph Fourier domain, when the graph signal is smooth, the dominant energy will be concentrated in the low pass region. However, when the graph signal is not smooth, it will give rise to lot of high frequency components. Now, the DOP of a defective product can be modeled as ρ (de f ) = ρ (temp) + ρ (err or ) , where ρ (err or ) denotes the error in DOP due to the defect. By knowing ρ (err or ) , the defective region can be localized. Let h hp (Λ) denote the graph filter frequency response of a high pass filter with a suitable cut-off that can be

Graph Signal Processing Based Product Inspection …

7

Fig. 1 Flow diagram of the proposed approach for product inspection, here TH = de f

determined by observing the ρ˜ (temp) i.e., the graph Fourier spectrum of the template DOP signal. Now, with this high pass filter, the error in the DOP image, ρ (err or ) is easily estimated using the following expression: ρˆ (err or ) = Ut h hp (Λ)ρ˜ (de f )

(9)

where ρ˜ (de f ) denotes the graph Fourier spectrum of ρ (de f ) .

3.4 Computational Complexity This proposed approach of defect detection and localization is computationally lightweight. In this proposed approach, the graph construction using (7) is the only computationally intensive task, but it is required to be done only once. The remaining task of defect detection and localization using (8) and (9), requires only few N th order matrix multiplications, where N denotes the total number of nodes. In practice, instead of representing each pixel by a node, a group of pixels can be combined and represented by a node which further helps in reducing the computations.

8

A. Gigie et al.

Fig. 2 Metal rod having scratch as defect (a) RGB image of 1 mm scratch (b) DOP image of 1 mm scratch (c) DOP image of 10 mm scratch

4 Results To compare the efficacy of the proposed approach for defect detection and localization, we considered a smooth polished metal rod with no defect as a reference product. In order to produce defective products, we created 10 products of the same material with different scratch widths ranging from 1 mm to 10 mm. Figure 2(a) and (b) shows the intensity image obtained using a standard camera and DOP image profile with the polarization imaging of a 1 mm scratch defect respectively. Notice that in the intensity image the defect is not clearly visible. However, in the DOP image due to the abrupt change in the polarization value, the defective region is clearly visible. For sake of comparison, Fig. 2(c) shows the DOP image of the defective product with 10 mm scratch. Next, we provide plots of GTV computed using (8) to test the sensitivity of GTV for defects and compare its performance against the standard MSE metric on the DOP image. Figure 3(a) and (b) show the results for the GTV and MSE for different scratch widths. Notice from the figures, the slope of the GTV metric is higher than the slope of MSE metric, which clearly indicates that GTV is more sensitive to smaller defects compared to MSE. Further, to demonstrate the robustness of the proposed approach to environmental aberrations, like illumination changes etc, an additional plot is also provided by randomly changing the illumination. From the plots, one can notice that MSE is extremely sensitive to aberrations, but the proposed approach is robust and the effect of aberrations is minimal. Finally, we compare the efficiency of the proposed approach for defect localization against the standard well known background subtraction based method, where the background model corresponding to the good quality product is used as a reference. The ground truth corresponding to scratch of 1 mm and 10 mm is shown in Fig. 4(a). Environmental aberration such as illumination change is then applied for both approaches to see its impact on the performance. The results corresponding to the background subtraction based approach and the proposed approach are shown in Fig. 4(b) and (c) respectively. From the results, notice that the defect localization of the proposed approach is better compared to the background subtraction based

Graph Signal Processing Based Product Inspection …

9

Fig. 3 (a) Graph Total Variation (GTV) (b) Mean Square Error (MSE), for varying size of Cut width and Lighting conditions

Fig. 4 Defect Localization for 1 and 10 mm scratch (a) Ground truth (b) Using background subtraction (c) Using Graph Filters

approach. For different scratch width ranging from 1 mm to 10 mm, the average MSE with respect to the ground truth for the background subtraction based approach and the proposed approach is around 0.1464 and 0.0129 respectively. Thus, from the above results we can conclude that the proposed inspection approach of polarization imaging combined with GSP framework is both efficient in detection and localization even for minor defects and also robust against environmental aberrations.

5 Conclusion This paper provided the necessary details, mathematical formulations and supporting results for the proposed framework for inspection of end-products by putting together polarization-based sensing and graph-based processing. By using the concepts of GTV and graph filters, we have shown that our framework is effective in the detection

10

A. Gigie et al.

and localization of minute scale defects with better sensitivity and robustness. In summary, an effort has been made to systematically combine the concepts of two active and useful areas address challenging scenarios.

References 1. Forsyth, D.A., Ponce, J.: Computer Vision: A Modern Approach. Prentice Hall Professional Technical Reference (2002) 2. Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Polarized 3d: High-quality depth sensing with polarization cues. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3370–3378 (2015) 3. Kim, D., Liu, J.J., Han, C.: Determination of steel quality based on discriminating textural feature selection. Chem. Eng. Sci. 66(23), 6264–6271 (2011) 4. LeMaster, D.A., Eissmann, M.T.: Multi-dimensional Imaging. Chapter Passive Polarimetric Imaging. Wiley, New York (2014) 5. Lucidvision: Going polarized - polarization adds a new perspective to the imaging industry. https://www.photonics.com/White_Papers/Going_Polarized_-_Polarization_Adds_ A_New/wpp1698 6. Piccardi, M.: Background subtraction techniques: a review. In: 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), vol. 4, pp. 3099–3104. IEEE (2004) 7. Schott, J.R.: Fundamentals of Polarimetric Remote Sensing, vol. 81. SPIE Press (2009) 8. Shi, Q., Xi, N., Sheng, W., Chen, Y.: Development of dynamic inspection methods for dimensional measurement of automotive body parts. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006, pp. 315–320. IEEE (2006) 9. Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013) 10. Singh, B., Jha, A., Kumar, V.: A holistic approach to quality success in tata steel. J. Comput. Intell. Bioinforma. 3(1), 11–20 (2010) 11. Song, K., Petrou, M., Kittler, J.: Texture defect detection: a review. In: Applications of Artificial Intelligence X: Machine Vision and Robotics. vol. 1708, pp. 99–106. International Society for Optics and Photonics (1992) 12. Stokes, G.G.: On the composition and resolution of streams of polarized light from different sources. Trans. Camb. Philos. Soc. 9, 399 (1851) 13. Tantaswadi, P., Vilainatre, J., Tamaree, N., Viraivan, P.: Machine vision for automated visual inspection of cotton quality in textile industries using color isodiscrimination contour. Comput. Ind. Eng. 37(1–2), 347–350 (1999) 14. Tominaga, S., Kimachi, A.: Polarization imaging for material classification. Opt. Eng. 47(12), 123201 (2008) 15. Vision, L.: Lucid vision polarization camera. https://thinklucid.com/phoenix-machine-vision 16. Vision, M.: Matrix vision polarization camera. https://www.matrix-vision.com/USB3-visioncamera-mvbluefox3-2.html 17. Wolff, L.B.: Polarization-based material classification from specular reflection. IEEE Trans. Pattern Anal. Mach. Intell. 12(11), 1059–1071 (1990) 18. Wolff, L.B., Boult, T.E.: Constraining object features using a polarization reflectance model. IEEE Trans. Pattern Anal. Mach. Intell. 13(7), 635–657 (1991) 19. Xie, X.: A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electron. Lett. Comput. Vis. Image Anal. 7(3), 1–22 (2008)

Real-Time Object Detection and Tracking from Videos Madhurima Ghosh, Aousnik Gupta, Mehdi Hossain, Adrish Bose, Dipak Das, and Manvendra Singh Chauhan

Abstract This paper presents a novel approach to solve the problem of detection and continuous tracking of object(s) of interest in real-time from videos taken using stationary and moving cameras for various purposes like surveillance, vehicle navigation, biotechnology, human–computer interaction and sports analytics. The major challenges of this problem are low computational latency and high accuracy in spite of occlusion, non-uniform illumination, low quality videos, change of scale and orientation of object(s) in motion. The two-fold problem of object detection and tracking is solved using a transformer encoder-decoder architecture with Convolutional Neural Network (CNN) backbone called DEtection TRansformer or DETR and Point Tracking techniques respectively. We have implemented and tested the most popular Bayesian filters for point tracking like Kalman filter, Unscented Kalman filter and Particle filter independently. However, the tracking algorithm using Particle filter that also integrates an Auto-regressive model (ARM) for motion prediction, gives best result as compared to the other approaches. Our algorithm demonstrates accuracy and near real-time results which are on par with the well-established algorithms. The high overall efficiency is achieved by implementing high-performance techniques at both the stages of object detection and tracking. Keywords Object detection · Object tracking · Kalman filter · Particle filter · Unscented Kalman filter · Convolutional neural network · Auto-regressive motion model

M. Ghosh (B) · A. Gupta · M. Hossain · A. Bose Department of Computer Science and Engineering, St. Thomas’ College of Engineering and Technology, Kolkata, India D. Das · M. S. Chauhan Department of Optronics, Integrated Test Range, Defence Research and Development Organisation, Chandipur, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_2

11

12

M. Ghosh et al.

1 Introduction In recent years, object tracking has gained immense importance in the field of computer vision due to its varied applications in video surveillance [1, 2], biotechnology [3], vehicle tracking [4, 5] and many more. It aims to locate one or multiple objects of interest or targets in each frame of a video [6]. In order to track an object, we first need to provide an image of the target object to the tracking algorithm which can be done by a detection algorithm or manually. A naive way of tracking involves running the detection algorithm for each frame individually. However, integrating a tracking algorithm along with the detector is advantageous as detection algorithm is computationally expensive and is not a viable option if there is motion blur and change in illumination in the video, occlusion problem and, change of scale and orientation of the target [7]. It should also be kept in mind that low resolution noisy input video should not affect the accuracy of object tracking. We have taken a novel approach to build an algorithm combining a low latency detector, an Auto-regressive motion (ARM) model [8] and Point tracking techniques, which works with accuracy and in near real-time. Some of the popular detectors are You Only Look Once (YOLO) [9], Fast R-CNN [10], Faster R-CNN [11], Histogram of Oriented Gradients (HOG) [12], Regionbased Convolutional Neural Networks (R-CNN), Region-based Fully Convolutional Network (R-FCN) [13], Single Shot Detector (SSD) [14] and Spatial Pyramid Pooling (SPP-net) [15]. In our case study, we have used Detection Transformer or DETR that has encoder-decoder transformer architecture with CNN backbone. Carion et al. [16] presented that DETR is on par with the well-established and highly efficient Faster R-CNN algorithms in terms of precision and significantly outperforms most of its competitive baselines in terms of latency or computational time delay. On the other hand, there are broadly three types of tracking techniques [17], namely, Point trackers, Kernel-based trackers and Silhouette-based trackers. We have limited our scope to three major Point tracking algorithms i.e., Kalman filter [18], Unscented Kalman filter [19] and Particle filter [20]. We have also implemented an ARM model that takes the coordinates of the target object as its input in three successive frames of the video and predicts the motion of the object in the next frame. This output i.e., predicted motion of the object is then fed to the Particle filter for more efficient tracking.

2 Solution Architecture This problem is largely divided into two parts, namely Object Detection and Object Tracking. The tracking algorithms use output of the detector to estimate the position of the target object in each frame of the video. The ARM model acts as an intermediate state which also takes output of the detector as its input. The motion of the target,

Real-Time Object Detection and Tracking from Videos

13

Fig. 1 Architecture and data flow of the solution

estimated by the ARM model is then utilized by Particle filter for tracking purposes. The uniqueness in our approach lies in the selection of optimized algorithms in each stage of the solution, thus improving overall accuracy and latency. The detailed architecture and data flow pipeline of the proposed solution is depicted in Fig. 1. These different point tracking approaches are finally compared and analysis of the results are used to conclude the best tracking algorithm for our use-case. In our case, we are dealing with tracking of Unmanned Aerial Vehicles (UAVs) in relatively homogenous background.

3 Object Detection The major constraint in object detection is to complete the computation in real-time. Thus, the objective is to remove many hand-designed components of vision problems like non-max suppression and anchor generation, which not only encodes our prior knowledge of the task, but also increases the computational and time complexities of any vision algorithm. The outline behind fulfilling the objective is to build an object detection model which would approach the problem as a direct set prediction problem—one that would have a global understanding of the target object(s) and correlate them with their surroundings, all at once. An Object Detection model is expected to output a fixed number of bounding boxes for a given image, along with their corresponding class prediction confidence values i.e., probabilities. Traditionally, Convolutional Neural Network (CNN) based vision models, namely, YOLO, Single Shot Detectors and Region based CNNs are employed for this task. However, convolutions have a local understanding of a different parts of an image, i.e., it is able to recognize only the most complex objects in an image. It does so by understanding the simpler patterns in the initial layers of the network and

14

M. Ghosh et al.

Fig. 2 End-to-end DETR model overview

correlating them together in the subsequent ones. However, this correlation extends only to a part of the image, i.e., it has a local understanding and is unable to correlate those complex features of an image that are far away from each other. This leads to unstable confidence values of prediction, i.e., duplicate/multiple bounding boxes for the same object, all of which having nearly the same confidence values, thus, causing overlaps. Therefore, it can be inferred that the model itself has almost no knowledge about the presence of only a single object present at the corresponding area of the image, which calls for the need of Non-Max Suppression (NMS) [21]. This algorithm selects the best bounding box, i.e., the one with the highest probability, for a given object and rejects or “suppresses” all the other bounding boxes, which have a high overlap (Intersection over Union score) with the “best” bounding box. Generally, an overlap of 0.5 is considered, however, the selection of that hyperparameter certainly affects the final results. Moreover, since each bounding box is compared with every other, the time complexity of NMS becomes O(n2), thus causing a non-negligible delay in the detection of the target object. Therefore, in an attempt to approach this problem as a direct set prediction problem and remove NMS and the like, a need to stabilize the confidence of predictions was established. To correlate the local features of an image, a Transformer-based architecture was used, the attention layers of which could leverage the features of an image, returned by a backbone CNN and correlate them together to have a global understanding of where the target object(s) are and how they correlate with the features (may be other target objects) throughout the image. The output of the transformer consists of a fixed number of bounding boxes with each bounding box having a confidence of prediction. The overview of the set prediction is depicted in Fig. 2. Since the confidence values are highly stable, the need for hand-designed components is non-existent.

3.1 Architecture Carion et al. depicted the architecture of the DEtection TRansformer [16] that uses the feature matrix returned by the CNN backbone, to correlate the features, globally depicted in Fig. 3. The backbone used over here is a pre-trained model of ResNet50

Real-Time Object Detection and Tracking from Videos

15

Fig. 3 Architecture and data flow of DETR

[22], which accepts an input image x ∈ R3×H ×W , with 3 color channels and generates     H W and W = 32 . This feature a feature matrix f ∈ RC×H ×W , where C = 2048, H = 32 C×H  ∗W  , which serves as the input to the transformer. matrix is flattened to z ∈ R We may optionally choose to reduce the channel dimension of f from C to a lower dimension D, by passing it through a 1 × 1 convolution layer. Moreover, since the   encoder expects a sequence as input, f is collapsed into z ∈ R D×H ∗W . The output of the Transformer consists of a fixed number (N ) of bounding boxes along with their corresponding classes, i.e., two matrices—class predictions c ∈ [0, 1] N ×1 , and normalized bounding box coordinates b ∈ [0, 1] N ×4 .

3.2 Encoder Each encoder layer has the standard architecture consisting a multi-head selfattention module and a feed forward network (FFN). Since the transformer architecture is permutation-invariant, fixed positional encodings are supplemented to it at the input of each attention layer. The encoder self-attention layers correlate the features globally with every other feature using pair-wise relations, thus understanding the dynamics throughout the image.

16

M. Ghosh et al.

3.3 Decoder Each decoder layer has the standard architecture of a transformer, consisting of multi-head self and encoder-decoder attention units, that transform the N embeddings of size d along with the encoder’s correlation of features into the required output. The conceptual difference with the Transformer used for Natural Language Processing (NLP) and the one used here is that the model yields the probabilities and bounding boxes at once, unlike the traditional transformer which is an autoregressive model, which predicts the output sequence one value at a time. The decoder is also permutation-invariant; hence, N input embeddings needs to be different to produce different results. These input embeddings are positional encodings, referred to as object queries, which instead of being fixed, are learnt while training. Similar to the encoder, object queries are added to the input of each attention layer of the decoder. The N object queries are then transformed into an output embedding by the decoder, which are independently decoded into box coordinates and class labels by a feed forward network, resulting in N number of final predictions. Using self-attention and encoder-decoder attention over these embeddings, the model globally reasons about all objects together using pair-wise relations, correlate object queries to the bounding boxes and minimize the duplication of predictions, thus, being able to use the whole image as context.

3.4 Feed Forward Network (FNN) FNN consists of three separate hidden layers with ReLU (Rectified Linear Unit) activation functions, the first one (optional) having a dimension of d, the second, having a hidden dimension of 1 which produces the confidence values and the third with a hidden dimension 4, which produces the bounding boxes.

3.5 Set Precision The major highlight besides using a Transformer model is the use of the Hungarian Matching algorithm. Since the number of predictions is fixed and permutation invariant, there exists a difficulty to score the predictions with respect to the ground truth bounding boxes. The Hungarian Matching algorithm overcomes these difficulties by finding an optimal bipartite matching between the predictions and the ground truth. To find an optimal bipartite matching, we need to find the optimal permutation, σ between the two sets, the predictions, y and the ground truth, y such that Lmatch (yi , y σ (i˙) ) is minimum, i.e., 





Real-Time Object Detection and Tracking from Videos 

σ = argmin

N

17 

i=1

Lmatch (yi , y σ (i˙) )

(1)



where, Lmatch (yi , y σ (i˙) ) is the pair-wise matching cost between the ground truth and  a prediction at the index σ i˙ , and yi = (ci , bi ), where, ci is the class of the object, bi is the bounding box coordinates and p σ (i˙) (ci ) is the class confidence prediction value at the ith index. For out purposes, we have 2 classes of prediction namely, background, denoted by ∅ and UAVs. The matching cost takes into consideration the class confidence values as well as the bounding box coordinates as, 

    Lmatch yi , y σ (i˙) = −1{ci =∅} p σ (i˙) (ci ) + 1{ci =∅} Lbox bi , bσ (i˙) 





(2)

where, class is not considered during the calculations and,  the background  Lbox bi , bσ (i˙) scores the bounding boxes which is a combination of L 1 loss and Generalized Intersection over Union loss, and is given by, 

    Lbox bi , bσ (i˙) = λ L1 ||bi − bσ (i˙) ||1 + λgiou LG I oU bi , bσ (i˙) 





(3)

  where, λ L1 , λgiou ∈ R are hyperparameters and, LG I oU bi , bσ (i˙) can be calculated using the algorithm of Fig. 4. This process of finding a one–one matching for direct set prediction makes sure that there are no duplicates present in the predictions, which stabilizes the output probabilities with continued training. After the optimal permutation, σ is found, the Hungarian Loss is given by, 



 N  L H ungarian y, y = [−log p σ (i˙) (ci ) + 1{ci =∅} L (bi , bσ (i˙) )] i=1 box 









(4)

4 Auto-Regressive Motion (ARM) Model The predictions of the position of the target(s) are not accurate by the traditional Bayesian filters used for point tracking in case of random motion. This is because Kalman filter and its variants i.e., Unscented Kalman filter and Extended Kalman filter [23] assume that their state variables are normally distributed. However, the “prediction” step of Particle filter can be improvised using an Auto-Regressive Motion (ARM) model for accurate and precise results. ARM model uses conditional maximum likelihood estimate of the model parameters for predicting future positions and orientation of moving objects in time varying environments.

18

M. Ghosh et al.

Fig. 4 Algorithm for Calculation of IoU and GIoU given predicted and ground truth bounding boxes

The position of a translating object at nth step computed by the ARM model can be given by, x(n) =

p 

α p, j xi (n − j) + eix (n)

(5)

j=1

where, x(n) is the position of the object at nth step, α p, j , j = 1, 2, …, p are the autoregressive parameters, and eix (n) is the zero-mean white Gaussian noise. Considering the time steps are very small which is almost always the case in real world visual sensing, we can assume that acceleration of the object is constant or slowly changing. Hence, Elnagar et al. [8] proposed another ARM model for change in acceleration as, x(n) ¨ = β1,1 x¨i (n − 1) + eix¨ (n)

(6)

where, x(n) ¨ is the acceleration in nth step, β1,1 is the autoregressive parameter and eix¨ (n) is the zero-mean white Gaussian noise. Again, by Newton’s equation of motion, 1 xi (n) = xi (n − 1) + x˙i (n − 1)T + x¨i (n − 1)(T )2 2

(7)

Real-Time Object Detection and Tracking from Videos

19

Fig. 5 Work-flow of ARM model

where, xi (n), x˙i (n), x¨i (n) are the position, velocity and acceleration at nth step respectively. Assuming, T = 1, x˙i (n) = xi (n)−x i (n − 1) and x¨i (n) = xi (n)−2x i (n − 1) + xi (n − 2) and using (6), we can reduce (7) to,     xi (n) = 2 + β 1,1 xi (n − 1) + −1−2β 1,1 xi (n − 2) + β1,1 xi (n − 3) + eix¨ (n) (8) Comparing (5) and (8) we get, ⎤ ⎡ ⎤ 2 + β1,1 α3,1 ⎣ α3,2 ⎦ = ⎣ −1 − 2β1,1 ⎦ α3,3 β1,1 ⎡

(9)

Hence, we can predict the position of an object using a third order ARM model i.e., using last 3 positions. Figure 5 demonstrates how this model works. For estimating the coefficients, the maximum likelihood approach is selected. Logarithmic likelihood function for computing the coefficients:

N  2   1 − 2σ12 (x¨i (n)−x¨i P (n)) 2 lc β1,1 , σ ; n = 1 . . . N x¨ = log √ e 2π σ n=4

(10)

Re-writing (10) we get: N − 3  2 N −3 log(2π ) − log σ 2 2 N 2 1  − x¨i (n) − β1,1 x¨i (n − 1) 2 2σ n=4

lc (β1,1 , σ 2 ; n = 1 . . . N ) = −

Maximizing (11) with respect to β1,1 and σ 2 we get:

(11)

20

M. Ghosh et al.

N β1,1 =

n=4

N

x¨i (n)x¨i (n − 1)

n=4

x¨i2 (n − 1)

2 1  x¨i (n) − β1,1 x¨i (n − 1) N − 4 n=4

(12)

N

σ2 =

(13)

Hence, using (12), (13) we could estimate the parameters, β1,1 and σ 2 and by using these values in (8) we get our ARM model. Similarly, using (12) and (13), we can also predict y coordinate of the object of interest.

5 Object Tracking The main objective of object tracking is to discover the route of the target(s) by detecting its position i.e., coordinates in 2-D in every frame of video [18]. Object tracking can be broadly categorized into Point tracking, Kernel-based tracking which includes Simple Template Matching [24], Mean Shift Method [25], Support Vector Machine (SVM) [26] and Layering-based tracking [27], and Silhouette-based tracking. However, we have dealt only with three popular point tracking approaches, namely Kalman filter, Unscented Kalman filter and Particle filter. In point tracking techniques, moving object(s) are represented using points on the coordinate plane. It is a simple and efficient tracking method to track tiny objects but the algorithm becomes complex when the events of occlusion and false detection occurs [28]. Since, UAV is a small object with respect to the environment taken under consideration, it can be considered as a point object in the video frame. Figure 6 illustrates the pipeline for data transmission in the object tracking algorithms implemented by us.

Fig. 6 Pipeline of Data Transmission in Object Tracking Algorithms

Real-Time Object Detection and Tracking from Videos

21

Fig. 7 Algorithm for Kalman filter

5.1 Kalman Filter (KF) Kalman Filter is a simple and fast algorithm for single object tracking that gives optimum results [29]. It is discrete in nature i.e.; it relies on measurement samples taken between repeated but constant periods of time. This algorithm is also recursive in nature and takes feedback in terms of noisy measurement i.e., its prediction of the future relies on the state of the present position, velocity and acceleration. [30] However, it is not very efficient for complete as well as partial occlusion. Figure 7 shows the algorithm for Kalman filtering.

5.2 Unscented Kalman Filter (U-KF) Kalman filter gives poor performance is highly non-linear models. In such cases, Unscented Kalman filter (U-KF) [17] which is a modification of the classical KF is used. Although it has higher precision, the latency or computational time delay is comparatively higher. U-KF uses a deterministic sampling technique known as the unscented transformation (UT) to pick a minimal set of sample points (called sigma points) around the mean. The sigma points are then propagated through the non-linear functions i.e., predict and update functions, from which a new mean and covariance estimate are formed. The resulting filter depends on how the transformed statistics of the UT are calculated and which set of sigma points are used.

22

M. Ghosh et al.

5.3 Particle Filter (PF) Particle filter is efficient in tracking objects in highly non-linear and non-Gaussian environments. It gives optimal results for multiple object tracking with occlusion problem [31]. Figure 8 depicts the algorithm for Particle filtering.

Fig. 8 Algorithm for particle filter

Fig. 9 Comparison of Predictions between DETR (blue bounding box) and EfficientDet-D1 (red bounding box)

Real-Time Object Detection and Tracking from Videos

23

However, for our use-case, the “particles” are getting scattered for some instances. Thus, we have considered the arithmetic mean of the particles as the estimated position of the point target(s), which has given very accurate results.

6 Implementation These algorithms of object detection and tracking are implemented in Python using TensorFlow and PyTorch frameworks on Intel i5 processor. The implementation code for this model can be found in [32].

7 Result and Discussion The performance of our object detection model is analyzed by the metrices of Frames per second (FPS), latency i.e., time taken to process each frame of the video and mean average precision (mAP). The value of mAP lies between 0 to 1, where 1 signifies that the detection model can recall all the objects correctly in the visual frame. Furthermore, the object tracking techniques are compared on the basis of FPS and mean squared error (MSE) i.e., mean of the squares of the deviation of the obtained output coordinates from the actual coordinates of the target(s). DETR model is compared to the previous most recent Object Detection Architecture by Google Research’s state of the art, EfficientDet-D1 [33] and Faster RCNN, and it can be inferred that after the removal of hand-designed components, and stabilization of the confidence of prediction, the latency has decreased by a maximum of 68.75% causing up to a 220% increase in observable speed. Hence, we discarded the other detection algorithms and used DETR for our use case. The values of comparison are shown in Table 1. On the other hand, the tracking algorithms were tested on Real World Object Detection Dataset for Quadcopter Unmanned Aerial Vehicle Detection [34] individually. Figures 10, 11 and 12 show tracking results of Kalman filter, Unscented Kalman filter and Particle filter respectively on randomly chosen frames of the input video. We have used synthetic videos for performance analysis and comparison of the filters in different kinds of simulations including variable speed, change of scale of the target, occlusion problem and non-linear movement. Figure 13 shows the detection cum tracking on synthetic videos for KF, U-KF and PF. Table 1 Comparative study of object detection models Model

Parameters

FPS

Latency (ms)

mAP

Detection Transformer

41 M

16

62.5

0.87

Faster RCNN-FPN

42 M

9

111.1

0.81

EfficientDet-D1

6.6 M

5

200.0

0.80

24

M. Ghosh et al.

Table 2 Comparative of filters with different simulations Algorithms FPS

Mean squared error Standard Variable Diagonal Diagonal simulation, variable simulation speed simulation, speed simulation constant speed, occlusion

KF

11–12 54.91

20.46

9205.99

8487.20

U-KF

3–4

59.31

25,502.55

333.05

3027.28

PF

6–7

0.15

981.79

15.41

210.25

Fig. 10 Output of Kalman filtering

Fig. 11 Output of unscented Kalman filtering

Real-Time Object Detection and Tracking from Videos

25

Fig. 12 Output of particle filtering

Fig. 13 Comparison of filtering approaches—Detector, Kalman filter, particle filter and Unscented Kalman filter respectively

26

M. Ghosh et al.

Figures 14, 15, 16, 17 along with Table II summarize the predictions of the three different tracking filters, and compare them altogether using Mean Squared Error of the predictions in 4 different simulated scenarios including, horizontal “8” movement

Fig. 14 Filter predictions alongside original coordinates for a horizontal “8” pattern of an UAV with constant speed

Fig. 15 Filter predictions alongside original coordinates for a horizontal “8” pattern of an UAV with variable speed

Real-Time Object Detection and Tracking from Videos

27

Fig. 16 Filter predictions alongside original coordinates for a diagonal “8” pattern of an UAV with constant speed

Fig. 17 Filter predictions alongside original coordinates for a diagonal “8” pattern of an UAV with constant speed and occlusion

28

M. Ghosh et al.

of drone with constant speed, horizontal “8” movement of drone with variable speed, diagonal “8” movement of drone with constant speed, and diagonal “8” movement of drone with constant speed and occlusion respectively. Considering the outputs on the real videos as well as the simulated videos, it can be clearly observed that the latency of Kalman filter is the least, but it has least accuracy as compared to the other filters. The accuracy and precision are increased by modifying it to Unscented Kalman filter, however, the computational time delay increases drastically. Overall, Particle filter, after integrating the efficient ARM model for prediction, gave optimum results considering both the factors of accuracy and latency for our use-case. The experimental results show that our approach using Particle filter is robust to track object(s) in different situations and adapt to change in speed and scale of the target, and also tackle occlusion problem efficiently.

8 Conclusion We proposed a novel approach for object detection and tracking that outperforms the well-established algorithms both in terms of accuracy and latency. Better performance is obtained by using “greedy” strategy where optimized algorithms are used for the individual tasks of object detection and tracking. In our study, we trained the model to track UAVs from standard RGB images only, however, the model can be trained with images from infrared low-light cameras and other object(s) of interest for multipurpose utilizations. Moreover, if the processor is upgraded and these algorithms are implemented using parallel programming languages such as CUDA on GPU, the latency of the algorithms will significantly decrease enabling the object tracker to work in real time i.e., speed higher than 25 FPS. The computational time delay of the tracking algorithm can be further decreased by developing a mechanism which will automatically enable the algorithm to switch between Kalman filter and Particle filter [35] considering the factors like occlusion and motion of the target(s) by utilizing the feedback from the ARM model i.e., when the motion is linear and there is no occlusion problem, the algorithm will use Kalman filter and otherwise, it will switch to Particle filter until the target becomes “stable”, which is a topic of our further research.

References 1. Tian, Y.-L., Lu, M., Hampapur, A.: In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) (2005) 2. Zhu, J., Lao, Y., Zheng, Y.F.: IEEE Trans. Circuits Syst. Video Technol. 20(2), 223–235 (2009) 3. Maggio, E., Smerladi, F., Cavallaro, A.: IEEE Trans. Circuits Syst. Video Technol. 17(10), 1348–1359 (2007) 4. Jazayeri, A., et al.: IEEE Trans. Intell. Transp. Syst. 12(2), 583–595 (2011)

Real-Time Object Detection and Tracking from Videos

29

5. Xiong, T., Debrunner, C.: IEEE Trans. Intell. Transp. Syst. 5(4), 324–328 (2004) 6. Nimade, B., Bharadi, V.: Adaptive automatic tracking, learning and detection of any real time object in the video stream. In: 5th IEEE International Conference-Confluence. The Next Generation Information Technology Summit (Confluence) 2014 7. Porikli, F., Yilmaz, A.: Object Detection and Tracking, Studies in Computational Intelligence (2012). 978-3-642-28597-4. https://doi.org/10.1007/978-3-642-28598-1_1 8. Elnagar, A., Gupta,K.: Motion prediction of moving objects based on autoregressive model. IEEE Trans. Syst., Man, Cybernet. - Part A: Syst. Humans 28(6), 803–810 (1998). https://doi. org/10.1109/3468.725351 9. Handalage, U., Kuganandamurthy, L.: Real-time object detection using YOLO: a review (2021). https://doi.org/10.13140/RG.2.2.24367.66723 10. Girshick, R.: Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2380–7504. https://doi.org/10.1109/ICCV.2015.169 11. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39(6) (2015). https://doi. org/10.1109/TPAMI.2016.2577031 12. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego (2005) 13. Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. arXiv:1605.06409 14. Liu, W. et al.: SSD: single shot MultiBox detector. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol. 9905. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2 15. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8691. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_23 16. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. arXiv:2005.12872 17. Hatwar, R.B., Kamble, S.D., Thakur, N.V., Kakde, S.: A review on moving object detection and tracking methods in video. Int. J. Pure Appl. Math. (Special Issue) 118(16), 511–526, 1314–3395 (2018) 18. Saho, K.: Kalman filter for moving object tracking: performance analysis and filter design (2017). https://doi.org/10.5772/intechopen.71731 19. Chen, X., Wang, X., Xuan, J.: Tracking multiple moving objects using unscented Kalman filtering techniques. In: 2012 International Conference on Engineering and Applied Science (ICEAS 2012). arXiv:1802.01235 20. Wang, F.: Particle filters for visual tracking. In: Shen, G., Huang, X., (eds.), Advanced Research on Computer Science and Information Engineering. CSIE 2011. Communications in Computer and Information Science, vol 152. Springer, Berlin (2011). https://doi.org/10.1007/978-3-64221402-8_17 21. Rothe, R., Guillaumin, M., Van Gool, L.: Non-maximum suppression for object detection by passing messages between windows. In: Cremers, D., Reid, I., Saito, H., Yang, M.H., (eds.), Computer Vision – ACCV 2014. ACCV 2014. Lecture Notes in Computer Science, vol. 9003. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16865-4_19 22. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90 23. Yang, S., Baum, M.: Extended Kalman filter for extended object tracking. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 4386–4390. https://doi.org/10.1109/ICASSP.2017.7952985 24. Jurie, F., Dhome, M.: A simple and efficient template matching algorithm. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 544–549. https://doi.org/10.1109/ICCV.2001.937673

30

M. Ghosh et al.

25. Wen, Z., Cai, Z.: Mean shift algorithm and its application in tracking of objects. In: 2006 International Conference on Machine Learning and Cybernetics, 2006, pp. 4024–4028. https:// doi.org/10.1109/ICMLC.2006.258803 26. Muralidharan, R., Chandrasekar.: Object recognition using support vector machine augmented by RST invariants. Int. J. Comput. Sci. 8 (2011) 27. Zhou, Y., Tao, H.: A background layer model for object tracking through occlusion, 0-76951950-4 (2003). https://doi.org/10.1109/ICCV.2003.1238469 28. Sebastian, P., VooiVoon, Y.: Comley colour space effect on tracking in video surveillance. Int. J. Electr. Eng. Inf. 2(4) (2010) 29. Balaji, S.R., Karthikeyan, S.: A survey on moving object tracking using image processing. In: 2017 11th International Conference on Intelligent Systems and Control (ISCO), pp. 469–474. IEEE (2017) 30. Robin, A., Moisan, L., Le Hegarat-Mascle, S.: An a-contrario approach for subpixel change detection in satellite imagery. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1977–1993 (2010) 31. Dung, L.R., Huang, Y.C., Huang, R.Y., Wu, Y.Y.: An adaptive particle filtering for solving occlusion problems of video tracking. In: Stephanidis, C., (eds.), HCI International 2015 - Posters’ Extended Abstracts. HCI 2015. Communications in Computer and Information Science, vol. 528. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21380-4_114 32. https://github.com/gittygupta/uav-detection-and-tracking 33. Tan, M., Pang, R., Le, Q.V.: EfficientDet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020) 34. Pawełczyk, M.Ł, Wojtyra, M.: Real world object detection dataset for quadcopter unmanned aerial vehicle detection. IEEE Access 8, 174394–174409 (2020). https://doi.org/10.1109/ACC ESS.2020.3026192 35. Lan, J.H., et al.: Combining the Kalman filter and particle filter in object tracking to avoid occlusion problems. In: Parinov, I., Chang, S.H., Long, B., (eds.), Advanced Materials. Springer Proceedings in Materials, vol. 6. Springer, Cham (2020). https://doi.org/10.1007/978-3-03045120-2_47

Deep Learning based Detection of Foot Lift Event Using a Single Accelerometer for Accurate Firing of FES Bijit Basumatary , Rajat Suvra Halder , and Ashish Sahani

Abstract Foot drop (FD) is defined as the inability to lift the front part of the foot from the ground. It can be corrected by functional electrical stimulation (FES) of the peroneal nerve by applying pulses of a given duration, amplitude, and frequency. For correcting the FD through FES, it requires accurate detection of the foot lift event (gait pattern) of the patient. The stimulation pulse of FES device has to be triggered only when the subject is trying to lift the foot. In a traditional FES system, foot lift is detected by sensor attached below the heel. However, such type of sensor requires cables which create discomfort to the patient while wearing the device. To avoid this cable complexity, different types of Inertial Measurement Unit (IMU) based sensors are used. However, those types of IMU based sensors have some disadvantages, such as false triggering, and still, there is a lack of a proper algorithm to detect proper foot lift event by the sensor. In this paper, we have proposed a Deep learning based foot lift detection algorithm using a single accelerometer. Our proposed algorithm has been implemented in our developed FES device, and it was successful in predicting the foot lift event. Keywords IMU · FES · Foot drop · Accelerometer · Deep learning · Foot lift · Gait detection

1 Introduction Sensors are the basic requirements of the FES devices to detect the foot lift event. Based on the sensor input, FES device decides when the pulse would be applied. All FES devices use different type of sensors to detect the foot lift event. Some B. Basumatary (B) · R. S. Halder · A. Sahani Department of Biomedical Engineering, Indian Institute of Technology Ropar, Rupnagar, India e-mail: [email protected] R. S. Halder e-mail: [email protected] A. Sahani e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_3

31

32

B. Basumatary et al.

commonly used sensors are—Pushbutton switch [1, 2], Gyroscopes [3], Goniometers [4, 5], Force Sensing Resistor (FSR) [5], Tilt sensors [6], Accelerometers [7], Electromyography sensor [8]. Usually, the foot sensors are placed below the heel of the shoes. However, sensor such as FSR, Pushbutton sensor requires cables that create discomfort to the patient while wearing the device. The methods of gait detection or foot lift detection can be divided into two, and these are Deep learning method and the threshold method. In Deep learning method, mainly LSTM model is used for time series data. In the threshold method, the gait phase is divided manually by threshold value based on the observation of data provided by the sensor [9]. Different methods and algorithm have been used and validated in various studies to detect a foot lift event and to study gait pattern. Some authors have divided the mechanisms into three major categories for studies of gait cycle such as floor sensor, vision, and wearable sensor [10]. FES device for foot drop requires sensor to trigger the pulses because the pulse has to be applied only if it is required. The sensor is usually incorporated with the electronic circuit [13]. In our proposed model, we have used Long Short-Term Memory (LSTM) recurrent neural network. Memory cell present in the LSTM architecture can remember the present and previous information for prediction [9]. Hence, it is suitable for the detection of time series data. In this paper, the proposed foot lift detection event aims at simplifying the FES system for foot drop. It also enables accurate detection of foot lift event with a single accelerometer sensor. The accelerometer placed below the knee measure x, y, z values which are typical time series data. Therefore, we have used the LSTM model to detect the foot lift event based on the accelerometer data. In our experiment, we have additionally used FSR (Force sensing resistor) sensor and Flex sensor to identify the label of the foot during lifting from the ground and ankle dorsiflexion, respectively. We have collected both the FSR and Flex sensor data simultaneously along with accelerometer data. Firstly, we tried to validate the correlation between the accelerometer data and the FSR data. Secondly, we want to explore whether our proposed model using LSTM is able to predict foot lift event as per the reference data provided by Flex sensor and FSR sensor.

2 Data Collection 2.1 Experimental Setup MMA7361 accelerometer sensor was interfaced with an ESP32 Microcontroller. The pin x, y, z of MMA7361 accelerometer were connected to the analog pins of ESP32 microcontroller. FSR (Force sensing resistor) sensor and Flex sensor were used to identify the label of foot lifting from the ground and ankle dorsiflexion, respectively. The FSR sensor was placed below the heel, and the Flex sensor was placed between tibia and ankle joint, as shown in Fig. 1. The accelerometer sensor was placed below the knee using breadboard and wearable strap. Figure 1 shows

Deep Learning based Detection …

33

Fig. 1 Experimental setup for data generation

a subject using three sensors (Accelerometer, Flex, FSR) to record the gait cycle (Walking) data. The accelerometer gives three values X, Y, Z. The directions of the X, Y, Z of the accelerometer have been shown in the Fig. 1. The Flex sensor gives the data of ankle dorsiflexion. More the ankle dorsiflexion more is the flex value. The FSR sensor provides information about whether the heel is touching the ground or not.

2.2 Data Acquisition The data was recorded using real-time PLX-DAQ software, and data were exported in Microsoft Excel as per the format. The experimental setup was interfaced with PLX-DAQ for data acquisition. Then the subjects were asked to walk by wearing the setup. The data were recorded from 5 healthy subjects. However, subjects were allowed to walk for 5 min to collect a large amount of data. The data were recorded at 100 ms interval. The data has a total of 3038 rows. Table1 shows only a portion of the final data.

3 Data Analysis The graph of the generated data was plotted in Fig. 2. Flex value indicates the ankle dorsiflexion. More is the flex value; more is the ankle dorsiflexion and vice versa. FSR value indicates whether the heel is touching the ground. When the heel touches the ground, it indicates zero, and when the heel is in the swing phase, it indicates some value greater than 0. The data was normalized by transforming the data based on the mean and standard deviation.

34

B. Basumatary et al.

Table 1 Data acquisition table X

Y

Z

Flex

FSR

1.91

10.25

0.42

88

0

2.44

10.61

−0.19

88

0

1.83

10.5

−0.28

92

0

2.36

10.26

0.04

93

0

2.22

9.95

−0.2

96

0

1.54

9.53

−1.72

99

0

1.72

9.71

−1

103

0

2.01

10.25

−1.17

103

0

0.67

11.88

−1.34

101

222

0.37

11.71

−0.42

88

423

4.52

10.33

−4.29

47

635

0.62

5.2

−8.74

39

858

0.3

3.1

−7.21

56

465

0.33

3.63

−7.34

66

336

2.48

8.6

−1.97

61

508

−4.13

5.18

−0.53

60

455

3.31

−1.14

60

0

12.25

−1.48

75

0

−2.28 2

Fig. 2 Graph of the collected data. In the first graph, blue, orange and green represent X, Y and Z values of the accelerometer respectively. In the second graph, blue and green represents Flex and FSR values respectively

Deep Learning based Detection …

35

4 Proposed Deep Learning Model Long Short-Term Memory (LSTM) was used for the detection of foot lift event based on the data collected from an accelerometer. LSTM is a type of Recurrent Neural Network (RNN) that can predict the future value based on the passed sequence of observations [11]. The Fig. 3 shows the architecture of the proposed LSTM model for foot lift event detection. Firstly, data were normalized and then was splitted for testing and validation. All the data of 5 subjects were combined. The data has a total of 3038 rows and 5 columns. We slice data by a moving window over the rows. Each slice W s has 15 rows. This slice is then divided into input and output sub-slices, namely W i and W o . W i has 10 rows and W o has 5 rows. All window slices still have 5 columns. Normalization is performed using Eq. (1). Wi N = (Wi − Wim )/Wis

(1a)

WoN = (Wo − Wim )/Wis

(1b)

where W iN and W oN are normalized input and output window sub-slices. W im is the mean of W i and W is is the standard deviation of W i .

Fig. 3 Architecture of the proposed LSTM model

36

B. Basumatary et al.

Fig. 4 Implemented LSTM model

While cleating windows, it was taken care that the windows which were overlapping at the transition between data from one to the next subject were not considered. The total number of slices (W s ) is 2010. These 2010 datasets were divided into training and testing set to train and evaluate the proposed model. 95% of the data was used for training and 5% for testing the model. The training data was used to train the proposed LSTM and Recurrent Neural Network (RNN) model that would predict Flex and FSR data from x, y, z values of the accelerometer.

Deep Learning based Detection … Table 2 Summary of the proposed LSTM model

Table 3 Comparison of RNN and LSTM model

37

Layers (types)

Output shape

Parameters

lstm (LSTM)

(None, 10, 64)

17,408

lstm_1 (LSTM)

(None, 64)

33,024

dense (Dense)

(None, 64)

4160

dense_1 (Dense)

(None, 64)

4160

dense_2 (Dense)

(None, 10)

650

dense_3 (Dense)

(None, 10)

110

reshape (Reshape)

(None, 5, 2)

0

Model

Minimum training loss

Minimum validation loss

RNN

0.0123

0.1233

LSTM

0.0066

0.0948

The LSTM model was built using Keras. Mean Squared Error (MSE) was used for calculating the loss. In MSE, the loss is calculated by taking the mean of squared differences between a predicted value and the actual value. The model has two LSTM layers and four dense layers along with input and output layers. The input layer has input of size (10, 3) and output of size (10, 3). The final output layer has the input of size (10, 3) and output of size (5, 2). The parameters of each layer are shown in Table 2.

5 Performance Analysis Two types of models were implemented, and both models were compared. After implementing the model, the result was plotted. The loss function of RNN and LSTM model were plotted, as shown in Figs. 5, 6. Loss function indicates how far the current values are from the optimal ones while training the LSTM model [12]. MSE was used as the loss function. The loss in each epoch is shown in Figs. 5 and 6. The value of loss should be minimized as much as possible. A lower loss value indicates that the output predicted by the proposed LSTM model is accurate. Two types of models were implemented, and both models were compared. In both models, batch size is 32, and the number of epoch is 500. The minimum training loss and validation loss for RNN model is 0.0123 and 0.1233 respectively, whereas, the minimum training loss and validation loss for LSTM model are 0.0066 and 0.0948 respectively. The Fig. 7 shows the results of the Simple RNN model for foot lift event detection. The RNN model could able to predict the Flex value and FSR value. However, LSTM

38

B. Basumatary et al.

Fig. 5 Loss function plot for RNN model. The blue colour represents training loss, and the orange colour represents testing loss

Fig. 6 Loss function plot for LSTM model. The blue colour represents training loss, and the orange colour represents testing loss

model has produced better results compared to simple RNN model. The Fig. 8 shows the results of the proposed LSTM model for foot lift event detection. The LSTM could able to predict the Flex value and FSR value. From the result, it was found that the proposed model could able to predict the next cycle of the gait. The result shows the predicted values of Flex and FSR using the data x, y, z values of the accelerometer. From the result, it was found that the proposed model could able to predict the next cycle of the gait. When the subject lifts the

Deep Learning based Detection …

39

Fig. 7 Predicted vs actual output of simple RNN model. In the first graph, blue, orange and green represent X, Y and Z values of the accelerometer respectively. In the second graph, blue and orange represent Flex predicted and Flex actual respectively. In the third graph, blue and orange represent FSR predicted and FSR actual respectively

foot, the value of accelerometer changes and the algorithm trigger the FES circuit to generate pulses based on the accelerometer data. The flex sensor and FSR were used to label the accelerometer data. Flex value indicates the ankle dorsiflexion. More is the flex value; more is the ankle dorsiflexion and vice versa. FSR value indicates whether the heel is touching the ground. When the heel touches the ground, it indicates zero, and when the heel is in the swing phase it indicates some value greater than 0. The algorithm predicts the next 0.5 s of data based on the previous 1 s of accelerometer data. Therefore, when the subject lifts the foot then only the FES circuit trigger the pulses and vice versa. TensorFlow Lite is used to run machine learning and deep learning models on microcontroller ESP32 with just a few KBs of memory. The core runtime just fits in 16 KB on 32-bit processor and they are capable of running many basic deep learning models [14]. The Fig. 9 shows the block diagram of the deployment of the deep learning model in ESP32 microcontroller. The model is designed using any platform such as Google colab or Jupyter notebook and then the model is converted to Tensorflow Lite. The converted model is then implemented in the ESP32 microcontroller.

40

B. Basumatary et al.

Fig. 8 Predicted vs actual output of LSTM model. In the first graph, blue, orange and green represent X, Y and Z values of the accelerometer respectively. In the second graph, blue and orange represent Flex predicted and Flex actual respectively. In the third graph, blue and orange represent FSR predicted and FSR actual respectively

Fig. 9 Block diagram of deploying model in ESP32

6 Conclusion We have successfully developed an algorithm that can detect the foot lift event based on the accelerometer data. The testing result shows that the developed method can be used for FES device for foot drop. This algorithm can be interfaced with FES device through a microcontroller. This reduces the device complexity and also makes the device comfortable for the patient. The stimulation of FES device has to be triggered only when the subject lifts the foot. In a traditional FES system, foot lift detection sensor is attached below the heel. However, such a type of sensor requires cables which create discomfort to the patient while wearing the device. To avoid this cable complexity, a single accelerometer will be integrated with the FES device. Therefore, we have developed a Deep learning based foot lift detection algorithm using a single accelerometer and our developed model was successfully able to predict the foot lift without using cable complexity.

Deep Learning based Detection …

41

Acknowledgements We would like to thank Department of Biomedical Engineering, Indian Institute of Technology Ropar for providing us with all the necessary infrastructure and support for carrying out this research.

References 1. Vodovnik, A.K., Stanic, U., Acimovic, R., Gros, N.: Recent applications of functional electrical stimulation to stroke patients in Ljubljana. Clin. Orthop.,131, 64–70 (1978) 2. Kobetic, R., Marsolais, E.B.: Synthesis of paraplegic gait with multichannel functional neuromuscular stimulation. IEEE Tran. Rehab. Eng. 2(2), 66–78 (1994) 3. Tong, K., Granat, H.M.: A practical gait analysis system using gyroscopes. Med. Eng. Phys. 21, 87–94 (1999) 4. Ng, S.K., Chizeck, H.J.: Fuzzy model identification for classification of gait events in paraplegics. IEEE Trans. Fuzzy Syst. 5(4), 536–544 (1997) 5. Kostov, A., Andrews, B.J., Popovic, D.B., Stein, R.B., Armstrong, W.: Machine learning in control of functional electrical stimulation systems for locomotion. IEEE Trans. Biomed. Eng. 42(6), 541–551 (1995) 6. Dai, R., Stein, R.B., Andrews, B.J., James, K.B., Wieler, M.: Application of tilt sensors in functional electrical stimulation. IEEE Trans. Rehab. Eng. 4, 63–72 (1996) 7. Willemsen, A., Bloemhof, F., Boom, H.: Automatic stance-swing phase detection from accelerometer data for peroneal nerve stimulation. IEEE Trans. Biomed. Eng. 37(12), 1201–1208 (1990) 8. Graupe, D., Kohn, K.H., Kralj, A., Basseas, S.: Patient controlled electrical stimulation via EMG signature discrimination for providing certain paraplegics with primitive walking functions. J. Biomed. Eng. 5(3), 220–226 (1983) 9. Ding, Z.: The real time gait phase detection based on long short-term memory. In: Proceedings IEEE 3rd International Conference Data Science Cyberspace, DSC 2018, pp. 33–38, (2018). https://doi.org/10.1109/DSC.2018.00014 10. Peinado-Contreras, A., Munoz-Organero, M.: Gait-based identification using deep recurrent neural networks and acceleration patterns. Sensors (Switzerland) 20(23), 1–18 (2020). https:// doi.org/10.3390/s20236900 11. Understanding LSTM Networks - colah’s blog, https://colah.github.io/posts/2015-08-Unders tanding-LSTMs/. Accessed 25 May 2021 12. Su, B., Gutierrez-Farewik, E.M.: Gait trajectory and gait phase prediction based on an LSTM network. Sensors (Switzerland) 20(24), 1–17 (2020). https://doi.org/10.3390/s20247127 13. Basumatary B, Halder RS, Sahani, A.: A microcontroller based charge balanced trapezoidal stimulus generator for FES system. In: 2021 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 2021, pp. 1–4. https://doi.org/10.1109/I2MTC50364. 2021.9459837 14. TensorFlow Lite for Microcontrollers, https://www.tensorflow.org/lite/microcontrollers. Accessed 02 Aug 2021

A NIRS Based Device for Identification of Acute Ischemic Stroke by Using a Novel Organic Dye in the Human Blood Serum Raktim Bhattacharya, Dalchand Ahirwar, Bidisha Biswas, Gaurav Bhutani, and Shubhajit Roy Chowdhury Abstract The paper presents the development of a portable instruments based on Near infrared spectroscopy based (NIRS) that can detect the occurrence of acute ischemia stroke through the estimation of albumin in human blood serum. Albumin has been estimated in human blood serum by mixing the serum with a novel organic dye. The novel organic dye named as CyG have an excitation wavelength (740 nm) and emission wavelength (805 nm) in the NIR-I region. This developed dye is highly specific and selective towards albumin and has the potential to determine concentration of albumin in blood serum Sample s. The developed device have been tested on human blood Sample with convincing results, where the device has found to show linear variation in output voltage with the concentration of albumin in human blood. Also the experimental results on albumin measurements indicates a good degree of repeatability. Keywords Blood serum · Albumin · CyG dye · LED · Photodiode

R. Bhattacharya (B) · D. Ahirwar · S. R. Chowdhury School of Computing and Electrical Engineering, Indian Institute of Technology Mandi, Mandi, India D. Ahirwar e-mail: [email protected] S. R. Chowdhury e-mail: [email protected] B. Biswas School of Basic Sciences, Indian Institute of Technology Mandi, Mandi, India G. Bhutani School of Engineering, Indian Institute of Technology Mandi, Mandi, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_4

43

44

R. Bhattacharya et al.

1 Introduction Acute Ischemic Stroke is a one of the major cause of death and disability around the world [1]. Globally, acute ischemic stroke mortality is not only high but continuously increasing which has been become a cause of concern. Around 7.5 million people suffers from stroke worldwide [2]. Among them 85–89% are found with ischemic stroke and 15–11% are found with hemorrhagic stroke [3]. The Indian stroke scenario is no less grim with a comparatively higher rate of incidence and prevalence owing to poor control of risk factors and a lack of public awareness and proper diagnosis [4]. Only MRI and CT scans are considered as gold standard around the world for detecting acute ischemic stroke but they are very costly and not suitable for Pointof-Care diagnosis [5]. EEG and NIRS methodology are emerging in such clinical applications and suited for point of care diagnosis. Early uses of NIRS were found in chemical analysis, food industries, later its popularity is gained in clinical application such as pediatric diagnosis, intensive care, ischemic stroke diagnosis, blood serum chromophore analysis, albumin level detection for its non-invasive measurement properties [6]. Hypoalbumin is a critical biomarker for identification of acute ischemic stroke. Albumin level in the blood serum below 35 g/L is the early indicator marker of acute ischemic stroke [7]. Ischemia modified albumin (IMA) is a modified form of albumin that forms as a result of conformational changes (IMA). Although IMA has been shown to rise in a variety of disorders, the specific mechanism that causes IMA to form is unknown. IMA has been identified as a marker for ischemia and other chronic diseases. This altered form of albumin, is created as a result of conformational changes [8]. The present study is aimed at detection of the concentration of albumin in blood serum, which can be used for early detection of ischemic stroke. From the literature survey, the author have found that the concentration of albumin in blood serum is below 35 g/L. The author’s group have developed a novel dye CyG, which specifically bind with the albumin having an excitation wavelength of 740 nm and emission wavelength of 805 nm [9]. By using the emission wavelength of this dye, the author has develop a near infrared spectroscopy (NIRS) based prototype instrument. Intensity of fluorescence spectroscopy and the concentration change can be derived from Beer’s law. By applying the Taylor series expansion where after grouping and rearranging logarithmic base conversion and basic assumption of dilute solution following equation can be obtained. F = 2.303.I0 .S..ε.c.d

(1)

where, relative fluorescence intensity (F), structural facture (S), fluorescence efficiency (F), molecular absorptivity (ε), concentration of the solution (c), distance between source and detector (d), initial intensity of light (I0 ) [10].

A NIRS Based Device for Identification of Acute Ischemic Stroke …

45

2 The Novel Organic Dye Since our current work focuses on the development of a NIRS spectroscopic device, we attempted to utilize a NIR emissive dye “CyG” developed and reported by Dey et. al. for conducting a number of experiments with our system [11]. Reportedly, the probe CyG in the presence of human serum albumin, displayed excitation (λex ) at 740 nm and emission (λem ) at 804 nm along with significant enhancement in the emission intensity [11]. Besides, the exceptional selectivity of CyG towards serum albumin in the diverse biological milieu as reported in the study [11], prompted us to explore the dye for our study. Considering the albumin specific fluorescence enhancement of CyG in the near infra-red-I (NIR-I) window (700–900 nm), we opted to utilize this probe for investigation of suitable albumin levels in the present study. All the detailed studies related to the CyG probe can be obtained in the previously published report.

3 Experimental Set-Up 3.1 Making of the Instrument The ray diagram of the optical instrument is shown in Fig. 1. In this Fig., the author

Fig. 1 Ray diagram of the NIR optical instrument

46

R. Bhattacharya et al.

Fig. 2 Developed Instrument

uses a LED of 730 nm to 750 nm with a peak wavelength of 740 nm. Two biconvex mirror have been used in this instrument, each having the focal length of 5 cm. The LED is placed at focus of biconvex lens I and II as shown in the Fig. 1. The cuvette is placed at the focus of both the biconvex lens. The photodiode is placed at the right angle of the cuvette. An optical filter is placed just before the photodiode.

3.2 Working Topology The LED is placed at the focal length of the first biconvex lens, so that all the light produced from the LED will be parallel after passing through the first biconvex lens and falls on the cuvette, which is containing the blood serum with the novel dye CyG. This novel dye having an excitation wavelength of 740 nm and the emission wavelength of 805 nm. So the LED light having the wavelength of 730 nm to 750 nm excites the Sample containing dye. Then all the light emitted from the Sample contains both 740 nm and 805 nm wavelengths of light. To separate these two light, a high pass optical filter of 790 nm has been used. So that all the light having the wavelength less than 790 nm will be blocked and rest other lights having wavelength more than 790 nm will be passed. Since the photodiode is placed at the right angle of the cuvette, so all the lights will fall on the photodiode. The photodiode is connected to a circuit which calibrates the concentration of albumin with the current generated with the help of photodiode. The developed instrument is shown in Fig. 2.

3.3 Specifications The specification of the instrument is shown below. • Source of light is replaceable LED and it’s wavelength ranging from 730 to 760 nm. • 5 W power supply can operate the whole process. • Two biconvex lens with 3 cm diameter and 5 cm focal length are used.

A NIRS Based Device for Identification of Acute Ischemic Stroke …

47

• Low cost cuvette has been used as Sample holder. • Photodiode of wavelength range 170 nm to 2600 nm.

4 Results and Discussions In this present work, clinical trials have been carried out, ten blood serum Sample s of healthy human subject have been collected from pathology lab of Medical Unit IIT Mandi with their prior consent following the Declaration of Helsinki (1964). Out of these one Samples have been neglected from the current study because of blood coagulation. Following steps of procedure have been followed. Sample testing for albumin concentration has been conducted at the pathology lab. It has been found that for subject id 1, 2, 3, 4, 5, 6, 7, 8 and 9, the albumin concentration is 4 g/dL, 4 g/dL/ 4.7 g/dL, 4.3 g/dL, 4.1 g/dL, 4.4 g/dL, 4.2 g/dL, 5 g/dL and 5 g/dL respectively as shown in Fig. 3. The concentration of the albumin in the blood serum Sample s has been obtained and then varying the concentration of albumin several tests have been conducted on the developed instrument. The experiments have been carried out at laboratory temperature of 25 °C (±7 °C). After obtaining the serum Sample s from the human blood for different subjects. The Sample s are sutaiblely diluted in the laboratory. 6 μL CyG dye has been added to the diluted solution to observe the variation of output voltage with change in concentration of albumin. Figures 4, 5, 6, 7, 8, 9, 10, 11 and 12 shows the graph of variation of voltage with respect to different albumin concentration of nine different subject IDs. The Table 1 shows the measurement statistics of nine Sample s which include percentage standard error and Adj.R2 value for linear fit model. The Sample 2, 3, 7, 8 and 9 have high adjusted R2 value, which shows high repeatability.

Fig. 3 Sample collected from IIT Mandi health center of different subject IDs

48

Fig. 4 Concentration of albumin versus voltage (Sample 1)

Fig. 5 Concentration of albumin versus voltage (Sample 2)

R. Bhattacharya et al.

A NIRS Based Device for Identification of Acute Ischemic Stroke …

Fig. 6 Concentration of albumin versus voltage (Sample 3)

Fig. 7 Concentration of albumin versus voltage (Sample 4)

49

50

Fig. 8 Concentration of albumin versus voltage (Sample 5)

Fig. 9 Concentration of albumin versus voltage (Sample 6)

R. Bhattacharya et al.

A NIRS Based Device for Identification of Acute Ischemic Stroke …

Fig. 10 Concentration of albumin versus voltage (Sample 7)

Fig. 11 Concentration of albumin versus voltage (Sample 8)

51

52

R. Bhattacharya et al.

Fig. 12 Concentration of albumin versus voltage (Sample 9)

Table 1 Percentage error in measurement for nine Sample Sample no

Standard error (%)

Adj. R2 value

1

72.61

0.74

2

29.21

0.9

3

13.01

0.94

4

17.13

0.78

5

64.19

0.77

6

50.87

0.73

7

37.25

0.96

8

23.09

0.97

9

45

0.9

5 Conclusions The NIRS prototype device has been developed and has been found to detect the albumin concentration from 0.5 g/dL to 5 g/dL. In all the Sample s of healthy human subject the albumin level was above 4 g/dL. Ischemia condition has been introduced by lowering the concentration of the albumin in the human serum. This device shows almost linear relationship between albumin concentration and the voltage. Hence, we can say that this type of instrument works in the region of NIR-1 spectroscopy.

A NIRS Based Device for Identification of Acute Ischemic Stroke …

53

This type of instrument is much cheaper and have high durability. It can work in any environment, easy to operate and fast to operate.

References 1. VanWie, C., Casiero, D.: Acute ischemic stroke in a male collegiate football athlete. Athlet. Train. Sports Health Care 13(4), e247–e251 (2021) 2. Li, N., Yang, F., Zhao, Z., Jiang, J. and Lei, Y.: Efficacy and safety of Naoxintong capsule for acute ischemic stroke: a protocol for systematic review and meta-analysis. Medicine 100(34); Jacobs, I.S., Bean, C.P.: Fine particles, thin films and exchange anisotropy. In: Rado, G.T., Suhl, H. (Eds.) Magnetism, vol. III, New York: Academic, 1963, pp. 271–350 (2021) 3. Ahirwar, D., Shakya, K., Banerjee, A., Khurana, D., Chowdhury, S.R.: Simulation studies for non invasive classification of ischemic and hemorrhagic stroke using near infrared spectroscopy. Biodevices, 192–198 (2019). https://doi.org/10.5220/0007413201920198 4. Pandian, J.D., Stroke, S.P.: Epidemiology and stroke care services in India. J Stroke 15(3), 128–134 (2013) 5. Henninger, N., Kuppers-Tiedt, L., Sicard, K.M., Gunther, A., Schneider, D.: Schwab, S Neuroprotective effect of hyperbaric oxygen therapy monitored by MR-imaging after embolic stroke in rats. Exp. Neurol. 201, 316–323 (2006) 6. Middleton, P.M., Chan, G.S.H., Steel, E., Malouf, P., Critoph, C., Flynn, G., O’Lone, E., Celler, B.G., Lovell, N.H.: Fingertip photoplethysmographic waveform variability and systemic vascular resistance in intensive care unit patients. Med. Biol. Eng. Comput. 49(8), 859–866 (2011) 7. Dziedzic, T., Pera, J., Slowik, A., Gryz-Kurek, E.A., Szczudlik, A.: Hypoalbuminemia in acute ischemic stroke patients: frequency and correlates. Eur. J. Clin. Nutr. 61(11), 1318–1322 (2007) 8. Kumar, P.A., Subramanian, K.: The role of ischemia modified albumin as a biomarker in patients with chronic liver disease. J. Clin. Diagn. Res. 10, 9–12 (2016) 9. Arora, Y., Mukherjee, S., Biswas, B., Bedi, V., Dey, G., Mondal, P., Ghosh, S., Roy Chowdhury, S.: A novel near infrared spectroscopy based device for albumin estimation. 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2020), Palais des congres de Montreal, Montreal, Quebec, Canada July 20–24, 2020 10. Miano, T.M., Senesi, N.: Synchronous excitation fluorescence spectroscopy applied to soil humic substances chemistry, 117–118, 41–51 (1992). https://doi.org/10.1016/0048-969 7(92)90071-y 11. Dey, G., et al.: Renal clearable new NIR probe: precise quantification of albumin in biofluids and fatty liver disease state identification through tissue specific high contrast imaging in vivo. Anal. Chem. 89(19), 10343–10352 (2017)

The Development of a Portable IoT-Enabled Aqueous Sulphur Sensor Brady Shearan, Fowzia Akhter, and S. C. Mukhopadhyay

Abstract The research proposed is in relation to real-time detection of Sulphur ions within aqueous mediums, with the future potential of detection of Hydrogen Sulfide (H2 S) in wastewater systems. Electrochemical Impedance Spectroscopy (EIS) is utilized alongside a novel interdigital capacitive sensor. The synthesis of graphenebased polymer composite thin films for Sulphur ion absorption are detailed. Sampled Sulphur measurements are conducted by using the presented thin-film coatings. The sensitivity curve obtained for a range of Sulphur concentrations from 0.5 ppm to 50 ppm, highlights the promising initial results, and strong potential for use within detection of H2 S. The coating of Reduced Graphene Oxide (rGO) with silver nanoparticles (AgNPs) exhibits an enhanced sensitivity towards Sulphur ions. A portable IoT-based system is also presented for detection of Sulphur irrespective of time and place. This paper provides a detailed approach from conception to employment of the developed system. Keywords Interdigital capacitive sensor · Sulphur · Hydrogen Sulfide (H2 S) · Reduced Graphene Oxide (rGO) · Internet of Things (IoT)

1 Introduction Hydrogen Sulfide (H2 S) is a toxic gas comprised of a singular Sulphur atom bonded to two hydrogen atoms and is commonly found in wastewater systems. H2 S gas exerts an offensive smelling odour at low concentrations of 1 ppm and can not only can be harmful to vision, skin and the respiratory system at above 5 ppm, but also significantly reduces the lifespan of liquid waste service infrastructure it is carried through [1]. At higher concentrations, H2 S reacts with bacteria on the lining of pipes which results in the biological oxidation of H2 S and the production of sulfuric acid, B. Shearan (B) · F. Akhter · S. C. Mukhopadhyay Department of Engineering, Macquarie University, Sydney, NSW, Australia e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_5

55

56

B. Shearan et al.

which then corrodes the piping causing cracks and fractures. Corrosion of sewerage infrastructure is estimated to cost billions of dollars every year in the United States alone [2]. Therefore, it is essential for the development of a system which is able to monitor H2 S levels in wastewater to see through both the reduction in waste infrastructure repair costs and the prevention of H2 S exposure to humans. Recently, significant research has been conducted surrounding the development of H2 S sensors. H2 S is traditionally detected utilizing semiconducting metal oxides or more recently through transition metal chalcogenides (TMDs) whilst in gas phase, but very few have the capability to detect H2 S in liquid phase [3]. A highly sensitive sensor for H2 S detection in aqueous sewerage samples using a microfluidic gas channel coupled with a metal-oxide semiconductor has been presented [4]. This system consists of an automated sampling system connected to a vaporization chamber. These types of systems are bulky, complex and require the use of expensive vaporization systems which are energy intensive. However, recently a thinfilm, graphene-based chemiresistor has been presented for H2 S detection in liquid phase [5]. The system aims to remove the requirement of a vaporization system, significantly reducing the cost of a detection system. Graphene has seen use within in numerous recent gas sensing applications, due to its well documented desirable range of physical properties [6]. Graphene is a material that consists of a singular 2-D monolayer of sp2 hybridized carbon atoms arranged in a honeycomb lattice [7]. The perfect arrangement of the carbon atoms within the honeycomb lattice provides exceptional mechanical and electrical properties. Though graphene itself is expensive to produce, therefore reduced graphene oxide (rGO) as a low-cost alternative has seen extensive use within electrochemical sensing due to its similar properties [8]. A comprehensive review of recent rGO sensing applications has been presented [9]. A number of low-cost rGO based sensors for detection of H2 S in gas form have been developed [10–14], though there remains little research surrounding aqueous applications of such sensors. It is desirable to be able to detect H2 S if present in water supplies before human exposure in gas form, though currently there are few existing sensors which can achieve this detection without extensive power consumption. Our lives are currently being transformed by the advancement of wireless communication architectures. This transformation is due to the emergence of Wireless Sensor Networks (WSNs) and the Internet-of-Things (IoT) concept, providing means for the rapid development of high-performance wireless sensing systems which maintain significantly low power consumption [15, 16]. Developments in such key resourceful technologies has revolutionized the field of smart sensing. Smart homes [17], smart health [18], smart agriculture [19] are an example of the important domains offered by the environmental monitoring within a smart city.

The Development of a Portable IoT-Enabled …

57

2 Materials and Methods This section represents the synthesis of the Sulphur sensing material, preparation of the sensing surface, Sulphur sample preparation, experimental procedure, design of the portable smart system for Sulphur ions detection using the developed sensor in water bodies.

2.1 Chemical Silver Acetate (99.999%), N-dimethylformamide (DMF) (99.6%), Polyethylene medium density (0.94 g/mL), Reduced graphene oxide powder (Chemically reduced by hydrazine), Graphene oxide dispersed in H2 O (2 mg/ml), Toluene anhydrous (99.8%) and Ethanol (99.99%) were purchased from Sigma-Aldrich. De-ionized (DI) Water was sourced from a Millipore (18 Mohms cm) dispensing system.

2.2 Synthesis of rGO/(RGO-AgNP)-PE At first, 50 mg of rGO and 25 mg of Silver Acetate were combined in 15 mL of DMF within a sonification container. This resulted in a silver acetate to rGO weight ratio of 1:2. The mixture was sonicated at a temperature of approximately 90 °C for 20 min. After allowing the mixture to cool to ambient temperature, the black-coloured compound in the flask was then poured into a boiling flask. Afterwards, the DMF was evaporated utilizing a rotary evaporator, resulting in the formation of a dark powder at the bottom of the flask. The sample then required washing with water and ethanol to extract any remaining silver acetate and unbound Ag nanoparticles. To achieve this, the sample was moved to a centrifuge tube and washed three times, respectively, with water and ethanol. The sample was dried carefully through slowly spraying nitrogen over the solvent surface and the final product was recovered as a dark black powder consisting of approximately 40 mg of AgNP coated rGO. Finally, 2 mg of rGO or rGO-AgNP powder was mixed with 1 ml of DMF and 0.5 ml of a PE (30 mg/ml in Toluene) mixture. The combined mixture was then sonificated for 30 min.

2.3 Preparation of Interdigital Sensor and Coating An interdigital sensor similar to a prior research [20] is used. Interdigital sensors function on the same parallel plate capacitor concept, although the electrodes are built

58

B. Shearan et al.

Fig. 1 a Developed interdigital sensor b Electrode layout of the interdigital sensors (1-5-50) configuration

on a solid, planar substratum as finger-like structures to allow single-sided access to the sample [21]. The benefit provided by the planar structure of these sensors is the ability to conduct one-sided and non-invasive impedance measurements [22]. A small AC signal is transmitted to the sensor’s positive electrodes, creating an electric field which pierces through the test sample and can be detected by the sensor electrodes. According to the characteristics of the material under test, the permittivity of the induced electric field changes. The change in the electric field can then be analyzed and utilized to identify the properties of the material. Figure 1a exhibits a diagrammatic representation of the sensor that has been used in this research. The planar interdigital sensor utilized in this experiment consist of a 1–5-50 configuration (Fig. 1b) which entails the design pattern consisting of five sensing electrodes per positive electrode, with a 50um gap between every two consecutive electrodes [23]. The sensing surface of the interdigital sensor was initially treated with methanol to eliminate any impurities prior to coating the sensor. On the sensing surface, 20 μL of the prepared suspensions were then pipetted and dried at 80 °C for 24 h. The suspension developed a solid polymer composite layer on top of the sensing surface because of the evaporation of the solvents.

2.4 Sample Preparation Initially, a 1000 ppm (parts per million) Sulphur solution was prepared mixing deionized water with Sodium Sulfite (Na2 SO3 ). Serial dilution method has been used to initially compose samples ranging from 0.5 to 50 ppm. The respective concentrations were calculated to maintain the same Sulphur ion concentrations as found in equivalent concentrations of H2 S. The sample water has been used to develop the standard curve and performing the experiments.

The Development of a Portable IoT-Enabled …

59

Fig. 2 Laboratory experiment setup

2.5 Electrochemical Impedance Spectroscopy (EIS) Electrochemical impedance spectroscopy (EIS) is a very popular method which is performed through the application of small AC electrical signals to closely calculate both the conductive as well as capacitive characteristics of materials. The sensor was mounted through a golden attachment clamp link to a ‘Hioki IM3536’ LCR Meter. To obtain data from the sensor, a bench-top computer was used. To initiate the measurement, the sensor was submerged in the sample water and kept for two minutes to allow for Sulphur absorption. The sensor was then removed from the sample, washed with deionized water and allowed to dry before the next measurement. The measurements were taken utilizing a frequency sweep between 10 Hz to 100,000 Hz at a 1 V peak to peak. Measurements were taken three times respectively and averaged. Figure 2 displays the laboratory setup for the measurement experiments.

2.6 Design and Development of the Portable Sensing System A portable system was designed to measure Sulphur concentrations from samples at any desired location. The proposed system consists of a box in which all electronics are stored and a small cut-out for a Liquid Crystal Display (LCD). The proposed system consists of an AD5933 impedance analyzer [22], Sulphur sensor, LCD, Arduino Uno Microcontroller [23] for processing data, LoRa shield [24] for wirelessly transmitting data to the ThingSpeak [25] cloud server and a Charging Shield for direct charging of the battery.

60

B. Shearan et al.

The high-precision impedance analyzer module, AD5933 calculates both the real and imaginary impedance at desired frequencies utilizing Discrete Fourier Transforms (DFT). The AD5933 requires calibration before initiating any measurements, to ensure a unity gain is maintained within the impedance analyzer. The gain factors are maintained to ensure accurate calculation of the sensor’s resistance and reactance values at each desired frequency. The system is operated at a particular frequency, determined based on the experimental outcomes discussed in the previous section. As this frequency operates under the minimum operation frequency achievable b y the stand-alone AD5933 IC, there required the use of an external clock. Therefore, the DS1077Z; a programmable, fixed-frequency oscillator was added to the circuit to provide an adjustable clock signal for the AD5933 to run from and conduct impedance analysis at frequencies ranging from 10 to 10,000 Hz. LoRa-WAN is a communication protocol that is utilized in the wireless sensing system as it provides the advantage of utilizing the lowest power consumption and transmission over large distances in comparison to other protocols such as Zigbee [26] and Wi-Fi. A rechargeable battery is used to prevent the need of battery replacement. This reduces the ongoing maintenance cost of the system when used over its lifetime. Figure 3 details the electronic connection diagram. Both the client and the server scripts are programmed within the Arduino IDE (Integrated Development Environment). The systems first initializes the AD5933 to operate the sensor and process an impedance measurement. After obtaining the data, it is then processed via the microcontroller utilizing a processing algorithm which converts the raw impedance data into both a temperature and Sulphur concentration respectively. After that, the resulting information is displayed on the LCD and then

Fig. 3 Connection diagram of the portable sensing system

The Development of a Portable IoT-Enabled …

61

Fig. 4 Flow diagram of sensing system

Fig. 5 Final portable prototype system

transmitted through the TTN (The Things Network) via a Dragino gateway [27] to the designated ThingSpeak channel. On the server-side of the program, both the channel ID and Application Process Interface (API) keys are set to ensure private transfer of the data to the correct allocated channel. The flow diagram of the system is displayed in Fig. 4 and the final system is Fig. 5.

3 Results and Discussions This section represents sensor’s response towards Sulphur ions and development of the calibration standards to calculate Sulphur concentrations in unknown samples.

62

B. Shearan et al.

3.1 Sensor’s Response Towards Sulphur Ions The sensors surface was coated with rGO-AgNP/PE to observe the effect of the coating in achieving detection of differing Sulphur ions concentrations from 0.5– 50 ppm. After each measurement, the coated sensing surface was cleaned thoroughly using deionized water to eliminate the particles from the sensing surface that have been adsorbed. Before the next measurement, the sensor then is dried at room temperature. The variation in the sensor’s resistance and reactance with the change in frequency for a wide range of Sulphur concentrations were recorded. We can see in Fig. 6a, b show that there is a distinguished difference in the measured resistance and reactance of the coated sensor when submerged in aqueous solutions with differing concentrations of Sulphur ions. The sensor shows considerable readings between 10 Hz to 1 kHz. Therefore, the calibration curves are developed at 250 Hz to determine Sulphur concentrations with the sensor’s resistance. Figure 7 show the calibration curves for measuring the low and high concentrations. The determination coefficient (R2 ) applying linear fit are found as 0.9814 and 0.9775, revealing a strong correlation between the Sulphur concentrations and the sensor’s resistance, particularly at low concentrations. Thus, Eq. 1 can be used for measuring low Sulphur ions concentrations from unknown samples: C=

Rsense − 13.469 4.4594

(1)

where, C (ppm) refers to the actual Sulphur concentrations, whereas Rsense (%) shows the sensor’s resistance calculated as sensitivity. Any unknown Sulphur concentrations in aqueous medium can be determined using Eq. (1). Two sensitivity equations are required as after 10 ppm, the sensors capacitance has reached a saturation point and the adsorption of Sulphur ions does not have as large of an impact upon the sensor’s electrical conductivity.

3.2 Sensor’s Response The response curve was produced for Sulphur ion concentrations, with a minimum limit of detection at 0.5 ppm. The following equation was utilized to calculate the sensitivity: Sensor’s Response(%) =

R D I W − R Sample × 100 RDI W

(2)

The Development of a Portable IoT-Enabled …

63

Fig. 6 a Variation in sensor’s resistance, and b Nyquist plot for various tested solutions using RGO-AgNP/PE coated sensor

RDIW is the resistance of the coated sensor in de-ionized water and Rsample is the resistance of the concentrated sample. The use of two different linear regression over one exponential regression is due to the simplicity in calculation and a higher degree of accuracy in concentration calculation is obtained.

64

B. Shearan et al.

Fig. 7 Calibration curves for measuring low concentrations and high concentrations using RGO-AgNP/PE coated sensor at 250 Hz

Fig. 8 Cross-Sensitivity results at 20 ppm concentrations

3.3 Selectivity of the Sensor The sensor was tested for cross-sensitivity against three other common analytes found in wastewater. All of the analytes were tested at concentrations of 20 ppm. The concentration of 20 ppm was chosen as a base-average concentration, with the concentrations commonly found in wastewater for each analyte being; Calcium (20 ppm) and maximum Nitrate (10–20 ppm) and Phosphate (10 ppm). The addition of the silver nanoparticles greatly improved the selectivity towards Sulphur ions. This is because Silver has a well-known affinity towards Sulphur ions and when adsorbed, significantly changes its electrical properties. Figure 8 shows the outcomes of this experiment.

3.4 Repeatability Testing of the Sensor The sensor was tested for its repeatability and accuracy in obtaining consistent measurements. The resistance (R) values of the sensor were measured three times respectively during exposure to different concentrations of Sulphur ions to confirm the stability and repeatability of the sensor. The results recorded during the experiment are exhibited in Fig. 9. It was found that the resistance response of the sensor

The Development of a Portable IoT-Enabled …

65

Fig. 9 Repeatability test for sensors resistance values

did not show a significant change for any of the tested concentrations. The Relative Standard Deviation (RSD) was calculated and found to remain below 1% for all concentrations, exhibiting the sensors exceptional response repeatability towards the sampled analyte.

3.5 Data Collection and Storage The purpose of this research is to develop a portable Sulphur ions detection system in a real-life scenario. Therefore, the portable system is used to measure the Sulphur concentration in drinking water. The Sulphur concentration of the sample determined by the sensor system is stored in the ThingSpeak cloud. This will allow to get the expert opinion if the water quality deteriorates. Figures 10 and 11, exhibits the measured concentration calculated by the final system over a short testing period, with measurements being taken of the same 6.5 ppm sample and battery voltage displayed. The measurement fluctuates occasionally, whilst retaining an error of under 5% for all readings. The addition of a battery voltage monitor (Fig. 12) allows

Fig. 10 Sulphur concentration measured by portable system

66

B. Shearan et al.

Fig. 11 Sulphur concentration stored on the ThingSpeak server

Fig. 12 Battery voltage gauge stored on the ThingSpeak server

for monitoring of the portable system battery and notification of when it requires charging.

The Development of a Portable IoT-Enabled …

67

4 Conclusion and Future Work In this paper, a low cost, sensitive IoT-based Sulphur sensor was produced and explained. As a sensitive coating on top of the sensor surface, the composite rGOAgNP/PE was used. The produced coating is capable of absorbing Sulphur ions from the water samples. EIS was utilized for the characterization of each developed coating, and a sensitivity curve was developed for the rGO-AgNP/PE coating. The concentrations found in water can be stored wirelessly or analysed locally utilizing the developed system. The future work of this research is to enable the sensor to achieve accurate measurements in mediums of differing pH levels, temperature. The effect of the reusability of the sensor should be studied and the sensor should be tested with samples of pure H2 S. Acknowledgements The authors would like to thank the School of Engineering, Macquarie University, for providing the laboratory facilities and supporting the research.

References 1. Baratto, C., Rigoni, F., Faglia, G., Comini, E., Zappa, D., Sberveglieri, G.: Zno and sno2 one-dimensional sensors for detection of hazardous gases 10, 1–3 (2017) 2. Li, X., O’Moore, L., Song, Y., Bond, P.L., Yuan, H., Wilkie, S., Hanzic, L., Jiang, G.: The rapid chemically induced corrosion of concrete sewers at high H2 S concentration. Water Res. 162, 95–104 (2019), ISSN 0043–1354. https://doi.org/10.1016/j.watres.2019.06.062 3. Luo, Y., Zhang, D., Fan, X.: Hydrothermal fabrication of ag-decorated mose2/reduced graphene oxide ternary hybrid for H2 S gas sensing. IEEE Sens. J., 1–1 (2020) 4. Montazeri, M.M., De Vries, N., Afantchao, A.D., O’Brien, A., Kadota, P., Hoorfar, M.: Development of a sensing platform for nuisance sewer gas monitoring: Hydrogen sulfide detection in aqueous versus gaseous samples. IEEE Sens. J. 18(19), 7772–7778 (2018) 5. Yavari, A., Tahmooressi, H., Hoorfar, M., Mohaghegh Montazeri, M., Tasnim, N., Farahani, A., Kadota, P., Markin, P., Dalili, A., Taatizadeh, E.: A graphene-based chemical sensor for hydrogen sulfide measurement in water 10, 1–4 (2019) 6. Zheng, H.: Application of graphene in elctrochemical sensing. Curr. Opin. Colloid Interface Sci. 20(5–6), 383–405 (2015). https://doi.org/10.1016/j.cocis.2015.10.011 7. Zubiarrain-Laserna, K.: Review—graphene-based water quality sensors. J. Electrochem. Soc. 167(3), 37539 (2020). https://doi.org/10.1149/1945-7111/ab67a5 8. Zheng, D., Hu, H., Liu, X., Hu, S.: Application of graphene in elctrochemical sensing. Curr. Opin. Colloid Interface Sci. 20(5–6), 383–405 (2015) 9. Rowley-Neale, S., Randviir, E., Dena, A., Banks, C.: An overview of recent applications of reduced graphene oxide as a basis of electroanalytical sensing platforms. Appl. Mater. Today 10, 218–226 (2018) 10. Cho, K.Y., Seo, H.Y., Yeom, Y.S., Kumar, P., Lee, A.S., Baek, K.-Y., Yoon, H.G.: Stable 2Dstructured supports incorporating ionic block copolymer-wrapped carbon nanotubes with graphene oxide toward compact decoration of metal nanoparticles and high-performance nanocatalysis. Carbon (New York) 105, 340–352 (2016). https://doi.org/10.1016/j.carbon.2016. 04.049

68

B. Shearan et al.

11. Peng, W.: Ultrasensitive ppb-level H2 S gas sensor at room temperature based on WO3/rGO hybrids. J. Mater. Sci. Mater. Electron. 31(6), 5008–5016 (2020). https://doi.org/10.1007/s10 854-020-03067-6 12. Some, X.: Highly sensitive and selective gas sensor using hydrophilic and hydrophobic graphenes. Sci. Rep. 3(1), 1868–1868 (2013). https://doi.org/10.1038/srep01868 13. Zhang, Z.: Highly sensitive H2 S sensors based on metal-organic framework driven γ-Fe2O3 on reduced graphene oxide composites at room temperature. Sens. Actuat. B Chem. 325, 128804 (2020). https://doi.org/10.1016/j.snb.2020.128804 14. MalekAlaie, J.: Selective hydrogen sulfide (H2 S) sensors based on molybdenum trioxide (MoO3) nanoparticle decorated reduced graphene oxide. Mater. Sci. Semicond. Process. 38, 93–100 (2015). https://doi.org/10.1016/j.mssp.2015.03.034 15. Akhter, F., Khadivizand, S., Siddiquei, H.R., Alahi, M.E.E., Mukhopadhyay, S.: IoT enabled intelligent sensor node for smart city: pedestrian counting and ambient monitoring. Sensors 19(15), 1–19 (2019). https://doi.org/10.3390/s19153374 16. Akhter, F., Khadivizand, S., Lodyga, J., Siddiquei, H.R., Alahi, Md.E.E., Mukhopadhyay, S.C.: Design and development of an IoT enabled pedestrian counting and environmental monitoring system for a smart city. In: Proceedings of the13th International Conference on Sensing Technology (ICST), Sydney, Australia, 2–4 Nov. 2019 17. Pirbhulal, S., Zhang, H., Alahi, M.E.E., Ghayvat, H., Mukhopadhyay, S.C., Zhang, Y.T., Wu, W.: A novel secure IoT-based smart home automation system using a wireless sensor network. Sensors 17(1), 1–19 (2017). https://doi.org/10.3390/s17010069 18. Afsarimanesh, N., Alahi, M.E.E., Mukhopadhyay, S.C., Kruger, M.: Development of IoT-based impedometric biosensor for point-of-care monitoring of bone loss. IEEE J. Emerg. Sel. Topics Circuit. Syst. 8(2), 211–220 (2018). https://doi.org/10.1109/JETCAS.2018.2819204 19. Akhter, F., Siddiquei, H.R., Alahi, M.E.E., Mukhopadhyay, S.C.: Recent advancement of the sensors for monitoring the water quality parameters in smart fisheries farming. Computers 10(26), 1–26 (2021). https://doi.org/10.3390/computers10030026 20. Akhter, F., Alahi, M.E.E., Siddiquei, H.R., Goonerate, C.P., Mukhopadhyay, S.C.: Graphene Oxide (GO) Coated Impedimetric Gas Sensor for Selective Detection of Carbon dioxide (CO2) with Temperature and Humidity Compensation IEEE Sens. J. Early Access (2020). https://doi. org/10.1109/JSEN.2020.3035795 21. Zia, A.I., Mukhopadhyay, S.C., Yu, P.L., Al-Bahadly, I.H., Gooneratne, C.P., Kosel, J.: Post annealing performance evaluation of printable interdigital capacitive sensors by principal component analysis. IEEE Sens. J. 15(6), 3110–3118 (2015). https://doi.org/10.1109/JSEN. 2014.2355224 22. Akhter, F., Nag, A., Alahi, M.E.E., Liu, H., Mukhopadhyay, S.C.: Electrochemical detection of calcium and magnesium in water bodies. Sens. Actuat. A 305, 1–10 (2020). https://doi.org/ 10.1016/j.sna.2020.111949 23. Analog Devices. (Sep. 4, 2017). AD5933: Impedance Analyzer. http://www.analog.com/media/ en/technicaldocumentation/data-sheets/AD5933.pdf 24. Arduino. (Sep. 4, 2017). Arduino Uno WiFi. https://store.arduino.cc/usa/arduino-uno-wifi 25. HOPERF Electronic. (Nov. 13, 2017). RFM95/96/97/98(W)—Low Power Long Range Transceiver Module V1.0 (V1.0 Edition). http://www.hoperf.com/upload/rf/RFM95_96_97_ 98W.pdf 26. Thingspeak. (2018, 15/11/2018). Thingspeak. 27. https://thingspeak.com/

FEA Analysis on the Optimum Placement of Sensor for Early Detection of Damage in Concrete Pavements Sakura Mukhopadhyay and Mohsen Asadnia

Abstract Structural Health Monitoring (SHM) involves with detection and evaluation of health of structures at early stage of any problem so that any kind of damage/accidents can be prevented. SHM has grown both in the research and development field in recent years to develop new technologies as well as use of conducting surveys and undertaking analysis for security, safety, and post-disaster of structures. It also allows to assess the external factors that affect the structure such as ageing, natural disasters, collisions, and environment. The application of SHM consists of selection and allocation of a suitable, appropriate smart sensor and sensing technology to measure the critical parameters not only affecting the damage but further the health and performance of the structure. The designed smart sensor system will have three main components being the sensing element, an energy harvester, and a wireless transmitter. The sensing elements will include the use of in-pavement sensors embedded within the concrete for internal monitoring of the pavement. The energy harvester will be a transmitter to transmit signals for sensors to harvest energy avoiding the use of cables and a power supply. Lastly, the system will use a planar-type radiator to receive high frequency energy and to radiate information. Keywords Structural health monitoring · Concrete pavement monitoring · Smart sensing · Finite element analysis

S. Mukhopadhyay (B) · M. Asadnia Arcadis Australia Pacific, Level 16/580 George Street, Sydney, NSW 2000, Australia e-mail: [email protected]; [email protected] M. Asadnia e-mail: [email protected] M. Asadnia Macquarie University, Macquarie Park, NSW 2109, Australia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_6

69

70

S. Mukhopadhyay and M. Asadnia

1 Introduction 1.1 Overview Civil structures play an essential role to society providing a safe environment to day-to-day life and enabling economic growth. They exist in various forms such as houses, office building, bridges, railways, roads, dams, tunnels and so on. These structures are found in a variety of dimensions, complex geometries, and various materials with different properties [1]. Failure of these structures can have catastrophic consequences affecting loss of lives, the safety of the public and economic growth. Early detection of the damage can avoid the failure, safety hazards, and need to further repair and maintain. Monitoring the health of the civil infrastructure can ensure the safe operation and longevity of the structures designed lifespan [2]. The suitable technology and the complete technique carried out to monitor and analyze the health of these civil infrastructures is known as structural health monitoring [3]. Structural health monitoring is defined as the process of implementing a method of early damage detection for civil infrastructures. It includes the selection and placement of sensors to measure critical parameters affecting the performance, health, and lifespan of the structural system to be monitored. The sensors selected must have the ability to measure and identify disturbances as well as external influences impacting the structural system. The factors of minimum cost, maximum efficiency and robustness that will not negatively affect the civil infrastructure itself must also be considered. For the application of structural health monitoring on concrete road pavements, non-destructive testing and evaluation is the only viable method [4]. Non-destructive testing is the evaluation of the test object by implementation of technology to monitor the health and performance of the structure without affecting the integrity of the objects so that future use or performance can be continued. Its basic principle is to determine the integrity of the object without destroying it [5]. A typical structural health monitoring system using non-destructive testing method is made of three major components of: a sensing unit, data processing unit and a health evaluation technique [6] shown in Fig. 1. The sensing unit of the system

Fig. 1 General architecture of a SHM system

FEA Analysis on the Optimum Placement of Sensor …

71

Fig. 2 Cracks on surface of concrete road pavement on Beecroft Road, NSW

is made of selected sensors that evaluate the structure to measure desired parameters. The data processing unit includes the processes of acquiring data, transmitting the data, and storing the data. The process of data acquisition includes the use of analog to digital converters, timers, signal conditioning circuits and use of storage devices. Storage devices and analysis and monitoring tools are used to manage the data collected with a reliable communication network to transfer necessary data to be further analyzed. Lastly, the data transferred is managed to be used within the health evaluation technique consisting of diagnostic algorithms for the data to be evaluated and used to form a conclusion. Concrete pavements have been implemented globally over the past fifty years and commonly used for roads transporting heavy traffic such as trucks. Due to the heavy loading concrete pavements require high maintenance and construction. Majority of the time, damage not visible to the naked eye has occurred within the structure internally. By the time damage has propagated to the surface it is usually too late and can cause a huge safety hazard to the public. Figure 2 shows cracks propagating to the surface on Beecroft Road. These cracks are safety hazards for the public causing the surface to be uneven and unable for public to walk over safely. The need for monitoring the lifecycle of the pavement is essential to ensure to identify the early indications of damage and possible causes that can be avoided before huge damage is caused. Mechanistic design guides only consider fatigue cracking that is located either mid-edge or from bottom-up as failure of the pavement [7]. Pavement design guides consider vehicle loads as static loads though they have a huge effect on the tensile stress induced within the pavement. There is a need of further research to investigate into the relationship between the distresses and responses of the pavement to the heavy loads applied to it.

72

S. Mukhopadhyay and M. Asadnia

1.2 Motivation and Challenges of SHM Damage in any product and structure affects the lifespan, economic and user-safety of it. In natural disasters such as earthquakes no method exists to safety check if buildings post disaster is still safe [7]. The requirement to monitor the performance and health of the structure is vital as time progresses. Damage in concrete pavements can appear as cracks, potholes and bumps affecting both vehicles, pedestrians, and the overall safety of the community. Cracking in road pavements is one of the main causes of damage affecting the quality and transportation purpose of it. Concrete pavements are used in mainly two uses of construction being: the road or the airport. For the scope of the project, the type of concrete pavements focused on are highway roads. More than eighty countries around the world have highways covering a global length of approximately two hundred million kilometers. With the increase of traffic and external factors, the damage of pavement roads has become more severe with possibilities to further expand in the future. This will long term affect the lifespan, durability and bearing capacity of the concrete pavement as well as affect many safety factors to users such as the vehicle speed and safety, fuel usage in long term [8]. There are several purposes of taking and assessing the measurements of the health of the structure. Firstly, a deeper understanding of the behavior and health of the structure through boundary conditions and applying a range of loads can be gained. From this, a mathematical model and relationships can be derived with the load conditions and response [9]. Secondly, monitoring the performance of the structure over long periods of time will allow for explanations and identifying the cause of deterioration in terms of characteristics and properties. For example, change in stiffness due to external factors such as water or cracking. Thirdly, mechanistic models can be produced to improve and validate possible losses preventing damage and predicting in advance when damage occurs. The models can be compared to different areas and types of roads.

1.3 Research Objectives The main objective of the research is to design and develop a smart sensing technique for early detection of pavement damage. To achieve this, various sensing technologies of strain gauge, temperature, moisture, and pH sensors are to be investigated to develop a suitable sensor array that collects desired parameters for damage detection in concrete pavements. Further to this, the signal processing techniques of ground penetrating radar, microwave and ultrasound will be investigated to extract the information of the damage. Through the research a suitable method that is efficient in time, cost, and accuracy of data collected will be used for detection of concrete cracks at various depths. Alternative methods to supply power to the sensors will be investigated through the technique of energy harvesting. Transmission of the data signals

FEA Analysis on the Optimum Placement of Sensor …

73

will be investigated using antennas in which optimum size and configurations will be found for various depths of the material. The general objectives for the area of study are defined as follows: • To investigate types of sensing technologies for early damage detection in concrete pavements • To explore different types of signal processing techniques to extract information on the damage • To design and develop a smart sensing technique for early detection of pavement damage (passive wireless sensor most ideal choice) • To determine best operating frequency to deliver energy to passive wireless sensor within the pavement • To determine optimum dimensions and structure of the sensing system for 1dimensional and 2-dimensional profiling and crack detection in concrete.

2 Sensors for SHM To assess, monitor and gain a measurement of the structure’s behavior, a suitable design and sensor with correct calibration with redundant quantity of sensors implementation is required for a reliable and efficient reading. For early damage detection in concrete pavements, the following sensors of fiber optics, strain gauge, temperature, moisture, and pH have been used in the area and researched further.

2.1 Fiber Optics Fiber optic sensors are used in NDT for the measurement of mainly strain and temperature. The sensor is durable and usually made from plastic fibers or glass allowing light to guide across the length of it. There are four types of existing fiber optic sensors known as point, multiplex, long base, and distributed fiber optic sensors. All four measure on the parameters of wavelength, wave intensity and wave interference of light. Point sensors are fiber optic sensors that use a single input point to undertake measurements where else multiplex sensors take multiple input points. The long base sensors can undertake measurements for large distances and lastly, distributed sensors take a measurement at any location of the fiber [10]. For structural health monitoring in concrete roads, long-base and multiplex sensors known as Fabry– Perot have been commonly used [11]. Fabry–Perot fiber optic sensor works on the operating principle of using two optical fibers separated by an airgap in a capillary tube [12]. The advantages of use of fiber optic sensors for monitoring in concrete is its high durability, its performance is independent of the external conditions and does not get affected by electromagnetic interference [10]. The limitations of the use of fiber optic sensors are the high cost and requirement of knowledge individuals to conduct experiments and maintenance purposes.

74

S. Mukhopadhyay and M. Asadnia

2.2 Strain Gauge Strain gauge sensors are utilized in structural health monitoring systems for measuring the strain of the structure. Strain gauge sensors operate when tightly connected to the measured object. The contraction of the measured object affects the dimension of the sensor producing strain [13]. The sensor’s measured data shows the relationship between the changes of the strain of the measured object allowing to identify initiation of cracking and damage. For concrete pavements, strain gauges are used for measuring of surface deflections to analyze the layer stiffness further calculating the strain and stresses [10]. Strain gauge sensors are easy to install, have high sensitivity and inexpensive allowing to identify and analyze the data collected to detect damage of the structure. Generally, strain gauge sensors off the shelf are wireless with a very limited range or wired to the sensor restricting its performance and increasing the cost of the system [14]. For the application of monitoring the structural health of road pavements, it is not practical to have power cables on the road as it is both a safety hazard and inconvenience. Another limitation is the detection limit in terms of depth of the structure as most of the time a single sensor cannot detect all cracks bottom to top of the structure [15]. For concrete monitoring, a large quantity of strain gauge sensors at different depths of concrete is required to ensure all cracks at all depths are detected.

2.3 Temperature Sensor In concrete pavements, temperature stresses are developed, and curling can occur due to changes of temperature in different layers of the pavement. Temperature control in concrete is essential especially during the curing stage as it affects the overall mechanical strength and volume stability of the structure. For monitoring of concrete pavements, temperature sensors have been utilized and measure the heat of hydration of the structure [16]. The function of temperature sensors is to measure the temperature of the objects external environment and monitor the change of performance over time. The most common temperature sensors used for monitoring concrete pavements are thermocouples and fiber temperature detection sensors. However, both temperature sensors are wired which is impractical for the purpose of road pavement concrete monitoring and high in cost and maintenance. They have the advantages of wireless, low power dissipation and affordable cost with the operating diagram for concrete monitoring. The RFID is embedded within the concrete pavement with the reader receiving the temperature data collected from the sensor.

FEA Analysis on the Optimum Placement of Sensor …

75

2.4 Moisture and pH Sensor Concrete pavements are affected by changes in moisture variations due to the external environment. It is essential to track and monitor the moisture in concrete pavements as ground water table should always be below 1 m from the pavement surface [17]. Similarly, measuring the pH level of the concrete indicates the alkalinity level of concrete which affects the performance of concrete. Current research has shown the development of temperature and pH level sensors for concrete monitoring with the base principle of microcantilever [18]. The developed sensor monitors concrete temperature and pH level by using a beam which measure deflection when curvature occurs. It contains a resistor, and the moisture and pH level are determined based on the relationship between the calculated stress and water content.

2.5 MEMS Sensor Micro-electromechanical sensors (MEMS) technologies have grown over the recent years being a possible alternative to efficient and long-term monitoring of SHM for concrete pavements. MEMs are microfabricated miniature systems that are made of actuators, microsensors and integrated circuits [10]. The dimensions of MEMs can vary from 1 µ to 1 mm [11]. The extremely small size has a low producing cost and are onboard microprocessors. The features include analog–digital conversion, digital processing, and computing. The advantages of using MEMS in SHM is the smaller size is relatively low in cost while allowing for fast and energy efficient monitoring. For SHM, the implementation of MEMS allows wireless monitoring within an intelligent system.

3 Finite Element Analysis COMSOL Multiphysics has been used to build and develop the experimental model and perform simulations of the concrete pavement. The module of structural mechanics was used which utilizes finite element modelling (FEM). The concrete pavement modelled in figure is made of a concrete slab representing concrete road pavements. The material properties and dimensions have been taken following Australian Road Standards [19]. A boundary condition is included in the simulation which is the surface of where the external load is being applied representing loads such as vehicles and pedestrians. Physics controlled mesh generation of extremely fine mesh is applied to the surface and corners of the sloid, The finite element mesh is made of 138,764 elements, with 32,756 boundary elements, 2084 edge elements, 103,870 domain elements and 54 nodes shown below in Figs. 3, 4 and 5.

76

S. Mukhopadhyay and M. Asadnia

Fig. 3 FEA mesh applied around concrete surface

Fig. 4 FEA mesh applied around strain gauge sensor

Fig. 5 FEA mesh applied around pavement crack

4 Results and Discussion The simulation ran involving observing the effect of strain on the concrete slab when different external loads were applied. The external loads applied were of a range that Australian Roads are designed to handle. The concrete slabs tested involved having no cracks, vertical and horizontal cracks both symmetric and asymmetric of different depths in which the value received by the strain gauges was observed and analyzed. When a crack is present the value of strain measured by the two strain gauges should be significantly different. The value of strain caused by stress is determined by measuring a change in resistance. A strain gauge used in a bridge configuration with a voltage source is known as a Wheatstone Bridge. Wheatstone Bridge Circuit is used to measure the small change in resistance. To enhance the sensitivity of the system, two strain gauges are used. From the results, the optimum placement of the two strain gauges to efficiently detect the presence of a crack was found to be three meters shown below in Fig. 6. This means a crack present in any position between the two strain gauges can be detected with strain gauges placed every three meters on the road.

FEA Analysis on the Optimum Placement of Sensor …

77

Fig. 6 Optimum strain gauge placement of three meters

Fig. 7 Strain measured by embedded strain gauges when no crack present in concrete

Figure 7 shows the strain measured by the two strain gauges when embedded in concrete with no crack present. The results show there is no significant change in the strain values measured implying no crack present. Figure 8 shows several strain measurements received by the two strain gauges at different depths of cracks. The crack depths of 18 cm, 13 cm, 8 cm, and 3 cm were simulated as these are the standard crack depths present in roads. Figure 8 shows for all depths there is a substantial difference in the strain measurements between the two strain gauges implying the presence of cracks. Figure 9 shows simulated results of strain measured at the strain gauges when embedded in concrete with horizontal cracks of different depth. The results show an increase in horizontal crack depth showed a significant difference in the strain results. At lower depths the difference of strain was present but not as significant especially when low external loads were applied. From the results we observe the lower external loads applied show a smaller difference between the strain measured by the strain gauges compared to higher loads. This shows that when heavier loads such as heavy vehicles are present the

78

S. Mukhopadhyay and M. Asadnia

Fig. 8 Strain measured by embedded strain gauges with vertical cracks present in concrete

Fig. 9 Strain measured by embedded strain gauges with horizontal cracks present in concrete

strain gauges can effectively show a change in strain but may be a bit weak when low loads such as pedestrians are present.

5 Sensor Selection From the research conducted, the sensor needs to be compact and small to minimize interference with concrete at a low cost. It needs to be high in efficiency and reliability with a low noise level and power consumption. Furthermore, the use of cables and power supply on the road should be eliminated. For concrete pavement

FEA Analysis on the Optimum Placement of Sensor …

79

monitoring, four main parameters of strain of the structure, temperature, moisture, and pH level have been selected based on the factors that affect the design of the pavement. For the sensor options for in pavement monitoring, the use of strain gauge, temperature sensor, moisture sensor and pH level sensors have been decided. The main experimental sensor focus of the system will be the strain gauge. The components of the smart sensor device will be made of three main parts of the sensing element, an energy harvester, and a wireless transmitter. The sensing elements are the use of the in-pavement sensors to be embedded within the concrete for internal monitoring of the characteristics of the pavement. The energy harvester will be a transmitter to transmit signals for sensor to harvest energy avoiding use of cables and power supply. Lastly, it will use a planar-type radiator to receive high frequency energy and to radiate information. The sensing elements assessed experimentally in the project are sensors fitted to the surface of the pavement for monitoring of the characteristics of the pavement. Sensors embedded to the top of the surface are unrealistic in the application of a road due to safety hazards and vehicles but are useful in verification of theoretical and simulation results. The strain gauge of TAL220 is a straight bar load cell made from aluminium-alloy that has the capability to sense up to 10 kg of force and provides a proportional electrical signal [13]. The strain gauge measures the electrical resistance which changes proportionally to the strain applied to it. This sensor can indicate the presence of a load and provides information on the stress–strain relationship on the structure. The load cells contain four strain gauges that are connected in the configuration of a Wheatstone bridge.

6 Preliminary Experimental Results The experimental work of the projects consists of three main parts: creating the concrete samples, the testing and mounting of the sensors, and applying the sample under the appropriate loading testing. Three-point bending test and compression test were carried out to understand and analyse the output signals from the sensors in identifying form of cracks. Sensors have been fit to the top of the slab for testing purposes and verification of the process of deformation occurring in the pavement. The three-point bending test was performed and repeated three times with an external load starting from 0 increasing progressively to 5 kN within one minute to the sample as shown in Fig. 11. The sensor output was measured before application of the external load and monitored intermittently as the load was applied on the sample. Originally, it was expected the sample would not crack due to the external load being applied is only 5 kN. However, after repeating the experiment three times, the sample snapped at the middle of the slab into two almost identical pieces as shown in Fig. 10. The cause of crack can be due to several factors such as composition of the sample being only sand, cement and water which is very weak compared to containing aggregates. Substances like this can handle very heavy loads but sometimes as seen in the experimental cannot handle any higher than 5 kN suddenly breaking from

80

S. Mukhopadhyay and M. Asadnia

Fig. 10 Sample cracked under three-point bending test

Fig. 11 Flexural load applied to sample over time

the centre. The centre is the weakest point of the sample meaning for any flexural test we expect any concrete structure to start to deteriorate from the middle [16]. The experimental results showed the sample cracked suddenly in a very short time within any early warning. This showed the sensor was not able to record any abrupt change of signal prior to the crack occurring. This is a challenge which needs to be investigated further of the possibility for making usefulness of the sensing system to detect any sudden crack such as the one in the experiment. The output of the sensor during the flexural test is shown in Fig. 12. The data shows as the load increased the resistance also increased as the sample is placed under tension but with a minimal change. It is observed that the output resistance of the sensor does not indicate any abrupt change of resistance. Figure 13 shows as the strain applied to the sample increases, the change in resistance also increases

FEA Analysis on the Optimum Placement of Sensor …

81

Fig. 12 Output of sensor during flexural test

Fig. 13 Change of resistance during flexural test

though very minimally. The strain gauges located in the centre of the sample show maximum change of resistance indicating maximum load is detected in the centre. This further validates both theoretical principle and simulation conducted. The compressive strength test was repeated three times with an external load starting from 0 and progressing to 100 kN at a rate of 1 kN/s. The sample size utilized was 250 mm * 150 mm *150 mm from the previous three-point bending test. The external load was fit to the surface area of the sample compressing down progressively shown in Fig. 14. During the test cracking noises were heard but no

82

S. Mukhopadhyay and M. Asadnia

Fig. 14 Compressive load applied to sample over time

cracks were observed on the sample. Figure 15 shows the output of the sensor under compressive load. It is observed the resistance decreases though minimally for both strain sensors. This validates simulation results where a compressive load shows a change in resistance however, similar to the three-point bending test does not give any information of an occurrence of crack or damage.

Fig. 15 Output of strain sensors during compressive test

FEA Analysis on the Optimum Placement of Sensor …

83

7 Communication The traditional practice of concrete pavement monitoring in SHM is implementing wired sensors due to being readily available off the shelf. Wired sensors are high in cost, maintenance and have low survivability in the application of road pavement monitoring restricting the long-term operation of the system. There is a huge safety issue of the lengthy cables suspended to the public with high risk of damage. For large roads, many sensors are required further increasing the installation cost, time, and an increase in the number of data connection ports. Due to these limitations impacting the potential of the monitoring system, an alternative method is essential to be investigated. Internet of things (IOT) have not been heavily applied to SHM systems in the past, but recent research has shown the use and potential for a long-term and cost-effective solution to SHM in road pavements [11]. The implementation of Internet of things allows the use of multiple sensors without the installation, cost, and maintenance of wiring. It offers an efficient and economical communication technology for SHM including efficient power consumption, high capability of data processing and precision of analog-to-digital conversion [20]. IOT in SHM are responsible for the collection and storage of structural measurements from sensors, the wireless transfer communication of the data and lastly the evaluation of the data directly to cloud [21]. The architecture of a IOT system in the application of SHM for pavement monitoring is shown in Fig. 16. The IOT system is made of a sensor interface connected to the sensors interrogating the structure and converting the analog signal received to a digital output.

Fig. 16 General architecture of IOT based system for SHM

84

S. Mukhopadhyay and M. Asadnia

IOT systems require to act as both a sensing and actuation interface. Analog to digital conversion is done by the microcontroller with the data transmitted wirelessly requiring a radio transceiver [22]. For the application of SHM of concrete pavements the most used radio frequencies are 2.4 GHz and 915 MHz. For a reliable wireless communication, the use of spread spectrum wireless signals has been commonly applied [23]. The advantage of the system is that it allows SHM to be implemented in large infrastructures and complex access areas. The main limitation is supplying power for both the wireless communication and sensors [24]. IOT has been applied in SHM for not only concrete road monitoring, but bridges, railway and aircrafts with the research and implementation in the area continuously growing [25].

8 Future Works For monitoring on long pavement roads, many sensors should be arranged to form large sensor arrays. The sensor-array layers can be developed to avoid the use of connecting wires to reduce electromagnetic interference, allow for consistent arranging of many sensors and layers can be surface-mounted on existing structures or embedded as extra layers in composites during manufacturing [26]. The sensors can be developed in-house with moderate cost of $40 with an approximate size of 1 cm × 1 cm. The experiments with the sensors will include creating different concrete samples to investigate the sensors full depth limits. By taking measurements of both the sensor and antenna response at different angles and depths of the concrete, the number of sensors required, and the optimum placements of the sensors can be estimated. For future internal concrete pavement monitoring the strain gauge of Geoken Model 4200 which is a 6-inch static strain sensor for concrete embedment can be used [27]. It contains an internal thermistor which can measure the temperature of the surrounding environment [28]. This sensor was specifically used and chosen due to its characteristics specifically made to be efficient in the concrete environment monitoring both strain and temperature. When embedded in the concrete sample, a change in strain will move the metal blocks relative to each other with the tension formed determining the vibration frequency of the steel wire. It is made of stainless steel in which it is corrosion free, water resistant and has a long lifespan.

9 Conclusion The simulated tests performed show the detection of presence of cracks in concrete road pavements with the use of strain gauges. The simulated results can be further compared with future experimental results and allow for a deep understanding in the feasibility of the project. The results show the application is promising but will face challenges that will need to be overcome for wide use. The advantage of simulation is the large number of experiments and different type of variables that can be simulated

FEA Analysis on the Optimum Placement of Sensor …

85

without the effect of cost or materials. The results and method of tests can be used to find the optimum size, number, and placement for different sensors for SHM and build on the knowledge of the behavior of concrete to prevent from damage. From the research and experimental work performed through the duration of the project, structural health monitoring is a promising field for the future. There are several challenges to be addressed for the application of roads, however with close collaboration of civil engineers it can be overcome to educate how sensing monitoring systems are beneficial for civil structures. Collaboration with civil engineers can lead to monitoring in applications wider than roads such as railways, buildings and more. Currently being an overhead wiring engineer, it has been observed the importance and potential for continuous monitoring of infrastructure to avoid unexpected damage directly affecting the wider community. Further work must be done in the testing of embedding sensors to ensure the lifespan and operation lasts once embedded within the structure. Though experimental work was conducted with the use of surface fitted sensor, further tests must be conducted with use of a Compression testing machine ranging of 300–3000 kN. Further analysis is required to distinguish the initiation of cracks and deterioration from the output signals of the sensors. Providing energy to the sensing elements must be done using an energy harvester and implementation of wireless communication to make the system smart.

References 1. Farrar, C.R., Worden, K.: An introduction to structural health monitoring. Philos. Trans. Royal Soc. A: Math. Phys. Eng. Sci. 365(1851), 303–315 (2007) 2. Frangopol, D.M., Liu, M.: Maintenance and management of civil infrastructure based on condition, safety, optimization, and life-cycle cost∗. Struct. Infrastruct. Eng. 3(1), 29–41 (2007) 3. Paul, S., Jafari, R.: Recent advances in intelligent-based structural health monitoring of civil structures. Adv. Sci. Technol. Eng. Syst. J. 3(5), 339–353 (2018). https://doi.org/10.25046/aj0 30540 4. Ansari, F.: JCSHM: invaluable resource for practice of civil structural health monitoring. J. Civil Struct. Health Monit. 8(1), 1–1 (2018). https://doi.org/10.1007/s13349-018-0271-x 5. Raj, B., Jayakumar, T., Thavasimuthu, M.: Practical Non-destructive Testing. Woodhead Publishing (2002) 6. McCann, D., Forde, M.: Review of NDT methods in the assessment of concrete and masonry structures. NDT and E Int. 34(2), 71–84 (2001) 7. Shull, P.J.: Nondestructive Evaluation: Theory, Techniques, and Applications. CRC Press (2016) 8. Karbhari, V.M., Zhao, L.: Use of composites for 21st century civil infrastructure. Comput. Methods Appl. Mech. Eng. 185(2), 433–454 (2000) 9. Domone, P., Illston, J.: Construction Materials: Their Nature and Behaviour. CRC Press (2010) 10. Wang, M.L, Lynch, J.P., Sohn, H.: Sensor Technologies for Civil Infrastructures: Sensing Hardware and Data Collection Methods for Performance Assessment. Elsevier (2014) 11. Scott, R. et al.: Development of low cost packaged fibre optic sensors for use in reinforced concrete structures. Measurement 135, 617–624 (2019). https://doi.org/10.1016/j.mea surement.2018.11.056 12. Afzal, M., Kabir, S., Sidek, O.: An in-depth review: structural health monitoring using fiber optic sensor. IETE Techn. Rev. 29(2), 105 (2012). https://doi.org/10.4103/0256-4602.95383. Accessed 4 April 2021

86

S. Mukhopadhyay and M. Asadnia

13. Cdn.sparkfun.com (2021). https://cdn.sparkfun.com/datasheets/Sensors/ForceFlex/TAL220 M4M5Update.pdf. Accessed 12 Oct 2021 14. "Strain Gauge Working Principle—Inst Tools”, Inst Tools (2021). https://instrumentationtools. com/strain-gauge-working-principle/. Accessed 05 Apr 2021 15. Alshandah, M., Huang, Y., Gao, J., Lu, P., Tolliver, D.: Experimental crack detection in concrete pavement using point strain sensors. Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2019 (2019). https://doi.org/10.1117/12.2513680. Accessed 5 March 2021 16. Alshandah, M., Huang, Y., Gao, Z., Lu, P.: Internal crack detection in concrete pavement using discrete strain sensors. J. Civil Struct. Health Monit. 10(2), 345–356 (2020). https://doi.org/ 10.1007/s13349-020-00388-2 17. Temperature Sensors: Types, How It Works, & Applications. Encardio Rite (2021) 18. Liu, Y., Deng, F., He, Y., Li, B., Liang, Z., Zhou, S.: Novel concrete temperature monitoring method based on an embedded passive RFID sensor tag. Sensors 17(7), 1463 (2017). https:// doi.org/10.3390/s17071463. Accessed 5 April 2021 19. Hou, X., Zheng, W.: Experiment and analysis on temperature distribution of prestressed concrete beams and slabs. In: Proceedings of the 2011 International Conference on Consumer Electronics, Communications and Networks (CECNet), Xianning, China, 16–18 April 2011; pp. 3156–3159 20. Blitz, J.: Electrical and Magnetic Methods of Non-Destructive Testing. Springer Science & Business Media (2012) 21. Abdulkarem, M., Samsudin, K., Rokhani, F., Rasid, M.A.: Wireless sensor network for structural health monitoring: a contemporary review of technologies, challenges, and future direction. Struct. Health Monit. 19(3), 693–735 (2019). https://doi.org/10.1177/147592171985 4528 22. Lan, C., Liu, W.: Structural health monitoring cloud and its applications for large-scale infrastructures App. Mech. Mater. 330, 418–425 (2013). https://doi.org/10.4028/www.scientific.net/ amm.330.418 23. Yi, T.H., Li, H.N.: Methodology developments in sensor placement for health monitoring of civil infrastructures. Int. J. Distrib. Sens. Netw. 8(8), 612–726 (2012) 24. Avci, O., Abdeljaber, O., Kiranyaz, S., et al.: Wireless and real-time structural damage detection: a novel decentralized method for wireless sensor networks. J. Sound Vibrat 424, 158–172 (2018) 25. Moreu, F., Kim, R.E., Spencer, B.F. et al.: Railroad bridge monitoring using wireless smart sensors. Struct. Contr. Health Monitor 24(2), e1863 (2017) 26. Liu, X., Cao, J., Song, W.Z., et al.: Distributed sensing for high-quality structural health monitoring using WSNs. IEEE Trans. Paral. Distr. Syst. 26(3), 738–747 (2015) 27. Guide to Pavement Technology. Austroads.com.au (2021). https://austroads.com.au/infrastru cture/pavements/guide-to-pavement-technology. Accessed 07 Mar 2021 28. Geokon.com, 2021. [Online] Available:https://www.geokon.com/content/datasheets/4200Se ries_Strain_Gages.pdf. Accessed 1 May 2021

Fusion of Radar Data Domains for Human Activity Recognition in Assisted Living Julien Le Kernec , Francesco Fioranelli , Olivier Romain , and Alexandre Bordat

Abstract Radar has long been considered an important technology for indoor monitoring and assisted living. As ageing has become a worldwide problem, it causes a huge burden on the government’s healthcare expenses and infrastructure. Radar-based human activity recognition (HAR) is foreseen to become a widespread sensing modality for health monitoring at home. Conventional radar-based HAR task usually adopts the amplitude of spectrograms as input to a convolutional neural network (CNN), which can limit the achieved performances. A hybrid fusion model is here proposed, which can integrate multiple radar data domains. The result shows that the proposed framework can achieve superior classification accuracy of 92.1% (+2.5% higher than conventional CNN) and a lighter computational load than the state-of-the-art techniques with 3D-CNN. Keywords Radar · Human activity recognition · Fusion · Machine learning

1 Introduction The growing population worldwide causes a huge burden on the government’s healthcare expenses and health infrastructure. Radar has long been considered an important technology for indoor monitoring and fall detection in assisted living. Compared with other competing technologies such as camera monitoring or ultrasonic sounding,

J. Le Kernec (B) James Watt School of Engineering, University of Glasgow, Glasgow, UK e-mail: [email protected] F. Fioranelli Department of Microelectronics, TU Delft, Delft, The Netherlands O. Romain · A. Bordat ETIS Lab, CY University, Cergy-Pontoise, France e-mail: [email protected] A. Bordat e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_7

87

88

J. Le Kernec et al.

radar has the advantages of non-intrusive sensing, insensitivity to lighting conditions, privacy preservation [1], and safety. These make radar very attractive in fields like human–machine interface, site surveillance and assisted living [2]. Radar data domains include but are not limited to raw data, range-time (RT), range-Doppler (RD), Doppler-time (DT) [3]. Traditional radar-based human activity recognition (HAR) focuses on a single radar domain, usually Doppler-time, and typically adopts a single-input network structure. Although this method can achieve a good accuracy, when activities are close to each other in Doppler-time representations, such as falling and tying a shoe lace, or if the signatures are different with elderly and young people in the training/testing data samples, the performance can significantly be impacted. Furthermore, only the amplitude is exploited and not the phase information, and other data domains can help distinguish activities with range spread for example.

2 Literature Review 2.1 Radar-Based Human Activities Recognition Radar can be used in indoor monitoring largely because of the progress of machine learning technology and the speed of graphic processing units [3]. Based on those technological advances, the use of deep neural networks (DNN) is now feasible for radar-based HAR. Usually, a set of handcrafted features from the micro-Doppler signature will be extracted from the radar signal, such as the Doppler bandwidth, or the centroid (torso frequency) for the DT domain. Then, statistical learning techniques, like support vector machine (SVM) [4] and random forest classifier [5] are used for classification. The DT signature is commonly used in radar-based HAR. In [4], the authors have classified human activities using SVM and a set of handcrafted features extracted form radar DT signatures. However, the effectiveness of this approach is largely dependent on some operational and situational factors [6], such as the transmit and pulse-repetition frequencies, dwell time and signal-to-noise ratio. Researchers gradually shifted their focus from statistical learning to deep learning, which extracts features automatically and performs classification simultaneously. In [7], the authors investigated the feasibility of using convolutional neural network (CNN) to categorise the radar DT signatures. Considering that the training of deep learning network requires a large amount of data samples, at the same time, radar signal processing and computer image processing often have strong similarities. Craley et al. [8] realised this problem and used transfer learning to pre-train the CNN aquatic activity classification to improve accuracy. In [9] a sparse auto-encoder was proposed to process the radar DT and RT signatures in parallel. Besides, [10, 11], both developed a hybrid network structure (convolutional and recurrent) to achieve a better recognition accuracy.

Fusion of Radar Data Domains for Human Activity …

89

However, all of the studies mentioned above have limited their scope to twodimensional domains and only combined the disjoint features inside the classifier. There are two innovative methods [12, 13] which integrate radar data from 2D multichannels into a 3D single channel. Both of them intend to transform the radar echoes into a 3D range-Doppler time (RDT) signature. But their RDT signatures lack sufficient resolution to extract sufficient information from micro-motions. To resolve this limitation, a geometric deep learning method based on the points formed by micromotion signatures was proposed in [14]. This method obtained a higher performance in both classification accuracy and noise robustness.

2.2 Multi-domain/Multi-modal Fusion Our experience of the world is multimodal, we see objects, through our five senses [15]. In general, “modal” refers to the way things happen or exist, and a research problem is characterised as multimodal when it includes multiple modalities. For the purpose of enabling the artificial intelligence to understand the world around it, we need to teach them to observe and to interpret the multimodal information like a human being. Recent research works are mainly dealing with sound, image and text multi-modal learning in such applications as speech recognition (audio + image) with challenges related to: • • • •

Representation—to find some unified representation of multi-modal information, Translation [16]—mapping one typical modality to another modality, Alignment [17]—finding the relationships between the modal sub-components, Fusion—obtaining more cross-features by integrating the multi-modal information, • Co-learning—use information-rich modalities to assist information-poor modalities. To sum up, there are mainly two advantages for implementing multimodal fusion into machine learning. 1.

2.

Information complementarity: Multi-modal fusion can complete the missing information of single mode to ensure the integrity of information to improve the model performance. Information crossing: Multi-modal fusion can fully explore the information interaction between different modalities, so even more abundant feature information can be obtained through fusion.

Modal fusion can be divided into four categories. They occur at different stages of the network training: • Signal/Pixel level fusion is the most intuitive fusion method. The data is preprocessed before entering the machine learning network and therefore, can fuse

90

J. Le Kernec et al.

the smallest particle of data. Some related research that implement pixel level fusion are [18–20]. • Feature level fusion, for instance [21, 22], includes Early and Late Fusion. These two fusion types occurr at different stages consists in the fusion of features extracted from the machine learning network. The difference is that early fusion refers to the fusion of features through concatenation, element-wise sum, elementwise average and then input to the full connection layer for processing, while late fusion mainly occurs in the full connected layer. Two advantages of fusing data at the feature level are that it can fully exploit the cross information between different modalities and also improve the robustness of the network. • Decision level, where the output of the classifier is combined to make a final decision. Since the output of different classifiers corresponds to different modalities, the results of different classifiers tend to have strong independence. Therefore, some problems of misclassification caused by the shortcoming of a single modality can be avoided. Common decision level fusion methods include maxfusion, average-fusion, Bayes’ rule based fusion and ensemble learning, among others [23]. • Hybrid combination of the three aforementioned fusion methods, so as to combine the advantages of different fusion methods. Since each single 2D radar data domain can provide supplementary information for other domains, recent studies have combined representationsin radar-based HAR. In [24], the authors have proposed an innovative architecture which implements a stacked auto-encoder (SAE) to fuse the multi-dimensional data at the feature level. In [15], the idea of pixel-level fusion to process the radar signal into a 3D point cloud was adopted. Combined with the PointNet [25, 26], they have achieved enhanced performance in HAR. Despite those existing improvements in radar-based HAR, the recognition ability can still be improved further by fully utilising the radar data domains. In this paper, we propose a novel multimodal fusion framework fusing radar information during network training and show improved performances compared to the state of the art.

3 Methodology We used the University of Glasgow (UoG) Human Activities dataset [27, 28] to validate our proposed network structure to benchmark performances against existing research. All data samples were collected from 72 volunteers (23 females, 49 males) aged between 25 and 98 years, and different locations from lab spaces to retirement homes. Each volunteer was asked to perform six activities: walking back and forth (A01), sitting down (A02), standing up (A03), picking up an object (A04), drinking water (A05), and simulated frontal fall (A06). The data was captured with a FMCW

Fusion of Radar Data Domains for Human Activity …

91

radar operating at 5800 MHz with 400 MHz instantaneous bandwidth, a pulse repetition frequency of 1 kHz, a transmitted power of 100 mW and Yagi antennas for transmit and receive gain of 17 dBi. Radar data domains include RT, DT, RD. Traditional radar-based HAR usually only retains the magnitude information and discards the phase information, i.e. only the magnitude of the information is finally used for the network training. From the prospective of physics, any slight motion of the target will give the radar echo a micro-Doppler shift. Taking the range-time phase information as a source of network input will undoubtedly increase the computation burden and complexity of our system, and therefore reduces the versatility. This is also one of the primary reasons why few researchers have utilised the phase information of radar signals to train the network. Inspired by the mechanism of “Attention”, we put forward a method of magnitude masking to combine the range information and phase information together. The core steps of this algorithm can be summarized as follows: Phase Unwrapping—One important thing that needs to be mentioned is that, during the process of phase information extraction, all phase information is automatically encapsulated in the range of [−π ; π ], which greatly limits the information continuity (as time is a continuous variable). With such a method, we can eliminate the phase discontinuity and protect the time-varying phase information. Threshold Filtering—The threshold allows to only focus on strong signals of interest and discard the noisy parts of the radar data domain representations below this threshold by setting them to 0. This attention-based magnitude masking is also used to identify the region of interest in the phase data for the RT data domain. The essence of this operation is to perform a pixel-level fusion on the phase and magnitude of RT domain. In addition, through threshold filtering, we only retain the most critical information in the RT representation. Therefore, during the fusion, we can give more weight to the important information in the phase map, so that our network can pay more “attention” in learning the information of the emphasised part during network training. Figure 1 summarizes the flow chart of the data preprocessing for the RT and DT domains as discussed in this section.

4 Network Training 4.1 Overall Network Structure Our network can be divided into 3 parts as shown in Fig. 2. It is composed of:

92

J. Le Kernec et al.

Fig. 1 The extraction process of the network input from the radar data

Fig. 2 Overall network structure

• A convolutional layer—this CNN consists of three structures, which are convolution, activation, and pooling. Together, they can automatically extract the feature map from the input data. • A fusion layer—it implements a feature-level fusion with typical modal fusion element (e.g. element_wise sum, element_wise multiplication, concatenation) as shown in Fig. 3. • A fully connected layer—it completes the mapping from feature information to label set (e.g. traditional fully connected layer, weight-shared fully connected layer). Our network has two inputs, each connected to a series of symmetric and identical convolution layers. In the actual operation, the processed RT and DT signatures are taken as the two inputs of the network respectively. Since high data resolution will burden our network and cause unnecessary waste of computation resources, both our input data are down-sampled to a 224 × 224 input image.

Fusion of Radar Data Domains for Human Activity …

93

Fig. 3 Four Fusion Elements

Firstly, the feature information is extracted by a series of 2D convolution and max-pooling layers. The output are the global features (with a size of 1 × 1024) each for RT and DT signatures. The next step is fusion.

4.2 Fusion Layer After obtaining the global features, we implement 4 different feature-level fusion elements to fully explore the capabilities of modal fusion, for instance, element-wise sum, element-wise product, element-wise average and concatenation as shown in Fig. 3. In most cases, multimodal feature-level fusion occurs just after the convolution layer. But in this project, we also tried to propose a fusion mechanism in the full connection layer (FCL), called deep fusion. As shown in Fig. 4, deep fusion is combined by three pairs of shared-weight fully connected layers and four fusion elements. In practice, those two structures alternate in the FCL of deep fusion, which enable more interactions among two sets of features by feeding back the average error in the fusion network to both branches of our fusion network.

94

J. Le Kernec et al.

Fig. 4 Two FCL structures

5 Results In this section, the experiment parameter settings and results of our proposed methodologies will be presented.

5.1 Dataset Separation The UoG dataset contains over 1700 radar signatures of six human activities. Among them, 1736 data sample were randomly picked to form two training sets and 192 samples divided into three separate validation sets. What needs to be stated here, is that those three validation sets will not participate in the network training, that is to say, we can use them to verify the effectiveness of the network we have trained. Young/Old Separation—Our dataset is relatively small compared to the huge data sets adopted in other deep learning tasks. Besides, the data set implemented in this project also contains a certain number of data samples collected from older people. Typically, there are certain differences between the elderly and the young participants in motion, posture and speed. For example, some elderly people need to use crutches, a cane or a walker for assistance in walking, which resulted in a few abnormal data samples for the elderly in our dataset. Based on the above situation, we infer that compared with the data set of the young volunteers, the data set of the elderly is more likely to have anomalies. Therefore, if we simply mix the data sets of the elderly and the young together, and then pack them randomly to form a validation set and a training set, we will not be able to guarantee that those abnormal data samples can be evenly distributed for the validation and training sets.

Fusion of Radar Data Domains for Human Activity …

95

Fig. 5 The data separation procedure

Once these abnormal data samples are concentrated in a certain set, the stability and robustness of our proposed model will be inevitably affected. To solve such a problem, we manually divided the UoG dataset into Young (below 65) and Old (above 65) groups according to the samples’ annotation. After that, we can randomly pick data from those two sets of data and compose a new data set. This separation process can be visualised as shown in Fig. 5. The proportion of people in the two age stages is evenly distributed in every one of the data sets. That is to say, in any training set or validation set, the proportion of samples of the elderly and the young is the same. This approach not only avoids the problems of system instability, but also allows our network to have a more comprehensive understanding of the movement characteristics of the elderly and the young people.

5.2 Data Preparation The signal processing consists of a 128-point Fast Fourier Transform (FFT) per sweep (1 ms) to obtain the range profiles from the raw I/Q data with a Hamming window. A 4th order Butterworth moving target indicator is implemented to remove static clutter. Then, 300 range profiles are accumulated to perform a zero-padded 1200point FFT for every range bin in the slow-time direction to obtain a range-Doppler map with an overlap factor of 0.90. After getting the RT and DT signatures, the threshold filtering and attention-based magnitude masking is applied with thresholding factor set at 0.6 of the maximum

96

J. Le Kernec et al.

value of the signatures. And finally, those processed data were resized to 224 × 224 and packed into 5 independent data sets which include three validation sets and two training sets.

5.3 Training Detail In the following sub-sections we will cover the details of network training, which includes the network parameter settings, comparison between different fusion elements, analysis toward the classification result and robustness analysis. Network Parameter Settings: Our network was trained with a batch size of 32 for 50 epochs. To make our network converge faster in the early stage and be more stable in the later training stage, we have introduced a dynamic learning rate into the training process. This dynamic learning rate will decay exponentially from 0.01 to 0.00001, and has a decay rate of 0.7 and decay step of 200,000. Fusion Element Analysis: In order to fully explore the influence of different fusion methods on the final classification accuracy of our network, we have implemented the four different fusion elements and demonstrate the evaluation results and the confusion matrices in Table 1 and Fig. 6. Similarity—The four fusion models have high recognition accuracy for A01, A02, A03 and A06. However, for A05 and A06, although our model improves the recognition accuracy of A05 and A06 compared with ordinary CNN networks, there is Table 1 Human activity classification accuracy using different fusion models Avg acc (%)

Avg A01 (%) class acc (%)

A02 (%)

A03 (%)

A04 (%)

A05 (%)

A06 (%)

Element-wise average

92.1

91.6

100.0

94.6

96.9

85.7

72.1

100.0

Element-wise product

88.5

90.5

100.0

91.9

96.8

85.2

69.6

100.0

Element-wise sum

91.4

91.3

100.0

97.3

96.8

67.8

86.0

100.0

Concatenation

89.1

87.8

94.7

96.9

91.8

74.2

81.25

100.0

Element-wise Average + Deep Fusion

92.2

92.7

100.0

97.1

91.7

75.8

91.4

100.0

PointNet [24]

92.2

92.8

96.6

96.7

86.7

83.3

83.3

100.0

CNN (no fusion applied)

89.6

90.4

97.2

97.1

88.9

93.3

65.7

100.0

Fusion of Radar Data Domains for Human Activity …

97

Fig. 6 Confusion matrices using four different fusion models

still an obvious confusion. This is probably caused by the high similarity between the picking up objects and drinking water activities. Difference Analysis—By comparing the above four basic fusion methods, element-wise average and element-wise sum have relatively higher classification accuracy compared to element-wise product and concatenation. On the one hand, this is because the two methods are very similar, and on the other hand, the two methods allow for deeper integration of radar multimodal data, and therefore can achieve higher recognition accuracy. But for method like concatenation, which leads to a high dimension intermediate layer. Such operation will not only increase the computation complexity but also decrease the interactions between two independent radar modalities. That is why it has the worst recognition accuracy. Since element-wise average has the best performance (92.1%), we have introduced a hybrid fusion model which can combine the element-wise average and deep fusion (Fig. 4) together. This method, as illustrated, yields quite good results, especially for identifying A01, A02, and A06. However, this method has a drawback, the weight

98

J. Le Kernec et al.

sharing of the fully connected layer will increase the computational load for network training (twice the computation complexity). Element-wise average seems to be a better choice as the performance decrease is only 0.1% with half the required computation. In addition, we also tried the method of extracting point clouds from RD sequence inspired from [15], and then do the classification on the point cloud using PointNet [25] and reported in [26]. We use the results of this experiment as a benchmark to compare with our proposed method. Compared with PointNet, our multimodal fusion method achieves similar recognition accuracy with element-wise average with fewer calculations.

6 Conclusion This paper investigated a method of radar multimodal fusion, which can enable CNN to improve the recognition of six kinds of human activities. In the data pre-processing stage, we put forward a data pre-processing pipeline, which can integrate the phase and range information at the pixel level and reduce the unwanted noise in the raw radar data to a certain extent. We believe that this method can effectively reduce the computing burden of the system and improve the robustness of the system at the same time. During the network training, we adopted four different fusion elements and two different FCL structures. Compared to traditional single-input CNN (89.6%), our fusion network has stronger HAR capability, especially when using element-wise average method (92% accuracy) to do the feature-level fusion. We have shown that we can achieve similar performances to more complex deep learning methods [26] with a lighter implementation by exploiting element-wise average feature-level fusion. The light implementation is in part achieved thanks to the reduced the number of network inputs through’Attention’ fusion. The next level of classification algorithms will need to integrate the complex form as a native data format as input as exemplified in [29] for example. The exploitation of the phase information has been shown in [26] to accelerate convergence in training as well as improving performances. Acknowledgements The authors would like to thank the British Council 515095884 and Campus France 44764WK—PHC Alliance France-UK, and PHC Cai Yuanpei—41457UK for their financial support. The authors would also like to give special thanks to Mr. J.G. for his valuable contribution to the elaboration of this article.

Fusion of Radar Data Domains for Human Activity …

99

References 1. Li, X., He, Y., Jing, X.: A survey of deep learning-based human activity recognition in radar. Remote Sens. 11(9), 1068 (2019) 2. Clemente, C., Balleri, A., Woodbridge, K., Soraghan, J.J.: Developments in target microdoppler signatures analysis: radar imaging, ultrasound and through-the-wall radar. EURASIP J. Adv. Sig. Process. 2013(1), 1–18 (2013) 3. Gurbuz, S.Z., Amin, M.G.: Radar-based human-motion recognition with deep learning: promising applications for indoor monitoring. IEEE Sig. Process. Mag. 36(4), 16–28 (2019) 4. Kim, Y., Ling, H.: Human activity classification based on micro-doppler signatures using a support vector machine. IEEE Trans. Geosci. Remote Sens. 47(5), 1328–1337 (2009) 5. Smith, K.A., Csech, C., Murdoch, D., Shaker, G.: Gesture recognition using mm-wave sensor for human-car interface. IEEE Sens. Lett. 2(2), 1–4 (2018) 6. Gürbüz, S.Z., Erol, B., Cagliyan, B., Tekeli, B.: Operational assessment and adaptive selection of micro-doppler features. IET Radar, Sonar Navigat. 9(9), 1196–1204 (2015) 7. Kim, Y., Moon, T.: Human detection and activity classification based on microdoppler signatures using deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 13(1), 8–12 (2015) 8. Craley, J., Murray, T.S., Mendat, D.R., Andreou, A.G.: Action recognition using micro-Doppler signatures and a recurrent neural network. In: 2017 51st Annual Conference on Information Sciences and Systems (CISS), pp. 1–5. IEEE (2017) 9. Jokanovi´c, B., Amin, M.: Fall detection using deep learning in range-doppler radars. IEEE Trans. Aerosp. Electron. Syst. 54(1), 180–189 (2018) 10. Wang, S., Song, J., Lien, J., Poupyrev, I., Hilliges, O.: Interacting with soli: exploring finegrained dynamic gesture recognition in the radio-frequency spectrum. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pp. 851–860 (2016) 11. Wang, M., Zhang, Y.D., Cui, G.: Human motion recognition exploiting radar with stacked recurrent neural network. Digit. Sig. Process. 87, 125–131 (2019) 12. Erol, B., Amin, M.G.: Radar data cube analysis for fall detection. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2446–2450. IEEE (2018) 13. Erol, B., Amin, M.G.: Radar data cube processing for human activity recognition using multisubspace learning. IEEE Trans. Aerosp. Electron. Syst. 55(6), 3617–3628 (2019) 14. Du, H., Jin, T., Song, Y., Dai, Y., Li, M.: A three-dimensional deep learning framework for human behavior analysis using range-doppler time points. IEEE Geosci. Remote Sens. Lett. 17(4), 611–615 (2019) 15. Baltrušaitis, T., Ahuja, C., Morency, L.-P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 423–443 (2018) 16. Hunt, A.J., Black, A.W.: Unit selection in a concatenative speech synthesis system using a large speech database. In: 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, vol. 1, pp. 373–376. IEEE (1996) 17. Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015) 18. Li, S., Kang, X., Fang, L., Hu, J., Yin, H.: Pixel-level image fusion: a survey of the state of the art. Inform. Fus. 33, 100–112 (2017) 19. Rockinger, O.: Pixel-level fusion of image sequences using wavelet frames. In: Proceedings of 16th Leeds Annual Statistical Research Workshop, Citeseer, pp. 149–154 (1996) 20. Naidu, V., Raol, J.R.: Pixel-level image fusion using wavelets and principal component analysis. Def. Sci. J. 58(3), 338 (2008) 21. Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3d object detection network for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1907–1915 (2017)

100

J. Le Kernec et al.

22. Kor, S., Tiwary, U.: Feature level fusion of multimodal medical images in lifting wavelet transform domain. In: The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 1, pp. 1479–1482. IEEE (2004) 23. Li, H., Mehul, A., Le Kernec, J., Gurbuz, S.Z., Fioranelli, F.: Sequential human gait classification with distributed radar sensor fusion. IEEE Sens. J. 21(6), 7590–7603 24. Jokanovic, B., Amin, M., Ahmad, F.: Radar fall motion detection using deep learning. In: 2016 IEEE Radar Conference (RadarConf), pp. 1–6. IEEE (2016) 25. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017) 26. Guo, J., Shu, C., Zhou, Y., Wang, K., Fioranelli, F., Romain, O., Le Kernec, J.: Complex fieldbased fusion network for human activities classification with radar. In: IET International Radar Conference 2020, Chongqing City, China, 4–6 November 2020, pp. 68–73 27. Fioranelli, F., Shah, S.A., Li, H., Shrestha, A., Yang, S., Le Kernec, J.: Radar signatures of human activities (2019) 28. Fioranelli, F., Shah, S.A., Li, H., Shrestha, A., Yang, S., Le Kernec, J.: Radar sensing for healthcare. Electron. Lett. 55(19), 1022-10 29. Gao, J., Deng, B., Qin, Y., Wang, H., Li, X.: Enhanced radar imaging using a complex-valued convolutional neural network. IEEE Geosci. Remote Sens. Lett. 16(1), 35–39 (2019)

Statistical Performance Analysis of Radar-Based Vital-Sign Processing Techniques Gabriel Beltrão , Mohammad Alaee-Kerahroodi, Udo Schroeder, Dimitri Tatarinov, and M. R. Bhavani Shankar

Abstract Radar-based vital-sign monitoring provides several advantages over standard methodologies. Despite the huge amount of recent work, the preference for particular technique(s) is in debt, due to lack of a formal comparison between them. In addition, collection of real data is a time-consuming process and therefore most of the proposed solutions are only evaluated under very limited scenarios. In this paper we present a simulation framework and a selection of results which allow easy performance comparison between radar-based vital-sign processing techniques. The proposed simulation tool scans over multiple breathing and heartbeat frequencies, and the combined effects along the entire signal processing chain can be analyzed, for different combinations of scenarios and techniques. The results have shown specific limitations for each method, thus indicating a need for proper selection based on operating conditions. In addition, while breathing estimation performance is only limited by noise, heartbeat estimation is limited by the presence of breathing harmonics and, despite promising results at specific breathing/heartbeat frequencies, the presented methods fail to fully mitigate this type of interference in all scenarios. Keywords Radar · Vital-signs · Breathing · Heartbeat · Signal processing

This work was supported by the National Research Fund of Luxembourg, under the FNR Industrial Fellowship Grant for Ph.D. projects (reference 14269859). G. Beltrão (B) · M. Alaee-Kerahroodi · M. R. Bhavani Shankar SnT—Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg, Luxembourg e-mail: [email protected] URL: https://wwwfr.uni.lu/snt U. Schroeder · D. Tatarinov IEE S.A., Bissen, Luxembourg © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_8

101

102

G. Beltrão et al.

1 Introduction Radar-based vital-sign monitoring provides several advantages over standard devices. Radar signals can penetrate through different materials and are not affected by the skin pigmentation or ambient light levels. In addition, radar devices are low-power, low-cost and privacy preserving [17]. These intrinsic characteristics have attracted the attention of the research community and, in line with recent technological advancements [4], a multitude of radar types are being used to address different applications such as sleep monitoring [16], rescue [9], in-car monitoring [13], and many others. To comply with these different scenarios, a wide range of radar frequencies have already been used. While using a lower frequency improves electromagnetic penetration and allows easier extraction of the displacement signal, higher frequencies result in better phase sensitivity and increased target reflectivity [17]. However, the signal processing framework needs to be adjusted to each scenario, and setting up correct algorithms and parameters for each application therefore remains a complicated task. Despite the huge amount of recent work, there is still a lack of a formal comparison between the basic vital-sign processing techniques. Particularly, given that each new proposal is evaluated in a very specific scenario, it is difficult to compare their performance most of the time, and to address strengths and limitations of each technique. In addition, collection of real data is a time-consuming process and therefore most of the proposed solutions are only evaluated under limited conditions. In this paper we aim to fill this gap by presenting a simulation framework and representative results which allow easy comparison between radar-based vital-sign processing techniques. The objective is to go beyond an isolated theoretical analysis, and understand the combined effects along the entire signal processing chain, for different combinations of scenarios, techniques and related parameters. The remainder of this paper is organized as follows. In Sect. 2, we introduce the signal modeling for vital-sign processing using radars. The simulation framework is presented in Sect. 3, while Sect. 4 presents our simulation results. Finally, in Sect. 5, some conclusions are drawn.

2 Vital-Sign Models The transmitted radar signal is modulated by the subtle chest-wall movements due to the breathing and heartbeat mechanisms, and is reflected with additional phase information related to this movement. Under ideal conditions, this time-varying phase can be written as 4πdb (t) 4πdh (t) + , (1) Δθ(t) = λ λ where λ is the operating wavelength, and db (t) and dh (t) represent the displacement signal associated to the periodic chest-wall movement due to the breathing and heartbeat, respectively (different periods). While this signal is directly received from

Statistical Performance Analysis of Radar-Based Vital-Sign . . .

103

the radar’s analog-to-digital converter (ADC) in continuous-wave (CW) systems, in frequency-modulated (FMCW) or phase-modulated (PMCW) systems it is the output of the range matched filter at the target range bin. In any case, the baseband I and Q signals from a target at nominal distance d0 can be represented as   4πd(t) + BI + n I (t), bI (t) = AI cos θ0 + λ

(2)

  4πd(t) + ϕIQ + BQ + n Q (t), bQ (t) = AQ sin θ0 + λ

(3)

where AI and AQ represent the I/Q amplitude imbalance, ϕIQ is the I/Q phase imbalance, θ0 = 4πd0 /λ is the constant phase shift, BI and BQ are DC offsets, and d(t) = db (t) + dh (t) represents the composite chest-wall movement. In addition, n I (t) and n Q (t) represent the noise component in each channel. Perfect recovery of the chest-wall motion d(t) allows for a precise estimation of the breathing and heartbeat frequencies by simple analysis of the movement periodicity. However, in practice, the received radar signal is usually mixed with additional reflections from the external environment and noise. These interfering signals are usually much stronger than those induced by the chest-wall millimeter displacement, thus rendering the accurate recovery and subsequent vital-sign frequency estimation a challenging task. Based on [3], the displacement due to breathing db (t) can be modeled as a lowpass filtered periodic sequence of quadratic inspiration and exponential expiration, expressed as ⎧ ⎨ −Pm t 2 + Pm T t, t ∈ [0, Ti ] Ti Te e  Ti T(t−T  db (t) = (4) Te i) ⎩ P−m Te e− τ − e− τ , t ∈ [Ti , T ] 1−e

τ

where Pm is a constant representing the pressure generated by the respiratory muscles, which controls the breathing displacement amplitude, Ti is the inspiration time, Te is the expiration time, T is the breathing period, and τ is a time constant. The inspiratory and expiratory times are considered fixed fractions of T . Hence, the profile is fully parameterized by Pm and the actual breathing period T . The value of τ was assumed to be equal to 1/5 of the expiratory time and a value of 0.6 was used for the inspiratoryexpiratory time ratio [3]. The standard physiological range of the breathing frequency is 10–25 bpm, while the amplitudes of the chest-wall motion can vary from 4 to 12 mm [6]. For modeling the heartbeat, a Gaussian pulse train can be used, based on the idea that the heartbeat is a short explosive motion, with pulsatile nature. The heartbeat displacement signal can thus be represented as [11] dh (t) =

n

ae−

(t−Tn )2 2c2

,

(5)

104

G. Beltrão et al.

where a is a normalization coefficient, c controls the pulse width, and Tn is the time interval between consecutive pulses. The standard physiological range of the heartbeat frequency goes from 60 to 100 bpm, while the amplitudes of the chest-wall motion can vary from 0.2 mm to 0.5 mm [6]. It is important to mention that the true chest-wall motion is a complex physiological phenomenon and it is unlikely that any model can fully characterize it in every situation. However, regardless of the displacement signal shape, its frequency content (spectral structure) is mainly determined by its inherent periodicity. The models used here are well-known and selected for illustrative purposes. In addition, the processing techniques described next are model agnostic and hence the chosen model does not disadvantage any one of them.

3 Simulation Framework The composite displacement signal is used to generate the I and Q samples according to (2) and (3), and additive white Gaussian noise (AWGN) is added to the complex samples according to a predefined signal-to-noise ratio (SNR) value. In addition, the parameters AI , AQ and ϕIQ can be used for modelling I/Q imbalance scenarios. Figure 1a shows 20 s of the generated displacements for both breathing and heartbeat motion, and the composite movement according to the described models. The amplitude and frequency were selected to be 10 mm and 12 bpm for breathing, and 0.5 mm and 60 bpm for the heartbeat. Given that standard amplitudes for the heartbeat displacement are usually 10–20 times smaller than the breathing displacement [2], the heartbeat main frequency component may eventually be masked by interfering harmonics from the breathing movement. Within the simulation framework, the amplitude relation between then can be independently controlled by the parameters Pm and a in (4) and (5) respectively, which will determine the signal-to-interference ratio (SIR) for heartbeat detection, where S I R = a/Pm . The simulator allows for scanning over multiple breathing and heartbeat frequencies, as well as SIR and SNR values, in a Monte Carlo approach. For all iterations of signal generation and processing, the final estimation root mean squared error (RMSE) is calculated, taking into consideration the entire signal processing chain. In this way, the techniques can be evaluated not only with respect to the noise, but also in relation to the combined effects of different methods and parameters. Finally, the performance can also be analyzed considering the influence of breathing over the heartbeat signal and vice-versa.

Statistical Performance Analysis of Radar-Based Vital-Sign . . .

105

Fig. 1 Simulated displacement (a) and I/Q (b) signals

4 Simulation Results 4.1 Phase Demodulation The initial signal processing step is commonly known as phase demodulation. It is essentially the process where the received I and Q signals are combined with the aim to recover the displacement signal d(t). Among several methods, the most used are the arctangent demodulation (AD) [12] and the complex signal demodulation (CSD) [7]. Complex Signal Demodulation. The CSD relies on small displacements (in relation to the operating wavelength) for recovering an approximation of the chest-wall motion. In this case, the displacement signal can be reconstructed as

 x(t) = bI (t) + j · bQ (t) = x + exp j [θ0 + Δθ(t)] ,

(6)

106

G. Beltrão et al.

where x = BI + jBQ represents a combined DC offset. Despite additional higher order harmonics, for small displacements (in relation to the operating wavelength), the recovered signal x(t) approximates the true chest-wall movement d(t), and the relevant frequency content is preserved. Arctangent Demodulation. On the other hand, the AD can be used for precise phase recovery. The AD directly extracts the phase of the received signal and, under ideal conditions, it recovers the displacement signal as x(t) =



 bQ (t) λ · unwrap arctan , 4π bI (t)

(7)

where the unwrap is necessary for removing possible phase discontinuities caused by the restricted codomain of the arctangent function. This operation is very sensitive to noise and interference, and may eventually accumulate errors, resulting in large distortions on the recovered displacement signal. DACM. To avoid the aforementioned limitations, the so called differentiate and crossmultiply (DACM) demodulation has also been proposed [15]. The DACM calculates the derivative of the arctangent function, followed by an integration step for recovering the phase. Its extended version can be efficiently implemented in the discrete form n bI [k](bQ [k] − bQ [k − 1]) − bQ [k](bI [k] − bI [k − 1]) , (8) x[n] = bI [k]2 + bQ [k]2 k=2 where the differentiation is approximated by a forward difference, and the integration is replaced by an accumulation. Linear Demodulation. Finally, the linear demodulation (LD) tries to suppress redundant information, and maximize the variance in the input signal [10]. It is based on the principal component analysis (PCA) of the input matrix

bI (t) , bQ (t)

MI Q =

(9)

where its first principal component is used as demodulated signal. Figure 2 shows the final RMSE for each of these demodulation techniques, considering breathing estimation at 24 and 60 GHz. In all cases, simple DFT-based estimation was used, where the estimated frequency is selected as the one yielding the maximum value of the DFT spectrum. The breathing frequencies are being scanned from 10 to 25 bpm with intervals of 1 bpm. The relevant simulation parameters are summarized in Table 1, and they will be used in the following simulations, unless explicitly stated otherwise. At 24 GHz, both the CSD and LD are robust to the noise, with small errors even for negative values of the SNR. The SNR threshold for correct operation is much higher for the AD, as well as for the DACM. At higher frequencies, when

Statistical Performance Analysis of Radar-Based Vital-Sign . . .

107

Fig. 2 RMSE (bpm) for breathing estimation considering different demodulation techniques at 24 GHz (a–d) and 60 GHz (e–h) Table 1 Standard simulation parameters Parameter Sampling frequency (Hz) Processing window (s) Breathing bandpass filter (Hz) Heartbeat bandpass filter (Hz) Zero-padding (samples) Breathing range (bpm) Heartbeat range (bpm) SNR range (dB) Number of trials

Value 100 20 0.1–0.7 0.75–3 2048 10–25 50–110 −20 to +20 100

the displacement amplitude is comparable to the operating wavelength, the CSD approximation does not hold anymore. At this point, intermodulation products and higher order harmonics start to dominate the spectrum, eventually being higher than the fundamental frequency component, which leads to estimation errors. The same effect seems to affect the LD. On the other hand, both AD and DACM presented better performance at 60 GHz.

4.2 Breathing Estimation Figure 3 shows the final RMSE considering the AD and different estimation techniques at 60 GHz (from now on, all the presented results will be related to this setup).

108

G. Beltrão et al.

Fig. 3 RMSE (bpm) for breathing estimation considering the AD and different estimation techniques at 60 GHz

In Fig. 3b the estimation was performed by time analysis (TA) of the displacement signal, where the average distance between peaks provides an estimate of the signal periodicity [1]. In Fig. 3c the ACC-FTPR technique from [11] was used. It is based on spectral estimation over the displacement signal autocorrelation, with the resolution being further improved by phase-regression using the complex sample of the dominant peak. Lastly, the nonlinear least squares (NLS) approach recently proposed in [5] was used in Fig. 3d. In this case, not only the dominant peak in the spectrum is used for estimation, but also its inherent harmonics which could eventually be detected. In this idealized case of breathing estimation without any external interference, the performance is limited only by noise and, after the SNR threshold, all techniques reach very low error values (always better than 1 bpm). However, by looking closer to the results in Fig. 4, it can be seen that the techniques have improved the frequency resolution and reduced the final error in relation to the standard DFT estimation. While Fig. 4a shows the error as a function of the reference breathing frequency, for the best SNR (20 dB), Fig. 4b shows the error as a function of the SNR, for a reference breathing frequency of 18 bpm. In addition, despite better results at higher SNR values, the TA-based estimation has poorer performance in the presence of noise and thus needs a higher SNR for correct estimation.

4.3 Heartbeat Estimation On the other hand, heartbeat estimation performance is influenced not only by the SNR, but mainly by the presence of high-order breathing harmonics. Figure 5 shows the heartbeat estimation performance at 60 GHz, using the AD and standard DFT estimation, and considering different breathing frequencies for a SIR of 1/12. It can be seen that, for smaller values of breathing frequency, the amount of harmonic interference is low and the heartbeat frequency can be precisely estimated given the minimum required SNR. However, for higher breathing frequencies, the interfering harmonics become dominant until a point where the heartbeat component is completely masked. At this point, the DFT estimation is in fact measuring breathing harmonics rather than the heartbeat frequency. This dynamic can be explained

Statistical Performance Analysis of Radar-Based Vital-Sign . . .

109

Fig. 4 a RMSE as a function of the reference breathing frequency. b RMSE as function of the SNR

Fig. 5 RMSE (bpm) for heartbeat estimation considering different breathing frequencies, with a SIR of 1/12

as follows: for lower breathing frequencies, the heartbeat estimation is competing with higher orders (5th, 6th and so on) of breathing harmonics which are already strongly attenuated. On the other hand, for higher breathing frequencies, the second and third harmonics are already over the heartbeat frequency range. These harmonics can be much stronger than the main heartbeat frequency component and thus prevent accurate estimation. In addition, Fig. 6 shows the results now considering a single breathing frequency of 12 bpm, but varying the SIR from 1/5 (Fig. 6a) to 1/20 (Fig. 6d). It can be seen in this case that, depending on the SIR, even a smaller breath-

110

G. Beltrão et al.

Fig. 6 RMSE (bpm) for heartbeat estimation considering different SIR, with a breathing frequency of 12 bpm

Fig. 7 RMSE (bpm) for heartbeat estimation considering different estimation methods, with a breathing frequency of 12 bpm, and a SIR of 1/15

ing frequency can generate harmonics capable of masking the heartbeat component and prevent correct estimation. Figure 7 shows the heartbeat estimation performance considering different estimation methods, when the breathing frequency is 12 bpm and the SIR is 1/15. In Fig. 7a, standard DFT estimation was used with no additional harmonic processing. In Fig. 7b, an algorithm for selecting spectrum peaks was used (DFT_PS), exploiting the knowledge of the already estimated breathing frequency and its harmonic-related positions. Instead of just selecting the maximum spectral peak in the heartbeat region, the algorithm looks for the higher peak which is not in a possible harmonic position. In Fig. 7c a simpler implementation of the RELAX algorithm [8] was used. In this case, the breathing harmonic components are iteratively estimated and removed from the displacement signal, until heartbeat peaks can be found. Lastly, Fig. 7d shows the results when spectral estimation is performed in the region of the second heartbeat harmonic (DFT_2nd). This algorithm was originally proposed in [14], and is based on the fact that the second heartbeat harmonic will probably be limited only by noise while the main heartbeat frequency component can be masked by low-order harmonics of breathing. Both DFT_PS and RELAX presented good performance with accurate estimation, except when the heartbeat frequency overlaps with the breathing harmonics positions. The DFT_2nd solves the overlap problem, however it has a higher SNR threshold and fails for higher heartbeat frequencies, when the main heartbeat frequency component is within the second harmonic region (ambiguity). In order to provide a complete picture of the harmonic relations, Fig. 8 finally shows the the RMSE for heartbeat estimation for all combinations between breathing and heartbeat frequencies, when the SNR and SIR are respectively 20 dB and 1/15.

Statistical Performance Analysis of Radar-Based Vital-Sign . . .

111

Fig. 8 RMSE (bpm) for heartbeat estimation considering different estimation methods, and all combinations of breathing and heartbeat frequency. The SNR and SIR were 20 dB and 1/15, respectively

Despite relatively good performance for specific frequency combinations, most of the time all techniques fail to provide accurate estimation, specially for higher frequency values. This indicates that robust solutions would need more powerful methods to deal with the harmonics problem.

5 Conclusions In this paper we presented a simulation framework and representative results to ease the comparison between radar-based vital-sign processing techniques. The proposed simulation tool allows for scanning over multiple breathing and heartbeat frequencies, and the resulting RMSE is calculated using a Monte Carlo approach. In this way, the combined effects along the entire signal processing chain could be analyzed, for different combinations of scenarios, techniques and parameters. We compared different demodulation techniques and the results have shown specific limitations for each method, thus indicating the need for a proper selection considering the operating frequency and expected SNR. In addition, in ideal conditions (no external interference), breathing estimation performance is only limited by noise and standard techniques may provide accurate results, with minor performance difference between them. On the other hand, the performance of heartbeat estimation is limited by the presence of breathing harmonics and, despite promising results at specific frequencies, the presented methods fail to fully mitigate this interference, specially at higher breathing/heartbeat frequencies. This indicates that specific harmonic mitigation techniques are thus needed in order to provide robust heartbeat estimation over all conditions.

112

G. Beltrão et al.

References 1. Ahmad, A., Roh, J.C., Wang, D., Dubey, A.: Vital signs monitoring of multiple people using a FMCW millimeter-wave sensor. In: 2018 IEEE Radar Conference, RadarConf 2018, no. 4, pp. 1450–1455 (2018). https://doi.org/10.1109/RADAR.2018.8378778 2. Al-Naji, A., Gibson, K., Lee, S.H., Chahl, J.: Monitoring of cardiorespiratory signal: principles of remote measurements and review of methods. IEEE Access 5, 15776–15790 (2017). https:// doi.org/10.1109/ACCESS.2017.2735419 3. Albanese, A., Cheng, L., Ursino, M., Chbat, N.W.: An integrated mathematical model of the human cardiopulmonary system: model development. Am. J. Physiol. Heart Circ Physiol 310(7), H899–H921 (2016). https://doi.org/10.1152/ajpheart.00230.2014 4. Beltrao, G., Alaee-Kerahroodi, M., Schroder, U., Bhavani, S.M.: Joint waveform/receiver design for vital-sign detection in signal-dependent interference. In: IEEE Radar Conference (2020) 5. Beltrão, G., Stutz, R., Hornberger, F., Martins, W.A., Tatarinov, D., Alaee–Kerahroodi, M., Lindner, U., Stock, L., Kaiser, E., Goedicke–Fritz, S., Schroeder, U., Bhavani Shankar M.R., Zemlin, M.: Contactless radar-based breathing monitoring of premature infants in the neonatal intensive care unit. Nat. Sci. Rep. (2022) 6. Kebe, M., Gadhafi, R., Mohammad, B., Sanduleanu, M., Saleh, H., Al-qutayri, M.: Human vital signs detection methods and potential using radars: a review. Sensors (Switzerland) 20(5) (2020). https://doi.org/10.3390/s20051454 7. Li, C., Lin, J.: Random body movement cancellation in doppler radar vital sign detection. IEEE Trans. Microw. Theory Tech. 56(12), 3143–3152 (2008). https://doi.org/10.1109/TMTT.2008. 2007139 8. Li, C., Ling, J., Li, J., Lin, J.: Accurate doppler radar noncontact vital sign detection using the RELAX algorithm. IEEE Transa. Instrument. Measur. 59(3), 687–695 (2010). https://doi.org/ 10.1109/TIM.2009.2025986 9. Li, J., Liu, L., Zeng, Z., Liu, F.: Advanced signal processing for vital sign extraction with applications in UWB radar detection of trapped victims in complex environments. IEEE J. Sel. Top. Appl. Earth Observat. Remote Sens. 7(3), 783–791 (2014). https://doi.org/10.1109/ JSTARS.2013.2259801 10. Massagram, W., Lubecke, V.M., Høst-Madsen, A., Boric-Lubecke, O.: Assessment of heart rate variability and respiratory sinus arrhythmia via doppler radar. IEEE Trans. Microw. Theory Tech. 57(10), 2542–2549 (2009). https://doi.org/10.1109/TMTT.2009.2029716 11. Nosrati, M., Tavassolian, N.: High-accuracy heart rate variability monitoring using doppler radar based on gaussian pulse train modeling and FTPR algorithm. IEEE Trans. Microw. Theory Tech. 66(1), 556–567 (2018). https://doi.org/10.1109/TMTT.2017.2721407 12. Park, B.K., Boric-Lubecke, O., Lubecke, V.M.: Arctangent demodulation with DC offset compensation in quadrature Doppler radar receiver systems. IEEE Trans. Microw. Theory Tech. 55(5), 1073–1078 (2007). https://doi.org/10.1109/TMTT.2007.895653 13. Park, J.K., Hong, Y., Lee, H., Jang, C., Yun, G.H., Lee, H.J., Yook, J.G.: Noncontact RF vital sign sensor for continuous monitoring of driver status. IEEE Trans. Biomed. Circ. Syst. 13(3), 493–502 (2019). https://doi.org/10.1109/TBCAS.2019.2908198 14. Rong, Y., Bliss, D.W.: Remote sensing for vital information based on spectral-domain harmonic signatures. IEEE Trans. Aerosp. Electron. Syst. 55(6), 3454–3465 (2019). https://doi.org/10. 1109/TAES.2019.2917489 15. Wang, J., Wang, X., Chen, L., Huangfu, J., Li, C., Ran, L.: Noncontact distance and amplitudeindependent vibration measurement based on an extended dacm algorithm. IEEE Tran. Instr. Measur. 63(1), 145–153 (2014). https://doi.org/10.1109/TIM.2013.2277530 16. Zakrzewski, M., Vehkaoja, A., Joutsen, A.S., Palovuori, K.T., Vanhala, J.J.: Noncontact respiration monitoring during sleep with microwave doppler radar. IEEE Sens. J. 15(10), 5683–5693 (2015). https://doi.org/10.1109/JSEN.2015.2446616 17. Zhang, Y., Qi, F., Lv, H., Liang, F., Wang, J.: Bioradar technology: recent research and advancements. IEEE Microw. Mag. 20(8), 58–73 (2019). https://doi.org/10.1109/MMM.2019.2915491

An Overview of Vital Signs Monitoring Based on RADAR Technologies Shahrokh Hamidi , Safieddin Safavi Naeini, and George Shaker

Abstract This paper is an overview on how to perform vital signs monitoring based on millimeter Wave (mmWave) Frequency Modulated Continuous Wave (FMCW) radars. We present the general ideas and tools to reconstruct the respiration as well as the heartbeat waveforms. Furthermore, we address the respiration-rate and heart-rate estimation. Finally, we present experimental results. Keywords Vital signs · Respiration and heart signal · FMCW radar

1 Introduction Vital signs detection using radar has been a popular topic for a few decades. The radar-based technology allows a non-contact measurement with high accuracy for respiration and heartbeat estimation. Compared to Electrocardiogram (ECG), Phonocardiogram (PCG) and peripheral blood flow pulse (PP) for heart monitoring, the radar-based approach consists of a non-contact device which transmits low-power and harmless electromagnetic waves toward the human subject and collects the reflections and analyses them with high accuracy. Moreover, the radar-based approach is also capable of monitoring the displacement of the chest of the human subject, which as a result, enables us to analyse the respiration signal as well as its waveform.

S. Hamidi (B) · S. S. Naeini · G. Shaker Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada e-mail: [email protected] S. S. Naeini e-mail: [email protected] G. Shaker e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_9

113

114

S. Hamidi et al.

Recently, Continuous Wave (CW) [1] and Frequency Modulated Continuous Wave (FMCW) [1, 2] radars have been extensively utilised for the task of vital signs estimation [3–10]. The CW radars can be considered as a subcategory of FMCW radars with a simpler structure. The negative aspect of the CW radars, however, is that they lack range resolution [1]. Apart from that, the vital sign analysis for both the FMCW and the CW radars are similar. In non-contact vital sign estimation based on radar signals, it is the displacement of the chest as well as the heart that are estimated. The chest and the heart displacement will modulate the phase of the radar signal. By analysing the modulation effect, we can estimate the respiration-rate as well as the heart-rate. Furthermore, the respiration and heartbeat waveforms can also be reconstructed, which, can be of high importance in detecting several different illnesses. Through studying the respiration waveform, for example, illnesses such as Bradypnea, Tachypnea, Hyperpnea, air trapping, and Cheyne-Stokes can be detected. This can be done by studying the distinct waveform that the respiration creates in each specific case. The purpose of this paper is to review how vital signs can be measured using FMCW radar-based technologies. We present the system model as well as the signal processing tools that are necessary for vital sign estimation. Then, we will use experimental data for verification. The paper has been organised as follows. In Sect. 2, we describe the system model. Section 3 has been dedicated to vital sign estimation. Finally, in Sect. 4, we present the experimental results in detail.

2 Model Description In FMCW radars, the signal transmitted by the transmitter is a chirp signal. After the transmitted signal bounces off a target, which has been located in the field of view of the radar, it is received at the location of the receiver and can be expressed as s(t) = e− j2π f c τ + 2πβtτ ,

(1)

where f c is the center frequency. The parameter β is given as b/T , where b and T stand for the bandwidth and the chirp time, respectively. Finally, τ is the round trip delay between the radar and the target. Upon taking a Fourier transform from (1), the energy of the target is concentrated at a frequency bin given as f b = βτ (Fig. 1). If we consider the energy of the target at this specific frequency bin over the course of different chirp signals, we will then obtain S(η) = e j2π f c τ (η) .

(2)

An Overview of Vital Signs Monitoring Based on RADAR Technologies

115

Fig. 1 The 2D fast-time-frequency versus slow-time diagram which shows the energy of a signal in a specific frequency bin (the grayed area) for different slow-time cells. The parameters η and f t represent the slow-time and fast-time-frequency, respectively

Table 1 Vital signs parameters Parameters Values Ar Ah fr fh

1–12 0.1–0.5 0.1–0.5 0.8–2

Units mm mm Hz Hz

In (2), the symbol η represents the slow-time parameter. The time delay τ (η) is . Moreover, the radial distance can be written as described as τ (η) = 2 R(η) c R(η) = R0 ± vr η + Ar sin(2π fr η) + Ah sin(2π f h η) + φ,

(3)

in which, R0 is the radial distance between the target and the radar. Also, the parameter vr is the radial velocity of the target. Moreover, fr and f h are the frequencies for the respiration and heart signals, respectively, with Ar and Ah representing the corresponding amplitudes. After substituting (3) in (2), we obtain ⎛ ⎝ j4π

S(η) = e

⎞

R0 ± vr η + Ar sin(2π fr η) + Ah sin(2π f h η) + φ ⎠ λ .

(4)

The goal in vital sign monitoring is to estimate the respiration-rate as well as the heartrate, namely, fr and f h . However, waveform reconstruction for both respiration and heart signals are also of high importance. For adults, the typical vital sign parameters are given in Table 1. As can be seen from (4), to estimate the respiration-rate and heart-rate, we need to calculate the phase of (4). However, the phase of the signal given in (4) is a combination of several different factors. In order to be able to estimate fr and f h , we apply different bandpass filters to the phase of the signal given in (4). In the next section, we explain these steps in more detail.

116

S. Hamidi et al.

3 Vital Signs Estimation The signal given in (4) contains the effect of several different components. In order to be able to correctly estimate the desired components, we will try to alleviate the effect of undesired components at each stage by applying band pass filers. Based on Table 1, to recover the respiratory signal, we apply an Infinite Impulse Response (IIR) bandpass filter, with bandwidth in the range 0.1–0.5 Hz, to the phase of the signal which has been described in (4). Consequently, the energy of the respiratory signal is preserved and the energy related to the heart signal as well as the noise of the system is removed. We will, then, analyse the result and extract the waveform and calculate the respiration-rate. The same procedure is followed for the heart-rate estimation. An IIR bandpass filter, with bandwidth in the range 0.8–2 Hz, is applied to the phase component of the signal which has been given in (4). As a result, the corresponding energy of the respiration signal is ignored and we can analyse the heart signal. Of course, it should be noted that, some of the harmonics of the breathing signal are still present and will contaminate the heart signal. To calculate the breathing rate, we can utilize several different methods such as peak finding, zero crossing, and Fourier-based techniques. Moreover, since the problem of respiration-rate and heart-rate estimation is sparse [11, 12], therefore, we can utilize this property and perform a incredibly high resolution estimation for both respiration-rate and heart-rate estimation. To achieve this goal, we will be using the following optimization problem [11, 12]. min .

1T ξ

s.t.

−ξ ≤ x ≤ ξ, A(f, η)x − (η)2 ≤ , f = [ f 1 , f 2 , · · · , f L ]T ,

x,¸

(5)

where  is the unwrapped phase of the signal given in (4) and the lth column of the N × L dictionary matrix A(f, η) is expressed as a( fl ), where, a( fl ) = T [1, e j2π fl η(1) , · · · , e j2π fl η(N −1) ] . To enforce sparsity, we should have N  L. The optimization problem given in (5) is a convex optimization problem [13]. Upon solving (5), the lth non-zero element of the L × 1 vector x corresponds to an estimate of the lth frequency component, fˆl = xl . After solving (5), the lth non-zero element of the L × 1 vector x corresponds to an estimate of the lth frequency component, fˆl = xl .

An Overview of Vital Signs Monitoring Based on RADAR Technologies

117

4 Experimental Results In this section, we present the experimental result based on the data gathered from AWR6843AOPEVM, which is a FMCW radar operating at frequencies ranging from 60 to 64 GHz, as well as MMWAVEICBOOST board from Texas Instruments (TI). To be able to have access to the raw ADC data, the DCA1000EVM real-time data-capture adapter for radar sensing evaluation module from TI is used. Figure 2 shows the radar and the necessary extra boards from TI. The AWR6843AOPEVM has 3 transmit and 4 receive antennas. For the vital signs estimation, however, we have used one Tx and one Rx. After taking a Fourier transform in the range direction from the raw data, the result is given in Fig. 3. The vertical axis shows the range samples while the horizontal axis represents the samples in the slow-time direction. The higher energy at the 29th range cell indicates the presence of the target, which in this case, is the person’s chest which has been located in front of the radar. The next step is Inphase (I) and Quadrature (Q) mismatch compensation. After I and Q mismatch compensation for the data contained in the 29th range cell, we calculate the phase of the signal. The next step is to unwrap the phase of the

Fig. 2 a The AWR6843AOPEVM FMCW radar as well as the MMWAVEICBOOST board from TI, b the DCA1000EVM real-time data-capture adapter for radar sensing evaluation module from TI

118

S. Hamidi et al.

Fig. 3 The 2D image which shows the range versus slow-time. The energy of the chest of the human subject, sitting in front of the radar, has been concentrated in the 29th range cell

signal. The result is shown in Fig. 4a. In Fig. 4b, we have applied de-trending and have also removed the DC component of the signal presented in Fig. 4a. To obtain the respiration signal, we apply a bandpass IIR Butterworth filter of order 4 to the signal illustrated in Fig. 4b. The result has been depicted in Fig. 5. After reconstructing the breathing signal, we can then provide an estimate of the respiration-rate. A simple yet robust technique for breathing rate estimation is the zero crossing method, in which, we find all the zero crossings of the signal shown in Fig. 5. Following that, we can easily find an estimate for the breathing rate. The result of the zero crossing method has been shown in Fig. 6a. Based on the zero crossing method, the breathing rate is estimated to be fr = 0.33 Hz. A different method to calculate the breathing rate is to consider the Fourier transform of the signal shown in Fig. 4b. Since the breathing pattern is periodic, therefore, the Fourier transform can provide us with an estimation for the breathing rate. Fig. 6b shows the result of the Fourier transform which has been taken from the signal given in Fig. 4b. The strong peak shown in Fig. 6b, provides us with the breathing rate which is fr = 0.33 Hz. The next data-set we have used for our analysis, is from [14]. The data-set presented in this article [14], consists of 24 h of synchronized data from a radar and a reference device. The implemented Continuous Wave (CW) radar system is based on the six-port technology and operates at 24 GHz in the ISM band. The same vital signs analysis for the FMCW radars can be performed for the CW radars as well. The drawback of the CW radars compared to FMCW radars is the lack of range resolution. Apart from that, the equation given in (4) is the same for both types of radars. Our ground-truth for the heart-rate is based on the electrocardiogram (ECG) signal. From the ECG signal, we are able to obtain the heartbeat and use it to validate the result that we acquire by processing the signal gathered from by the radar.

An Overview of Vital Signs Monitoring Based on RADAR Technologies Fig. 4 a The unwrapped phase of the signal represented in the 29th range cell which has been shown in Fig. 3 and b the unwrapped phase signal after de-trending and DC removal

Fig. 5 The signal representing the displacement of the chest of the human subject in mm

119

120

S. Hamidi et al.

Fig. 6 a The unwrapped phase of the signal corresponding to the 29th range cell of Fig. 3, b the unwrapped phase signal after de-trending and DC removal

The data is from a human subject and the radar is located in front of the chest of the person under test. Figure 7 shows the unwrapped phase of the radar signal. The ECG signal related to the radar data shown in Fig. 7, has been depicted in Fig. 8. Based on Fig. 8, the heart-rate, which can be estimated by calculating the time difference between the peaks in the ECG signal, is f h = 1.66 Hz. Now, we can consider the Fourier transform of the signal shown in Fig. 7 which has been depicted in Fig. 9. We can see the respiration-rate as well as its harmonics. We can also see the heart-rate at f h = 1.66 Hz. Figure 10 illustrates the result of applying the IIR bandpass filter to the signal presented in Fig. 7. Figure 10a depicts a signal representing the displacement of the chest. In Fig. 10a, a bandpass IIR Butterworth filter of order 4 has been applied to the unwrapped signal shown in Fig. 7. The lower and upper cut-off frequencies for the filter have been chosen as, fl = 0.1 Hz and f u = 0.5 Hz, respectively. To extract the heart signal,

An Overview of Vital Signs Monitoring Based on RADAR Technologies Fig. 7 The signal representing the displacement of the chest as well as heart in mm

Fig. 8 a The ECG signal, b the blow-up of the ECG signal with red crosses indicating the location of the R peaks. By calculating the time interval between the R peaks in the ECG signal, the hear-rate can be estimated

121

122 Fig. 9 The result of taking Fourier transform from the phase signal represented in Fig. 7. The fundamental as well as the harmonics of the respiration signal can be seen in the figure. The fundamental frequency related to the heartbeat is also present in the plot

Fig. 10 The result of the radar signal analysis for, a the respiratory signal, b the heartbeat signal

S. Hamidi et al.

An Overview of Vital Signs Monitoring Based on RADAR Technologies

123

Fig. 11 a The respiration-rate and b the heart-rate estimation based on (5)

we have designed a bandpass IIR Butterworth filter of order 4 with fl = 0.8 Hz and f u = 2 Hz. The result for the heart signal has been illustrated in Fig. 10b. It should be noted that Fig. 10b contains not only the heart signal but several harmonics of the respiration signal as well. Alternatively, to estimate the respiration-rate and heart-rate with high resolution and high accuracy, we can apply (5) to the unwrapped phase of the signal given in Fig. 7. In this approach, we do not need to apply bandpass filter anymore since the filtering can be easily performed by choosing specific range of frequencies for the dictionary matrix A(f, η). The result of applying (5) to the unwrapped phase of the signal given in Fig. 7 have been illustrated in Fig. 11. From Fig. 11, we have estimated the respiration-rate and heart-rate as fr = 0.295 Hz and f h = 1.66 Hz, respectively.

124

S. Hamidi et al.

5 Conclusion In this paper, we presented the general idea behind vital signs monitoring based on mmWave FMCW radars. Moreover, we described the necessary processing tools to extract the respiration and heart waveforms as well as the respiration-rate and heart-rate. Finally, we used experimental data and presented the results in detail. Acknowledgements The authors would like to thank NSERC, OCI, and Tidal Medical for their financial support.

References 1. Mahafza, B.R.: Radar Signal Analysis and Processing Using MATLAB. Chapman and Hall/CRC (2008) 2. Hamidi S., Nezhad-Ahmadi, M., Safavi-Naeini, S.: TDM based virtual FMCW MIMO radar imaging at 79 GHz. In: IEEE 18th International Symposium on Antenna Technology and Applied Electromagnetics ANTEM, pp. 1–2 (2018) 3. Prat, A., Blanch, S., Aguasca, A., Romeu, J., Broquetas, A.: Collimated beam FMCW radar for vital sign patient monitoring. IEEE Trans. Antennas Propagat. 67(8), 5073–5080 (2019) 4. Wu, S., Sakamoto, T., Oishi, K., Sato, T., Inoue, K., Fukuda, T., Mizutani, K., Sakai, H.: Personspecific heart rate estimation with ultra-wideband radar using convolutional neural networks. IEEE Access 7, 168484–168494 (2019) 5. Alizadeh, M., Shaker, G., Almeida, J.C.M.D., Morita, P.P., Safavi-Naeini, S.: Remote monitoring of human vital signs using mm-Wave FMCW radar. IEEE Access 7, 54958–54968 (2019) 6. Ahmad, A., Roh, J.C., Wang, D., Dubey, A.: Vital signs monitoring of multiple people using a FMCW millimeter-wave sensor. In: IEEE Radar Conference (RadarConf18), pp. 1450–1455 (2018) 7. Tu, J., Hwang, T., Lin, J.: Respiration rate measurement under 1-D body motion using single continuous-wave Doppler radar vital sign detection system. IEEE Trans. Microw. Theory Tech. 64(6), 1937–1946 (2016) 8. Vinci, G., Lindner, S., Barbon, F., Mann, S., Hofmann, M., Duda, A., Weigel, R., Koelpin, A.: Six-port radar sensor for remote respiration rate and heartbeat vital-sign monitoring. IEEE Trans. Microw. Theory Tech. 61(5), 2093–2100 (2013) 9. Tan, H., Qiao, D., Li, Y.: Non-contact heart rate tracking using doppler radar. In: International Conference on Systems and Informatics (ICSAI2012), pp. 1711–1714 (2012) 10. Li, C., Ling, J., Li, J., Lin, J.: Accurate Doppler radar noncontact vital sign detection using the RELAX algorithm. IEEE Trans. Instr. Measur. 59(3), 687–695 (2010) 11. Candes, J.R.E.J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006) 12. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006) 13. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004) 14. Schellenberger, S., Shi, K., Steigleder, T., Malessa, A., Michler, F., Hameyer, L., Neumann, N., Lurz, F., Weigel, R., Ostgathe, C., Koelpin, A.: A dataset of clinically recorded radar vital signs with synchronised reference sensor signals. Sci. Data 7, 291 (2020)

A C-Band Intermodulation Radar System for Target Motion Discrimination Ashish Mishra and Changzhi Li

Abstract This paper discusses the design and test of an intermodulation radar receiver and a nonlinear tag. To discriminate reflections at the fundamental and the third-order intermodulation frequencies, the receiver was designed to attenuate the fundamental reflections and amplify the lower 3rd-order tone generated by the nonlinear target. The nonlinear tag was designed on a flexible substrate for wearable applications. Experiments were performed with two types of targets, i.e., the nonlinear tag and a metal reflector. The results were recorded and analyzed to demonstrate the clutter noise rejection capability of the intermodulation radar. Keywords Clutter rejection · Intermodulation · Nonlinear tag · Nonlinear radar

1 Introduction Non-linear radars are used to track targets of interest and reject clutters. These radars identify the nonlinear characteristics present in many electronic components, such as diodes and transistors. Since naturally occurring things are linear in behavior, they can be distinguished from the nonlinear components, which are man-made. In nonlinear radars, fundamental tone(s) is sent towards a nonlinear tag, which in return reflects nonlinear tone(s) along with the fundamental tone(s). In the receivers, the fundamental tone(s) is separated from nonlinear response to distinguish between targets and clutters [1–3]. Many nonlinear radars detect harmonics of the transmitted tone(s), which lead to some major challenges in design, e.g., radio spectrum licensing issues [3, 4] and high path loss of the harmonic tones compared with the fundamental tone (s). Since

A. Mishra (B) · C. Li Department of Electrical Engineering, Texas Tech University, Lubbock, TX 79409, USA e-mail: [email protected] C. Li e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_10

125

126

A. Mishra and C. Li

conventional filters can hardly meet the high-quality factor requirement, diplexers were used, making the system bulky and expensive. Since harmonic frequency tones occupy a bandwidth at least twice of the fundamental tones, many harmonic radars need to be very broadband. Another commonly used nonlinear radar is subharmonic [5], here the fundamental response from the radar f is captured by the tag, and nonlinear response f/n, is reflected along with the fundamental. Here n, denotes the order of the subharmonic response. In these radars, the nonlinear responses have lower path loss at the cost of the larger receiver and tag size. Recently, a new nonlinear response based on intermodulation response for target localization and clutter rejection purpose has been discussed [6, 7]. In these nonlinear radars, the two fundamental tones are sent out towards the tag/nonlinear target; the tag absorbs these fundamental tones and reflects a series of intermodulation and harmonic responses. The receiver of these radars’ separate 3rd order harmonic response and attenuate other responses. To achieve high rejection, diplexers were generally utilized. This leads to bulky and expensive radar systems. In this paper, an intermodulation-based nonlinear radar is discussed to separate clutters from the target of interest. Here, diplexers were replaced with a series of gain blocks and filters to achieve comparable rejection for separating the target from the clutter. The receiver part of the radar is designed on a printed circuit board (PCB) with a 50-dB attenuation to fundamental tones and a 20-dB gain to the lower 3rdorder tone. This intermodulation radar can overcome some drawbacks of harmonic radars. Its 3rd-order tones are close to the fundamental tones, sharing almost the same path loss as the fundamental tones. Compared with the harmonic radar, which requires its nonlinear target to operate in two largely separated frequency bands, the intermodulation radar only needs a nonlinear target with a simple matching circuit to work in a single band and avoids licensing issues [8–10]. This paper is divided into four sections. Section 2 discusses the theory and design of the intermodulation radar and passive nonlinear tag. Section 3 presents the measurement results demonstrating motion discrimination. A conclusion is drawn in Sect. 4.

2 Intermodulation Theory and Receiver Design When two frequency tones f 1 and f 2 ( f 1 < f 2 ) pass through a nonlinear device; additional frequency tones are generated at m f 1 ± n f 2 , where m and n could be any integer numbers. The lower 3rd-order tone fr 1 = 2 f 1 − f 2 is used in this radar as they are close to fundamentals. The receiver is designed to attenuate frequencies above fr 2 = 2 f 1 − f 2 .

A C-Band Intermodulation Radar System for Target …

127

2.1 Radar System Figure 1 shows the complete block diagram of the intermodulation radar. To maintain coherence and achieve required sensitivity based on the range correlation effect [11], all the three signal generators are tied to the same reference, which was provided by signal generator SG3. Two fundamental tones of equal amplitudes were generated using SG1 and SG2, and fed to the transmitting antenna through the power combiner. The two transmitted tones are denoted as: T (t) =

2 

cos[2π f i t + φ(t)]

(1)

i=1

where φ(t) is the phase noise of the two signal generators. The transmitted signal is sent to the target at a distance xo . The nonlinear tag used in this project is passive and has Schottky diodes to generate nonlinear responses. As a result, the radar received signal can be expressed as:    4π x(t) 2xo 4π xo − −φ t − R(t) = cos 2π fr 1 t − λ λ c

(2)

where λ is the wavelength corresponding to fr 1 , x(t) is the mechanical displacement of the target, and the signal amplitude is normalized to 1. The fundamental components, the higher 3rd-order component fr 2 , and the other nonlinear tones are ignored in (2) as they will be at least 50-dB weaker than the fr 1 tone at the input of the low noise amplifier (LNA) in the radar receiver. The LO port of the mixer is Ref in TX Antenna

SG1 (5.85 GHz) Ref in SG2 (6 GHz)

Nonlinear TAG

Power Combiner

RF Signal Chain LNA

Mixer Ref out

RF

LO

SG3 (5.7 GHz)

Channel I

IF

DAQ

Gain Block

Filter

Gain Blocks

Filter

Channel Q

Fig. 1 The C-band intermodulation radar and its non-linear target

Gain Blocks

RX Antenna

128

A. Mishra and C. Li

driven by a signal with frequency fr 1 , which also provides additional rejection of other frequency tones. The radar is AC-coupled at the mixer output to remove the DC offset, and the I and Q signals from the baseband amplifier can be expressed as: 

4π x(t) + φ(t) B I /Q (t) = A I /Q cos θ + λ

 (3)

where θ = 4πλxo + θo and θo accounts for the phase shift when the signal is reflected from the tag and propagates along building blocks in the radar signal chain. φ(t) is the residual phase noise, which can be ignored for short-range detection due to the range correlation effect [7]. A I and A Q are the amplitudes of the I/Q channel baseband outputs. The use of I /Q channels solves the null detection point issue [7]. These baseband signals are converted from analog to digital using NI-USB 6009 (DAQ).

2.2 Receiver RF Signal Chain The receiver was designed with f 1 = 5.85 GHz and f 2 = 6 GHz. A combination of filters and amplifiers is used to attenuate the fundamental tones and amplify the lower 3rd-order tone. One of the key challenges of cascading amplifiers and filters is system stability. The receiver chain was finalized by performing system-level simulations in NI-AWR. The fabricated receiver RF signal chain is shown in Fig. 2. The RF signal chain attenuates the fundamentals at 5.85 and 6 GHz by 50 dB and provides 20.72-dB gain at 5.7 GHz. To achieve this, a 9th-order Chebyshev filter was designed to attenuate the tones at f 1 , f 2 and fr 2 . The filter was designed and simulated with NI-AWR for RT/Duroid 5880 material. The filter provides over 40-dB attenuation to the fundamental tones. Fig. 2 The receiver RF signal chain of the intermodulation radar

A C-Band Intermodulation Radar System for Target …

129

Fig. 3 Measured S-parameter of the receiver RF signal chain

Apart from attenuating the fundamental tone(s), the isolation is also a key criterion for the receiver design because the leakage from the local oscillator (LO) port to the radio frequency (RF) port can radiate back to the surrounding. Because of that, the corresponding clutter reflection can be comparable to the 3rd-order tone generated by the tag, leading to clutter interference. In this work, the reverse isolation from port 2 to port 1 is above 50 dB across the 5–6 GHz band (Fig. 3).

2.3 Nonlinear Tag The passive nonlinear tag was designed for operation without requiring any battery. Figure 4 shows the nonlinear tag fabricated on RT/Duroid 5880 substrate and the schematic of its unit cell with the Infineon BAT 15-03 W Schottky diode. The S-parameter model of the diode with 0-V and 0-mA bias was used to design the microstrip line matching circuit for each unit cell. A C-band patch antenna is also integrated into each unit cell. The tag contains a 4 × 9 array of unit cells.

3 Experiment Two types of targets were used in the experiments, i.e., the fabricated nonlinear tag with a size of 26.67 cm × 19.56 cm as shown in Fig. 4, and a copper plate with a size of 30.48 cm × 22.86 cm. The copper plate was used to mimic an undesired clutter with a size larger than the nonlinear tag. A computer-controlled actuator and a motion phantom were used to generate mechanical movements. The actuator can generate an arbitrary movement amplitude and frequency. At the same time, the phantom is a battery-powered device with a rotating oval disc that generates a complex motion mimicking human breathing. All the measurements were recorded with the targets placed at 0.5-m away from the intermodulation radar.

A. Mishra and C. Li

26.67 cm

130

19.56 cm

Fig. 4 Fabricated nonlinear tag with a unit shown in the inset

3.1 Nonlinear Tag Versus Copper Plate In this measurement, the nonlinear tag and the copper plate were mounted sequentially on the actuator moving with 1-mm peak-to-peak amplitude and 0.8-Hz frequency in front of the radar. The baseband data was recorded, and Fast Fourier Transform (FFT) was performed. The results are shown in Fig. 5. From this graph, it can be seen that the signal detected from the nonlinear tag was 18-dB stronger than that detected from the copper plate, although the copper plate has a larger physical size than the nonlinear tag. The weak motion signal visible for the copper plate was mainly due to the nonlinearity of the amplifiers and mixer in the receiver chain.

3.2 Target Detection in the Presence of Clutters The second experiment demonstrated motion discriminating when a small motion of the nonlinear tag and a large motion of the copper plate coexist. In this case, the copper plate was placed on the motion phantom, which moved with approximately 10-mm peak-to-peak amplitude and a frequency of 0.2 Hz, while the nonlinear tag was placed on the actuator moving with a 2-mm peak-to-peak amplitude and 0.9-Hz frequency. The two objects were placed side by side in front of the radar. Figure 6 shows the baseband signal detected by the radar and its corresponding spectrum. Although the copper plate has a larger size and moves with an amplitude

A C-Band Intermodulation Radar System for Target …

131

Fig. 5 Nonlinear tag versus metal plate: a Baseband output for metal and nonlinear tag motion. b FFT of baseband signal

five times higher than the nonlinear tag, the radar successfully detected the 0.9-Hz nonlinear tag motion. The radar also suppressed the 0.2-Hz strong clutter motion to 6-dB below the desired signal level.

4 Conclusion An intermodulation radar was designed and experimentally tested with different targets. The receiver board was able to amplify the 3rd-order frequency tone and attenuate the fundamental tones. The radar can successfully differentiate between the motions of a nonlinear tag from a large clutter. Compared to harmonic radars, the proposed intermodulation radar operates in the same frequency band to transmit and receive signals, which simplifies the hardware design and avoids licensing issues due to multiple-band operation.

132

A. Mishra and C. Li

Fig. 6 Target detection in the presence of clutters: a Baseband output for metal and nonlinear tag motion. b FFT of baseband signal

Acknowledgements The authors would like to acknowledge grant support from National Science Foundation (NSF) ECCS-2030094 and ECCS-1808613.

References 1. Mandel, C., Schuster, C., Kubina, B., Schüßler, M., Jakoby, R.: Dual frequency selective multiple access with quasi-chipless/powerless RFID mixer tags. IEEE Microw. Wirel. Compon. Lett. 24(8), 572–574 (2014) 2. Mishra, A., Li, C.: A third-order harmonic radar design for mm-wave frequencies. In: Proceedings of SPIE, Radar Sensor Technology XXV, 117421A, vol. 11742, 12 April 2021 3. Tsai, Z., et al.: A high-range-accuracy and high-sensitivity harmonic radar using pulse pseudorandom code for bee searching. IEEE Trans. Microw. Theory Techn. 61(1), 666–675 (2013) 4. Peng, Z., Psychogiou, D., Li, C.: Investigation of the roles of filters for a harmonic FMCW radar. In: International Applied Computational Electromagnetics Society Symposium (ACES), pp. 1–2, Suzhou, August 2017 5. El Agroudy, N., El-Shennawy, M., Joram, N., Ellinger, F.: Design of a 24 GHz FMCW radar system based on sub-harmonic generation. IET Radar Sonar Navigat. 12(9), 1052–1057 (2018)

A C-Band Intermodulation Radar System for Target …

133

6. Mishra, A., McDonnell, W., Wang, J., Rodriguez, D., Li, C.: Intermodulation-based nonlinear smart health sensing of human vital signs and location. IEEE Access 7, 158284–158295 (2019). https://doi.org/10.1109/ACCESS.2019.2950347 7. Mishra, A., Li, C.: A low power 5.8-GHz ISM-band intermodulation radar system for target motion discrimination. IEEE Sens. J. 19(20), 9206–9214 (2019). https://doi.org/10.1109/JSEN. 2019.2926189 8. Mishra, A., Wang, J., Rodriguez, D., Li, C.: Utilizing passive intermodulation response of frequency-modulated continuous-wave signal for target identification and mapping. IEEE Sens. J. 21(16), 17817–17826 (2021). https://doi.org/10.1109/JSEN.2021.3084205 9. Mishra, A., Li, C.: 5.8-GHz ISM band intermodulation radar for high-sensitivity motionsensing applications. In: 2018 IEEE Radio and Wireless Symposium, pp. 4–6, January 2018 10. Kuo, N.C., Zhao, B., Niknejad, A.M.: A 10-Mb/s uplink utilizing rectifier third-order intermodulation in a miniature CMOS tag. IEEE Microw. Wirel. Compon. Lett. 27(11), 1031–1033 (2017) 11. Droitcour, A.D., Boric-Lubecke, O., Lubecke, V.M., Lin, J., Kovacs, G.T.A.: Range correlation and I/Q performance benefits in single-chip silicon Doppler radars for noncontact cardiopulmonary monitoring. IEEE Trans. Microw. Theory Techn. 52(3), 838–848 (2004)

Lossy Mode Resonances Supported by Nanoparticle-Based Thin-Films Ignacio Vitoria, Carlos Coronel, Aritz Ozcariz, Carlos Ruiz Zamarreño, and Ignacio R. Matias

Abstract Optical fiber sensors based on the lossy mode resonance (LMR) effect obtained by means of the utilization nanoparticle-based thin-films are presented for the first time in this work. Tungsten oxide III nanoparticles (WO3NP) and diamond nanoparticles (DNP) have been deposited separately onto cladding removed multimode fiber to obtain LMRs. The fabrication of the thin-films was performed using the layer by layer (LbL) technique. Poly(allylamine hydrochloride) PAH was used in both cases to embed the nanoparticles. The LMRs obtained were characterized during the thin-film fabrication process allowing to precisely tuning the resonance in the desired part of the spectrum. Thin-film properties were studied using a scanning electron microscope (SEM) and optical microscope to verify their homogeneity and roughness. The response of the devices to changes in the surrounding medium refractive index (SMRI) was studied showing sensitivities of 5940 and 1368 nm/RI for WO3NP and DNP respectively, which opens the door to the utilization of these devices for novel sensing applications. Keywords LMR · Optical fiber sensor · Nanoparticle · Tungsten oxide · Nanodiamonds

1 Introduction Optical fiber sensors have distinctive advantages, including their ability to measure in remote places or rough conditions (high temperature, humidity, corrosive, or hazardous atmospheres) as well as their excellent sensitivity, small size and flexibility [1]. Different and heterogeneous technologies use optical fiber as sensors such as optical fiber interferometers or optical fiber gratings and filters [2]. Among them, sensors based on Lossy Mode Resonances (LMR) have been growing in interest [3]. I. Vitoria (B) · C. Coronel · A. Ozcariz · C. R. Zamarreño · I. R. Matias IEEC Department, Public University of Navarre, 31006 Pamplona (NA), Spain e-mail: [email protected] I. Vitoria · A. Ozcariz · C. R. Zamarreño · I. R. Matias Institute of Smart Cities, Jeronimo de Ayanz Building, 31006 Pamplona (NA), Spain © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_11

135

136

I. Vitoria et al.

LMR sensors have similar characteristics to the more mature and popular sensors based on Surface Plasmon Resonances (SPR) [4]. However, they differ on the optical properties of the materials used for SPR or LMR generation. In SPR the permittivity of the thin-film has a negative real part and is greater in magnitude than the imaginary part of the thin-film itself and that of the environment that surrounds it. Good conductor metals, such as gold and silver, generally meet this condition. On the other hand, with LMR, the real part of the permittivity of the thin-film must be positive and of greater magnitude than its own imaginary part and the imaginary part of the external medium. Metallic oxides and polymers are generally used for LMR generation [5, 6]. The optical properties and the thickness of the thin-films are crucial to tune the LMRs (wavelength position and width), which has been widely studied by researchers using a wide range of materials and fabrication methods, such as sputtering [7, 8], atomic layer deposition [9], dip coating [10], layer by layer (LbL) [11], among others [12]. The advantages of the LMR in contrast with other resonance modes are the possibility of generating the resonances with both transverse modes, the Transverse Electric TE and Transverse Magnetic TM in polarized light. Additionally, various peaks of different orders, corresponding with the modes guided in the coating, appear for each transverse mode, producing multiple resonance peaks that can be monitored independently in different parts of the spectrum. Another advantage is, as previously described, the wide variety of materials that meet the conditions for the generation of the effect, enabling the use of sensitive materials in LMR sensors for the detection of gases, humidity, pH, chemical species, as well as biosensors among others [3]. The Layer-by-Layer electrostatic self-assembly method, popular due to its simplicity and no need of special equipment, has been widely explored in literature for the fabrication of polymeric coatings with controlled thickness. In general, the use of NP for the formation of thin-films allows to obtain rougher and more porous surfaces leading to an increase of the surface area of the sensor as well as improving the sensing performance. Most thin-film fabrication techniques cannot be interrupted during the fabrication process, so it is only possible the decoration with NP in the surface of the film as a post-treatment. In contrast, LbL, based on electrostatic attraction of oppositely charged polymers, enables the possibility to include nanoparticles (NP) within the thin-film structure. Previous approach has been presented in different works as a valuable method to improve the sensing properties of the final structure [5, 12–14]. Alternately, LbL technique also permits to incorporate NP in the interlayers during the construction or even substituting one of the polymers forming layers of only NP. The use of a polymer in combination with nanoparticles for the generation of LMRs was firstly described in 2010 using TiO2 NPs and the polymer poly(sodium 4-styrenesulfonate) (PSS) [15]. The aim of this work is to analyse this type of structures by means of the utilization of tungsten oxide nanoparticles (WO3NP) and diamond nanoparticles (DNP), which have been chosen due to their peculiarities as it will be mention in the paragraphs below. Tungsten oxide III (WO3 ) is a well know material for gas sensors [16]. The generation of LMRs with WO3 has been proven feasible with cover slips as a waveguide [17], although for the best of our knowledge it has never been reported before an

Lossy Mode Resonances Supported by Nanoparticle-Based Thin-Films

137

optical fiber LMR-based device using WO3 nanoparticles (WO3NPs). In addition, previous works have reported optical fiber gas sensors using WO3 as sensing material for the detection of different gaseous compounds, such as ethanol [18], hydrogen [19], nitrogen dioxide [20] methanol [21], among others, which opens the door to a multiplicity of applications. Diamond nanoparticles (DNPs) have been chosen because the permittivity of this material is within the margins that allow generating a LMR. Further, diamond has good thermal, mechanical, electrical and chemical qualities, which makes it a very versatile and resistant material for harsh environment applications. Among its many applications, DNPs permit its use for the manufacturing of different types of sensors like: electrochemical sensors [22, 23], surface acoustic wave (SAW) sensors [24] or optical sensors [25]. Different type of sensors based on DNPs have been recently developed for the detection of toxic gases, such as NO2 [26] and NH3 [27]. In addition, there are many techniques that permit the deposition of DNPs, such as dip coating, LbL electrostatic self-assembly, vapour deposition and drop casting [28]. The similarities of the construction of both nanoparticle based thin-films, such as the technique (LbL) and the polymer used, enables an easy comparison of the results obtained with both approaches. To the best of the authors’ knowledge, this is the first time that WO3NP and DNP are used for the generation of LMRs.

2 Materials and Methods 2.1 Chemical Materials The polymer used for the sensing layer was poly(allylamine hydrochloride) (PAH) with a molecular weight of ∼7500 purchased from Sigma-Aldrich. WO3NPs were obtained from nanopowder with a particle size 0, ∀k ∈ K , ∀i ∈ I, ψCi − 1 −

(14)

k=1

xik = {0, 1}, ∀k ∈ K , ∀i ∈ I, K 

(15)

xik = 1, ∀i ∈ I,

(16)

xik = 1, ∀k ∈ K ,

(17)

k=1 I  i=1

yik = {0, 1}, ∀k ∈ K , ∀i ∈ I,

(18)

A Lightweight Security Scheme for Failure Detection …

Γi =

 I  K   Cik + RiC + Ti K + TiC + E T ≤ di ∀i ∈ I.

403

(19)

i=1 k=1

The Eqs. (11)–(17) are the constraints where deadline and resources are determined in the system. Equation (18, 19) are the constraints which determine that each workload can assign to one node at a time and each node can run one workload at a time. Proof. To prove that the Γi is an NP-complete problem, we can show that Γi is a decision problem, Where, Offloader Engine decides where to offload the workload of an application either to the cloudlet or public cloud. Whereas, the decision problem can be illustrated in the following way: taken a positive value di , is it promising to find the feasible solution S= {yik | ∀i ∈ I, ∀k ∈ K } such that Γi ≤ di and constraints (11)– (17) are satisfied well? That one may demonstrate the Γi is an NP-complete, MobCloud restricted by only two homogeneous cloudlet with similar resource capacities, i.e., ξk (e.g., storage, CPU). Meanwhile, let us suppose that T → ∞, i.e., Γi ≤ di is for all time satisfied for every solution of S. Then, the decision problem of Γi can be converted into a partition problem, because of Offloader Engine offloads workload to the cloudlet in such way that the average arrival rates of two sets to cloudlet DC remain the same. Thus, the partition problem is reduced to the decision problem of Γi . It is a noteworthy partition problem is well known as an NP-complete problem. Hence, it is proved that Γi is an NP-complete problem.

4 Proposed Schemes The study suggested three different schemes to solve the workload assignment problem with lightweight secure and failure aware schemes in the cloudlet cloud network. It will start from an input of all applications which are encrypted by modified fully homomorphic encryption scheme. The encrypted workload execution based on assignment strategies. If there is any failure of workload on the scheduled node, the transient failure schemes will recover them under their deadlines. Algorithm 1 encrypted all workload tasks based on RSA [27] scheme with 256 bits asymmetric schemes. The primary key used for the encryption and private key used for the decryption by the initial node. The proposed MFHE is secure and lightweight as compared to existing GoldwasserMicali [28] and Benaloh [26] homomorphic methods. After the encryption on workloads, the workload assignment (WA) strategy is a greedy approach where all workloads assignment to the available optimal nodes as shown in Algorithm 2. Thus, S ← yi∗∗ k ∗ and Γi∗ ← S shows that Algorithm 2 assigned workloads on the optimal way on the cloudlet cloud network. Suppose any failure of workload occurs in the cloudlet cloud network, the status of the workloads to be determined based on Eq. (2). The transient failure aware scheme monitors the temporary failure of the workload in the network and recovers them until sik will become 1. To deal with any transient failure, the study devises the workload

404

A. H. Sodhro et al.

Algorithm 1: Modified Fully Homomorphic Encryption Scheme

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Input : (ce = (tu 1 .tu 2 ) mod n) ; Output: (m = cd mod n) ; begin p ← large integer; q ← large integer; p = q; n ← p×q ; φ(n) = ( p − 1) × (q − 1); Choose an integer  when 1 < φ(n); gcd(, φ(n)=1; d =  −1 modφ(n); public key← (, n); private key← d; ce = (tu 1 .tu 2 ) mod n; m = cd mod n; return c = (tu 1 .tu 2 )e mod n; return m = cd mod n;

Algorithm 2: Workload Assignment

1 2 3 4 5 6 7 8 9 10 11 12 13

Input : i ∈ I, k ∈ K ; Output: minΓi ; begin Q i ← sort the workloads descending order; S ← 0 initial feasible solution; Γi ← 0; foreach (i ∈ Q i ) do I K if ( i=1 k=1 Ci + λi ci ≤ ξk + ξk ) then if (E T + Cik + Ti K ≤ T ) then sort FHE tasks {tu i , ...., T }; and other tasks Q i based on equation (19); Set yi∗∗ k ∗ = E T + Cik + Ti K ; Assign optimal cloudlet based on equation (20); S ← yi∗∗ k ∗ ; Γi∗ ← S;

15

else Γi∗ = E T + RiC + TiC ≤ T remote execution;

16

Γi ← Γi + Γi∗ ;

14

17

return Γi ;

assignment transient failure aware (WATFA) schemes that initially identify the failure and store them into the list of failures. The ExponentialBackoff method gives three maximum options to recover tasks under its given downtime must be less than the deadline. The failure aware policy will decide if the workload recovered workload

A Lightweight Security Scheme for Failure Detection …

405

within the deadline, it will be continued, other WATFA will ask for Algorithm 2 reschedule workload from scratch. Algorithm 3: Workload Assignment Transient Failure Aware (WATFA) 1 2 3 4 5 6 7 8 9

Input : erlist[ex] ; begin at p ← 0 attemp; maxattempt ← 3; t ← downtime delay; foreach (erlist[ex] in vi ) do if (tia .TiK + t ≤ T ) then ExponentialBackoff(at p, tia , t) application failure; else if (tin .Ti K + t ≤ T ) then ExponentialBackoff(at p, tin , t) network failure;

10 11

else if (tic .Ti K + t ≤ T ) then ExponentialBackoff(at p, tic , t) node failure;

12 13

else if (λi .TiK + t > T ) then apply resubmission if not satisfy equation (21);

14 15

if (at p > maxattempt) then exist request;

16 17 18

at p + +; counter increment; r etr y Strategy ← E x ponential Backo f f ;

19 20 21 22 23 24 25 26 27 28

return r etr y Strategy; F A Policy ← r etr y Strategy; F A Policy.E xecute Action(Asyn N avigateto); foreach (data in erlist[vi , r esour ce, ex]) do transient failure cloudlet Asyn N avigateto(); { G E T ← data idempotent httprequest method; PU T ← data idempotent httprequest method; Delete ← data idempotent method; H ead ← data idempotent modify method; }

Detect (err or s[ex],

N 

tia,n,c .Ti K , T ) + t,

t=1

(20)

e.g., tia = 1 or tin = 1 or tic = 1. Equation (20) calculates downtime when any task failed (tia,n,c , tu i ) at application or network or cloud to recover from failed states. The retry strategy mainly identifies how much time to retry the failed business tasks along with at what intervals. The proposed retry strategy gives option of retrying the failure with the given maximum finite attempts in the system, We denote failed operation of a task with these notations

406

A. H. Sodhro et al.

tia , the failure could be transient failure, tin due to network failure, and tic node failure in the Mob-Cloud. The key parameters such as allowable attempts, downtime, and counter are used as a variables in Algorithm 3. Retry strategy exploited Random exponential back-off interval algorithm [27] to retry any transient failure error in the Mob-Cloud. The downtime variable t will record the delay during retry operation with all possible attempts and must be less than the given threshold value.

5 Performance Evaluation The study conducted the experiment based on Ifogsim [29] simulator. The healthcare dataset of different healthcare tasks [25] implemented in ifogsim with 500, 1000, 1500 and 2000 tasks. The performances of all applications evaluated based on statistical relative deviation parameter (RPD) methods in the Eq. (21). R P D%

Γi∗ − Γi ∀I. Γi∗

(21)

Figure 1 shows that the modified fully homomorphic encryption is lightweight than existing GoldwasserMicali and Benaloh schemes. The main reason is that the MFHE is encrypted and decrypted with rich resources. However, the existing solutions didn’t find which node has a resource which one has no resource to process the encryption and decryption. Figure 2 shows the RPD% of WATFA outperformed all existing workload assignment schemes such as checkpointing [21] (B1), primary backup [25] (B2) and failure balance [29] (B3). The main reason is that, all existing schemes only focused on persistent failure, however, WATFA focused both persistent and transient failure in the network.

Fig. 1 MFHE Schemes for healthcare workload

A Lightweight Security Scheme for Failure Detection …

407

Fig. 2 Workload assignment transient failure schemes performance

6 Conclusion This paper proposed a new lightweight microservices mobile cloud (Mob-Cloud) framework that replaces the heavyweight VM-based MCC. The study devises MFHE (Modified Fully Homomorphism Encryption) and WATFA (Workload Assignment Transient Fault aware) schemes to deal with security and failures. Simulation results showed that the proposals are practical for the considered problem and achieved the optimal solutions as compared to existing schemes. Acknowledgements This work is financially supported by the Research grant of PIFI 2020 (2020VBC0002), Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences (SIAT,CAS), Shenzhen, China. This work is also partially supported by the Research Council of Norway through the SFI Norwegian Centre for Cybersecurity in Critical Sectors (NORCICS), project no. 310105. Authors from the Norwegian Computing Center (NR) would like to express great appreciation to the NORCICS team and ICT Research Department at NR.

References 1. Giannakopoulos, I., Konstantinou, I., Tsoumakos, D., Koziris, N.: Cloud application deployment with transient failure recovery. J. Cloud Comput. 7(1), 1–20 (2018) 2. Sodhro, A.H., Pirbhulal, S., De Albuquerque, V.H.C.: Artificial intelligence-driven mechanism for edge computing-based industrial applications. IEEE Trans. Indus. Inform. 15(7), 4235–4243 (2019)

408

A. H. Sodhro et al.

3. do Nascimento Dantas, A.P., Cavalcante, A.L.B.: Evaluation of antecedent rainfall effects in the analysis of the probability of transient failure in unsaturated slopes. In: MATEC Web of Conferences, vol. 337, p. 03016. EDP Sciences (2021) 4. Lynn, T., Rosati, P., Lejeune, A., Emeakaroha, V.: A preliminary review of enterprise serverless cloud computing (function-as-a-service) platforms. In: 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pp. 162–169. IEEE (2017) 5. Hafeez, I., Antikainen, M., Ding, A.Y., Tarkoma, S.: Iot-keeper: securing iot communications in edge networks. arXiv preprint arXiv:1810.08415 (2018) 6. Xu, R., Nikouei, S.Y., Nagothu, S., Fitwi, A., Chen, Y.: Blendsps: a blockchain-enabled decentralized smart public safety system. Smart Cities 3(3), 928–951 (2020) 7. Lakhan, A., Dootio, M.A., Groenli, T.M., Sodhro, A.H., Khokhar, M.S.: Multi-layer latency aware workload assignment of e-transport iot applications in mobile sensors cloudlet cloud networks. Electronics 10(14), 1719 (2021) 8. Hussain, M., Wei, L.-F. et al.: Energy and performance-efficient task scheduling in heterogeneous virtualized cloud computing. Sustain. Comput. Inform. Syst. 30, 100517 (2021) 9. Ordóñez-Camacho, D.: Reducing the iot security breach with a microservice architecture based on tls and oauth2 reduciendo la brecha de seguridad del iot con una arquitectura de 10. Xu, R., Nikouei, S.Y., Chen, Y., Blasch, E., Aved, A.: Blendmas: a blockchain-enabled decentralized microservices architecture for smart public safety. In: 2019 IEEE International Conference on Blockchain (Blockchain), pp. 564–571. IEEE (2019) 11. Ahmadi, A., Sodhro, A.H., Cherifi, C., Cheutet, V., Ouzrout, Y.: Evolution of 3c cyber-physical systems architecture for industry 4.0. In: International Workshop on Service Orientation in Holonic and Multi-Agent Manufacturing, pp. 448–459. Springer (2018) 12. Borhani, M., Liyanage, M. et al.: Secure and Resilient Communications in the Industrial Internet. pp. 219–242 (2020) 13. Yun, J., Goh, Y., Chung, J.-M.: Dqn based optimization framework for secure sharded blockchain systems. IEEE Int. Things J. (2020) 14. Zhang, F., Wang, M.M.: Stochastic congestion game for load balancing in mobile edge computing. IEEE Int. Things J. (2020) 15. Bolton, T., Dargahi, T., Belguith et al.: On the security and privacy challenges of virtual assistants. Sensors 21(7), 2312 (2021) 16. Khoso, F.H. et al.: A microservice-based system for industrial internet of things in fog-cloud assisted network. Eng. Technol. App. Sci. Res. 11(2), 7029–7032 (2021) 17. Omoniwa, B., Hussain, R., Javed, M.A., Bouk, S.H., Malik, S.A.: Fog/edge computing-based iot (feciot): architecture, applications, and research issues. IEEE Inte. Things J. 6(3), 4118– 4149 (2018) 18. Sodhro, A.H., Pirbhulal, S., Luo, Z., Muhammad, K., Zahid, N.Z.: Toward 6G architecture for energy-efficient communication in iot-enabled smart automation systems. IEEE Int. Things J. 8(7), 5141–5148 (2020) 19. Talat, R., Obaidat, M.S., Muzammal, M.: A decentralised approach to privacy preserving trajectory mining. Future Gener. Comput. Syst., 102, 382–392 (2020) 20. Rongxu, X., Jin, W., Kim, D.: Microservice security agent based on api gateway in edge computing. Sensors 19(22), 4905 (2019) 21. Magsi, H. et al.: Analysis of signal noise reduction by using filters, pp. 1–6 (2018) 22. Mujeeb-ur Rehman, A.L., Hussain, Z., Khoso, F.H., Arain, A.A.: Cyber security intelligence and ethereum blockchain technology for e-commerce. Int. J. 9(7) (2021) 23. Nykvist, L.M., Carl et al.: A lightweight portable intrusion detection communication system for auditing applications 33, 4327. Wiley Online Library (2020) 24. Ahmad, I., Shahabuddin, S., Malik, H., Harjula, E., Leppänen, T., Loven, L., Anttonen et al.: Machine learning meets communication networks: current trends and future challenges, vol. 8, pp. 223418–223460. IEEE (2020) 25. Islam, J., Harjula, E., Kumar, T., Karhula, P., Ylianttila, M.: Docker enabled virtualized nanoservices for local iot edge networks, pp. 1–7 (2019)

A Lightweight Security Scheme for Failure Detection …

409

26. Hina, M. et al.: Evolution of 5G in internet of medical things. In: 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pp. 1–7. IEEE (2018) 27. Zhang, T., et al.: A joint deep learning and internet of medical things driven framework for elderly patients. IEEE Access 8, 75822–75832 (2020) 28. Muzammal, M., Talat, R., et al.: A multi-sensor data fusion enabled ensemble approach for medical data from body sensor networks. Inform. Fusion 53, 155–164 (2020) 29. Sodhro, A.H., Luo, Z., Sangaiah, A.K., Baik, S.W.: Mobile edge computing based qos optimization in medical healthcare applications. Int. J. Inform. Manag. 45, 308–318 (2019)

Comparison of the Routing Algorithms Based on Average Location Error and Accuracy in WSN P. Sakthi Shunmuga Sundaram and K. Vijayan

Abstract Nowadays everything becomes wireless sensors, WSNs used in many applications like health care systems, surveillance of natural disasters mainly in the military, in the wireless sensor network the most challenging parameters are average localization error, time, and accuracy of the network, theaccuracy will play the major role in the network. In this paper, we are discussing the different types of algorithms and compare the accuracy of the target detection algorithms. Here taken some of the sample algorithms namely followed EPSM, PCA-RNN, and ANN, the main goal of this research work is to provide a survey of the algorithms. The detailed analysis of the algorithm is represented in a graphical model. Keywords Wireless sensor network · Accuracy · Artificial neural networks · Clustering · Scheduling · Quality of service

1 Introduction Wireless sensor networks include research in IOTs [1], inwhich WSNs harvest the energy with the information decode the received RF signals [2, 3]. The dynamic networking environment is integrated with a wireless sensor network [4]. The sensor nodes are communicated through a wireless medium for transmission [5]. Wireless sensor networks are used in many real-time scenarios [6]. ALocation identification error will occur when it is computed in mobile phone users for the cellular model [7]. Terminal devices are used to identifythe location of each user [8]. Using the infrared we can improve the [9, 10] performance of location identification [11]. In the real-time location identified [12] method the device wants to carry out by the users [13]. P. S. S. Sundaram · K. Vijayan (B) Department of Electronics and Communication Engineering, SRM Institute of Science and Technology, Kattankulathur, Chennai, Tamil Nadu 603203, India e-mail: [email protected] P. S. S. Sundaram e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_32

411

412

P. S. S. Sundaram and K. Vijayan

Fig. 1 Wireless sensor network

The device-free method used in the wireless sensor network to identify a location [14] will utilize the variation in RSS [15] sensor nodes having the power of finding the problem solutions [16]. Wireless sensor networks have a monitoring region [17], the target that comes inside the region will receive the signals [18] to make batter RSS variation in location-based identification [19]. Using the device-free method in compressive sensing and Bayesian technic for identified the location [20] the basic wireless sensor network is shown in Fig. 1.

2 Methodology We are discussing the different types of methodologies, algorithmsused in that methodologies and compare the average localization error, time, and accuracy of algorithms. Here take some of the sample algorithms namely followed EPSM, PCA-RNN, and ANN Methods in Fig. 2. Fig. 2 Types of methodology algorithms

Comparison of the Routing Algorithms …

413

3 ANN Methodology One of the main important things is to find the Hotspot issues when the transmission happens on Wireless Sensor Networks (WSNs). Location identification is focusing on localization, the localization is used to find the target terminal. Artificial Neural Networks (AAN) identified the location [1]. The Received Signal Strength (RSS)having the parameter with the collected data within the range of communication. ⎤ ⎡ α1,1 · · · α1,P ⎥ ⎢ (1) RSSmat = ⎣ ... . . . ... ⎦ α P,1 · · · α P,P

The RSS different matrix is computed as αi = |αi − RSS|, i = 1, . . . , M

(2)

The main difference is gathered datausing this method, output isa nonlinear function based on location identification. The output value it’s in the region, using an ANN model the indexed matrix is trained all sensor, and the location identification is fixed.

4 ANN Algorithm The ANN model implementsan identified free device location system in the wireless sensor networks [1], the target assessed without device terminal,the wireless sensor network is constructed with ZigBee and gathered data of RSS. ANN algorithm flow chart is shown in Fig. 3. During transmissionmonitoring range the K value in a peak, the value of RSS variations, and the index matrix collected input coordinator for a location identified out of ANN outline. Theposition error in calculating the location sensor nodes when the improvement of the scheming distance of location empathy [1]. The output of the ANN Algorithmis plotted in Fig. 4 between transmission radius and average localization error. In Fig. 5 between the total number of sensor nodes and total time in seconds. In Fig. 6 computing location in meters versus accuracy.

5 EPSM Methodology In Energy Preserving Secure Measure (EPSM) in contradiction of the Wormhole SpellNetworks, energy measurement ofthe network depending on the connectivity of detectiona wormhole attack, tested the detection accuracy is 90% [20]. EPSM

414 Fig. 3 ANN algorithm flow chart

Fig. 4 Transmission radius in meter versus average localization error ANN-model

P. S. S. Sundaram and K. Vijayan

Comparison of the Routing Algorithms …

415

Fig. 5 Total number of sensor nodes versus Total time in seconds ANN-model

Fig. 6 Computing location in meters versus accuracy (%) ANN-model

algorithm flow chart is shown in Fig. 7. Wireless sensor network designed by sensors, sensor sensing and absorb the surrounding information and sending to the base station through the cluster formation in the network with the connection of an internet. Sensors are distributed and these sensors are low cost and having limited batteries, computational ability, and memory size. For the security purpose the sensors are dispassionate ability to expose by attacked, in the network there are different types of attack will occur in that wormhole attack is also one of the attacks it will attack the routing layers. The wormhole attacks the network and generating the fake short root. It’s formed when two sensor nodes are established with a wired or wireless connection between them, the energy preserving method detects the wormhole attack from the network, and this method is used in the AODV protocol [20]. This method has two stages to apply all the sensors in the selected path in the transmission. This method is achieved an accuracy level of 90%. By using this method we will get a better energy-efficient, throughput, and end-to-end delay. The output of

416

P. S. S. Sundaram and K. Vijayan

Fig. 7 Flow chart EPSM

the EPSMAlgorithmis plotted in Fig. 8 between transmission radius and average localization error. In Fig. 9 between the total number of sensor nodes and total time in seconds. In Fig. 10 computing location in meters versus accuracy. Fig. 8 Transmission radius in meter versus localization error average EPSM-model

Comparison of the Routing Algorithms …

417

Fig. 9 Total number of sensor nodes versus total time in seconds EPSM-model

Fig. 10 Computing location in meters versus accuracy (%) EPSM-Model

6 PCA-RNN Algorithm A PCA-RNN (Principal Component Analysis-Recurrent Neural Network) is to categorizethe DDoS that spells the network, selecting acharacteristic to describe the network traffic of the network. The PCA algorithm reduces the time complexity of detection. When put on to PCA, a prediction time will be reduced and the original information will be contained. CA-RNN is used to give better accuracy, sensitivity, and precision. The DDoS influenceis a malicious effortthat clopsthe normal traffic in the targeted network. This model detects DDoS attacks in the network [20]. PCA-RNN algorithm flow chart is shown in Fig. 11. The accuracy will provide the relation between the attacks and its confidential into total no of attacks [20]. Precision conveys the portion of forecasts isprecise. When Memory is one of the portions of DDoS spells and that is forecast. F-score is regular sensitivity and precision.

418

P. S. S. Sundaram and K. Vijayan

Fig. 11 Flow chart for PCA-RNN algorithm

Accuracy (acc.) =

TP +TN T P + T N + FP + FN

TP T P + FP TP Sensitivity (sen.) = T P + FN  pr e · sen F − score = 1 + β 2 2 β pr e + sen

(3)

Precision (pre) =

Note, TP-No of true DDoS attacks in the network, TN-No of true normal traffic in the network, FP-No of false DDoS attacks in the network, FN-No of false normal traffic in the networks.

(4)

Comparison of the Routing Algorithms …

419

7 Discussion A review of EPSM, PCA-RNN, and ANN algorithm, the accuracy of each algorithm and the Average Location Error of each algorithm are discussed based on the transmission radius (m) the Average location error going to be varied. Here we are taken the transmission radius as follows 10, 15, 20, 25, 30, 35, 40, 45 based on this radius what’s the average location error going to get in the different algorithms. the location error performance comparison tabulated in Table 1, accuracy presentage performance comparison tabulated in Table 2, and the overall performance is also tabulated in Table 3. The output of the PCA-RNN Algorithm plotted in Fig. 12 between transmission radius and average localization error. In Fig. 13 between the total number of sensor nodes and total time in seconds. In Fig. 14 computing location in meters versus accuracy (Figs. 15, 16 and 17). Table 1 Location error performance comparison S. no

Algorithm type

Location error

1

ANN-Model

7,6.3,6,5.5,5.3,5,3.8,3,2.4,1.2

2

PCA-RNN Model

7.5,6.8,6.4,5.9,5.6,5.5,4.5,3.5,3,1.6

3

EPSM

8,7.6,7.5,6.8,6.7,6.5,5.2,4.5,3.6,2.4

Transmission radius as follows 10,15,20,25,30,35,40,45

Table 2 Accuracy presentage performance comparison S. no

Algorithm type

Accuracy (%) Transmission radius as follows 10,15,20,25,30,35,40,45

1

ANN-Model

93,92.8,92.1,91.4,90,89.8,89.1,88.8

2

PCA-RNN Model

92,90.5,90,89.5,88.3,88,87.8,87.4

3

EPSM

90,89.5,89,88.8,88.5,88,87.5,87.2

Table 3 An overall performance comparison S. no

Algorithm type

Location error

Time(s)

Accuracy (%)

1

ANN-model

Less error

Fast time

Good accuracy

2

PCA-RNN model

Average error

Average time

Average accuracy

3

EPSM

More error

Low time

Low accuracy

420 Fig. 12 Transmission radius in meter versus localization error average PCA-RNN-Model

Fig. 13 Total number of sensor nodes versus total time in seconds PCA-RNN-model

Fig. 14 Computing location in meters versus accuracy (%) PCA-RNN-Model

P. S. S. Sundaram and K. Vijayan

Comparison of the Routing Algorithms … Fig. 15 Comparison of ANN, EPSM, and PCA-RNN-model transmission radius in meter versus average localization error

Fig. 16 Comparison of ANN, EPSM, and aPCA-RNN-model total number of sensor nodes versus total time in seconds

Fig. 17 Comparison of ANN, EPSM, and A PCA-RNN-model computing a location (m) versus accuracy (%)

421

422

P. S. S. Sundaram and K. Vijayan

8 Conclusion This paper discussed these three types of algorithms and compared the accuracy of the target detection of EPSM, PCA-RNN, and ANN algorithms. The simulation result for Transmission radius in Meter versus average localization error for each algorithm for ANN, EPSM, and PCA-RNN-Model,The simulation result for Total number of sensor nodes versus Total time in seconds for each algorithmfor ANN, EPSM, and PCA-RNN-Model,The simulation result Computing a location (m) versus accuracy (%) for each algorithm for ANN, EPSM, and PCA-RNN-Model. The analyzed output of the comparison is representation in the form of graphics and is plotted. The performance analysis table of the EPSM, PCA-RNN, and ANN algorithms will lead to future work to find some novel algorithms. This analysis results will help researchers to propose novel solutions.

References 1. Harold Robinson, Y., Ganesh Babu, R., Lakshmi Narayanan, K., Krishnan, R., Santhana Krishnan, R., Paramaiyappan, M.: Enhanced location identification technique for wireless sensor networks. 2021 5th International Conference on Trends in Electronics and Informatics (ICOEI)|978–1–6654–1571–2/21/$31.00 ©2021 IEEE. https://doi.org/10.1109/ICOEI51242. 2021.9452861 2. Zanella, Bui, N., Castellani, A.P., Vangelista, L., Zorzi, M.: Internet of things for smart cities. IEEE Int. Things J. 1(1), 22–32 (2014) 3. Harold Robinson, Y., Rajaram, M.: Energy-aware multipath routing scheme based on particle swarm optimization in mobile ad hoc networks. Scient. World J., 1–9 (2015) 4. Robinson, Y.H., Vimal, S., Julie, E.G. et al.: 3-dimensional manifold and machine learningbased localization algorithm for wireless sensor networks. Wirel. Pers. Commun. (2021). https://doi.org/10.1007/s11277-021-08291-9 5. Li, Q., Meng, L., Zhang, Y., Yan, J.: DDoS attacks detection using machine learning algorithms, digital TV and multimedia communication. Communications in Computer and Information Science. Springer, Singapore, vol. 1009, pp. 205–216 (2019). https://doi.org/10.1007/978-98113-8138-6_17 6. Zhou, M., Tang, Y., Tian, Z., Xie, L., Nie, W.: Robust neighborhood graphing for semisupervised indoor localization with light-loaded location fingerprinting. IEEE Int. Things J. (2017) 7. M Zhou Y Tang Z Tian X Geng 2017 Semi-supervised learning for indoor hybrid fingerprint database calibration with low effort IEEE Access 5 4388 4400 8. Santhana Krishnan, R., Golden Julie, E., Harold Robinson, Y., Kumar, R., Son, L.H., Tuan, T.A., Long, H.V.: Modified zone-based intrusion detection system for security enhancement in mobile ad-hoc networks. Wirel. Netw., 1–15 (2019) 9. Zhu, Y.J, Li, Y.Q., Fan, Q.G., Wang, Z.: Ad hoc on-demand distance vector routing protocol based on load balance. In: Proceedings MATEC Web Conference, Art. no. 02090 (2016) 10. Huang, Y., Jin, L., Zhong, Z., Lou, Y., Zhang, S.: Detection and defense of active attacks for generating a secret key from wireless channels in a static environment. ISA Trans. (2019) 11. Robinson, Y.H., Julie, E.G., Jacob, I.J. et al.: Enhanced energy proficient encoding algorithm for reducing medium time in wireless networks. Wirel. PersCommun. (2021). https://doi.org/ 10.1007/s11277-021-08421-3

Comparison of the Routing Algorithms …

423

12. S-H Fang T-N Lin 2010 Cooperative multi-radio localization in heterogeneous wireless networks IEEE Trans. Wirel. Commun. 9 5 1547 1551 13. M Zhou Y Tang W Nie L Xie X Yang 2017 Grass MA: graph-based semi-supervised manifold alignment for indoor WLAN localization IEEE Sens. J. 17 21 7086 7095 14. Harold Robinson, Y., Golden Julie, E., Saravanan, K., Kumar, R., Son, L.H.: DRP: Dynamic routing protocol in wireless sensor networks. Wirel. Person. Commun., 1–17. Springer (2019) 15. Y Sun W Meng C Li N Zhao K Zhao N Zhang 2017 Human localization using multi-source heterogeneous data in indoor environments IEEE Access 5 812 822 16. Z Deng W Si Z Qu X Liu Z Na 2017 Heading estimation fusing inertial sensors and landmarks for indoor navigation using a smartphone in the pocket EURASIP J. Wirel. Commun. Netw. 1 2017 17. J Wang D Fang Z Yang 2016 E-HIPA: an energy-efficient framework for high-precision multitarget -adaptive device-free localization IEEE Trans. Mob. Comput. 16 3 716 729 18. WA Aliady SA Al-Ahmadi 2019 Energy preserving secure measure against wormhole attack in wireless sensor networks IEEE Access 7 84132 84141 https://doi.org/10.1109/ACCESS.2019. 2924283 19. N Patwari J Wilson 2010 RF sensor networks for device-free localization: measurements, models, and algorithms Proc. IEEE 98 11 1961 1973 20. J Wang Q Gao P Cheng Y Yu K Xin H Wang 2014 Lightweight robust device-free localization in wireless networks IEEE Trans. Industr. Electron. 61 10 5681 5689

Application of Variational Mode Decomposition to FMCW Radar Interference Mitigation Thilina Balasooriya, Prateek Nallabolu , and Changzhi Li

Abstract As the automotive industry progresses towards implementing advanced driver-assistance systems (ADAS) and autonomous vehicles (AVs) that involve radars, the probability of radar interference between vehicles will increase tremendously in the future. To address this issue, novel interference mitigation methods for real-time interpretation are necessary. This work presents an evaluation of variational mode decomposition (VMD) based baseband reconstruction for interference mitigation in frequency-modulated continuous-wave (FMCW) radars using a MATLAB simulation. The VMD algorithm is applied to decompose the interference-prone baseband signal into multiple frequency components. The interference generated component of the baseband signal is segregated from the interference-free baseband response. The authors find that VMD has potential for this application, with the simulation results indicating an improvement in the noise floor of the recovered interference-free baseband signal. Keywords Automotive radar · Frequency-modulated continuous-wave (FMCW) radar · Radar interference · Variational mode decomposition (VMD)

1 Introduction With the increasing interest for advanced driver-assistance systems (ADAS) and autonomous vehicles (AV), frequency-modulated continuous-wave (FMCW) radars have emerged as a key sensing modality due to their all-weather detection capability, low cost, and wide coverage [1, 2]. State-of-the-art millimeter-wave (mmWave) automotive radars have been employed for ADAS functionalities such as adaptive cruise control (ACC), blind-spot detection (BSD), and collision avoidance systems, T. Balasooriya Hamilton High School, Chandler, AZ 82548, USA T. Balasooriya · P. Nallabolu (B) · C. Li Texas Tech University, Lubbock, TX 79409, USA e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_33

425

426

T. Balasooriya et al.

to mention a few. Typically, each vehicle is equipped with multiple radars that serve various ADAS functions. With the increasing number of automotive radars on the road, interference among these radars needs to be addressed and tackled. The main manifestations of interference in FMCW radars are the occurrence of ghost targets and elevated noise floor, leading to reduced sensitivity in detecting weak targets. Several methods have been proposed to address interference mitigation in FMCW radar systems. In [3], a frequency hopping technique was proposed in which the operating frequency of the FMCW radar was randomly assigned in the 76–81 GHz band, such that it was mathematically improbable for two radars to operate in the same frequency sub-band. However, this is not foolproof, and as the environments become more crowded, random frequency hopping will be a less viable solution. Time-domain zeroing of interfered samples was proposed in [4], in which the interfered section of the baseband signal was identified by applying an amplitude threshold and replacing the corresponding data samples with zeros to get rid of the interference. This method compromises the accuracy by completely ignoring a section of data, which is not the optimal desired solution. To combat this issue, several works were proposed that estimate the interference pattern using techniques such as mathematical analysis [5] and adaptive noise cancellation [6]. However, the works mentioned above [5, 6] require complex baseband architecture radar for optimal performance. Since the interference removal can be considered as a classical denoising problem, sparse reconstruction techniques such as orthogonal matching pursuit (OMP) and Bayesian learning were used in [7, 8], respectively. If the number of interference-generated data samples increases, the sparse reconstruction methods will fail to recover the interference-free signal. A recurrent neural network (RNN) that implements a self-attention model was proposed in [9], which predicts how well the target output attends to the baseband input. Convolution neural networks (CNN) based interference mitigation was proposed in [10, 11], where interference added range-Doppler (RD) frames were given as input, and the CNNs were trained to output the interference-free RD frame. However, to generate an extensive training set that included all the real-world interference patterns would be an impossible task. This paper performs a feasibility study on the application of variational mode decomposition (VMD) for interference mitigation in FMCW radars. MATLAB simulations were performed to evaluate the performance of VMD to remove interference generated baseband data for various interference patterns. Unlike existing methods, the proposed VMD approach doesn’t require identifying the interference-affected data samples beforehand. The rest of this work is organized as follows: Sect. 2 presents the theory of FMCW radar interference and VMD. Section 3 discusses the MATLAB simulation setup and the obtained simulation results. Finally, conclusions are drawn in Sect. 4.

Application of Variational Mode Decomposition …

427

2 Theory 2.1 FMCW Interference Model Figure 1 shows a real-world scenario of the interference caused in an automotive radar by a surrounding radar. An automotive FMCW radar transmits a chirp signal characterized by a bandwidth B and chirp duration T. This chirp signal gets reflected off a target, undergoes amplification on the radar’s receiver chain, and upon deramping generates an intermediate frequency (IF) signal (also referred to as baseband signal). The de-ramping operation refers to the mixing of the received signal with a portion of the transmitted chirp signal. The time delay between the transmitted and received chirp signal translates to the frequency of the baseband signal, referred to as beat frequency. The generated beat frequency is unique depending on the range of the target. The generated baseband signal is passed through a low-pass filter (LPF) and then further amplified before it is digitized for further signal processing. Ideally, the cut-off frequency of the LPF is chosen based on the maximum detectable range of the radar, which in turn dictates the maximum beat frequency generated by the de-ramping operation. The baseband signal generated at the output of the LPF can be mathematically represented as:    SIF (t) = HLPF (t) × Re STX (t) × SRX (t) ,

(1)

where t represents the so-called fast time, Re[.] denotes the real part of the complex number, and (.)´ denoted complex conjugate operation. S TX (t), S RX (t), and S IF (t) represent the transmitted and received chirp signal, and the generated baseband signal, respectively. The transfer function on the LPF is represented as H LPF (t). H LPF (t) can be considered as unity for the chirp signals reflected off a true target

Fig. 1 Graphical illustration of the interference scenario in an automotive FMCW radar caused by a nearby radar

428

T. Balasooriya et al.

Fig. 2 The region of interference between the victim chirp and the interference chirp

because the generated beat frequencies lie within the cut-off frequency of the LPF. In the presence of an interfering radar transmitting a chirp signal S * TX (t), the interference added baseband signal S * IF (t) generated after the LPF of the victim radar is given as:   ∗  (t) = SIF (t) + HLPF (t) × Re STX (t) × STX (t − τ ∗ ) , SIF

(2)

where τ * represents the one-way signal propagation delay due to the arbitrary distance between the victim and interference radar. The similarity between the slope (bandwidth/chirp duration) of the victim radar chirp and the interfering radar chirp, and the bandwidth of the LPF determine the impact of the interference on the generated baseband signal. The more identical the slopes, more the number baseband data samples affected by the interfering radar. Also, the higher the LPF bandwidth, the greater the interference’s impact on the generated baseband data. Figure 2 shows the impact of the cut-off frequency of the LPF on the region of interference caused due to the interfering radar. It should also be noted that the similarity between the slopes of the victim and the interference chirp also affects the region of interference.

2.2 Variational Mode Decomposition Variational mode decomposition algorithm decomposes a non-stationary signal into consequent variational mode functions (VMFs) [12]. Each VMF is characterized by a center frequency and bandwidth. VMD consists of three major steps: computing the analytic signal for each mode μk using Hilbert transform, shift each mode’s frequency spectrum to baseband by using an exponential tuned to the mode’s estimated center frequency, and then perform Gaussian smoothing to estimate the bandwidth. The number of VMFs is directly inputted to the algorithm using the parameter k, while the parameter α decides the bandwidth and center frequency of each VMF. To summarize,

Application of Variational Mode Decomposition …

429

the higher the value of α, the narrower the bandwidth of each mode, and the center frequency increases with the mode number. Additional parameters include noise tolerance which can be set to zero for optimal results. From Fig. 2, the interference region is very narrow and results in high-frequency components in the baseband signal within the interference region. Using VMD, the interference-prone baseband signal can be decomposed into multiple modes such that the highest mode consists of the interference data samples, while the lower modes represent the baseband signal components generated by the true targets.

3 Simulation Results Based on the interference model presented in Sect. 2.1, a simulation setup was created in MATLAB. To ease the computational load, the operating band of the FMCW radars was limited to 1–500 MHz. The victim radar was configured with a bandwidth of 100 MHz (200–300 MHz) and a chirp duration of 1 ms. The cut-off frequency of the LPF in the victim radar was set to 50 kHz. In the first scenario, the victim radar was configured with a bandwidth of 130 MHz (190–320 MHz) and a chirp duration of 1 ms. A single true target was considered at 3 m, and the signal strength of the interfering chirp was three times that of the reflections from the true target. This reflects a realworld scenario, where the interference chirp must travel one way, undergoing lower path loss and thus having higher signal strength at the receiver port of the victim radar, compared to the reflections from true targets. Figure 3a demonstrates the baseband signal with added interference. The signal-to-noise (SNR) was set to 30 dB. The VMD algorithm was applied to the interference-prone baseband data with the parameters set to k = 2 and α = 5000. With k = 2, the signal was decomposed into two VMFs, where the first VMF corresponds to the interference-free baseband signal. Figure 3b shows the decomposed VMF modes 1 and 2. Mode 1 represents the interference-free baseband, and mode 2 represents the interference plus noise. Figure 3c shows a comparison of the range maps of the interference-prone and interference-free baseband signals along with the ground truth (ideal scenario with (a)

(b)

(c)

Interference region

T

Fig. 3 a The generated interference-prone baseband signal, b VMD decomposition output modes, and c obtained range map for simulation scenario 1

430

T. Balasooriya et al.

no interference). The increase in the noise floor of the signal with added interference and the significant improvement in the noise floor in the interference-free range map can be observed. In the second scenario, the FMCW parameters of the interference radar were set to a bandwidth of 100 MHz (195–305 MHz) and a chirp duration of 1ms. Due to the high correlation between the victim and interference chirp, the interference region will be broader in this scenario. A true target at 35 m was considered, and the ratio of signal strength of the interference signal to that of the reflection from the true target was set to ten. The VMD parameters were unchanged. As seen in Fig. 4a, the interference region was broader, and the interference samples’ amplitude was much higher than scenario 1. As the beat frequency generated due to the true target at 35 m is higher, the decomposed mode 1 in Fig. 4b, representing the interference-free baseband signal, still has some distortion. However, the range map for the interference-free baseband data has a much lower noise floor than the interference added range map, as shown in Fig. 4c. In the third simulation scenario, two real targets were considered at 5 and 25 m. The bandwidth and time duration of the interference chirp was set to 130 MHz (190– 320 MHz) and 1 ms, respectively. The VMD algorithm was applied to decompose the baseband signal shown in Fig. 5a into three VMFs by setting k to 3. However, since α = 5000 represents a moderate bandwidth constraint, both the target responses were decomposed into mode 1, as seen in Fig. 5b. Ideally, for targets at a farther distance, their response would be decomposed into mode 2, and summing mode 1 and 2 output (a)

(b)

(c)

Fig. 4 a The generated interference-prone baseband signal, b VMD decomposition output modes, and c obtained range map for simulation scenario 2 (a)

(b)

(c)

Ghost peak

Fig. 5 a The generated interference-prone baseband signal, b VMD decomposition output modes, and c obtained range map for simulation scenario 3

Application of Variational Mode Decomposition …

431

would provide an estimate of the interference-free baseband signal. From the range maps shown in Fig. 5c, it can be observed that the noise floor in the interference-free case was improved until the 100 m range. However, around 125 m, the noise floor hits a peak and appears like a ghost target. In this scenario, given the correct set of VMD parameters, the ghost peak could be avoided. The VMD approach poses few drawbacks. The recovery of the interference-free baseband signal is dependent on various factors like the similarity between the victim and interference chirp slopes, the distance of the true targets, the number of true targets, and the signal strength of the interference chirp. Different scenarios might require different VMD parameters to obtain the optimal interference-free baseband reconstruction. Therefore, it is necessary to either find a standardized set of parameters that work almost universally or create an algorithm to decide the appropriate VMD parameters given the situation.

4 Conclusion As radar-based automotive features and self-driving cars become more prevalent, the probability of radar interference will increase, causing many potential driving hazards. A radar interference mitigation technique based on a variational mode decomposition approach was evaluated using MATLAB simulation to address this need. The use of VMD for removing interference-affected samples from the baseband signal has proven promising. The recovered interference-free baseband signals exhibited improvement of the noise floor. Future work includes creating a standardized set of VMD parameters for any interference scenario and addressing the issue of ghost peaks. Acknowledgement The authors wish to acknowledge the National Science Foundation (NSF) for funding support under grant ECCS-2028863 and ECCS-1808613.

References 1. Vossiek, M., Heide, P., Nalezinski, M., and Magori, V.: Novel FMCW radar system concept with adaptive compensation of phase errors. In: 26th European Microwave Conference, pp. 135–139 (1996) 2. Ju, Y., Jin, Y., Lee, J.: Design and implementation of a 24 GHz FMCW radar system for automotive applications. In: International Radar Conference, pp. 1–4 (2014) 3. Aydogdu, C., et al.: Radar interference mitigation for automated driving: exploring proactive strategies. IEEE Signal Process. Mag. 37(4), 72–84 (2020) 4. Brooker, G.M.: Mutual interference of millimeter-wave radar systems. IEEE Trans. Electromagn. Compat. 49(1), 170–181 (2007) 5. Bechter, J., Biswas, K.D., Waldschmidt, C.: Estimation and cancellation of interferences in automotive radar signals. In: 18th International Radar Symposium (IRS), pp. 1–10 (2017)

432

T. Balasooriya et al.

6. Jin, F., Cao, S.: Automotive radar interference mitigation using adaptive noise canceller. IEEE Trans. Veh. Technol. 68(4), 3747–3754 (2019) 7. Correas-Serrano, A., Gonzalez-Huici, M.A.: Sparse reconstruction of chirplets for automotive FMCW radar interference mitigation. In: IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), pp. 1–4 (2019) 8. Chen, S., Taghia, J., Kühnau, U., Fei, T., Grünhaupt, F., Martin, R.: Automotive radar interference reduction based on sparse Bayesian learning. In: IEEE Radar Conference (RadarConf20), pp. 1–6 (2020) 9. Mun, J., Ha, S., Lee, J.: Automotive radar signal interference mitigation using RNN with self attention. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3802–3806 (2020) 10. Fuchs, J., Dubey, A., Lübke, M., Weigel, R., Lurz, F.: Automotive radar interference mitigation using a convolutional autoencoder. In: IEEE International Radar Conference (RADAR), pp. 315–320 (2020) 11. Rock, J., Toth, M., Messner, E., Meissner, P., Pernkopf, F.: Complex signal denoising and interference mitigation for automotive radar using convolutional neural networks. In: 22nd International Conference on Information Fusion (FUSION), pp. 1–8 (2019) 12. Dragomiretskiy, K., Zosso, D.: Variational mode decomposition. IEEE Trans. Signal Process. 62(3), 531–544 (2014)

Design of a Microwave Planar Device for Humidity Detection Durga Nand Mahaseth, Tarikul Islam, and Upendra Mittal

Abstract This paper presents a new design of planar resonator, which is used for humidity sensing application. A microwave based narrow band pass filter is designed for humidity detection with change in effective permittivity. Effective permittivity is considered as a combined permittivity of sensing film and absorbed water molecule permittivity. Resonator changes resonance frequency linearly to water vapor absorption. In this paper, we have selected the substrate materials such as FR4 for making planar resonator. Different design parameters like length and width and effective permittivity of the resonator, which depends on operating resonance frequency as well as material used for sensing. The resonator is then exposed to the different level of humidity and gets the significant frequency shift with change in relative humidity. Keywords Humidity sensor · Planar resonator · Band pass filter

1 Introduction Humidity plays an important role in lab. While performing an experiment, this affects the results of any type of research. This is one of the main important factors, which affects almost everything present in the environment including weather condition. It affects the human life directly or indirectly in different ways such as healthcondition, food storage, building, electronic gadgets, vehicle performance and many more significantly. Many different types of the sensors are reported in the literature to measure humidity but still research is continuously going on to optimize the device parameters like size, sensitivity, drift, power consumption etc. [1–9]. In this paper we discussed about microwave frequency-based planer humidity sensor. The advantage D. N. Mahaseth (B) · T. Islam Jamia Millia IslamiaA Central University, New Delhi 110025, India e-mail: [email protected] T. Islam e-mail: [email protected] U. Mittal SSPL, DRDO, New Delhi, India © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_34

433

434

D. N. Mahaseth et al.

of this resonator based senor is easy to design due to Planar structure, easy to interface with electronic circuitry, very compatible to any prototype measuring system with cost effectiveness and easy to calibrate [10]. An approach to compact humidity sensor is reported here. It is based on change in permittivity with different level of humidity. Base resonance frequency is changed with change in effective permittivity. The concept compromises with resonator structure directly depend on resonating frequency, that is resonator length and width would be different for different operating frequencies. The width of the micro-stripline is worked as a feed line, which is to be optimized for impedance matching with 50 connector [11–13]. The effective permittivity is described in the Sect. 1. The complete mathematical background of designed parameters is illustrated in the Sect. 2. The all designed parameters such as effective permittivity, microstripline width, and resonator length for FR4 substrate is calculated in the Sect. 3. The practical response of the sensor with humidity and result analysis is reported in the Sect. 4.

2 Proposed Humidity Sensor Proposed microwave planer resonator for humidity detection is designed as shown in Fig. 1. Resonance frequency depends on the length of the resonator through which wave travel. Effective length of the resonator stub (L2 + 2L1 ) would be equal to wavelength divided by two (λ/2). So that, maximum magnitude will be appear at both ends A and B. Hence, it acts like a band pass filter (BPF). If stub length will equal to λ/4, one ends (either A or B) behaves like a open and other end as the short, so the signal attenuates at the short end. Therefore, it acts as a band reject filter (BRF). As a result, we could say that, if the length is even number multiple of λ/4, the resonators behaves like BPF, and if the length is odd number multiple of λ/4, it acts as a BRF. Also, the width of the resonating stub affects the bandwidth (wide/narrow) of the filters. As the width increases the bandwidth decreases. Similarly, with decrease in width, the bandwidth increases. So, there is an inverse relation between the resonating stub width and the bandwidth of the designed filters. Fig. 1 Proposed planar resonator based microwave humidity sensor

Design of a Microwave Planar Device for Humidity Detection

435

2.1 Coupling Effects Coupling defines a major role in the designing of filter. If the coupling is weak, the load does not receive enough power, so the frequency selectivity will be poor. On the other hand, if the coupling will strong, not only frequency but also the wide range of frequency will be selected, it means wide bandwidth. So, we cannot say that which particular frequency will be resonance frequency. Hence, resonance phenomenon will be erased. So, there is a need to optimize the coupling effect. The coupling can be of different types such as an end coupling for bandwidth less than 5%, a line coupling for bandwidth 5–20% and a direct coupling for bandwidth greater than 20% respectively.

2.2 Relationship Between Strip Width (W) and Substrate Thickness (D) The microstripline, for given characteristic impedance (Z0 ) and a dielectric constant (r ), the w/d ratio can be found using Eqs. (1) and (2). w = d



 8e A , e2 A − 2

f or w/d ≤ 2

(1)

    w 0.61 2 εr − 1 ln(B − 1) + 0.39 − , = B − 1 − ln(2B − 1) + d π 2εr εr f or w/d ≥ 2 (2)    0.11 Z 0 εr + 1 εr − 1 + 0.23 + wher e, A = 60 2 εr + 1 εr 377π and B = √ 2Z 0 εr

2.3 Microstrip Series Gap (g) A gap (g) can be useful in microstrip resonators. It is used to couple RF energy from input line to the resonator. It can also be used in direct current (DC) blocking circuit if the frequency is high enough. Single gap is rarely enough to provide enough coupling in PCB technology therefore, an interdigital structure can be used to boost coupling. The equivalent circuit of microstrip gap discontinuity is illustrated in Fig. 2.

436

D. N. Mahaseth et al.

Fig. 2 Proposed microstripline series gap discontinuity equivalent circuit

Cp =

1 Ce 2

(3)

Cp is parallel plate capacitance between microstrip line and ground plane, and capacitance Cg is formed due to fringing effect between microstrip lines separated by a gap (g).   1 1 C0 − Ce and Cg = 2 2

C0 εr 0.8 g m0 wher e, × e K0 × pF/mm = w w 9.6



g me Ce εr 0.9 = and × e Ke × pF/mm w w 9.6

(4)

where, Ce is even mode capacitance, when exciting the lines symmetrically (even mode) and Co is odd mode capacitance, when exciting the lines anti symmetrically (odd mode). Parameters mo , me , ko and ke are numerical value (unit less) for odd and even mode of excitation. This numerical value depends upon ratio of width (w) of the microstripline line and thickness (d) of the substrate.

w 0.619 log w d − 0.3853 , i f 0.1 ≤ g w ≤ 0.1 d m e = 0.8675 i f 0.1 ≤ g w ≤ 0.3 1.565 =  0.16 − 1 i f 0.3 ≤ g w ≤ 1.0 w d  K 0 = 4.26 − 1.453 log w d i f 0.1 ≤ g w ≤ 1.0  0.12 K e = 2.043 × w d i f 0.1 ≤ g w ≤ 0.3 0.03 = 1.97 − i f 0.3 ≤ g w ≤ 1.0 w d m0 =

Design of a Microwave Planar Device for Humidity Detection

437

2.4 Effective Permittivity The effective dielectric constant value depends on the dielectric layer on the substrate. The dielectric layer may be a simple substrate, or sensing film. The dielectric constant of the sensing film changes due to deposition of water vapor in the film. The effective dielectric (eff ) of a bare substrate without sensing film (air) is shown in (5). The  effective dielectric with sensitive material εe f f is shown in Eq. (6). Where, r is represented as relative dielectric constant of the substrate material used for fabrication  whereas εe f f is used for composite dielectric constant including sensitive coating on the planer resonator (see Fig. 3). εe f f =

1 εr + 1 εr − 1 + × 2 2 1+ 



εe f f

(5)

12 w/d



εr − ε 1 εr + ε + × = 2 2 1+

(6)

12 w/d

Now we calculated length of the resonator for required resonance frequency (fr ). Let us assume a resonator with resonance frequency 1 GHz frequency. So, for resonance, effective resonator length (l) should be equal to half wavelength (λ/2) or phase difference 180°. Parameter ‘n’ denotes the effective loop presents in designed resonator. Here we considered only one resonator loop (n = 1) i.e. one U-shape stub line resonator. 1800 = βle f f = 2π fr wher e, K 0 = nc

√

εe f f K 0 le f f

√ where, ‘β’ is phase constant ( εe f f K 0 ) and symbol ‘c’ represents speed of light in m/sec. Hence, the resonance frequency can be given by 1 nc × fr = √ εe f f 2le f f

(a) Fig. 3 Microstrip line a original geometry b equivalent geometry

(b)

438

D. N. Mahaseth et al.

3 Working Principle 3.1 Microstripline Length Calculation Different types of substrate are used for the design of the microwave resonators of different frequencies. A resonator was designed on FR4 substrate material. In this paragraph we have discussed about all the parameters like width (w), length (L) and effective dielectric constant for required resonance frequency. The geometrical parameters of the FR4 sheet are relative permittivity (εr = 4.4), thickness (d = 0.8) mm and loss tangent (tan δ = 0.02). The S-parameters measuring device like Network analyser is having 50 connector in our lab. So microstrip feeding line width (w) is designed as with perfectly matched impedance to 50 connector.    50 4.4 + 1 4.4 − 1 0.11 A= + 0.23 + = 1.52 60 2 4.4 + 1 4.4 w 8 × e1.52 so, = 2×1.52 = 1.91 ≤ 2 d e −2 Hence, the width of the microstripline can be calculated by using the formula (w = 0.8 × 1.91 = 1.53 mm). The effective permittivity (εe f f ) of the substrate can be calculated as below in air surrounding. εe f f =

1 5.4 3.4 + × 2 2 1+

12 1.91

= 3.33

The effective resonator length at 1 GHz resonance frequency is calculated by using these values below. Resonance frequency is inversely proportion to effective permittivity at constant resonator length. 2 × 3.14 × 1 × 109 = 21 1 × 3 × 108 1 3.14 1 so, le f f = 180 × ×√ = 82 mm × 180 3.33 21 K0 =

To reduce physical dimension of resonator, we can use bend type structure like Ushape, square, circular and many more. Here, we can see that if we increase required resonance frequency on same substrate, the effective length of the resonator can be reduced, so the size of the sensor can be reduced accordingly.

Design of a Microwave Planar Device for Humidity Detection

439

Fig. 4 Planar resonater based humidity sensor with 10% PVA

3.2 Fabrication of Polyvinyl Alcohol Coated Microwave Humidity Sensor Polyvinyl alcohol (PVA) solution having -OH group, so it is hydrophilic in nature. PVA is used for significant sensitivity, low cost, high resolution and accurate RH sensing application [14]. Dielectric properties of PVA in aqueous solution are investigated up to 20 GHz frequency. It forms hydrogen–hydrogen (−H–H−) bond in presence of water molecule to change its dielectric and conductivity properties. It can be made more sensitivity by adding proper polymer catalyst. The PVA polymer was purchased from Sigma Aldrich. The solution was made by adding PVA flakes in the DI water/ethanol solution in different ration to make 10% solution. The solution was stirred for three to furs hours at normal temperature on magnetic stirrer continuously. The flakes were completely dissolved and transparent. Dip coater make SDC 2007C (Apex Instrument) was used for coating the planar resonator. Sensing film was deposited between microstripline and U-shape resonator stub. Series capacitance formed between microstripline and resonator will be changed due to sensing film exposed to humidity. The planar resonator was dipped in the solution for 5 min, then lifting and dry at 600C for 15 min. The film was deposited ten times at particular constant dipping parameters.

4 Experimental Work and Result Analysis The planar microwave sensor was fabricated according to the designed parameters for humidity sensing at 1 GHz resonance frequency. The resonator was coated with 10% PVA solution. Figure 4 shows the photograph of the PVA film microwave humidity sensor. Experiments were conducted at three distinct humidity levels such as 10% RH (Silica gel), 32.8% RH (Magnesium chloride) and 43% RH (Potassium carbonate). The sensor was placed in a sealed desigator with dry silica. The sensor was connected to the connectors through the shielded cables of a Vector network analyzer

440

D. N. Mahaseth et al.

Fig. 5 Experimental setup in lab for Humidity planar sensor

Fig. 6 Experimental results in lab for Humidity planar sensor

(HP 8753 VNA). When the humidity in the chamber was stable, the ‘S21 ’parameter was acquired through the data acquisition system. Experiments were then conducted with two saturated salt solutions. The magnitude variation with frequency of the ‘S21 ’ parameter at three different humidity levels is shown in Figs. 5 and 6. As Fig. 4 shows that the resonating frequency was 977 MHz with molecular sieve (10% RH). When, increase in % RH, the resonating frequency was shifted to 975 MHz. There is almost 2 MHz change in frequency due to nearly 23% RH change in humidity. Detail experimental results and the response parameters will be the further scope of the work.

Design of a Microwave Planar Device for Humidity Detection

441

References 1. Mittal, U., Islam, T., Nimal, A.T., Sharma, M.U.: A novel sol–gel γ-Al2 O3 thin-film-based rapid SAW humidity sensor. IEEE Trans. Electron Devices 62(12), 4242–4250 (2015) 2. Yao, Y., Chen, X., Guo, H., Wu, Z.: Graphene oxide thin film coated quartz crystal microbalance for humidity detection. Appl. Surf. Sci. 257(17), 7778–7782 (2011). https://doi.org/10.1016/j. apsusc.2011.04.028 3. Lu, L., et al.: High performance SnO2 /MoS2 -based surface acoustic wave humidity sensor with good linearity. IEEE Sens. J. 19(23), 11027–11033 (2019). https://doi.org/10.1109/JSEN. 2019.2937186 4. Wang, F., et al.: A miniaturized integrated SAW sensing system for relative humidity based on graphene oxide film. IEEE Sens. J. 20(17), 9733–9739 (2020). https://doi.org/10.1109/JSEN. 2020.2989787 5. Xu, Z., Li, Z.: Design and fabrication of ZnO-based SAW sensor using low power homo-buffer layer for enhanced humidity sensing. IEEE Sens. J. 21(6), 7428–7433 (2021). https://doi.org/ 10.1109/JSEN.2021.3049350 6. Y. Su et al.: Surface acoustic wave humidity sensor based on three-dimensional architecture graphene/PVA/SiO2 and its application for respiration monitoring. Sens. Actuators B Chem. 308, 127693 (September 2019). https://doi.org/10.1016/j.snb.2020.127693 7. Mahaseth, D.N., Kumar, L., Islam, T.: An efficient signal conditioning circuit to piecewise linearizing the response characteristic of highly nonlinear sensors. Sens. Actuators A Phys. 280 (2018). https://doi.org/10.1016/j.sna.2018.08.001 8. Alam, S., Mittal, U., Islam, T.: The oxide film coated surface acoustic wave resonators for the measurement of relative humidity. IEEE Trans. Instrum. Meas. 70 (2021). https://doi.org/10. 1109/TIM.2021.3072108 9. Alam, S., Islam, T., Mittal, U.: A sensitive inexpensive SAW sensor for wide range humidity measurement. IEEE Sens. J. 20(1), 546–551 (2020). https://doi.org/10.1109/JSEN.2019.294 2521 10. Bernou, C., Rebière, D., Pistré, J.: Microwave sensors: a new sensing principle. Application to humidity detection. Sens. Actuators B Chem. 68(1), 88–93 (2000). https://doi.org/10.1016/ S0925-4005(00)00466-4 11. Bhartia, K.C.G.I.B.P.: Microstrip lines and slot lines.pdf (1996) 12. D. M. Pozar, Microwave Engineering. Wiley (2011) 13. Thomson, A.F., Gopinath, A.: Calculation of microstrip discontinuity inductances. IEEE Trans. Microw. Theory Tech. 23(8), 648–655 (1975). https://doi.org/10.1109/TMTT.1975.1128643 14. Penza, M., Cassano, G.: Relative humidity sensing by PVA-coated dual resonator SAW oscillator. Sens. Actuators B Chem. 68(1), 300–306 (2000). https://doi.org/10.1016/S0925-400 5(00)00448-2

A Compact Wideband Planar Monopole Antenna with Defected Ground Structure for Modern Radar Sensing Systems Zaheer Ahmed Dayo , Muhammad Aamir , Shoaib Ahmed Dayo , Permanand Soothar , Imran A. Khoso , Zhihua Hu, and Guan Yurong

Abstract This paper presents a compact and wideband planar monopole antenna with defected ground structure for modern radar sensing systems. Two antenna structures are designed and simulated. The first developed antenna model contains a miniaturized square patch; microstrip feed line and simple partial ground plane. The proposed antenna is fed with 50  microstrip feeding network. A low-cost FR4 epoxy substrate material is used with constant values of relative permittivity 4.4 and di-electric loss tangent 0.02. The designed antenna possesses an overall compact dimension of 31.68 × 18.95 mm2 printed on the thick substrate. Moreover, the second proposed antenna prototype is constructed by engraving inverted L-shaped slots on the upper surface of the simple partial ground plane hence; the planar antenna with defected partial ground plane is formed. The size and gap between the L-shaped slots influences the impedance bandwidth and radiation performance of the antenna. The parametric study of the different variables involved in the proposed antenna is performed. The antenna’s key features including impedance bandwidth and radiation pattern are simulated and analyzed. A good improvement of 15.35% in impedance bandwidth at S11 < –10 dB and stable near monopole like radiation pattern with gain Z. A. Dayo (B) · M. Aamir · Z. Hu · G. Yurong College of Computer Science, Huanggang Normal University, Huangzhou 438000, People’s Republic of China e-mail: [email protected] M. Aamir e-mail: [email protected] Z. Hu e-mail: [email protected] S. A. Dayo Department of Industrial Engineering, Universitá degli Studi di Salerno (University of Salerno), Via Giovanni Paolo II, 132-84084, Fisciano (SA), Italy P. Soothar School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, People’s Republic of China I. A. Khoso College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, People’s Republic of China © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 N. K. Suryadevara et al. (eds.), Sensing Technology, Lecture Notes in Electrical Engineering 886, https://doi.org/10.1007/978-3-030-98886-9_35

443

444

Z. A. Dayo et al.

performance around 4.0 dBi have been observed. The significant improvement in the antenna’s performance is attained particularly with the defected partial ground plane. The performance analysis results reveal that the proposed antenna is a good contender for modern radar sensing systems. Keywords Planar monopole antenna · Defected partial ground plane · Impedance bandwidth · Radiation pattern and modern radar sensing systems

1 Introduction In modern communication systems, there is a need to design a smart antenna, which is economical, small in size, light weight with wideband features and can be easily integrated with RF and microwave circuits [1]. Recently, microstrip patch radiators with different shapes are good choice for wireless communication systems. However, there are some disadvantages of microstrip patch antenna such as poor efficiency, narrower impedance bandwidth, radiation pattern performance, reasonable gain and large dimensions [2]. Therefore, in order to overcome the above-mentioned limitations, researchers are working on different techniques to achieve the optimal performance of patch antennas. An Ultra-wideband (UWB) spectrum covers a broad frequency ranges from 3.1 to 10.6 GHz allocated by the Federal Communication Commission (FCC) in the February 2002 [3]. Moreover, electromagnetic (EM) spectrum covers wireless applications including modern radio detection and ranging (RADAR) sensing systems [4], television (TV) broadcast satellites, wireless local area network (WLAN), spectrum sensing [5], world-wide interoperability through microwave access (WiMax), Wireless fidelity (Wifi), microwave devices, amateur radio and many more [6]. These modern wireless applications reside in the super high frequency (SHF) band of the EM spectrum which ranges from 3 to 30 GHz. Hence, it has become a motivation for antenna design engineers to model an efficient antenna system which operate within the specified frequency range and may exhibit broader bandwidth, higher data transmission rate at low operating power density spectrum. In addition, low cost, small volume, high accuracy, fewer difficulties, material availability in the market and easy fabrication process make the printed planar monopole antenna expands to be good contender for the modern radar sensing communication systems. Numerous studies on the compact wideband antennas have been proposed in Refs. [7–9]. The broadband wireless sensing system was developed in Ref. [10]. A simple low power transmitter has been introduced for the biomedical radar sensing applications [11]. The different shapes of the radiator and the modified ground plane were reported in [12–14]. However, the proposed antenna designs exhibits favorable results with the expensive cost and larger dimensions. Besides, wideband single unit antennas, multiband and multiple input multiple output (MIMO) antenna systems with notch band, tunable, defected ground structure (DGS) and reconfigurablity features were reported for cognitive radio, UWB, microwave imaging techniques, mod-

A Compact Wideband Planar Monopole Antenna with Defected …

445

ern wireless sensing and heterogeneous wireless applications [15–23]. The authors also worked on the reconfigurable antenna for cognitive radio and wireless sensing applications reported in Ref. [24]. Further, to improve the impedance bandwidth the researchers proposed different antennas including planar inverted F antenna (PIFA) and monopole antenna with DGS by incorporating the different shapes of slots, tuning stubs and proper selection of the feed line [25, 26]. Additionally, the existing studies proposed a heart shaped patch-based planar monopole antenna with re-configurable notch band features [27]. The authors have filtered the specific frequency band by loading the circular shaped slot engraved on the patch. An stepped open slot antenna for broadband applications has been proposed in Ref. [28]. The reported work achieved the wideband results on the large dimensions substrate. Recently, Zere Iman proposed an antenna with symmetrical slot loaded DGS [29]. The antenna exhibited wideband features and can be used for microwave image applications. Moreover, miniature wideband antennas with heart shaped patch, rectangle, hexagonal slots etched out from the ground plane and loaded different shaped stubs were proposed in Refs. [30, 31]. However, to analyze the modern antenna topology, the compact size and adjustment in the designed antenna dimension in terms of variables is required. This adjustment can be achieved by sophisticated EM high frequency structure (HFSS) simulation software which has the capability of the rigorous optimization. In this study the authors have designed and simulated the compact wideband antenna with defected ground structure (DGS) for modern radar sensing wireless communication systems. The first designed antenna contains the miniature square patch, feed line and the simple partial ground plane (SPGP). The antenna is feed by 50  microstrip feed-line. Inverted L-shaped slots are etched on the upper side of SPGP and form an antenna with defected partial ground plane (DPGP). The alteration in the designed antenna is realized to ensure the improvement in bandwidth. Moreover, the parametric optimization of different variables has been carried out. The antenna has achieved near monopole like stable radiation pattern, improved impedance bandwidth, and the gain around 4.0 dBi. The simulation results of the two designed antenna prototypes i.e. antenna with SPGP and DPGP, are compared and the substantial improvement in the results has been observed.

2 Antenna Design Geometry, Performance Analysis and Mathematical Strategy The top and bottom view of designed antennas is depicted in the Fig. 1a, b. The antenna structure comprises of a compact square patch, microstrip feed line, simple partial ground plane (SPGP). The antenna model is designed on the top surface of thick substrate with compact dimensions 31.68 × 18.95 × 1.39 mm3 . A low cost FR4 epoxy laminate is used as substrate material with constant value of relative permittivity εr = 4.4 and dielectric loss tangent δ = 0.02. The square radiating patch has

446

Z. A. Dayo et al.

Fig. 1 Structural design of antenna with SPGP, a top view, b bottom view and c Simulated reflection coefficient of designed antenna with SPGP across the specified frequency

miniaturized dimensions of 9.49 × 9.49 mm2 . Moreover, radiating patch is placed at a certain distance apart from the left and right side of the dielectric substrate and is also separated from the DPGP by the length of 1.58 mm. The feed line has the dimension of 16.59 × 1.98 mm2 which is responsible to achieve the proper impedance matching. As can be seen from Fig. 1a, b, an SPGP is etched on the flipside of dielectric substrate and is taken as optimized value for broader impedance bandwidth. The variables of the radiator are adopted to adjust the resonances in addition to operating impedance bandwidth over the given wide frequency range. Figure 1c elucidates the simulated reflection coefficient of the designed antenna. It can be analyzed that the proposed antenna exhibited the broad frequency ranges from 3.4 to 10.4 GHz at 6 dB return loss which constitutes 7.0 GHz impedance bandwidth. Three resonances at 4.227 GHz, 6.571 GHz and 8.019 GHz have been observed. These resonances have

A Compact Wideband Planar Monopole Antenna with Defected …

(a)

447

(b)

(c) Fig. 2 Structural design of antenna with DPGP, a top view, b bottom view and c Simulated reflection coefficient of designed antenna with DPGP across the specified frequency

deeper return loss of 25.6 dB, 27.4 dB and 38.42 dB. Moreover, the broad impedance bandwidth of 6.06 GHz ranging from 3.69 to 9.75 GHz has been observed at 10 dB return loss. Moreover, an inverted L-shape slot is etched on the upper side of SPGP therefore making the antenna with DPGP as elucidated in Fig. 2a, b. An inverted L-shaped slot has the size of 5.23 × 0.5 mm2 . Figure 2c demonstrates the simulated reflection coefficient performance of the proposed antenna with DPGP. The two resonances are observed at 3.8 GHz and 9.8 GHz with deep return loss of 31 dB and 29.9 dB. It can be analyzed that the proposed antenna exhibited the impedance bandwidth ranging from 3.25 to 11 GHz at 6 dB return loss which constitutes 7.75 GHz broad impedance bandwidth. Further, the antenna exhibited the broad impedance bandwidth of 6.99 GHz ranging from 3.53 to 10.52 GHz at 10 dB return loss.

448

Z. A. Dayo et al.

Table 1 Variables assigned to the antenna lattice S. no. Variable name Variable acronym 01 02 03 04 05 06

Length of patch Width of patch Length of feed line Width of feed line Length of ground Width of ground

L1 W1 L2 W2 LDPGP WDPGP

Optimized values (mm) 9.49 9.49 16.59 1.98 14.23 18.95

It is noted that DPGP, gap between the square radiating patch and the dimensions of inverted L-shape slots have resulted in improved impedance bandwidth and excitation of multiple frequencies are due to an extra electromagnetic (EM) coupling between radiating element and the PGP. The proposed antenna is designed and simulated in the EM solver Ansys high frequency structure simulator (HFSS) version 13.0. Moreover, the calculated values have been obtained from the equations explained below. After multiple experimental simulations the optimal values of variables obtained with parametric study are listed in Table 1.

2.1 Utilized Equations in the Proposed Antenna Design Structure The dimensions of the radiating patch, dielectric substrate, feed line and partial ground plane of the proposed antenna can be obtained by the following approximated Eqs. (1–10). The value of patch length can be obtained by using the formula. L 1 = L e f f − 2 × ΔL

(1)

where, L 1 represents the length of patch, L e f f is the effective length and ΔL is the normalized extension in length. Moreover, L e f f can be calculated as follows: Lef f =

v0 √ 2 f εr e f f

(2)

where, v0 is the speed of light in free space, f is the resonant frequency and εr e f f is the effective dielectric constant. Furthermore, εr e f f can be calculated as follows. εr e f f =

εr e f f + 1 εr − 1 +    2 2 1 + 12 wh

(3)

A Compact Wideband Planar Monopole Antenna with Defected …

449

where, εr is the dielectric constant of the substrate, h and w, represents the height and width of the dielectric substrate respectively. Moreover, normalized extension in length due to fringing effect can be calculated as follows.  

   εr e f f + 1 wh1 + 0.264   ΔL = 0.412 × h  εr e f f − 0.258 wh1 + 0.8

(4)

where, W 1 represents the width of patch. Moreover, width of patch can be calculated as follows.   2 v0 W1 = (5) 2f εr + 1 The length of defected partial ground plane (DPGP) can be calculated as follows. L DPG P = L1 + 6 × h

(6)

where, L 1 the length of patch is, however h can be calculated as follows. h=

0.606 × λ √ εr

(7)

The DPGP width can be calculated as follows. W D P G P = W1 + 6 × h

(8)

The feed line length can be calculated as follows. L2 =

λg 4

(9)

where, λg is the guided wavelength, and can be calculated as follows. λ λg = √ εr e f f

(10)

3 Results and Discussion In this section, the analysis of the simulation results including parametric study and radiation pattern of the designed antennas are discussed. Furthermore, the parametric study is verified by the extensive simulations using defined variables such as: length and width of patch, feed-line and DPGP respectively.

450

Z. A. Dayo et al.

3.1 Variation of Patch with Respect to Length and Width The dimensions of the patch have great influence in achieving the proper impedance matching. Figure 3a, b demonstrates the variation of patch length and width from 8.49 to 9.49 mm. It is analyzed from Fig. 3a, b that the optimized values for length and width of patch have been achieved at 9.49 mm.

3.2 Variation of Feed Line with Respect to Length and Width Microstrip feed line is the key part of the proposed antenna design structure, from which excitation of the voltage or current to the antenna radiator can be realized. It is very important to set the proper dimensions of feed line so that the perfect impedance matching can be achieved. Figure 4a shows the different optimetric values of from 15.59 to 16.59 mm. It is analyzed that the obtained optimum results have been achieved at final value. Figure 4b demonstrates the width of feed line. It is experimentally verified at 1.38–1.98 mm and finally the proper matching is obtained at 1.98 mm.

3.3 Variation of DPGP with Respect to Length and Width Length of the DPGP plays an important role to achieve the broadband impedance bandwidth. The defined dimensions of PGP are almost half of the dimensions of the dielectric substrate. Figure 5 (a) demonstrates the optimal result of S11 < –10 dB for length of DPGP which is obtained at 14.23 mm. Figure 5b shows the optimized value

(a)

(b)

Fig. 3 Patch parameter variation versus frequency a length of patch, b width of patch

A Compact Wideband Planar Monopole Antenna with Defected …

(a)

451

(b)

Fig. 4 Feedline parameter variation versus frequency, a length of feedline, b width of feedline

(a)

(b)

Fig. 5 DPGP parameter variation versus frequency, a length of DPGP, b width of DPGP

of the width of DPGP which is achieved at 18.95 mm. It is observed from the Fig. 5a, b that the change in the dimensions of DPGP influences the wide impedance bandwidth and proper impedance matching. Finally, the optimized results of radiating patch, feed line and PGP are suggested for the accomplishment of the proposed antenna geometry particularly for broadband and modern radar sensing systems.

3.4 Radiation Pattern The two dimensional (2D) far-field radiation pattern at multiple frequencies particularly at θ = 90◦ of the designed antennas with SPGP and DPGP are shown in the Fig. 6a–c. It is analyzed that at multiple frequencies the designed antenna has nearly consistent and symmetrical monopole like radiation pattern. Further, it can

452

Z. A. Dayo et al.

(a)

(b)

(c) Fig. 6 Radiation performance of the planar monopole antennas, a at 3.8 GHz, b at 6.57 GHz and c at 9.8 GHz resonaces

be seen that the designed antennas has the good gain performance almost 4.0 dBi. The proposed antenna key characteristics are compared with the latest state of the art reported work. Table 2 clearly shows the advantages of the proposed work i.e. antenna dimensions and impedance bandwidth in comparison to the latest literature.

A Compact Wideband Planar Monopole Antenna with Defected … Table 2 Comparison table Reported References Antenna dimensions   (year) (L × W × H ) mm3 This Work (2022) [29] (2020) [31] (2019) [28] (2018) [27] (2017)

31.68 × 18.95 × 1.39 30 × 30 × 1.5 150 × 130 × 1.6 32 × 30 × 0.8 42 × 30 × 0.508

453

Bandwidth (GHz)

Substrate material

3.53–10.52 2.5–12.5 0.5–4.5 2.99–10.87 2.8–11.9

FR4 Epoxy Rogers 5880 PTFE FR4 Epoxy Rogers 5880

4 Conclusion and Future Work In this article, a compact wideband planar monopole antenna with defected ground structure for modern radar sensing systems has been proposed. Two antenna models were designed and simulated. The designed antennas consist of the miniaturized patch, ground plane and 50  microstrip feeding structure. The proposed antenna with defected ground structure (DGS) was realized by engraving the inverted L-shaped slot on the simple partial ground plane (SPGP). An alteration in the first antenna design was mainly to achieve the broadband and stable radiation features. The designed antenna key parameters such as radiation pattern, gain and bandwidth were analyzed. The proposed antenna exhibited the broad impedance bandwidth 3.53–10.52 GHz, which constitutes 6.99 GHz at S11 < –10 dB, stable nearly monopole like radiation performance and gain almost 4.0 dBi. The impedance bandwidth of the antenna with DPGP has been increased up to 15.35% as compared to the antenna with SPGP. The simulation results reveal that the proposed antenna is the good candidate for modern radar sensing systems. The manufacturing cost, soldering of sub-miniature-A (SMA) connector and testing the fabricated antenna model are challenging. The designed antennas may further extend to test the results in real time environment.

References 1. Kim, S., Nam, S.: Compact ultrawideband antenna on folded ground plane. IEEE Trans. Antennas Propag. 68(10), 7179–7183 (2020) 2. Dayo, Z.A., Cao, Q., Wang, Y., Ur Rahman, S., Soothar, P.: A compact broadband antenna for civil and military wireless communication applications. Int. J. Adv. Comput. Sci. Appl. 10(9), 39–44 (2019) 3. Yang, Y., Zhao, Z., Ding, X., Nie, Z., Liu, Q.H.: Compact UWB slot antenna utilizing travelingwave mode based on slotline transitions. IEEE Trans. Antennas Propag. 67(1), 140–150 (2019) 4. Dayo, Z.A., Cao, Q., Wang, Y., Pirbhulal, S., Sodhro, A.H.: A compact high-gain coplanar waveguide-fed antenna for military RADAR applications. Int. J. Antennas Propag. 2020(8024101), 1–10 (2020) 5. Gayatri, T., Anveshkumar, N., Sharma, V.K.: A compact planar UWB antenna for spectrum sensing in cognitive radio. In: 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), pp. 1–5 (February 2020)

454

Z. A. Dayo et al.

6. Soothar, P., Wang, H., Xu, C., Dayo, Z.A., Muneer, B., Kanwar, K.: A compact broadband and high gain tapered slot antenna with stripline feeding network for H, X, Ku and K band applications. Int. J. Adv. Comput. Sci. Appl. 11(7), 239–244 (2020) 7. Karthika, K., Kavitha, C., Kavitha, K., Thaseen, B., Anusha, G., Nithyaanandhan, E.: Design of a novel UWB antenna for wireless applications. In: 5th International Conference on Inventive Computation Technologies (ICICT) on Proceedings, pp. 937–941. IEEE (2020) 8. Dayo, Z.A., Cao, Q., Wang, Y., Soothar, P., Muneer, B., Chowdhry, B.S.: A compact broadband high gain antenna using slotted inverted omega shape ground plane and tuning stub loaded radiator. Wirel. Pers. Commun. 113(1), 499–518 (2020) 9. Eid, A.M., Wallace, J.W.: Ultrawideband speech sensing. IEEE Antennas Wirel. Propag. Lett. 8, 1414–1417 (2009) 10. Jha, A.K., et al.: Broadband wireless sensing system for non-invasive testing of biological samples. IEEE J. Emerg. Sel. Top. Circuits Syst. 8(2), 251–259 (2018) 11. Wang, X., Dinh, A., Teng, D.: Low-power low-complexity carrier-based UWB transmitter in 90 nm CMOS for wireless biomedical radar sensing applications. In: IEEE International Symposium on Circuits and Systems: Nano-Bio Circuit Fabrics and Systems, pp. 3477–3480 (2010) 12. Dancila, D., et al.: Wide band on-chip slot antenna with back-side etched trench for W-band sensing applications. In: 7th European Conference on Antennas and Propagation (EuCAP), pp. 1576–1579 (2013) 13. Shan, X., Shen, Z.: Miniaturized UHF/UWB tag antenna. IEEE Antennas Wirel. Propag. Lett. 18(12), 2453–2457 (2019) 14. Wang, Y., Wang, N., Denidni, T.A., Zeng, Q., Wei, G.: Integrated ultrawideband/narrowband rectangular dielectric resonator antenna for cognitive radio. IEEE Antennas Wirel. Propag. Lett. 13, 694–697 (2014) 15. Atallah, H.A., Abdel-Rahman, A.B., Yoshitomi, K., Pokharel, R.K.: Novel compact UWB monopole RDRA for cognitive radio spectrum sensing applications. In: IEEE International Conference on Consumer Electronics, ICCE, Berlin, pp. 462–466 (2015) 16. Dayo, Z.A., Cao, Q., Wang, Y., Soothar, P.: A compact high gain multiband bow-tie slot antenna. In: International Applied Computational Electromagnetics Society Symposium (ACES), Nanjing, China, vol. 1, pp. 1–2 (2019) 17. Gangwar, D., Sharma, A., Kanaujia, B.K., Singh, S.P., Lay-Ekuakille, A.: Characterization and performance measurement of low RCS wideband circularly polarized MIMO antenna for microwave sensing applications. IEEE Trans. Instrum. Meas. 69(6), 3847–3854 (2020) 18. Trivedi, K.: Mutual Coupling Reduction using Defected Ground Structure in UWB DRA Array for MIMO Application, pp. 1879–1880 (2020) 19. Tang, Z., Wu, X., Zhan, J., Hu, S., Xi, Z., Liu, Y.: Compact UWB-MIMO antenna with high isolation and triple band-notched characteristics. IEEE Access 7, 19856–19865 (2019) 20. Dayo, Z.A., Cao, Q., Soothar, P., Lodro, M.M., Li, Y.: A compact coplanar waveguide feed bowtie slot antenna for WIMAX, C and X band applications. In: IEEE International Conference on Computational Electromagnetics (ICCEM), Shanghai China, vol. 26(3), pp. 1–3 (2019) 21. Hussain, R., Sharawi, M.S.: A cognitive radio reconfigurable MIMO and sensing antenna system. IEEE Antennas Wirel. Propag. Lett. 14, 257–260 (2015) 22. Emadeddin, A., Salari, M.A., Zoghi, M., Darvazehban, A., Manoochehri, O.: A compact ultrawideband multibeam antenna system. IEEE Trans. Antennas Propag. 66(1), 125–131 (2018) 23. Srivastava, G., Mohan, A., Chakrabarty, A.: Compact reconfigurable UWB slot antenna for cognitive radio applications. IEEE Antennas Wirel. Propag. Lett. 16, 1139–1142 (2017) 24. Tawk, Y., Costantine, J., Christodoulou, C.G.: A rotatable reconfigurable antenna for cognitive radio applications. In: IEEE Radio and Wireless Symposium, pp. 158–161 (2011) 25. Chatterjee, A., Kumar, A., Midya, M.: Design of a Novel 2-D Printed Inverted-F Antenna with Defected Ground Structure for Ultra Wide Band Applications, pp. 3–6 (2019) 26. Elhabchi, M., Srifi, M.N., Touahni, R.: Bandwidth enhancement of trapezoid antenna using an open F-rotated and I-shaped slots defected ground structures (DGS). In: International Symposium on Advanced Electrical and Communication Technologies, pp. 1–4 (2019)

A Compact Wideband Planar Monopole Antenna with Defected …

455

27. Hua, C., Lu, Y., Liu, T.: UWB heart-shaped planar monopole antenna with a reconfigurable notched band. Prog. Electromagn. Res. Lett. 65(2016), 123–130 (2017) 28. Yeo, J.: Miniaturized UWB Stepped Open-Slot Antenna, vol. 78(8), pp. 119–127 (2018) 29. Iman, Z., Dautov, K., Hashmi, M., Chaudhary, M.A.: Symmetrically Slotted Ground Defected UWB Antenna Configurations for Microwave Imaging Techniques, pp. 825–827 (2020) 30. Keshwala, U., Ray, K.: A novel small-scale heart shape monopole antenna with DGS for UWB. In: International Conference on Computational Intelligence and Knowledge Economy, vol. 4, pp. 36–39 (2019) 31. Kadam, P.A., Deshmukh, A.A.: Modified Compact Broadband High Gain Defected Ground Plane Microstrip Antenna, vol. 1, pp. 8–11 (2019)